Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

How to retrieve the option in actions_intent_option #52

Closed
chaoranxie opened this issue Jan 23, 2018 · 10 comments
Closed

How to retrieve the option in actions_intent_option #52

chaoranxie opened this issue Jan 23, 2018 · 10 comments

Comments

@chaoranxie
Copy link
Contributor

from the doc https://developers.google.com/actions/assistant/helpers it looks like we should be able to access actions_intent_option value by app.getContextArgument('actions_intent_option', 'OPTION').value

How do we do the same thing with flask-assistant? Thanks

@treethought
Copy link
Owner

Hey @chaoranxie. I believe those helper functions are new in Actions V2 API. Unfortunately the library does not yet support v2. This means you need to uncheck the "Use Actions on Google Conversation API V2" option in DialogFlow's google assistant integration settings.

I am working on updating the library to support V2, but in the meantime you can access the information from google assistant via flask-assistant's request object.

from flask_assistant import ask, tell, request
...
google_data = request['originalRequest']['data']

The current implementation of list selectors pass the selected option as a "User Says" phrase to dialogflow. So if you select an option with the key of "monday", then your agent will receive a query of "monday" which will be used to match an appropriate intent.

Sorry for the inconvenience, I'll try to have the V2 support ready soon

@chaoranxie
Copy link
Contributor Author

chaoranxie commented Jan 25, 2018

Hi @treethought I am using v1 of the API right now. The current implementation of list selectors pass the selected option as a "User Says" phrase to dialogflow. , this is exactly what i want, i have a help intent, which build a list of sample user says, the idea is that user can select from the list and the key/title of the selected item will be treated as "user says" and match my find-event intent.

though that's not what's happening. any idea on how i can troubleshoot? here is the information i found.

  1. does not matter which item i click, assistant will always say What was that?, but if i click on the item again(or any other item), it will treat it as "user say". seem like the first click is broken.

  2. I tested that each item's key/title will result in trigging find-event intent if i say it or text it instead of select from list

  3. I could add actions_intent_OPTION event to my find-event intent, then it will always trigger my find-event intent but because it is not treated as "user say" so none of the parameters will be resolved.

@chaoranxie
Copy link
Contributor Author

so the help intent is the one building the list, and the find-event intent is the one supposed to handle the selected item. do you know if there is anything i need to do to make sure, the key/title of the item gets treated as "user say", i am not sure if i need to set something on help intent or find-event intent.

@chaoranxie
Copy link
Contributor Author

analyzing the request parameter
it looks like the first time i click the item, this is the info

"resolvedQuery": "actions_intent_OPTION",

"data": {
      "inputs": [
        {
          "rawInputs": [
            {
              "query": "event for Young Members",
              "inputType": "VOICE"
            }
          ],
          "intent": "actions.intent.OPTION",
          "arguments": [
            {
              "textValue": "event for Young Members",
              "name": "OPTION"
            }
          ]
        }

while the second time i click the item on the list

    "resolvedQuery": "event for Young Members",
"data": {
      "inputs": [
        {
          "rawInputs": [
            {
              "query": "event for Young Members",
              "inputType": "TOUCH"
            }
          ],
          "intent": "actions.intent.TEXT",
          "arguments": [
            {
              "rawText": "event for Young Members",
              "textValue": "event for Young Members",
              "name": "text"
            }
          ]
        }
      ],

@treethought
Copy link
Owner

Interesting. Thanks for this write up.
Is your previous comment showing the info received after you added actions_intent_OPTION as an event for your find-event intent?

@treethought
Copy link
Owner

treethought commented Jan 25, 2018

I just tested the actions_demo from this repo and found that everything works as expected (including list option selection) when interacting with the agent via the DialogFlow console. (On the right side you can click the down arrow next to "Default response" to view what google assistant responses should look like).

However, I am running into the same issue when testing inside the Actions on Google Simulator.

This leads me to believe that somewhere in the 2 platforms' migration to V2 of actions on google, something was changed to where flask-assistant integrates properly with Dialogflow and Dialogflow integrates properly with Actions. But the communication from Actions back to DIalogflow and then to webhook is no longer aligned with the request/response format that was followed when originally implementing the integration inside flask-assistant.

I'm sorry I don't have a solid answer right now. I will look into this tomorrow and get back to you. Thanks again for your info, it's really helpful.

@chaoranxie
Copy link
Contributor Author

Yes, my previous comments show when i do add actions_intent_OPTION event to find-event intent, instead of receiving resolvedQuery of event for Young Members, i got actions_intent_OPTION, which i guess is why the parameters/slots for the are not populated.

I confirmed that it works on the DialogFlow console for me too, didn't know you can do that ^__^.

Don't be sorry, you are doing amazing work, and all the hard groundworks so the rest of us can just enjoy building assistant. This is not a deal breaker for me, just thought I would let you know.

Feel free to post any information you find here, like blog article, stackoverflow questions if any here. i am still very new to the ecosystem, would love to learn more and maybe help out.

@treethought
Copy link
Owner

Thanks for the kind words @chaoranxie. Though not always relevant to flask-assistant the DialogFlow forums are often a great resource. As you become more familiar with the space (or before) always feel free to contribute!

In regards to the issue, I now have simple responses, cards, and lists working for actions v2 :). Not quite ready to merge into master, but will be soon. In the mean time you can checkout the actions-v2 branch if you feel like it. There's a few API differences which I can help you with until I have the docs updated.

@treethought
Copy link
Owner

treethought commented Dec 19, 2018

While troubleshooting this issue with dialogflow V2, I have found that selecting an item automatically creates a context named actions_intent_option. Therefore, to easily handle the selection you can set up action functions like the following:

Note that this applies only when interacting inside google assistant or the actions simulator. Within the dialogflow, the selected item's key is simply sent as a user phrase and should invoke the corresponding intent

@assist.action('create-list')
def show_list():
    speech = "Please select an option"
    resp = ask(speech)
    my_list = resp.build_list("List Title")
    for i in my_items:
        my_list.add_item(i['title'], i['key'])

    # set a new context in case you have multiple intents with lists
    context_manager.add('my-list-1-selection')
    return my_list

@assist.context("my-list-1-selection")
@assist.action("Option-Select-Intent")
def handle_list_option_selection():
    """Special intent to handle option selection when using Google Assistant

    Intent needs to have the actions_intent_OPTION event,
    which will be invoked when an item is selected.
    The action func parses the option key and then calls
    the corresponding action function for the selection
    """
    # this context was automatically created
    c = context_manager.get("actions_intent_option")
    key = c.get("OPTION")
    return some_action_function(key)

# then to handle a different list's selection
# require the context set in that list's intent
# all list selections need to trigger the same intent
# via the action_intent_OPTION event

@assist.context("my-list-2-selection")
@assist.action("Option-Select") 
def handle_other_list_option_selection():
    c = context_manager.get("actions_intent_option")
    key = c.get("OPTION")
    return some_other_action_func(key)

@treethought treethought pinned this issue Dec 19, 2018
@chaoranxie
Copy link
Contributor Author

Hey @treethought, thanks for the update. I should start refactoring my code to decouple it from DialogFlow v1,v2 API so that I can move it to v2 without too much changes.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

2 participants