I'm trying to create a multilanguage bot by detecting the language and selecting the proper set of LUIS keys and strings. My problem is, that my LuisDialog serializes itself and the MakeRoot method is not being called anymore.
My code (roughly):
public async Task<HttpResponseMessage> Post([FromBody]Activity activity)
{
if (activity.Type == ActivityTypes.Message)
{
var languageData = DetectLanguage(activity); // here I have the keys, strings etc.
await Conversation.SendAsync(activity, () => new Dialogs.RootDialog(languageData));
}
else
{
HandleSystemMessage(activity);
}
var response = Request.CreateResponse(HttpStatusCode.OK);
return response;
}
I've tried with the intermediate dialog, which selects the language and context.Forward everything to the LuisDialog, but I'm struggling with managing that. If this is a good strategy, I can share more code. I'm also considering scoreables.
If your need is about switching the language of your LUIS Dialog at its start
You have to make a method to get each LUIS parameter by language and as you know the language with your DetectLanguage, choose the right ones.
Then pass them to your LuisDialog like the following:
public async Task<HttpResponseMessage> Post([FromBody]Activity activity)
{
if (activity.Type == ActivityTypes.Message)
{
var languageData = DetectLanguage(activity); // here I have the keys, strings etc.
var luisService = new LuisService(new LuisModelAttribute("yourLuisAppIdGivenTheLanguageData", "yourLuisAppKeyGivenTheLanguageData", domain: "yourLuisDomainGivenTheLanguageData"));
await Conversation.SendAsync(activity, () => new Dialogs.RootDialog(luisService));
}
else
{
HandleSystemMessage(activity);
}
var response = Request.CreateResponse(HttpStatusCode.OK);
return response;
}
And your RootDialog shoud look like this:
public class RootDialog : LuisDialog<object>
{
public RootDialog(params ILuisService[] services) : base(services)
{
}
If you have a more complex project
We implemented a complex project allowing language switch at any time. Due to this possibility, activity's locale field could not be fully trusted even if you override it on the beginning of its processing.
The strategy is the following:
Detect user's language in MessagesController's Post method, for every incoming message
Store the user language in bot state (userData) when it's changing
Then:
When displaying a text, get the value in the right language using userData
When using a language specific tool like LUIS, get the right parameter using userData
You will need an intermediate RootDialog to handle this LUIS language switch, and you have to complete your LuisDialog after every detection (or to check the language before every MessageReceived on your LuisDialog).
Related
I have a bot written in C# that is using LUIS to determine intents. I have a method that makes a call to the LUIS service and then looks for an 'Open_Case' intent. The model has a CaseNumber entity defined which may or may not be included in the response from the LUIS service.
If the response doesn't have a case number entity I start a dialog to ask the user for the case number.
Once I have a case number I then want to return a card with case information.
Here's the code I have:-
/// <summary>
/// Dispatches the turn to the requested LUIS model.
/// </summary>
private async Task DispatchToLuisModelAsync(ITurnContext context, string appName, DialogContext dc, CancellationToken cancellationToken =
default (CancellationToken)) {
var result = await botServices.LuisServices[appName].RecognizeAsync(context, cancellationToken);
var intent = result.Intents ? .FirstOrDefault();
string caseNumber = null;
if (intent ? .Key == "Open_Case") {
if (!result.Entities.ContainsKey("Case_CaseNumber")) {
var dialogResult = await dc.BeginDialogAsync(CaseNumberDialogId, null, cancellationToken);
} else {
caseNumber = (string)((Newtonsoft.Json.Linq.JValue) result.Entities["Case_CaseNumber"].First).Value;
var cardAttachment = botServices.CaseInfoServices.LookupCase(caseNumber);
var reply = context.Activity.CreateReply();
reply.Attachments = new List < Attachment > () {
cardAttachment
};
await context.SendActivityAsync(reply, cancellationToken);
}
}
}
What I'm struggling with is where the code send the card response should sit.
In the code I currently have I send the card if the number was returned in the LUIS response, but if there was no number and I start the dialog then I only get access to the number either in the final step of the dialog or in the dialog result in the root turn handler. I've currently duplicated the reply inside the final step in the dialog, but it feels wrong and inelegant.
I'm sure there must be a way that I can collect the number from LUIS or the dialog and THEN send the response from a single place instead of duplicating code.
Any suggestions gratefully received...
I came to the conclusion that I need to put the code that displays the card into a method on the bot class, then call it from the else in code snippet and also from the turn handler when the dialogTurnStatus is equal to Complete
I have this code but but i think its over-complicated and can be simplified.
Also is there a way to go back to a spefici waterfall step if ever the user type "back" without restarting the whole dialog? I am new to this and it's hard to find a guide or online course on botframework v4 since it is new. Any help would be appreciated thanks!
public GetNameAndAgeDialog(string dialogId, IEnumerable<WaterfallStep> steps = null) : base(dialogId, steps)
{
var name = "";
var age = "";
AddStep(async (stepContext, cancellationToken) =>
{
return await stepContext.PromptAsync("textPrompt",
new PromptOptions
{
Prompt = stepContext.Context.Activity.CreateReply("What's your name?")
});
});
AddStep(async (stepContext, cancellationToken) =>
{
name = stepContext.Result.ToString();
return await stepContext.PromptAsync("numberPrompt",
new PromptOptions
{
Prompt = stepContext.Context.Activity.CreateReply($"Hi {name}, How old are you ?")
});
});
AddStep(async (stepContext, cancellationToken) =>
{
age= stepContext.Result.ToString();
return await stepContext.PromptAsync("confirmPrompt",
new PromptOptions
{
Prompt = stepContext.Context.Activity.CreateReply($"Got it you're {name}, age {age}. {Environment.NewLine}Is this correct?"),
Choices = new[] {new Choice {Value = "Yes"},
new Choice {Value = "No"},
}.ToList()
});
});
AddStep(async (stepContext, cancellationToken) =>
{
var result = (stepContext.Result as FoundChoice).Value;
if(result == "Yes" || result == "yes" || result == "Yeah" || result == "Correct" || result == "correct")
{
var state = await (stepContext.Context.TurnState["FPBotAccessors"] as FPBotAccessors).FPBotStateAccessor.GetAsync(stepContext.Context);
state.Name = name;
state.Age = int.Parse(age);
return await stepContext.BeginDialogAsync(MainDialog.Id, cancellationToken);
}
else
{
//restart the dialog
return await stepContext.ReplaceDialogAsync(GetNameAndAgeDialog.Id);
}
});
}
public static string Id => "GetNameAndAgeDialog";
public static GetNameAndAgeDialog Instance { get; } = new GetNameAndAgeDialog(Id);
}
And this is my accessors code:
public class FPBotAccessors
{
public FPBotAccessors(ConversationState conversationState)
{
ConversationState = conversationState ?? throw new ArgumentNullException(nameof(conversationState));
}
public static string FPBotAccessorName { get; } = $"{nameof(FPBotAccessors)}.FPBotState";
public IStatePropertyAccessor<FPBotState> FPBotStateAccessor { get; internal set; }
public static string DialogStateAccessorName { get; } = $"{nameof(FPBotAccessors)}.DialogState";
public IStatePropertyAccessor<DialogState> DialogStateAccessor { get; internal set; }
public ConversationState ConversationState { get; }
//
public static string ConversationFlowName { get; } = "ConversationFlow";
public IStatePropertyAccessor<ConversationFlow> ConversationFlowAccessor { get; set; }
}
So, there are a couple issues with your code and things you can do to make it better.
State within the dialog
First, let's start with the fact that you're closing over local variables in your constructor and accessing those from the closures that represent your steps. This "works" right now but is ultimately flawed. The initial flaw is different depending on the approach you've taking with instancing your GetNameAndAgeDialog dialog.
If you're using it as a singleton, that means all active conversations between users and your bot would be going through that one instance and you would have a concurrency issue where two users talking to the bot at the same time would be storing their values into the same memory (those variables) and stepping on each other's data.
It's also possible, depending on which samples you're following, that you're instead instantiating your GetNameAndAgeDialog on every turn. This would mean that those variables are then initialized to an empty string on every turn of the conversation and you'd lose track of the original values.
Ultimately, regardless of the instancing used, the approach ends up being flawed no matter what when it comes to scale out because at best your state would be pegged to a single server instance and if one turn of the conversation took place on ServerA and the next turn of the conversation took place on ServerM then ServerM would not have the values from the previous turn.
Alright, so clearly you need to store them with some kind of proper state management mechanism. You're clearly somewhat familiar with using BotState (be it conversation or user scope) already being that you're already using the state property accessors, but it's probably premature to store values you're collecting throughout a multi-turn prompt into someplace more permanent until you're at the end of the collection process. Luckily, dialogs themselves are stored into state, which you may have figured out when you set up a state property accessor for DialogState, and therefore offer a temporarily persistence mechanism that is tied to each dialog's lifetime on the dialog stack. Using this state is not obvious or documented well (yet), but WaterfallDialog actually goes a step further and exposes a first class Values collection via its WaterfallStepContext companion class which is fed into each step. This means that each step of your waterfall flow can add values into the Values collection and access values that previous steps may have put into there. There is a pretty good sample of this in the documentation page titled Create advanced conversation flow using branches and loops.
Not Making The Best Use of Prompts
You're using a TextPrompt for name which is perfect and you'll get a string from it and be all set. Though you might want to consider throwing a validator on there to make sure you get something that looks like a name instead of just allowing any old value.
You appear to be using a NumberPrompt<T> for the age (judging by the name "numberPrompt" at least), but then you .ToString() the step.Result and ultimately do an int.Parse in the final step. Using a NumberPrompt<int> would guarantee you get an int and you can/should just use that value as is rather than turning it back into a string and then parsing it yourself again later.
You've got a prompt named "confirmPrompt", but it does not appear to be an actual ConfirmPrompt because you're doing all the Choice work and positive value detection (e.g. checking for variations of "Yes") yourself. If you actually use a ConfirmPrompt it will do this all of this for you and its result will be a bool which you can then just easily test in your logic.
Minor stuff
Currently you're using stepContext.Context.Activity.CreateReply to create activities. This is fine, but long winded and unecessary. I would highly recommend just using the MessageFactory APIs.
I would always make sure to pass the CancellationToken through to all the XXXAsync APIs that take it... it's just good practice.
Your final step either restarts the GetNameAndAgeDialog if they don't confirm the details or starts the MainDialog if they do confirm the details. Restarting with ReplaceDialogAsync is awesome, that's the right way to do it! I just wanted to point out that by using BeginDialogAsync to start the MainDialog means that you're effectively leaving the GetNameAndAgeDialog at the bottom of the stack for the remainder of the conversation's lifetime. It's not a huge deal, but considering you'll likely never pop the stack back to there I would instead suggest using ReplaceDialogAsync for starting up the MainDialog as well.
Refactored Code
Here is the code rewritten using all of the advice above:
public GetNameAndAgeDialog(string dialogId, IEnumerable<WaterfallStep> steps = null) : base(dialogId, steps)
{
AddStep(async (stepContext, cancellationToken) =>
{
return await stepContext.PromptAsync("textPrompt",
new PromptOptions
{
Prompt = MessageFactory.Text("What's your name?"),
},
cancellationToken: cancellationToken);
});
AddStep(async (stepContext, cancellationToken) =>
{
var name = (string)stepContext.Result;
stepContext.Values["name"] = name;
return await stepContext.PromptAsync("numberPrompt",
new PromptOptions
{
Prompt = MessageFactory.Text($"Hi {name}, How old are you ?"),
},
cancellationToken: cancellationToken);
});
AddStep(async (stepContext, cancellationToken) =>
{
var age = (int)stepContext.Result;
stepContext.Values["age"] = age;
return await stepContext.PromptAsync("confirmPrompt",
new PromptOptions
{
Prompt = MessageFactory.Text($"Got it you're {name}, age {age}.{Environment.NewLine}Is this correct?"),
},
cancellationToken: cancellationToken);
});
AddStep(async (stepContext, cancellationToken) =>
{
var result = (bool)stepContext.Result;
if(result)
{
var state = await (stepContext.Context.TurnState["FPBotAccessors"] as FPBotAccessors).FPBotStateAccessor.GetAsync(stepContext.Context);
state.Name = stepContext.Values["name"] as string;
state.Age = stepContext.Values["age"] as int;
return await stepContext.ReplaceDialogAsync(MainDialog.Id, cancellationToken: cancellationToken);
}
else
{
//restart the dialog
return await stepContext.ReplaceDialogAsync(GetNameAndAgeDialog.Id, cancellationToken: cancellationToken);
}
});
}
Also is there a way to go back to a spefici waterfall step if ever the user type "back" without restarting the whole dialog?
No, there is not a way to do this today. The topic has come up in internal discussions with the team, but nothing has been decided yet. If you think this is a feature that would be useful, please submit an issue over on GitHub and we can see if it gains enough momentum to get the feature added.
I'm trying to build a Dialog using the Microsoft Bot Framework which helps users consult purchase order status (currently, just a mock). I am using a LuisDialog which, when it detects the "ConsultPO" intent, it's supposed to ask for the user's 'customer id' and wait a follow up message from the user. However, it keeps going back to the start of the Luis Dialog and processing the intent instead of resuming from the waited method. This is the intent's code, which runs correctly:
[LuisIntent("ConsultPO")]
public async Task POIntent(IDialogContext context, LuisResult result)
{
string PO = "";
foreach (var entity in result.Entities)
{
if (entity.Type == "purchaseOrder")
PO = entity.Entity;
}
if (PO.Length != 0)
{
po_query = PO;
}
await context.PostAsync("Ok, can you confirm your customer id and I'll check for you?");
context.Wait(confirmCustomer_getPO);
}
This is the code I would expect to be executed after the user responds with a follow up message:
public async Task confirmCustomer_getPO(IDialogContext context, IAwaitable<object> argument)
{
await context.PostAsync("DEBUG TEST");
IMessageActivity activity = (IMessageActivity)await argument;
customer_query = activity.Text;
if (po_query.Length > 0)
{
PurchaseOrder po = POservice.findPO(po_query, customer_query);
await buildSendResponse(po, context);
//more non relevant code
When I answer to the bot's inquiry after context.Wait(confirmCustomer_getPO) is executed, it just goes into LUIS then runs the code respective to "None" intent. The message "DEBUG TEST" is never sent.
Why is "confirmCustomer_getPO" never getting called?
EDIT:
I added a debug message in the StartAsync method. I'm not sure whether this is supposed to happen but it pops up every time I send a message to the bot, which makes me believe the Dialog is simply restarting every time I message the bot:
public class EchoDialog : LuisDialog<object>
{
public EchoDialog() : base(new LuisService(new LuisModelAttribute(
ConfigurationManager.AppSettings["LuisAppId"],
ConfigurationManager.AppSettings["LuisAPIKey"],
domain: ConfigurationManager.AppSettings["LuisAPIHostName"])))
{
}
public override Task StartAsync(IDialogContext context)
{
context.PostAsync("I'm in startAsync");
return base.StartAsync(context);
}
Local debugging shows no exceptions are occurring and that any breakpoint in the waited method is never reached, although the context.Wait call does happen.
I figured out the issue myself after fighting with it for a while. The issue was with the bot store. I was using an InMemoryDataStore which was not working - switching to TableBotDataStore fixed the problem. The issue with the DataStore meant that states weren't being saved so my "waits" and "forwards" were not being saved into the dialog stack - any new incoming message was sent to the RootDialog.
Broken - not working while this was in global.asax.cs:
Conversation.UpdateContainer(
builder =>
{
builder.RegisterModule(new AzureModule(Assembly.GetExecutingAssembly()));
var store = new InMemoryDataStore(); // volatile in-memory store
builder.Register(c => store)
.Keyed<IBotDataStore<BotData>>(AzureModule.Key_DataStore)
.AsSelf()
.SingleInstance();
});
GlobalConfiguration.Configure(WebApiConfig.Register);
As soon as I updated store to:
var store = new TableBotDataStore(ConfigurationManager.AppSettings["AzureWebJobsStorage"]);
Having a valid "AzureWebJobsStorage" setting in web.config from my application settings in Azure, the problem was fixed without any other changes in the code.
We're experimenting with the MS Bot Framework and haven't quite worked out how to do this scenario:
We have a LUIS Dialog (type <object>), which is working correctly and is trained properly. To use the common sandwich example, the basics of what LUIS intent is looking for is the user asking for the status of an order. If the order number was provided in the question ("What is the status of order 1234?"), then the LUIS dialog does the lookup and reports the status directly (which is all currently working).
However, if the user just triggers the intent without providing the order number ("I'd like to look up the status of an order."), I'd like to launch another dialog/form to ask the user if they'd like to look up the order by address or order number, and then do the appropriate DB lookup based on how they answer.
I'm just not sure how to configure the Form/Dialog (or even which is best in this case) to do a different lookup based on if they choose address or number lookup.
Here's the intent so far:
private readonly BuildFormDelegate<OrderStatusDialog> OrderStatusDelegate;
[LuisIntent(nameof(LuisIntents.OrderStatus))]
public async Task OrderStatus(IDialogContext context, LuisResult result)
{
// Order number(s) were provided
if (result.Entities.Any(Entity => Entity.Type == nameof(LuisEntityTypes.OrderNumber)))
{
// Loop in case they asked about multiple orders
foreach (var entity in result.Entities.Where(Entity => Entity.Type == nameof(LuisEntityTypes.OrderNumber)))
{
var orderNum = entity.Entity;
// Call webservice to check status
var request = new RestRequest(Properties.Settings.Default.GetOrderByNum, Method.GET);
request.AddUrlSegment("num", orderNum);
var response = await RestHelper.SendRestRequestAsync(request);
var parsedResponse = JObject.Parse(response);
if ((bool)parsedResponse["errored"])
{
await context.PostAsync((string)parsedResponse["errMsg"]);
continue;
}
// Grab status from returned JSON
var status = parsedResponse["orderStatus"].ToString();
await context.PostAsync($"The status of order {orderNum} is {status}");
}
context.Wait(MessageReceived);
}
// Order number was not provided
else
{
var orderStatusForm = new FormDialog<OrderStatusDialog>(new OrderStatusDialog(), OrderStatusDelegate,
FormOptions.PromptInStart);
context.Call<OrderStatusDialog>(orderStatusForm, CallBack);
}
}
private async Task CallBack(IDialogContext context, IAwaitable<object> result)
{
context.Wait(MessageReceived);
}
And the form:
public enum OrderStatusLookupOptions
{
Address,
OrderNumber
}
[Serializable]
public class OrderStatusDialog
{
public OrderStatusLookupOptions? LookupOption;
public static IForm<OrderStatusDialog> BuildForm()
{
return new FormBuilder<OrderStatusDialog>()
.Message("In order to look up the status of a order, we will first need either the order number or your delivery address.")
.Build();
}
}
The FormFlow route is a valid option. What is missing in your form flow is asking for the address/order number after the lookup option is selected.
What you can do in that case is adding two more fields to the OrderStatusDialog class: OrderNumber and DeliveryAddress.
Then you need to use the selected OrderStatusLookupOptions to activate/deactivate the next field.
The code, from the top of my head, would be something like:
[Serializable]
public class OrderStatusDialog
{
public OrderStatusLookupOptions? LookupOption;
public int OrderNumber;
public string DeliveryAddress
public static IForm<OrderStatusDialog> BuildForm()
{
return new FormBuilder<OrderStatusDialog>()
.Message("In order to look up the status of a order, we will first need either the order number or your delivery address.")
.Field(nameof(OrderStatusDialog.LookupOption))
.Field(new FieldReflector<OrderStatusDialog>(nameof(OrderStatusDialog.OrderNumber))
.SetActive(state => state.LookupOption == OrderStatusLookupOptions.OrderNumber))
.Field(new FieldReflector<OrderStatusDialog>(nameof(OrderStatusDialog.DeliveryAddress))
.SetActive(state => state.LookupOption == OrderStatusLookupOptions.Address))
.Build();
}
}
Then on your Callback method you will receive the form filled and you can do the DB lookup.
Alternatively, you can just use PromptDialogs and guide the user through the same experience. Take a look to the MultiDialogs sample to see the different alternatives.
I added a working sample on this here.
I started learn the Microsoft Bot Framework recently, so I started make a Chatbot and I guess I making it wrong
I making the chatbot this way:
--> I get the message's user
--> send to LUIS
--> get the intent and the entities
--> select my answer and send it.
it's ok, but get the following situation:
USER: I wanna change my email. --> intent : ChangeInfo entities:
email/value:email
CHATBOT: Tell me your new Email please. --> intent: noIntent
entities: noEntities
USER: email#email.com --> intent: IDon'tKnow entities:
email/value:email#email.com
I take this situation, when the USER send his email, I send to LUIs, but a email dont have a intent, just have a entity, but a email can be used in a lot Different situations, My question is, How My bot know the context of conversation to understand this email is for change email and not send a email, or update this email or another thing.
my code on gitHub here, its a ugly code, i know, but i make this just to understand the bot framework, after I will let this code more beautiful
This should be as simple as using a LuisDialog and a set of Prompts to manage the users flow. Below you will find some quick code I put together to show you how this could be done. You don't need extra steps or adding extra entities, or going to Luis with the email provided by the user.
I would recommend you to read a bit more about LuisDialog and Dialogs in general as the way you are using Luis in your controller I don't think is the way to go.
Here is a good Luis Sample and here a good one around multi-dialogs.
Sample Code
namespace MyNamespace
{
using System;
using System.Threading.Tasks;
using Microsoft.Bot.Builder.Dialogs;
using Microsoft.Bot.Builder.Internals.Fibers;
using Microsoft.Bot.Builder.Luis;
using Microsoft.Bot.Builder.Luis.Models;
using Microsoft.Bot.Connector;
[Serializable]
[LuisModel("YourModelId", "YourSubscriptionKey")]
public class MyLuisDialog : LuisDialog<object>
{
[LuisIntent("")]
[LuisIntent("None")]
public async Task None(IDialogContext context, LuisResult result)
{
string message = "Não entendi, me diga com outras palavras!";
await context.PostAsync(message);
context.Wait(this.MessageReceived);
}
[LuisIntent("ChangeInfo")]
public async Task ChangeInfo(IDialogContext context, IAwaitable<IMessageActivity> activity, LuisResult result)
{
// no need to go to luis again..
PromptDialog.Text(context, AfterEmailProvided, "Tell me your new email please?");
}
private async Task AfterEmailProvided(IDialogContext context, IAwaitable<string> result)
{
try
{
var email = await result;
// logic to store your email...
}
catch
{
// here handle your errors in case the user doesn't not provide an email
}
context.Wait(this.MessageReceived);
}
[LuisIntent("PaymentInfo")]
public async Task Payment(IDialogContext context, IAwaitable<IMessageActivity> activity, LuisResult result)
{
// logic to retrieve the current payment info..
var email = "test#email.com";
PromptDialog.Confirm(context, AfterEmailConfirmation, $"Is it {email} your current email?");
}
private async Task AfterEmailConfirmation(IDialogContext context, IAwaitable<bool> result)
{
try
{
var response = await result;
// if the way to store the payment email is the same as the one used to store the email when going through the ChangeInfo intent, then you can use the same After... method; otherwise create a new one
PromptDialog.Text(context, AfterEmailProvided, "What's your current email?");
}
catch
{
// here handle your errors in case the user doesn't not provide an email
}
context.Wait(this.MessageReceived);
}
}
}
In my bot flow, I'm using a step variable that I change from the front-end. And another step variable that I change from the bot. This helps me identify which step I am in the conversation. You can do the same to identify what your bot is asking the user.
var data = {step: "asked_email"};
var msg = builder.Message(session).addEntity(data).text("Your message.");
session.send(msg);
If you don't want to send a specific step to LUIS for recognition, you can handle that in the onBeginDialog handler:
intents.onBegin(function (session, args, next) {
if (session.message.step !== "email") {
next();
} else {
//Do something else and not go to LUIS.
session.endDialog();
}
});
You can find the reference to LUIS onBeginDialog here:
https://docs.botframework.com/en-us/node/builder/chat/IntentDialog/#onbegin--ondefault-handlers
Details about message data can be found here:
https://docs.botframework.com/en-us/node/builder/chat-reference/classes/_botbuilder_d_.message.html#entities