I have a bot written in C# that is using LUIS to determine intents. I have a method that makes a call to the LUIS service and then looks for an 'Open_Case' intent. The model has a CaseNumber entity defined which may or may not be included in the response from the LUIS service.
If the response doesn't have a case number entity I start a dialog to ask the user for the case number.
Once I have a case number I then want to return a card with case information.
Here's the code I have:-
/// <summary>
/// Dispatches the turn to the requested LUIS model.
/// </summary>
private async Task DispatchToLuisModelAsync(ITurnContext context, string appName, DialogContext dc, CancellationToken cancellationToken =
default (CancellationToken)) {
var result = await botServices.LuisServices[appName].RecognizeAsync(context, cancellationToken);
var intent = result.Intents ? .FirstOrDefault();
string caseNumber = null;
if (intent ? .Key == "Open_Case") {
if (!result.Entities.ContainsKey("Case_CaseNumber")) {
var dialogResult = await dc.BeginDialogAsync(CaseNumberDialogId, null, cancellationToken);
} else {
caseNumber = (string)((Newtonsoft.Json.Linq.JValue) result.Entities["Case_CaseNumber"].First).Value;
var cardAttachment = botServices.CaseInfoServices.LookupCase(caseNumber);
var reply = context.Activity.CreateReply();
reply.Attachments = new List < Attachment > () {
cardAttachment
};
await context.SendActivityAsync(reply, cancellationToken);
}
}
}
What I'm struggling with is where the code send the card response should sit.
In the code I currently have I send the card if the number was returned in the LUIS response, but if there was no number and I start the dialog then I only get access to the number either in the final step of the dialog or in the dialog result in the root turn handler. I've currently duplicated the reply inside the final step in the dialog, but it feels wrong and inelegant.
I'm sure there must be a way that I can collect the number from LUIS or the dialog and THEN send the response from a single place instead of duplicating code.
Any suggestions gratefully received...
I came to the conclusion that I need to put the code that displays the card into a method on the bot class, then call it from the else in code snippet and also from the turn handler when the dialogTurnStatus is equal to Complete
Related
The bot in Discord
As you can see, it responds to the request, but then doesn't do anything with the number I've entered. It just sits there.
Here is my code for that command (currently it's just copied from the DSharp tutorial website, I planned to modify it significantly, but it won't even do the standard):
[Command("waitforcode"), Description("Waits for a response containing a generated code.")]
public async Task WaitForCode(CommandContext ctx)
{
// first retrieve the interactivity module from the client
var interactivity = ctx.Client.GetInteractivity();
// generate a code
var codebytes = new byte[8];
using (var rng = RandomNumberGenerator.Create())
rng.GetBytes(codebytes);
var code = BitConverter.ToString(codebytes).ToLower().Replace("-", "");
// announce the code
await ctx.RespondAsync($"The first one to type the following code gets a reward: `{code}`");
// wait for anyone who types it
var msg = await interactivity.WaitForMessageAsync(xm => xm.Content.Contains(code), TimeSpan.FromSeconds(60));
if (!msg.TimedOut)
{
// announce the winner
await ctx.RespondAsync($"And the winner is: {msg.Result.Author.Mention}");
}
else
{
await ctx.RespondAsync("Nobody? Really?");
}
}
If anybody could help me, that would be great!
In JSON the tag "isContextOnly" is present (found in https://learn.microsoft.com/en-us/azure/cognitive-services/qnamaker/how-to/multiturn-conversation)
But with c# using Microsoft.Bot.Builder.Dialogs there is no way to get for example response.Answers[0].isContextOnly
Is there any solution?
One of the samples in the Microsoft Botframework dotnet samples repo, #49.QnABot-All-Features, is designed to handle the different features offered by QnA maker, including active learning and context-only. More specifically, the dialog class in the sample QnAMakerBaseDialog, inherits from QnAMakerDialog.
In the parent QnAMakerDialog, there's a method that checks for multi-turn context:
private async Task<DialogTurnResult> CheckForMultiTurnPromptAsync(WaterfallStepContext stepContext, CancellationToken cancellationToken)
{
var dialogOptions = ObjectPath.GetPathValue<QnAMakerDialogOptions>(stepContext.ActiveDialog.State, Options);
if (stepContext.Result is List<QueryResult> response && response.Count > 0)
{
// -Check if context is present and prompt exists
// -If yes: Add reverse index of prompt display name and its corresponding QnA ID
// -Set PreviousQnAId as answer.Id
// -Display card for the prompt
// -Wait for the reply
// -If no: Skip to next step
var answer = response.First();
if (answer.Context != null && answer.Context.Prompts.Count() > 0)
{
var previousContextData = ObjectPath.GetPathValue(stepContext.ActiveDialog.State, QnAContextData, new Dictionary<string, int>());
foreach (var prompt in answer.Context.Prompts)
{
previousContextData[prompt.DisplayText] = prompt.QnaId;
}
ObjectPath.SetPathValue(stepContext.ActiveDialog.State, QnAContextData, previousContextData);
ObjectPath.SetPathValue(stepContext.ActiveDialog.State, PreviousQnAId, answer.Id);
ObjectPath.SetPathValue(stepContext.ActiveDialog.State, Options, dialogOptions);
// Get multi-turn prompts card activity.
var message = QnACardBuilder.GetQnADefaultResponse(answer, dialogOptions.ResponseOptions.DisplayPreciseAnswerOnly);
await stepContext.Context.SendActivityAsync(message, cancellationToken).ConfigureAwait(false);
return new DialogTurnResult(DialogTurnStatus.Waiting);
}
}
return await stepContext.NextAsync(stepContext.Result, cancellationToken).ConfigureAwait(false);
}
This is part of the internal waterfall of getting a QnA answer from QnAMaker.ai. It does look for a context in if (answers.Context != null ....
In QueryResult.cs, Context is defined as:
/// <summary>
/// Gets or sets context for multi-turn responses.
/// </summary>
/// <value>
/// The context from which the QnA was extracted.
/// </value>
[JsonProperty(PropertyName = "context")]
public QnAResponseContext Context { get; set; }
So to answer your question, to get the isContextOnly tag results in C#, debug into the results from a QnAMaker query, and you'll find a attribute called Context. This is what you're looking for.
I am trying to customize the CoreBot example (https://github.com/microsoft/BotBuilder-Samples/tree/master/samples/csharp_dotnetcore/13.core-bot) , so it can also receive images in addition to text.
While there are plenty of good documentation (below) and responses on stackoverflow, i am new to C# and have difficulties to combine several pieces of code with the C# syntax.
https://learn.microsoft.com/en-us/azure/bot-service/bot-builder-howto-add-media-attachments?view=azure-bot-service-4.0&tabs=csharp
https://learn.microsoft.com/en-us/azure/bot-service/dotnet/bot-builder-dotnet-add-media-attachments?view=azure-bot-service-3.0
https://learn.microsoft.com/en-us/azure/bot-service/nodejs/bot-builder-nodejs-send-receive-attachments?view=azure-bot-service-3.0
how to send images which are in local folder in microsoft botframework sdk v4 using c#
Can a Bot receive image as message or attachment from a user
On the code below, i am inserting this piece of code in the CoreBot :
var activity = stepContext.Context.Activity
var reply = activity.CreateReply();
if (activity.Attachments != null && activity.Attachments.Any())
{
var messageText = stepContext.Options?.ToString() ?? "this seems to be an image an i am not yet able to understand it";
var promptMessage = MessageFactory.Text(messageText, messageText, InputHints.ExpectingInput);
return await stepContext.PromptAsync(nameof(TextPrompt), new PromptOptions { Prompt = promptMessage }, cancellationToken);
}
Below is the block of code in which i have inserted the "if image, then"
private async Task<DialogTurnResult> ActStepAsync(WaterfallStepContext stepContext, CancellationToken cancellationToken)
{
if (!_luisRecognizer.IsConfigured)
{
// LUIS is not configured, we just run the BookingDialog path with an empty BookingDetailsInstance.
return await stepContext.BeginDialogAsync(nameof(BookingDialog), new BookingDetails(), cancellationToken);
}
var activity = stepContext.Context.Activity;
if (activity.Attachments != null && activity.Attachments.Any())
{
var messageText = stepContext.Options?.ToString() ?? "this seems to be an image an i am not yet able to understand it";
var promptMessage = MessageFactory.Text(messageText, messageText, InputHints.ExpectingInput);
return await stepContext.PromptAsync(nameof(TextPrompt), new PromptOptions { Prompt = promptMessage }, cancellationToken);
}
// Call LUIS and gather any potential booking details. (Note the TurnContext has the response to the prompt.)
var luisResult = await _luisRecognizer.RecognizeAsync<FlightBooking>(stepContext.Context, cancellationToken);
switch (luisResult.TopIntent().intent)
{
case FlightBooking.Intent.BookFlight:
await ShowWarningForUnsupportedCities(stepContext.Context, luisResult, cancellationToken);
// Initialize BookingDetails with any entities we may have found in the response.
var bookingDetails = new BookingDetails()
{
// Get destination and origin from the composite entities arrays.
Destination = luisResult.ToEntities.Airport,
Origin = luisResult.FromEntities.Airport,
TravelDate = luisResult.TravelDate,
};
// Run the BookingDialog giving it whatever details we have from the LUIS call, it will fill out the remainder.
return await stepContext.BeginDialogAsync(nameof(BookingDialog), bookingDetails, cancellationToken);
I have also added AddDialog(new AttachmentPrompt(nameof(AttachmentPrompt))); in the waterfall declaration as below
public MainDialog(FlightBookingRecognizer luisRecognizer, BookingDialog bookingDialog, ILogger<MainDialog> logger)
: base(nameof(MainDialog))
{
_luisRecognizer = luisRecognizer;
Logger = logger;
AddDialog(new TextPrompt(nameof(TextPrompt)));
AddDialog(bookingDialog);
AddDialog(new AttachmentPrompt(nameof(AttachmentPrompt)));
AddDialog(new WaterfallDialog(nameof(WaterfallDialog), new WaterfallStep[]
{
IntroStepAsync,
ActStepAsync,
}));
// The initial child Dialog to run.
InitialDialogId = nameof(WaterfallDialog);
}
The issue is that the code the piece of code I added is not doing anything.
As mentionned, i am a noob to C# and any suggestion or observation would be greatly appreciated!
I'm surprised it even lets you compile with Any(). In my testing, Visual Studio threw build errors.
Change:
if (activity.Attachments != null && activity.Attachments.Any())
to
if (activity.Attachments != null && activity.Attachments.Count > 0)
The above answer assumes that the activity contains an attachment, but just isn't being caught. If activity doesn't even contain an attachment, there's something else wrong. In which case, please include your whole dialog or preferably, a link to your code/repo.
How to disable input/submit button actions in the previous conversation of BotChat - AdaptiveCards in the Microsoft Bot Framework (C#)
I'm imagining you want to display a card to the user that's meant to be used only once, such as a calendar reminder like the one seen in this example.
Bots are mostly meant to have the same kind of access to a channel that a human would, so they can't go back and modify the messages that have already been sent (unless the specific channel allows edits like Slack does). While you can't disable a button in a card that's already part of the conversation history, you can change the way your bot responds to the messages that are generated by that card. What you'll want to do is keep track of whether a button has been clicked and then respond differently when the button is clicked subsequent times.
Here's a basic example of some Dialog code that can respond to messages in three ways. If you type any message and send it to the bot, it will display a card with a button on it. If you click the button, it will say "You did it!" along with the ID of the button you clicked. If you click the same button again, it will say "You already did that!" again attaching the ID.
/// <summary>
/// You'll want a label like this to identify the activity
/// that gets generated in response to your submit button.
/// </summary>
private const string DO_SOMETHING = "DoSomething";
/// <summary>
/// This is passed into context.Wait() in your StartAsync method.
/// </summary>
private async Task MessageReceivedAsync(IDialogContext context,
IAwaitable<IMessageActivity> result)
{
var msg = await result;
if (!string.IsNullOrWhiteSpace(msg.Text))
{
// If msg.Text isn't null or white space then that means the user
// actually typed something, and we're responding to that with a card.
var reply = context.MakeMessage();
var attachment = MakeAdaptiveCardAttachment();
reply.Attachments.Add(attachment);
await context.PostAsync(reply);
}
else
{
// If the user didn't type anything then this could be an activity
// that was generated by your submit button. But we want to make sure
// it is by checking msg.Value.
dynamic value = msg.Value;
try
{
// If value doesn't have a type then this will throw a RuntimeBinderException
if (value != null && value.type == DO_SOMETHING)
{
string id = value.id;
// Check the ID to see if that particular card has been clicked before.
if (!context.PrivateConversationData.ContainsKey(id))
{
// This is how your bot will keep track of what's been clicked.
context.PrivateConversationData.SetValue(id, true);
await context.PostAsync("You did it! " + id);
}
else
{
await context.PostAsync("You already did that! " + id);
}
}
}
catch (Microsoft.CSharp.RuntimeBinder.RuntimeBinderException)
{
// Respond to messages that don't have values with a type (or id).
}
}
context.Wait(MessageReceivedAsync);
}
private static Attachment MakeAdaptiveCardAttachment()
{
var card = new AdaptiveCard();
// We need to identify this specific card if we want to allow multiple
// instances of the card to be clicked.
// A timestamp could work but a GUID will do.
var cardId = Guid.NewGuid().ToString();
card.Body.Add(new TextBlock() { Text = cardId });
card.Actions.Add(new SubmitAction()
{
Title = "Do something",
// The data we put inside this action will persist.
// I've found setting DataJson to be more reliable than using the Data property.
// Note that if your WebApiConfig.cs has a CamelCasePropertyNamesContractResolver
// (which is a default) and you use capitalized (Pascal case) identifiers,
// they may be converted to camel case and you won't be able to retrieve
// the data with the same identifiers.
DataJson = JsonConvert.SerializeObject(new
{
// We need a type to differentiate this action from other actions.
type = DO_SOMETHING,
// We need an id to differentiate this card from other cards.
id = cardId,
}),
});
return new Attachment()
{
ContentType = AdaptiveCard.ContentType,
Content = card,
};
}
Here's what it looks like in Bot Framework Emulator. Note that even after you've clicked one card and can't get the first response from that card, you can still get the first response from the other card.
I have a subdialog in a bot built using MS bot framework that starts as follows - the standard way:
public async Task StartAsync(IDialogContext context)
{
var msg = "Let's find your flights! Tell me the flight number, city or airline.";
var reply = context.MakeMessage();
reply.Text = msg;
//add quick replies here
await context.PostAsync(reply);
context.Wait(UserInputReceived);
}
This dialog is called using two different ways, depending on whether in the previous screen the user tapped a button that says "Flights" or immediately entered a flight number. Here is the code from the parent dialog:
else if (response.Text == MainOptions[2]) //user tapped a button
{
context.Call(new FlightsDialog(), ChildDialogComplete);
}
else //user entered some text instead of tapping a button
{
await context.Forward(new FlightsDialog(), ChildDialogComplete,
activity, CancellationToken.None);
}
Question: how can I know (from within the FlightsDialog) whether that dialog was called using context.Call() or context.Forward()? This is because in the case of context.Forward(), StartAsync() shouldn't output the prompt asking the user to enter the flight number - they already did this.
The best idea I have is to save a flag in the ConversationData or user data, as below, and access it from the IDialog, but I thought there could be a better way?
public static void SetUserDataProperty(Activity activity, string PropertyName, string ValueToSet)
{
StateClient client = activity.GetStateClient();
BotData userData = client.BotState.GetUserData(activity.ChannelId, activity.From.Id);
userData.SetProperty<string>(PropertyName, ValueToSet);
client.BotState.SetUserDataAsync(activity.ChannelId, activity.From.Id, userData);
}
Unfortunately Forward actually calls Call (and then does some other stuff afterwards), so your Dialog wouldn't be able to differentiate.
void IDialogStack.Call<R>(IDialog<R> child, ResumeAfter<R> resume)
{
var callRest = ToRest(child.StartAsync);
if (resume != null)
{
var doneRest = ToRest(resume);
this.wait = this.fiber.Call<DialogTask, object, R>(callRest, null, doneRest);
}
else
{
this.wait = this.fiber.Call<DialogTask, object>(callRest, null);
}
}
async Task IDialogStack.Forward<R, T>(IDialog<R> child, ResumeAfter<R> resume, T item, CancellationToken token)
{
IDialogStack stack = this;
stack.Call(child, resume);
await stack.PollAsync(token);
IPostToBot postToBot = this;
await postToBot.PostAsync(item, token);
}
From https://github.com/Microsoft/BotBuilder/blob/10893730134135dd4af4250277de8e1b980f81c9/CSharp/Library/Dialogs/DialogTask.cs#L196