I am trying to extract dateTime using LUIS.
Environment: BotFramework V4
Lang: c#/ .NET
I am using the built in recognizer to extract Date / Time provided by the user.
var recognizerResult = await_services.LuisServices[LuisKey].RecognizeAsync(turnContext, cancellationToken);
Newtonsoft.Json.Linq.JObject Entities1 = recognizerResult?.Entities;
Now when I am printing this JSON file I am not getting the " Value " field in the Resolution portion.This is the JSON response I am getting when i extract the entities.
Query String : fly on 20 mar return on 24 mar Entities Returned { "$instance": { “datetime”: [ { “startIndex”: 7, “endIndex”: 13, “text”: “20 mar”, “type”: “builtin.datetimeV2.date” }, { “startIndex”: 24, “endIndex”: 30, “text”: “24 mar”, “type”: “builtin.datetimeV2.date” } ] }, “datetime”: [ { “type”: “date”, “timex”: [ “XXXX-03-20” ] }, { “type”: “date”, “timex”: [ “XXXX-03-24” ] } ] }
I have tried using the includeAPIResult : true parameter when creating the recognizer. Still the same results. Kindly suggest what am I doing wrong.
And how can I extract proper Date time from the json response.
Yeah so this is an known issue right now if you choose to use the LuisRecognizer package. Essentially it "eats" important details of the raw LUIS response.
My professional recommendation would be to eschew LuisRecognizer and just use the full on LuisClient directly. The only real reason to use LuisRecognizer is if you want that extra level of abstraction that allows you to work with other recognizer implementations. If you're "all in" on LUIS, then you probably want access to the full fidelity of LUIS anyway.
I do something like this:
luisResults.Entities.MyDateEntity?[0].Expressions?[0]
It will then result to a string that I parse to dateTime. Given that luis successfully sets MyDateEntity value.
Related
I have a file. The file has the following text:
[
{
_id: ObjectId("5da08d49949b4c000100b90b"),
ModifiedOn: ISODate("2019-10-11T14:10:17.461Z"),
DateOfCreation: ISODate("2019-10-11T14:10:17.459Z"),
DateModified: ISODate("2019-10-11T14:10:17.459Z"),
Region: null,
UniqueNumber: Long("465561"),
Numiration: 1,
Code: '001Е',
User: { ID: ObjectId("someid") },
}
]
I tried to do the following.
JsonConvert.DeserializeObject(text);
But it's not working. It will throw exception.
Unexpected character encountered while parsing value: O. Path
'[0]._id', line 3, position 9.
What i can do? I wanna convert this text to bsonarray. Without custom class.
I also tried to do that:
BsonArray array = BsonSerializer.Deserialize<BsonArray>(str);
but it throw exception JSON reader was expecting a value but found 'Long'.
ObjectId(...), Long(...), and ISODate(...) are not valid JSON.
If you have the object in the mongo shell, you might try using the tostrictjson built-in function to convert it, but note that the mongo shell will want you to use NumberLong() instead of Long()
In strict JSON, those values would look something like"
{
"_id" : { "$oid" : "5da08d49949b4c000100b90b" },
"ModifiedOn" : { "$date" : "2019-10-11T07:10:17.461-0700" },
"UniqueNumber" : { "$numberLong" : "465561" }
}
I'm querying a mySQL (MariaDB) database and use JSON_ARRAY and JSON_OBJECT to gather json objects as array.
I have an api to retrieve the data from the DB, by using GetAsync.
Afterwards, in the api consumer, I want to parse the Json into an object,
using JArray (Newtonsoft).
I can't manage to parse the Json due to problem in the JSON_OBJECT (it seems like a quote wrapping the array sign "["). See error code below.
What am I doing wrong and how can I fix it ?
When getting another Json from another data source (not using JSON_ARRAY and JSON_OBJECT), parsing is working (see example below)
This is my code:
public JARrray parse(HttpResponseMessage response)
{
return JArray.Parse(response.Content.ReadAsStringAsync().Result);
}
This is the string that works:
[
{
"remark":"",
"driver": [
{
"username":"0000000",
"uid":"00000000",
"fname": "000000",
"lname": ""
}
]
}
]
And this is the string that get the message
After parsing a value an unexpected character was encountered: u. Path '[0].driver', line..."
I can see in the stack that this is happening on the line:
JsonTextReader.ParsePostValue(Boolean ignoreComments):
Json:
[
{
"remark":"",
"driver": "[{\"username\": \"0000000\", \"uid\": \"00000000\", \"fname\":\"ZZZZZ\", \"lname\":\"\"}]"
}
]
or even, when I removing the "" :
[
{
"remark":"",
"driver": "[{"username": "0000000", "uid": "00000000", "fname":"ZZZZZ", "lname":""}]"
}
]
\"fname\":"\ZZZZZ\"
The escaped double quote in front of ZZZZZ is wrong. Change it from "\ to \".
I'm pulling intents back from API.AI and parsing these to C# objects using Newtonsoft.Json in the following way:-
intentListModel = JsonConvert.DeserializeObject<List<IntentListModel>>(intentList);
intentList is a JSON string from the webrequest. However at Line 1, position 161, it fails. The bit of JSON concerned is:-
"contextIn": [
"Employed"
],
"events": [{
"name": "Occupation_DOB"
}],
NB: This is only part of the JSON, and the JSON opens and closes with [] as it is a list of JSON items.
specifically the opening { on events. I'm stumped, I've run it through a validator and I see valid JSON.
Could anyone suggest what I can try, or is there a setting somewhere for this? Or is the error message actually looking another area of the JSON string?
Thanks in advance!
UPDATE
Whole JSON sample posted
[
{
"id":"18b025c5-3567-49c1-a9e9-25583f9156ca",
"name":"IncomeProtection - Employed? - Occupation/DOB/Email",
"state":"LOADED",
"contextIn":[
"Employed"
],
"events":[
{
"name":"Occupation_DOB"
}
],
"parentId":"ad5f0007-c084-4615-93dd-6c82ca5e7602",
"parameters":[
{
"required":true,
"dataType":"#Occupation",
"name":"Occupation",
"value":"$Occupation",
"prompts":[
"Whatu0027s your Occupation?"
],
"isList":false
},
{
"required":true,
"dataType":"#sys.date",
"name":"date",
"value":"$date",
"prompts":[
"Whatu0027s your date of birth?"
],
"isList":false
}
],
"contextOut":[
{
"name":"OccupationDOB",
"parameters":{
},
"lifespan":1
}
],
"actions":[
"IncomeProtection:Occupation/DOB"
],
"priority":500000,
"fallbackIntent":false
}
]
This issue was down to one of the items from the API being returned in a list, but in the particular example I looked at, the API returned a list of 1 item. I misread the brackets and created a class property of type string instead of List<string>, hence the failure of code.
Hope this helps people in the future.
I want to take all the key-value pairs after the \"tags\" under "instanceData" and make them key-value pairs under "properties".
I have this...
{
"id": "/subscriptions/1234abcd-ab12-12ab-12ab-abcdfghi1234/Daily_BRSDT_20161214_0000",
"name": "Daily_BRSDT_20161214_0000",
"type": "Microsoft.Commerce/UsageAggregate",
"properties": {
"subscriptionId": "1234abcd-ab12-12ab-12ab-abcdfghi1234",
"usageStartTime": "2017-03-08T00:00:00+00:00",
"usageEndTime": "2017-03-09T00:00:00+00:00",
"meterName": "Standard IO - File Read Operation Units (in 10,000s)",
"meterCategory": "Data Management",
"unit": "10,000s",
"instanceData": "{\"Microsoft.Resources\":{\"resourceUri\":\"/subscriptions/1234abcd-ab12-12ab-12ab-abcdfghi1234/resourceGroups/default-resource-group67/providers/Microsoft.Storage/storageAccounts/defaultstorage67\",\"location\":\"ussouthcentral\",\"tags\":{\"ProjectName\":\"default Portal\",\"billTo\":\"Technology\",\"component\":\"Persistant Storage\",\"department\":\"Technology\",\"displayName\":\"default Portal Storage Account\",\"enviornment\":\"default\",\"function\":\"Reporting\",\"matterNumber\":\"999999\",\"primaryowner\":\"john#internet.com\",\"productLine\":\"Information Components\",\"secondaryowner\":\"mary#internet.com\",\"version\":\"1.0.0.0\"}}}",
"meterId": "12345ab-259d-4206-a6ae-12345678abcd",
"infoFields": {},
"quantity": 0.0004
}
}
I want this...
{
"id": "/subscriptions/1234abcd-ab12-12ab-12ab-abcdfghi1234/Daily_BRSDT_20161214_0000",
"name": "Daily_BRSDT_20161214_0000",
"type": "Microsoft.Commerce/UsageAggregate",
"properties": {
"subscriptionId": "1234abcd-ab12-12ab-12ab-abcdfghi1234",
"usageStartTime": "2017-03-08T00:00:00+00:00",
"usageEndTime": "2017-03-09T00:00:00+00:00",
"meterName": "Standard IO - File Read Operation Units (in 10,000s)",
"meterCategory": "Data Management",
"unit": "10,000s",
"instanceData": "{\"Microsoft.Resources\":{\"resourceUri\":\"/subscriptions/1234abcd-ab12-12ab-12ab-abcdfghi1234/resourceGroups/default-resource-group67/providers/Microsoft.Storage/storageAccounts/defaultstorage67\",\"location\":\"ussouthcentral\"}}",
"ProjectName":"default Portal",
"billTo":"Technology",
"component":"Persistant Storage",
"department":"Technology",
"displayName":"default Portal Storage Account",
"enviornment":"default",
"function":"Reporting",
"matterNumber":"999999",
"primaryowner":"john#internet.com",
"productLine":"Information Components",
"secondaryowner":"mary#internet.com",
"version":"1.0.0.0",
"meterId": "12345ab-259d-4206-a6ae-12345678abcd",
"infoFields": {},
"quantity": 0.0004
}
}
Is there a simple way to convert this? I am attempting to do this with RegEx with no luck.
I would recommend looking at something like this:
How can I deserialize JSON to a simple Dictionary<string,string> in ASP.NET?
Serialize List<KeyValuePair<string, string>> as JSON
Effectively you're going to need to strip out the one key you want to be parsed, and re-add that to your JSON object.
Json.NET - Newtonsoft's tool is great for working with JSON.
http://www.newtonsoft.com/json
Easiest way to do it:
Convert your entire JSON string to a Dictionary or to a List<KeyValuePair<string,string>>.
Take the instanceData bit you want to split apart, and then parse it into another C# object.
Merge both objects together using some logic to ensure no duplicate keys.
Serialize your object back into JSON
This is an easy way to do it, though not the most efficient way.
I'm playing around with the jQuery Week Calendar and am trying to get this to work, but I can't figure out why this is throwing an error.
The calendar has a method that returns a list of events to populate itself with.
The method (which uses preset events for demo purposes) looks like this:
function getEventData() {
var year = new Date().getFullYear();
var month = new Date().getMonth();
var day = new Date().getDate();
return {
events : [
{
"id":1,
"start": new Date(year, month, day, 12),
"end": new Date(year, month, day, 13, 30),
"title":"Lunch with Mike"
},
...
{
"id":6,
"start": new Date(year, month, day, 10),
"end": new Date(year, month, day, 11),
"title":"I am read-only",
readOnly : true
}
]
};
}
I changed it so that it used a jQuery GET to another page that would return a list of real dates. For testing, I kept the same test data just to see if the post was being done properly.
The method will call the page, and it will return this in the responseText property.
{
events : [
{
'id':1,
'start': new Date(2010, 2, 22, 12),
'end': new Date(2010, 2, 22, 13, 30),
'title':'Lunch with Mike'
},
...
{
'id':6,
'start': new Date(2010, 2, 28, 10),
'end': new Date(2010, 2, 28, 11),
'title':'I am read-only',
readOnly : true
}
]
}
Is there a difference between the two methods that I'm missing? The two objects look exactly the same to me, except the second one uses ' instead of " since it's being written through C#, and it doesn't use the year/month/day variables.
The problem is that the second method throws an error saying that "G is undefined", is there any reason jQuery wouldn't like the JSON object that I'm returning?
EDIT: I think I found the source of the error. Inside one of the methods there is an if statement like this:
if (typeof options.data == 'string') {
if (options.loading) options.loading(true);
var jsonOptions = {};
jsonOptions[options.startParam || 'start'] = Math.round(weekStartDate.getTime() / 1000);
jsonOptions[options.endParam || 'end'] = Math.round(weekEndDate.getTime() / 1000);
$.getJSON(options.data, jsonOptions, function(data) {
self._renderEvents(data, $weekDayColumns);
if (options.loading) options.loading(false);
});
}
else if ($.isFunction(options.data)) {
options.data(weekStartDate, weekEndDate,
function(data) {
self._renderEvents(data, $weekDayColumns);
});
}
When just outputting the JSON object in getEventData() (the first way) it hit the else statement because it's recognized as a function (I'm guessing), but when I do the ajax GET, it's being considered a string and going into the if.
Is there anyway to get this recognized as a function? I tried wrapping the return in braces, changing the dataType to script and using eval() to cast the result as a function, but none of that seemed to work.
Have you tried enclosing events in quotes as well?
'events': [ ....
Put your response text into JSON validator and see, where the problem lies.
Also note, that outputting new Date(2010, 2, 28, 10) directly into text is considered bad practice, rather use real values and then parse them on client side with JS. (I believe this is the most obvious error, as long as you don't use eval to parse your JSON string :))
since jQuery 1.4.0, the library is very strict, on how JSON is formatted (your json text misses quotes for keys)
After searching through Google I came across the jQuery Week Calendar Google Group and this discussion had the exact same problem I was having.
I think the main problem was the improper formatting of DateTimes. Returning the events JSON object directly converted the date properly, but my ajax call was returning new Date(2010, 02, 23, 12) as the times. It caused it to fail silently. Changing the datetimes to something like 2010-02-23T12:15:00.000+10:00 seems to have fixed it.
However that user does appear to be incorrect about the events : [ breaking the JSON. I left it in my ajax call and it still works fine.