Related
I'm trying to insert a json like this (fieldname with a "."), in a Net Core Console Project
{"name.field" : "MongoDB", "type" : "Database"}
Using the C# code belove:
-with InsertManyOptions with BypassDocumentValidation in true
var options = new InsertManyOptions
{
BypassDocumentValidation = true,
IsOrdered = false
};
await _collection.InsertManyAsync(items, options);
But I have this exception:
Element name 'name.field' is not valid
I´m using :
C# Mongo Driver 2.5
Net Core Project
MongoDB version 4.0.3
Any idea? Thanks!
The BypassDocumentValidation can be used to bypass the JSON Schema validation. The issue you are facing, however, is due to the C# driver which explicitly prevents the use of the dot symbol . as part of a field name.
This used to be required up until MongoDB v3.6 which officially added support for fields with ".".
Looking into the internals of the C# driver you can see that the BsonWriter.WriteName method calls contains this code which throws the Exception you're seeing:
if (!_elementNameValidator.IsValidElementName(name))
{
var message = string.Format("Element name '{0}' is not valid'.", name);
throw new BsonSerializationException(message);
}
The _elementNameValidator is something that is managed internally by the driver which in fact comes with a NoOpElementNameValidator that doesn't do any validations. The driver, however, won't use this validator for "normal" collections.
All that said, I would strongly advise against the use of field names with "unusual" characters anyway because this is likely to set you up for unexpected behaviour and all sorts of other issues down the road.
In order to get around this you can do one of the following things:
a) Write your own custom serializer which is an option that I would personally steer clear off if possible - it adds complexity that most of the time shouldn't be required.
b) Use the below helper extension (copied from one of the unit testing projects inside the driver) to convert the BsonDocument into a RawBsonDocument which can then successfully written to the server:
public static class RawBsonDocumentHelper
{
public static RawBsonDocument FromBsonDocument(BsonDocument document)
{
using (var memoryStream = new MemoryStream())
{
using (var bsonWriter = new BsonBinaryWriter(memoryStream, BsonBinaryWriterSettings.Defaults))
{
var context = BsonSerializationContext.CreateRoot(bsonWriter);
BsonDocumentSerializer.Instance.Serialize(context, document);
}
return new RawBsonDocument(memoryStream.ToArray());
}
}
public static RawBsonDocument FromJson(string json)
{
return FromBsonDocument(BsonDocument.Parse(json));
}
}
And then simply write the RawBsonDocument to the server:
RawBsonDocument rawDoc = RawBsonDocumentHelper.FromJson("{\"name.field\" : \"MongoDB\", \"type\" : \"Database\"}");
collection.InsertOne(rawDoc);
I want to be able to use the dnnclassifier (estimator) on top of IIS using tensorflowsharp. The model has previously been trained in python. I got so far that I can now generate PB files, know the correct input/outputs, however I am stuck in tensorflowsharp using string inputs.
I can create a valid .pb file of the iris dataset. It uses the following feate_spec:
{'SepalLength': FixedLenFeature(shape=(1,), dtype=tf.float32, default_value=None), 'SepalWidth': FixedLenFeature(shape=(1,), dtype=tf.float32, default_value=None), 'PetalLength': FixedLenFeature(shape=(1,), dtype=tf.float32, default_value=None), 'PetalWidth': FixedLenFeature(shape=(1,), dtype=tf.float32, default_value=None)}
I have created a simple c# console to try and spin it up. The input should be an "input_example_tensor" and the output is located in "dnn/head/predictions/probabilities". This I discoved after alex_zu provided help using the saved_model_cli command here.
As far as I am aware all tensorflow estimator API's work like this.
Here comes the problem: the input_example_tensor should be of a string format which will be parsed internally by the ParseExample function. Now i am stuck. I have found TFTensor.CreateString, but this doesn't solve the problem.
using System;
using TensorFlow;
namespace repository
{
class Program
{
static void Main(string[] args)
{
using (TFGraph tfGraph = new TFGraph()){
using (var tmpSess = new TFSession(tfGraph)){
using (var tfSessionOptions = new TFSessionOptions()){
using (var metaGraphUnused = new TFBuffer()){
//generating a new session based on the pb folder location with the tag serve
TFSession tfSession = tmpSess.FromSavedModel(
tfSessionOptions,
null,
#"path/to/model/pb",
new[] { "serve" },
tfGraph,
metaGraphUnused
);
//generating a new runner, which will fetch the tensorflow results later
var runner = tfSession.GetRunner();
//this is in the actual tensorflow documentation, how to implement this???
string fromTensorflowPythonExample = "{'SepalLength': [5.1, 5.9, 6.9],'SepalWidth': [3.3, 3.0, 3.1],'PetalLength': [1.7, 4.2, 5.4],'PetalWidth': [0.5, 1.5, 2.1],}";
//this is the problem, it's not working...
TFTensor rawInput = new TFTensor(new float[4]{5.1f,3.3f,1.7f,0.5f});
byte[] serializedTensor = System.Text.Encoding.ASCII.GetBytes(rawInput.ToString());
TFTensor inputTensor = TensorFlow.TFTensor.CreateString (serializedTensor);
runner.AddInput(tfGraph["input_example_tensor"][0], inputTensor);
runner.Fetch("dnn/head/predictions/probabilities", 0);
//start the run and get the results of the iris example
var output = runner.Run();
TFTensor result = output[0];
//printing response to the client
Console.WriteLine(result.ToString());
Console.ReadLine();
}
}
}
}
}
}
}
This example will give the following error:
An unhandled exception of type 'TensorFlow.TFException' occurred in TensorFlowSharp.dll: 'Expected serialized to be a vector, got shape: []
[[Node: ParseExample/ParseExample = ParseExample[Ndense=4, Nsparse=0, Tdense=[DT_FLOAT, DT_FLOAT, DT_FLOAT, DT_FLOAT], dense_shapes=[[1], [1], [1], [1]], sparse_types=[], _device="/job:localhost/replica:0/task:0/device:CPU:0"](_arg_input_example_tensor_0_0, ParseExample/ParseExample/names, ParseExample/ParseExample/dense_keys_0, ParseExample/ParseExample/dense_keys_1, ParseExample/ParseExample/dense_keys_2, ParseExample/ParseExample/dense_keys_3, ParseExample/Const, ParseExample/Const, ParseExample/Const, ParseExample/Const)]]'
How can I serialize tensors in such a way that I can use the pb file correctly?
I also posted the issue on github, here you can find the iris example python file, pb file and the console applications. In my opinion solving this creates a
neat solution for all tensorflow users having ancient production environments (like me).
The Expected serialized to be a vector, got shape: [] error can be fixed by using an overload of the TFTensor.CreateString function: Instead of directly taking a string, the model apparently expects a vector containing a single string:
TFTensor inputTensor = TFTensor.CreateString(new byte[][] { bytes }, new TFShape(1));
The input_example_tensor in your case now expects a serialized Example protobuf message (see also the docs and the example.proto file).
Using the protobuf compiler, I've generated a C# file containing the Example class. You can download it from here: https://pastebin.com/iLT8MUdR. Specifically, I used this online tool with CSharpProtoc and replaced the import "tensorflow/core/example/feature.proto"; line by the messages defined in that file.
Once you've added the file to your project, you'll need a package reference to Google.Protobuf. Then, you can pass serialized examples to the model like this:
Func<float, Tensorflow.Feature> makeFeature = (float x) => {
var floatList = new Tensorflow.FloatList();
floatList.Value.Add(x);
return new Tensorflow.Feature { FloatList = floatList };
};
var example = new Tensorflow.Example { Features = new Tensorflow.Features() };
example.Features.Feature.Add("SepalLength", makeFeature(5.1f));
example.Features.Feature.Add("SepalWidth", makeFeature(3.3f));
example.Features.Feature.Add("PetalLength", makeFeature(1.7f));
example.Features.Feature.Add("PetalWidth", makeFeature(0.5f));
TFTensor inputTensor = TFTensor.CreateString(
new [] { example.ToByteArray() }, new TFShape(1));
runner.AddInput(tfGraph["input_example_tensor"][0], inputTensor);
runner.Fetch("dnn/head/predictions/probabilities", 0);
//start the run and get the results of the iris example
var output = runner.Run();
TFTensor result = output[0];
I am using CANoe tool for CAPL scripting.
And I have referred this link for help given by Vector: https://vector.com/portal/medien/cmc/application_notes/AN-IND-1-011_Using_CANoe_NET_API.pdf
Now I am facing issue at step or section: 2.7 Calling CAPL Functions.
There are no syntax errors, but I think, the code is not able to retrieve the function from the CAPL file as it is giving this error when I pass values to the function from C# code.
Error statement:
An exception of type
'Microsoft.CSharp.RuntimeBinder.RuntimeBinderException' occurred in
System.Core.dll but was not handled in user code
Additional information: Cannot convert null to 'int' because it is a
non-nullable value type
If there is a handler for this exception, the program may be safely continued.
Can some one tell me what exactly the issue is?
Code:
public void Init()
{
CANoe.Application myApp;
myApp = new CANoe.Application();
CANoe.Measurement mymeasure;
mymeasure = (CANoe.Measurement)myApp.Measurement;
myApp.Open(#"D:\Planter CAPl\CANoeCAPLdll.cfg", true, true);
CANoe.OpenConfigurationResult ocresult = myApp.Configuration.OpenConfigurationResult;
if (ocresult.result == 0)
{
CANoe.CAPL CANoeCAPL = (CANoe.CAPL)myApp.CAPL;
CANoeCAPL.Compile(null);
CANoe.CAPLFunction mydata;
mydata = (CANoe.CAPLFunction)CANoeCAPL.GetFunction("Data");//CANoeCAPL.GetFunction("DataShare");
mymeasure.Start();
Thread.Sleep(2000);
int Result = (int)mydata.Call(10,20,30);// Exeception error is coming at this point*
if (mymeasure.Running)
{
mymeasure.Stop();
}
}
}
I think the error message is relatively self-explanatory, you're trying to convert a null value to an int type. Since int is a value type, it cannot be null, therefore, you get an exception.
In the line
int Result = (int)mydata.Call(10,20,30);
The value returned by mydata.Call(10,20,30) is null. You should check if that value is null and do something.
var myDataResult = mydata.Call(10,20,30);
if(myDataResult == null)
// do something
else
int result = (int)myDataResult;
As to why the call to the Call method returns null, you'll have to check the API you're using.
I'm sending a SecurityListRequest and I am receiving confirmation. I am unable to get the symbol from the response. I'm getting response like this.
8=FIXT.1.1|9=795|35=y|34=3|49=TMATCH=YYYYY|52=20160804-09:39:56.534|56=zzzzz|320=0001|322=zzzz->zzzzx|393=19|560=0|893=Y|
146=2|
55=xxxxx|48=xxxxx|22=8|167=xxxx|762=PERIOD|15=xxx|423=1|20000=1W|
55=xxxx|48=xxxxx|22=8|167=xxxx|762=PERIOD|15=xxx|423=1|20000=1M|
10=234
I tried to get like this
QuickFix.FIX50.SecurityList.NoRelatedSymGroup symbolGroup
= new QuickFix.FIX50.SecurityList.NoRelatedSymGroup();
int noofsymbols = m.Get(new NoRelatedSym()).getValue();
symbolGroup.Get(new Symbol());
and
m.get(new symbol()).getvalue();
and
m.getfield(new stringfield(55));
Every time it is throwing exception saying "Field Not Found" for tag 55.
You created a new empty NoRelatedSymGroup object, but you forgot to populate it from your message.
I think you are using the C++ QF with the C# wrapper, so this is probably the code you'd want:
var sym1 = new Symbol();
var sym2 = new Symbol();
m.getGroup(1, symbolGroup); // first group
symbolGroup.get(sym1);
m.getGroup(2, symbolGroup); // second group
symbolGroup.get(sym2);
Currently, I have some issues. I'm using C# with Json.NET. The issue is that I always get:
{"Unexpected character encountered while parsing value: e. Path '', line 0, position 0."}
So the way I'm using Json.NET is the following. I have a Class which should be saved. The class looks like this:
public class stats
{
public string time { get; set; }
public string value { get; set; }
}
public class ViewerStatsFormat
{
public List<stats> viewerstats { get; set; }
public String version { get; set; }
public ViewerStatsFormat(bool chk)
{
this.viewerstats = new List<stats>();
}
}
One object of this class will be filled and saved with:
File.WriteAllText(tmpfile, JsonConvert.SerializeObject(current), Encoding.UTF8);
The saving part works fine and the file exists and is filled. After that the file will be read back into the class with:
try
{
ViewerStatsFormat current = JsonConvert.DeserializeObject<ViewerStatsFormat>(tmpfile);
//otherstuff
}
catch(Exception ex)
{
//error loging stuff
}
Now on the current= line comes the exception:
{"Unexpected character encountered while parsing value: e. Path '', line 0, position 0."}
I don't know why this comes. The JSON file is the following -> Click me I am the JSON link
Does anyone have any ideas?
Possibly you are not passing JSON to DeserializeObject.
It looks like from File.WriteAllText(tmpfile,... that type of tmpfile is string that contain path to a file. JsonConvert.DeserializeObject takes JSON value, not file path - so it fails trying to convert something like #"c:\temp\fooo" - which is clearly not JSON.
I solved the problem with these online tools:
To check if the Json structure is OKAY: http://jsonlint.com/
To generate my Object class from my Json structure: https://www.jsonutils.com/
The simple code:
RootObject rootObj= JsonConvert.DeserializeObject<RootObject>(File.ReadAllText(pathFile));
In my case, the file containing JSON string had BOM. Once I removed BOM the problem was solved.
I experienced the same error in my Xamarin.Android solution.
I verified that my JSON was correct, and noticed that the error only appeared when I ran the app as a Release build.
It turned out that the Linker was removing a library from Newtonsoft.JSON, causing the JSON to be parsed incorrectly.
I fixed the error by adding Newtonsoft.Json to the Ignore assemblies setting in the Android Build Configuration (screen shot below)
JSON Parsing Code
static readonly JsonSerializer _serializer = new JsonSerializer();
static readonly HttpClient _client = new HttpClient();
static async Task<T> GetDataObjectFromAPI<T>(string apiUrl)
{
using (var stream = await _client.GetStreamAsync(apiUrl).ConfigureAwait(false))
using (var reader = new StreamReader(stream))
using (var json = new JsonTextReader(reader))
{
if (json == null)
return default(T);
return _serializer.Deserialize<T>(json);
}
}
Visual Studio Mac Screenshot
Visual Studio Screenshot
I have also encountered this error for a Web API (.Net Core 3.0) action that was binding to a string instead to an object or a JObject. The JSON was correct, but the binder tried to get a string from the JSON structure and failed.
So, instead of:
[HttpPost("[action]")]
public object Search([FromBody] string data)
I had to use the more specific:
[HttpPost("[action]")]
public object Search([FromBody] JObject data)
This issue is related to Byte Order Mark in the JSON file. JSON file is not encoded as UTF8 encoding data when saved. Using File.ReadAllText(pathFile) fix this issue.
When we are operating on Byte data and converting that to string and then passing to JsonConvert.DeserializeObject, we can use UTF32 encoding to get the string.
byte[] docBytes = File.ReadAllBytes(filePath);
string jsonString = Encoding.UTF32.GetString(docBytes);
I had the same problem with webapi in ASP.NET core, in my case it was because my application needs authentication, then it assigns the annotation [AllowAnonymous] and it worked.
[AllowAnonymous]
public async Task <IList <IServic >> GetServices () {
}
I ran into this issue and it ended up being because of BOM characters in my input string.
Here's what I ended up doing:
String.Trim(new char[] { '\uFEFF', '\u200B' });
This resolved the issue for me.
In my case, I was getting an error on JsonConvert.PopulateObject().
My request was returning JSON that was wrapped in an extra pair of '[ ]' brackets, making my result an array of one object rather than just an object. Here's what I did to get inside these brackets (only for that type of model):
T jsonResponse = new T();
var settings = new JsonSerializerSettings
{
DateParseHandling = DateParseHandling.DateTimeOffset,
NullValueHandling = NullValueHandling.Ignore,
};
var jRslt = response.Content.ReadAsStringAsync().Result;
if (jsonResponse.GetType() == typeof(myProject.Models.MyModel))
{
var dobj = JsonConvert.DeserializeObject<MyModel[]>(jRslt);
var y = dobj.First();
var szObj = JsonConvert.SerializeObject(y);
JsonConvert.PopulateObject(szObj, jsonResponse, settings);
}
else
{
JsonConvert.PopulateObject(jRslt, jsonResponse);
}
If you are using downloading data using url...may need to use
var result = client.DownloadData(url);
In my scenario I had a slightly different message, where the line and position were not zero.
E. Path 'job[0].name', line 1, position 12.
This was the top Google answer for the message I quoted.
This came about because I had called a program from the Windows command line, passing JSON as a parameter.
When I reviewed the args in my program, all the double quotes got stripped.
You have to reconstitute them.
I posted a solution here. Though it could probably be enhanced with a Regex.
I had a similar error and thought I'd answer in case anyone was having something similar. I was looping over a directory of json files and deserializing them but was getting this same error.
The problem was that it was trying to grab hidden files as well. Make sure the file you're passing in is a .json file. I'm guessing it'll handle text as well. Hope this helps.
I had simular problem. In my case the problem was in DateTime format. It was just numbers and it is also know as EpochFormat or UnixTimestamp.
A part from my JSON:
"direction": "outbound",
"date_archive": 1554691800224,
"date_doc": 1524700800000,
"date_sent": 1524704189000,
"date_received": 1524704189000,
"date_store_till": 1712544600224,
So I've used an attribute like this:
[JsonProperty("date_received")]
[JsonConverter(typeof(MicrosecondEpochConverter))]
public DateTime? DateReceived { get; set; }
You can find MicrosecondEpochConverter code here: https://stackoverflow.com/a/19972214/4324624
I faced similar error message in Xamarin forms when sending request to webApi to get a Token,
Make sure all keys (key : value) (ex.'username', 'password', 'grant_type') in the Json file are exactly what the webApi expecting, otherwise it fires this exception.
Unhandled Exception: Newtonsoft.Json.JsonReaderException: Unexpected character encountered while parsing value: <. Path '', line 0, position 0
Please check the model you shared between client and server is same. sometimes you get this error when you not updated the Api version and it returns a updated model, but you still have an old one. Sometimes you get what you serialize/deserialize is not a valid JSON.
In my case, it was the lack of a default parameterless constructor !!!
In my case, I was calling the async service method without using await, so before Task is completed I was trying to return the result!
Suppose this is your json
{
"date":"11/05/2016",
"venue": "{\"ID\":12,\"CITY\":Delhi}"
}
if you again want deserialize venue, modify json as below
{
"date":"11/05/2016",
"venue": "{\"ID\":\"12\",\"CITY\":\"Delhi\"}"
}
then try to deserialize to respective class by taking the value of venue
This error occurs when we parse json content to model object. Json content type is string.
For example:
https://dotnetfiddle.net/uFClKj
Some times, an api that we call may return an error. If we do not check the response status, but proceed to parse the response to model, this issue will occur.
When I encountered a similar problem, I fixed it by substituting &mode=xml for &mode=json in the request.