I'm trying to insert a json like this (fieldname with a "."), in a Net Core Console Project
{"name.field" : "MongoDB", "type" : "Database"}
Using the C# code belove:
-with InsertManyOptions with BypassDocumentValidation in true
var options = new InsertManyOptions
{
BypassDocumentValidation = true,
IsOrdered = false
};
await _collection.InsertManyAsync(items, options);
But I have this exception:
Element name 'name.field' is not valid
I´m using :
C# Mongo Driver 2.5
Net Core Project
MongoDB version 4.0.3
Any idea? Thanks!
The BypassDocumentValidation can be used to bypass the JSON Schema validation. The issue you are facing, however, is due to the C# driver which explicitly prevents the use of the dot symbol . as part of a field name.
This used to be required up until MongoDB v3.6 which officially added support for fields with ".".
Looking into the internals of the C# driver you can see that the BsonWriter.WriteName method calls contains this code which throws the Exception you're seeing:
if (!_elementNameValidator.IsValidElementName(name))
{
var message = string.Format("Element name '{0}' is not valid'.", name);
throw new BsonSerializationException(message);
}
The _elementNameValidator is something that is managed internally by the driver which in fact comes with a NoOpElementNameValidator that doesn't do any validations. The driver, however, won't use this validator for "normal" collections.
All that said, I would strongly advise against the use of field names with "unusual" characters anyway because this is likely to set you up for unexpected behaviour and all sorts of other issues down the road.
In order to get around this you can do one of the following things:
a) Write your own custom serializer which is an option that I would personally steer clear off if possible - it adds complexity that most of the time shouldn't be required.
b) Use the below helper extension (copied from one of the unit testing projects inside the driver) to convert the BsonDocument into a RawBsonDocument which can then successfully written to the server:
public static class RawBsonDocumentHelper
{
public static RawBsonDocument FromBsonDocument(BsonDocument document)
{
using (var memoryStream = new MemoryStream())
{
using (var bsonWriter = new BsonBinaryWriter(memoryStream, BsonBinaryWriterSettings.Defaults))
{
var context = BsonSerializationContext.CreateRoot(bsonWriter);
BsonDocumentSerializer.Instance.Serialize(context, document);
}
return new RawBsonDocument(memoryStream.ToArray());
}
}
public static RawBsonDocument FromJson(string json)
{
return FromBsonDocument(BsonDocument.Parse(json));
}
}
And then simply write the RawBsonDocument to the server:
RawBsonDocument rawDoc = RawBsonDocumentHelper.FromJson("{\"name.field\" : \"MongoDB\", \"type\" : \"Database\"}");
collection.InsertOne(rawDoc);
Related
I am using MongoDB.Driver 2.11.0 and .Net Standard 2.1. To ensure that a database exists and a collection exists, I have the following code:
IMongoClient client = ...; // inject a Mongo client
MongoDatabaseSettings dbSettings = new MongoDatabaseSettings();
IMongoDatabase db = client.GetDatabase("MyDatabase", dbSettings);
MongoCollectionSettings collectionSettings = new MongoCollectionSettings()
{
GuidRepresentation = GuidRepresentation.Standard,
};
IMongoCollection<MyClass> collection = db.GetCollection<MyClass>("MyClasses", collectionSettings);
In earlier versions of MongoDB.Driver, this code would compile without any warnings. In v2.11.0 I am now getting a warning that "MongoCollectionSettings.GuidRepresentation is obsolete: Configure serializers instead" but I have not been able to find any samples illustrating the new way of setting the Guid serialization format. Does anyone know of other ways to set the serializers for a collection?
If you want to define GuidRepresentation for a specific property, you can do it during the registration of the class map, like so:
BsonClassMap.RegisterClassMap<MyClass>(m =>
{
m.AutoMap();
m.MapIdMember(d => d.Id).SetSerializer(new GuidSerializer(GuidRepresentation.Standard));
});
If you want to do it globally:
BsonSerializer.RegisterSerializer(new GuidSerializer(GuidRepresentation.Standard));
It was changed in the latest release, see details here: https://mongodb.github.io/mongo-csharp-driver/2.11/reference/bson/guidserialization/
I want to be able to use the dnnclassifier (estimator) on top of IIS using tensorflowsharp. The model has previously been trained in python. I got so far that I can now generate PB files, know the correct input/outputs, however I am stuck in tensorflowsharp using string inputs.
I can create a valid .pb file of the iris dataset. It uses the following feate_spec:
{'SepalLength': FixedLenFeature(shape=(1,), dtype=tf.float32, default_value=None), 'SepalWidth': FixedLenFeature(shape=(1,), dtype=tf.float32, default_value=None), 'PetalLength': FixedLenFeature(shape=(1,), dtype=tf.float32, default_value=None), 'PetalWidth': FixedLenFeature(shape=(1,), dtype=tf.float32, default_value=None)}
I have created a simple c# console to try and spin it up. The input should be an "input_example_tensor" and the output is located in "dnn/head/predictions/probabilities". This I discoved after alex_zu provided help using the saved_model_cli command here.
As far as I am aware all tensorflow estimator API's work like this.
Here comes the problem: the input_example_tensor should be of a string format which will be parsed internally by the ParseExample function. Now i am stuck. I have found TFTensor.CreateString, but this doesn't solve the problem.
using System;
using TensorFlow;
namespace repository
{
class Program
{
static void Main(string[] args)
{
using (TFGraph tfGraph = new TFGraph()){
using (var tmpSess = new TFSession(tfGraph)){
using (var tfSessionOptions = new TFSessionOptions()){
using (var metaGraphUnused = new TFBuffer()){
//generating a new session based on the pb folder location with the tag serve
TFSession tfSession = tmpSess.FromSavedModel(
tfSessionOptions,
null,
#"path/to/model/pb",
new[] { "serve" },
tfGraph,
metaGraphUnused
);
//generating a new runner, which will fetch the tensorflow results later
var runner = tfSession.GetRunner();
//this is in the actual tensorflow documentation, how to implement this???
string fromTensorflowPythonExample = "{'SepalLength': [5.1, 5.9, 6.9],'SepalWidth': [3.3, 3.0, 3.1],'PetalLength': [1.7, 4.2, 5.4],'PetalWidth': [0.5, 1.5, 2.1],}";
//this is the problem, it's not working...
TFTensor rawInput = new TFTensor(new float[4]{5.1f,3.3f,1.7f,0.5f});
byte[] serializedTensor = System.Text.Encoding.ASCII.GetBytes(rawInput.ToString());
TFTensor inputTensor = TensorFlow.TFTensor.CreateString (serializedTensor);
runner.AddInput(tfGraph["input_example_tensor"][0], inputTensor);
runner.Fetch("dnn/head/predictions/probabilities", 0);
//start the run and get the results of the iris example
var output = runner.Run();
TFTensor result = output[0];
//printing response to the client
Console.WriteLine(result.ToString());
Console.ReadLine();
}
}
}
}
}
}
}
This example will give the following error:
An unhandled exception of type 'TensorFlow.TFException' occurred in TensorFlowSharp.dll: 'Expected serialized to be a vector, got shape: []
[[Node: ParseExample/ParseExample = ParseExample[Ndense=4, Nsparse=0, Tdense=[DT_FLOAT, DT_FLOAT, DT_FLOAT, DT_FLOAT], dense_shapes=[[1], [1], [1], [1]], sparse_types=[], _device="/job:localhost/replica:0/task:0/device:CPU:0"](_arg_input_example_tensor_0_0, ParseExample/ParseExample/names, ParseExample/ParseExample/dense_keys_0, ParseExample/ParseExample/dense_keys_1, ParseExample/ParseExample/dense_keys_2, ParseExample/ParseExample/dense_keys_3, ParseExample/Const, ParseExample/Const, ParseExample/Const, ParseExample/Const)]]'
How can I serialize tensors in such a way that I can use the pb file correctly?
I also posted the issue on github, here you can find the iris example python file, pb file and the console applications. In my opinion solving this creates a
neat solution for all tensorflow users having ancient production environments (like me).
The Expected serialized to be a vector, got shape: [] error can be fixed by using an overload of the TFTensor.CreateString function: Instead of directly taking a string, the model apparently expects a vector containing a single string:
TFTensor inputTensor = TFTensor.CreateString(new byte[][] { bytes }, new TFShape(1));
The input_example_tensor in your case now expects a serialized Example protobuf message (see also the docs and the example.proto file).
Using the protobuf compiler, I've generated a C# file containing the Example class. You can download it from here: https://pastebin.com/iLT8MUdR. Specifically, I used this online tool with CSharpProtoc and replaced the import "tensorflow/core/example/feature.proto"; line by the messages defined in that file.
Once you've added the file to your project, you'll need a package reference to Google.Protobuf. Then, you can pass serialized examples to the model like this:
Func<float, Tensorflow.Feature> makeFeature = (float x) => {
var floatList = new Tensorflow.FloatList();
floatList.Value.Add(x);
return new Tensorflow.Feature { FloatList = floatList };
};
var example = new Tensorflow.Example { Features = new Tensorflow.Features() };
example.Features.Feature.Add("SepalLength", makeFeature(5.1f));
example.Features.Feature.Add("SepalWidth", makeFeature(3.3f));
example.Features.Feature.Add("PetalLength", makeFeature(1.7f));
example.Features.Feature.Add("PetalWidth", makeFeature(0.5f));
TFTensor inputTensor = TFTensor.CreateString(
new [] { example.ToByteArray() }, new TFShape(1));
runner.AddInput(tfGraph["input_example_tensor"][0], inputTensor);
runner.Fetch("dnn/head/predictions/probabilities", 0);
//start the run and get the results of the iris example
var output = runner.Run();
TFTensor result = output[0];
I am trying to write a generator plugin for specflow to modify the steps or tags that end up in the generated feature.cs files.
I have tried to follow the method described here:
SpecFlow- is it possible to programmatically add lines to a scenario?
relevant code:
public new CodeNamespace GenerateUnitTestFixture(Feature feature, string testClassName, string targetNamespace)
{
foreach (var scenario in feature.Scenarios)
{
scenario.Steps.Insert(0, new Given { Text = "Given I have <Theme> set as my current theme" });
//add any other steps you need....
}
return base.GenerateUnitTestFixture(feature, testClassName, targetNamespace);
}
The definition of GenerateUnitTestFixture has since changed in UnitTestFeatureGenerator, but using roughly the same method I am unable to insert steps as the entire GherkinDocument argument passed in to the function is readonly including steps so they cannot be modified or added to.
public new CodeNamespace GenerateUnitTestFixture(SpecFlowDocument document, string testClassName, string targetNamespace)
{
var feature = document.Feature;
foreach (ScenarioDefinition scenarioDefinition in feature.Children)
{
scenarioDefinition.Steps.///Cannot insert
}
return base.GenerateUnitTestFixture(document, testClassName, targetNamespace);
}
The same problem occurs for tags. I cant find much up-to-date documentation on this stuff, is it still possible to modify steps/tags or is it no longer supported?
Can any one suggest another way to integrate NodeJs in .Net MVC application? I am now using the following code:
public class Startup
{
public async Task<object> Invoke(dynamic input)
{
DepartmentRep person = new DepartmentRep(new MvcAppUsingEdgeJSMongoDbContext());
var department= person.GetAllDepartments();
//var department = "hello";
return department;
}
}
public class DepartmentController : Controller
{
DepartmentRepository departmentRepository = new DepartmentRepository(new MvcAppUsingEdgeJSMongoDbContext());
string connectionString = ConfigurationManager.AppSettings["connectionString"].ToString();
public ViewResult Index()
{
// var clrMethod = Edge.Func("DepartmentRep.cs");
var getData = Edge.Func("./DepartmentRep.dll");
// return View(clrMethod);
return View(departmentRepository.GetAllDepartments());
}
}
It seems to me, you may have a misunderstanding of the EdgeJs use case.
Your Startup/Invoke class/Signature is meant to be called from Node(JavaScript),
And from the code you are showing it looks like you are loading .Net from .Net
Also , as the Invoke signature suggest, It should be asynchronous.
If you want to use node from .Net side. You should check the project documentation from
scripting-nodejs-from-clr downwards.
var func = Edge.Func(#"
return function (data, callback) {
callback(null, 'Node.js welcomes ' + data);
}
");
As you can see there the wrapped code is Javascript, this time running in .Net more specifically running in Node.
The perfect use case IMMO is the Socket-Server, that is, something Node does better than .Net (IMMO again)
Which is in perfect contrast with the .Net Ado Sql Server access from NodeJs, a .Net Specialization from NodeJs context
In my project I use ServiceStack to allow user to export data in csv format. It's ServiceStack that makes the job but it always sorts my dictionary by alphabetical and I don't want that. I want that my csv file have his columns in exact order that I inserted my data in the dictionary.
There is a way to configure ServiceStack to don't make this sort ?
My dictionary :
var excelResult = new Dictionary<string, string>();
excelResult["Id"] = x.Id.ToString();
excelResult["IBS Account Id"] = x.IBSAccountId.ToString(CultureInfo.InvariantCulture);
excelResult["IBS Order Id"] = x.IBSOrderId.ToString();
And in my csv file, instead of "Id, IBS Account Id,IBS Order Id" I have "IBS Account Id,IBS Order Id,Id"
Thank you very much !!
Here is how to unregister the default 'text/csv' content-type and serializer.
In AppHost.cs,
using ServiceStack.Common;
In AppHost Configure()...
SetConfig(new EndpointHostConfig
{
EnableFeatures = Feature.All.Remove(Feature.Csv),
});
Your apphost will no longer do any special handling for text/csv. At that point, you should be able to register it again as you wish, as explained in How to register your own custom format with ServiceStack.
Substitute your TapasSerializer for CsvSerializer and you should be good.
I discovered a clue how to do it from this SO answer.
Finally, I discover it's impossible(?) to override or custom ServiceStack CsvSerializer.
So, my solution (because I have to do something after all) : Create a new custom type, serialize it and write it in a csv file :)
Step 1 : (AppHOst.cs) Register my new type "tapas" (because I like tapas but I don't think my lead will be ok with that :p)
ContentTypeFilters.Register("text/x-tapas",
TapasSerializer.SerializeToStream, TapasSerializer.DeserializeFromStream);
Step 2 : (AppHOst.cs) Configure the response for the creation of the file
ResponseFilters.Add((req, res, dto) =>
{
if (req.ResponseContentType == "text/x-tapas")
{
res.AddHeader(HttpHeaders.ContentDisposition,
string.Format("attachment;filename={0}.csv", req.OperationName));
}
});
Step 3 : (myPage.handlebars) Swap csv with my new type in my request link (in java for my project)
<li>{{localize "ExportOrders"}}</li>
Step 4 : Your Tapas Serializer !
public class TapasSerializer
{
static TapasSerializer()
{
}
public static void SerializeToStream(IRequestContext requestContext, object response, Stream stream)
{
//TO DO
}
public static object DeserializeFromStream(Type type, Stream stream)
{
//TO DO
}
}
Thank you for reading !