How to serialize tensor input required by dnnclassifier (serving_input_reciever) - c#

I want to be able to use the dnnclassifier (estimator) on top of IIS using tensorflowsharp. The model has previously been trained in python. I got so far that I can now generate PB files, know the correct input/outputs, however I am stuck in tensorflowsharp using string inputs.
I can create a valid .pb file of the iris dataset. It uses the following feate_spec:
{'SepalLength': FixedLenFeature(shape=(1,), dtype=tf.float32, default_value=None), 'SepalWidth': FixedLenFeature(shape=(1,), dtype=tf.float32, default_value=None), 'PetalLength': FixedLenFeature(shape=(1,), dtype=tf.float32, default_value=None), 'PetalWidth': FixedLenFeature(shape=(1,), dtype=tf.float32, default_value=None)}
I have created a simple c# console to try and spin it up. The input should be an "input_example_tensor" and the output is located in "dnn/head/predictions/probabilities". This I discoved after alex_zu provided help using the saved_model_cli command here.
As far as I am aware all tensorflow estimator API's work like this.
Here comes the problem: the input_example_tensor should be of a string format which will be parsed internally by the ParseExample function. Now i am stuck. I have found TFTensor.CreateString, but this doesn't solve the problem.
using System;
using TensorFlow;
namespace repository
{
class Program
{
static void Main(string[] args)
{
using (TFGraph tfGraph = new TFGraph()){
using (var tmpSess = new TFSession(tfGraph)){
using (var tfSessionOptions = new TFSessionOptions()){
using (var metaGraphUnused = new TFBuffer()){
//generating a new session based on the pb folder location with the tag serve
TFSession tfSession = tmpSess.FromSavedModel(
tfSessionOptions,
null,
#"path/to/model/pb",
new[] { "serve" },
tfGraph,
metaGraphUnused
);
//generating a new runner, which will fetch the tensorflow results later
var runner = tfSession.GetRunner();
//this is in the actual tensorflow documentation, how to implement this???
string fromTensorflowPythonExample = "{'SepalLength': [5.1, 5.9, 6.9],'SepalWidth': [3.3, 3.0, 3.1],'PetalLength': [1.7, 4.2, 5.4],'PetalWidth': [0.5, 1.5, 2.1],}";
//this is the problem, it's not working...
TFTensor rawInput = new TFTensor(new float[4]{5.1f,3.3f,1.7f,0.5f});
byte[] serializedTensor = System.Text.Encoding.ASCII.GetBytes(rawInput.ToString());
TFTensor inputTensor = TensorFlow.TFTensor.CreateString (serializedTensor);
runner.AddInput(tfGraph["input_example_tensor"][0], inputTensor);
runner.Fetch("dnn/head/predictions/probabilities", 0);
//start the run and get the results of the iris example
var output = runner.Run();
TFTensor result = output[0];
//printing response to the client
Console.WriteLine(result.ToString());
Console.ReadLine();
}
}
}
}
}
}
}
This example will give the following error:
An unhandled exception of type 'TensorFlow.TFException' occurred in TensorFlowSharp.dll: 'Expected serialized to be a vector, got shape: []
[[Node: ParseExample/ParseExample = ParseExample[Ndense=4, Nsparse=0, Tdense=[DT_FLOAT, DT_FLOAT, DT_FLOAT, DT_FLOAT], dense_shapes=[[1], [1], [1], [1]], sparse_types=[], _device="/job:localhost/replica:0/task:0/device:CPU:0"](_arg_input_example_tensor_0_0, ParseExample/ParseExample/names, ParseExample/ParseExample/dense_keys_0, ParseExample/ParseExample/dense_keys_1, ParseExample/ParseExample/dense_keys_2, ParseExample/ParseExample/dense_keys_3, ParseExample/Const, ParseExample/Const, ParseExample/Const, ParseExample/Const)]]'
How can I serialize tensors in such a way that I can use the pb file correctly?
I also posted the issue on github, here you can find the iris example python file, pb file and the console applications. In my opinion solving this creates a
neat solution for all tensorflow users having ancient production environments (like me).

The Expected serialized to be a vector, got shape: [] error can be fixed by using an overload of the TFTensor.CreateString function: Instead of directly taking a string, the model apparently expects a vector containing a single string:
TFTensor inputTensor = TFTensor.CreateString(new byte[][] { bytes }, new TFShape(1));
The input_example_tensor in your case now expects a serialized Example protobuf message (see also the docs and the example.proto file).
Using the protobuf compiler, I've generated a C# file containing the Example class. You can download it from here: https://pastebin.com/iLT8MUdR. Specifically, I used this online tool with CSharpProtoc and replaced the import "tensorflow/core/example/feature.proto"; line by the messages defined in that file.
Once you've added the file to your project, you'll need a package reference to Google.Protobuf. Then, you can pass serialized examples to the model like this:
Func<float, Tensorflow.Feature> makeFeature = (float x) => {
var floatList = new Tensorflow.FloatList();
floatList.Value.Add(x);
return new Tensorflow.Feature { FloatList = floatList };
};
var example = new Tensorflow.Example { Features = new Tensorflow.Features() };
example.Features.Feature.Add("SepalLength", makeFeature(5.1f));
example.Features.Feature.Add("SepalWidth", makeFeature(3.3f));
example.Features.Feature.Add("PetalLength", makeFeature(1.7f));
example.Features.Feature.Add("PetalWidth", makeFeature(0.5f));
TFTensor inputTensor = TFTensor.CreateString(
new [] { example.ToByteArray() }, new TFShape(1));
runner.AddInput(tfGraph["input_example_tensor"][0], inputTensor);
runner.Fetch("dnn/head/predictions/probabilities", 0);
//start the run and get the results of the iris example
var output = runner.Run();
TFTensor result = output[0];

Related

Is there a way to verify an Antlr input file without running it?

I have the following code taking the input of my input file:
var inputStream = new AntlrInputStream(File.ReadAllText(fileName));
var lexer = new LegitusLexer(inputStream);
var commonTokenStream = new CommonTokenStream(lexer);
var parser = new LegitusParser(commonTokenStream);
parser.AddErrorListener(this);
var context = parser.program();
var visitor = new LegitusVisitor(_io.GetDefaultMethods(), _io.GetDefaultVariables())
{
Logger = _logger
};
visitor.Visit(context);
But when I call parser.program(), my program runs as it should. However, I need a way to validate that the input file is syntactically correct, so that users can verify without having to run the scripts (which run against a special machine).
Does Antlr4csharp support this easily?
The Antlr tool can be used to lint the source.
The only difference from a 'standard' tool run is that no output files are generated -- the warnings/errors will be the same.
For example (Java; string source content w/manually applied file 'name'):
Tool tool = new Tool();
tool.removeListeners();
tool.addListener(new YourLintErrorReporter());
ANTLRStringStream in = new ANTLRStringStream(content);
GrammarRootAST ast = tool.parse(name, in);
Grammar g = tool.createGrammar(ast);
g.fileName = name; // to ensure all err msgs identify the file by name
tool.process(g, false); // false -> lint: don't gencode
The CS implementation is equivalent.

What is the best way to load a model in Tensorflow.NET

I saved a tensorflow.keras model in python and need to use in in C# / Tensorflow.NET 0.15
var net = tf.keras.models.load_model(net_name) does not seem to be implemented
var session = tf.Session.LoadFromSavedModel(net_name);
var graph = sess.graph;
seems to work but I have then a session / graph not a keras model
I would ideally like to call something like net.predict(x), how can I get there from a graph/session ?
Yes, i Did. The best way is to convert you package to the ONNX format. ONNX is a open source format that is supposed to run on any framework (tensorflow, torch...)
In python, add the package onnx and keras2onnx:
import onnx
import keras2onnx
import onnxruntime
net_onnx = keras2onnx.convert_keras(net_keras)
onnx.save_model(net_onnx, onnx_name)
Then in C# .NET, install the nuget Microsoft.ML.
var context = new MLContext();
var session = new InferenceSession(filename);
float[] sample;
int[] dims = new int[] { 1, sample_size};
var tensor = new DenseTensor<float>(sample,dims);
var xs = new List<NamedOnnxValue>()
{
NamedOnnxValue.CreateFromTensor<float>("dense_input", tensor),
};
using (var results = session.Run(xs))
{
// manipulate the results
}
Note that you need to call explicitly the fist layer or the input layer of your network to pass on the sample. best is to give it a nice name in Keras. You can check the name in python by running net_keras.summary()

Execution error when calling Matlab compiled function from c#

I am using Matlab 2015a and have developed, trained and tested a classification ensemble (boosted tree) and saved the best trained model (.mat file).
As I wish to use this model in a .Net C# application I have create a .m file to load the .mat file containing the trained classifier and use it to predict an outcome based upon the features passed in (see code for .m file below).
function [ypredict score] = fnTrainedClassifer( input_args )
load ('trainedClassifier.mat');
[ypredict score] = predict(trainedClassifier,input_args);
end
I then used the Matlab compiler to create a .Net assembly and made sure to include the .mat file in the section for files required for your library to run. All good so far... I then created a quick C# console app to test the library (see code of the app below) and ran it.
using System;
using MathWorks.MATLAB.NET.Arrays;
using TrainedClassifierComp;
namespace ConsoleApplication2
{
class Program
{
static void Main(string[] args)
{
MLTestClass theModel = null; /* Stores deployment class instance */
MWStructArray inputs = null; /* Sample input data */
MWArray[] result = null; /* Stores the result */
MWNumericArray prediction = null; /* Ouptut data extracted from result */
MWNumericArray score = null; /* Ouptut data extracted from result */
/* Create the new deployment object */
theModel = new MLTestClass();
/* Create an MWStructArray */
String[] myFieldNames = { "1", "2", "3", "4", "5", "6", "7", "8" };
inputs = new MWStructArray(1, 8, myFieldNames);
/* Populate struct with some sample inputs */
inputs["1", 1] = 1;
inputs["2", 2] = 2;
inputs["3", 3] = 3;
inputs["4", 4] = 4;
inputs["5", 5] = 5;
inputs["6", 6] = 6;
inputs["7", 7] = 7;
inputs["8", 8] = 8;
/* Show some of the sample data */
Console.WriteLine("Inputs: ");
Console.WriteLine(inputs.ToString());
/* Pass it to a MATLAB function */
result = theModel.fnTrainedClassifier(1, inputs);
prediction = (MWNumericArray) result[0];
score = (MWNumericArray)result[1];
/* Show the results */
Console.WriteLine("Prediction: ");
Console.WriteLine(prediction.ToString());
Console.WriteLine("Score: ");
Console.WriteLine(prediction.ToString());
}
}
}
Unfortunately i come up with the following execution time error, which searching on Google bears no useful results:
Warning: Variable 'trainedClassifier' originally saved as a classreg.learning.classif.ClassificationEnsemble cannot be instantiated as an object and will be read in as a uint32.
In fnTrainedClassifier (line 4)
Error using predict (line 84)
Systems of uint32 class cannot be used with the "predict" command. Convert the system to an identified model first, such as by using the "idss" command.
Error in fnTrainedClassifier (line 5)
Any guidance on how to resolve this issue info on how you were able to achieve something similar would be greatly appreciated.
Additional Info: I have tried a couple of other AI systems e.g. neural network and I am having the same problem. It seems to me that I am not approaching this correctly - is there another way of using Matlab functions in external compiled languages such as c#?
It has been a while the question asked, but I faced the same problem. I hope the following solution helps some to save some time.
So, when a class object is loaded from a .mat file it seems Matlab compiler reads the class types but does not load the class itself. Therefore, somehow we need to add the class definition of instantiated object.
To force the compiler to load the required classes, I created an empty object of the class, then I loaded object from the .mat file and assigned it to the empty object.
Your mat code should be something like following;
function [ypredict score] = fnTrainedClassifer( input_args )
trainedClassifier = classreg.learning.classif.ClassificationEnsemble.empty; % only this line is added.
load ('trainedClassifier.mat');
[ypredict score] = predict(trainedClassifier,input_args);
end

Api for working with a classes as OOP?

I'm writing a 3rd party app that needs to read in .cs files and be able to manipulate classes, then ultimately save back to file.
The type of code I am looking at would be something like:
var classManager = new classManager();
var classes = classManager.LoadFromFile(filePath);
var class = classes[0]; // Just illustrating more than 1 class can exist in a file
var prop = new ClassProperty {Type=MyType.GetType() };
prop.AddGet("return x+y < 50");
//stuff like prop.ReadOnly = true;
class.AddProperty(prop);
var method = new ClassMethod {signature="int id, string name"};
method.MethodBody = GetMethodBodyAsString(); //not writing out an entire method body here
class.AddMethod(method);
class.SaveToFile(true); //Format code
Does such a library exist?
The .NET Compiler Platform Roslyn is what you're looking for. It supports parsing and editting cs files. Check out this post for an example

DynamicActivity - How to invoke a workflow that stored in Database?

This is proof of concept project - The goal is to create an application that receive some system wide events and based on some business rules invokes a specific workflow.
The workflows are created separately and the xaml source is stored in a database.
Following is the code that used to invoke the workflow:
public void RaiseEvent(IEvent e, IEventData eventData)
{
var typeName = e.GetType().FullName;
// Query Db for all workflows for the event
var repo = new WorkflowRepository();
var workflows = repo.GetActiveWorkflowsByEvent(typeName);
foreach (var wf in workflows)
{
var condition =
ConditionEvaluator.PrepareCondition(wf.Condition.Expression, eventData);
var okToStart = ConditionEvaluator.Evaluate(condition);
if (okToStart)
{
// Next line is throwing an exeption
object o = XamlServices.Parse(wf.WorkflowDefinition.Expression);
DynamicActivity da = o as DynamicActivity;
WorkflowInvoker.Invoke(da,
new Dictionary<string, object>
{{ "EventData", eventData }});
}
}
We have created very simple workflow that runs without problems on its own. But when xaml is being loaded using XamlService.Parse it throw following exception:
System.Xaml.XamlObjectWriterException was unhandled
Message='No matching constructor found on type 'System.Activities.Activity'.
You can use the Arguments or FactoryMethod directives to construct this type.'
Line number '1' and line position '30'.
Any idea what is wrong?
Thank you.
Not sure what is causing your problem, I have used XamlServices.Load() in the past without any problems, but the easiest way of loading a workflow XAML at runtime is by using the ActivityXamlServices.Load(). See here for an example.
Ok I have solved this by using ActivityXamlServices
So Instead of this line:
object o = XamlServices.Parse(wf.WorkflowDefinition.Expression);
I am using following snippet:
var mStream = new memoryStream(
ASCIIEncoding.Default.GetBytes(wf.WorkflowDefinition.Expression));
object o = ActivityXamlServices.Load(mStream);

Categories

Resources