I'm attempting to import a model created in Keras/Tensorflow and use it for inference in a Unity project.
I have successfully imported the model and validated by printing names of input and output nodes in the graph. Though, when I try to get the output value from the runner, I get this exception.
TFException: Attempting to use uninitialized value action_W
[[Node: action_W/read = IdentityT=DT_FLOAT, _class=["loc:#action_W"], _device="/job:localhost/replica:0/task:0/cpu:0"]]
TensorFlow.TFStatus.CheckMaybeRaise (TensorFlow.TFStatus incomingStatus, System.Boolean last) (at <6ed6db22f8874deba74ffe3e566039be>:0)
TensorFlow.TFSession.Run (TensorFlow.TFOutput[] inputs, TensorFlow.TFTensor[] inputValues, TensorFlow.TFOutput[] outputs, TensorFlow.TFOperation[] targetOpers, TensorFlow.TFBuffer runMetadata, TensorFlow.TFBuffer runOptions, TensorFlow.TFStatus status) (at <6ed6db22f8874deba74ffe3e566039be>:0)
TensorFlow.TFSession+Runner.Run (TensorFlow.TFStatus status) (at <6ed6db22f8874deba74ffe3e566039be>:0)
RecordArbitraryData.ModelPredict (System.Single[,] input) (at Assets/Scripts/Spells/RecordArbitraryData.cs:230)
RecordArbitraryData.FixedUpdate () (at Assets/Scripts/Spells/RecordArbitraryData.cs:95)
Here are the two functions I use. InstantiateModel is called OnStart() in my unity script. And ModelPredict is called when the user passes an input to the script.
void InstantiateModel(){
string model_name = "simple_as_binary";
//Instantiate Graph
graphModel = Resources.Load (model_name) as TextAsset;
graph = new TFGraph ();
graph.Import (graphModel.bytes);
session = new TFSession (graph);
}
void ModelPredict(float[,] input){
using (graph) {
using (session) {
//Assign input tensors
var runner = session.GetRunner ();
runner.AddInput (graph [input_node_name] [0], input);
//Calculate and access output of graph
runner.Fetch (graph[output_node_name][0]);
Debug.Log ("Output node name: " + graph [output_node_name].Name);
float[,] recurrent_tensor = runner.Run () [0].GetValue () as float[,];
//var results = runner.Run();
//Debug.Log("Prediciton: " + results);
}
}
}
Any help appreciated - TensorflowSharp is very new to me.
I was able to figure out most of my problems. I'm currently at a point where my model is predicting in unity, but only predicting the first of four classes. My guess is it has something to do with the weights not getting initialized correctly from the checkpoint files? Edit: My values weren't being normalized before being passed to neural network.
Preface: Mozilla Firefox works best for displaying tensorboard; it took me a long time to realize that google chrome was causing my graph to be invisible (tensorboard is how I was able to figure out the nodes that needed to be used for input and output).
First Issue: I was renaming a .pb file into a .bytes file. This is incorrect because the model’s weights come from the checkpoint file, and are given to the nodes held in the .pb file. This was causing the uninitialized variables. These variables were used for training, which were removed after using the freeze_graph function.
Second Issue: I was using the file created called ‘checkpoint’, which was throwing an error. I then changed the name of the checkpoint to ‘test’ and used this in the freeze_graph function. When calling the checkpoints file, I was required to use ‘test.ckpt’. I assume this function knows to grab the three files automatically based on the .ckpt? ‘Test’ without ‘.ckpt’ did not work.
Third Issue: when using the freeze_graph function, I needed to export the .pb file in keras/tf with text=False. I tested True and False; True threw an error about “bad wiring”.
Fourth Issue: Tensorboard was very difficult to use without any organization. Using tf.name_scope helped a lot with not only visualization, but making sure I was using/referencing the correct nodes in TensorFlowSharp. In keras I found it helpful to separate the final Dense layer and Activation into their own scopes so I could find the correct output node. The rest of my network was put into a ‘body’ scope, and the sole input layer in ‘input’ scope. The name_scope function prepends ‘scopename/’ to the node name. I don't think it’s necessary, but it helped me.
Fifth Issue: The version of tensorflowsharp released as a unity package is not up to date. This caused an issue with a keras placeholder for ‘keras_training_phase’. In keras, you pass this along with an input like [0] + input. I tried to do the same by creating a new TFTensor(bool), but I was getting an error ‘inaccessible due to its protection level. This was an error with the implicit conversion between bool and TFTensor in my unity TensorFlowSharp version. To fix this I had to use a function found in this stackoverflow solution where the .bytes file is read in, the placeholder for keras_training_phase is found, and is swapped out for a bool constant set to false. This worked for me because my model was pretrained in python, so it may not be a great fix for someone that’s trying to train and test the model. A condition for removing this node with the freeze_graph function would really be useful.
Hope someone finds this useful!
Related
in the ARFoundation Unity samples I found the following code snippet:
public void BackButtonPressed()
{
if (Application.CanStreamedLevelBeLoaded("Menu"))
{
SceneManager.LoadScene("Menu", LoadSceneMode.Single);
LoaderUtility.Deinitialize();
}
}
Can somebody please explain to me when and why to use Application.CanStreamedLevelBeLoaded(String string)? In the Unity docs I found this as explanation: Can the streamed level be loaded?
But that tells me nothing.
And when and why should one also use LoadSceneMode.Single and LoaderUtility.Deinitialize()?
I found this for LoaderUtility.Deinitialize(): Deinitializes the currently active XR Loader, if one exists. This destroys all subsystems.
I appreciate any help. Thanks!
Application.CanStreamedLevelBeLoaded does exactly what you think it should do when reading the function name, it checks if the streamed level can be loaded. Use it BEFORE attempting to actually load a streamed level. Streamed in this case means downloading the level for the WebGL client.
The progress of the stream means how far the download of your level has progressed.
Use this function to check the download state, just like you see in the doc example, to prevent an exception when trying to load an unfinished level.
Behind the scenes it uses a float called downloadProgress of the UnityWebRequest to determine if the file is downloaded.
https://docs.unity3d.com/ScriptReference/Networking.UnityWebRequest-downloadProgress.html
Deeper down you will most probably find a FileStream, hence the name of the function.
I'm trying to update XAxis on Active CATIA document using C#. Anyone knows how to do that? somehowGetActiveAxisSysObject() is placeholder in pseudo code:
Array xMatrix = Array.CreateInstance(typeof(double), 3);
xMatrix.SetValue(5.0, 0);
xMatrix.SetValue(0.0, 1);
xMatrix.SetValue(0.0, 2);
MECMOD.AxisSystem targetAxisSys = **somehowGetActiveAxisSysObject();**
targetAxisSys.PutXAxis(xMatrix);
THANK YOU!
It's been a while since I worked with Catia in C# so the code below may not run properly.
First, I will assume you are working in a part document and that you have created an axis system from the menu: Insert->AxisSystems->AxisSystem (or named something similar). Next, I will also assume that you went into this items properties and renamed it to "Larry".
MECMOD.AxisSystems thisPartsAxisSysCollection = (MECMOD.AxisSystems)Part.AxisSystems;
MECMOD.AxisSystem oneAxisSys = thisPartsAxisSysCollection(1); //Index is name unknown
//or
MECMOD.AxisSystem oneAxisSys = thisPartsAxisSysCollection("Larry");
oneAxisSys.PutXAxis("your data");
If this is not what you are trying to do, but instead are trying to move the part's origin then you will need to access Part.OriginElements instead and use that object's PlaneXY, PlaneYZ, and PlaneZX. Unfortunately those are read only if I remember correctly.
I am not a c# guy. Needless to say that I don't have experience on this topic.
I bought a software and I installed it on my computer. Now I thought of using some of it's functions on my software(that I make to plan in c#). So I messaged the person who sold me the software and he send me a two page pdf file explaining what to do.
It states:
This software features a COM interface.
And it goes saying it's API contains a function "stackAPI".
and the parameters used are apiname type string, apipass type string.
Return values type long. 0 for sucess and 1 for error.
That's all it states, I tired searching google, it could not help me at all. So how do I start?
when I write the following code on c# it gives me error.
string[] apiname;
string[] apipass;
stackAPI(apiname, apipass);
I know if I was using dll I would import it as
[DllImport("example.dll"]
But no dll is provided.
Do I need to add the path to the folder where the software is installed to call the API ?
To get started:
using example.dll;
Then in your main class:
example.CustomService api = new example.CustomService();
var response = api.Dostuff();
Console.WriteLine(response);
If anyone wants to know how I did it. After a week of searching I was able to find the solution today. I wanted to call the function stackAPI(apiname, apipass);.
stackAPI(apiname, apipass) was the member of stack.callapi
So I wrote:
dynamic lifestohack = Activator.CreateInstance(Type.GetTypeFromProgID("stack.callapi"));
than just you can call the function like this.
int rv;
string[] apiname;
string[] apipass;
rv=lifestoahck.stackAPI(apiname, apipass);
Hope it may help someone.
I started with the solution here http://social.technet.microsoft.com/wiki/contents/articles/20547.biztalk-server-dynamic-schema-resolver-real-scenario.aspx
which matches my scenario perfectly except for the send port, but that isn't necessary. I need the receive port to choose the file and apply a schema to disassemble. From their the orchestration does the mapping, some of it custom, etc.
I've done everything in the tutorial but I keep getting the following error.
"There was a failure executing the receive pipeline... The body part is NULL"
The things I don't get from the tutorial but don't believe they should be an issue are:
I created a new solution and project to make the custompipeline component (reference figure 19) and thus the dll file. Meaning it is on it's own namespace. However, it looks like from the tutorial they created the project within the main biztalk solution (ie the one with the pipeline and the orchestration) and thus the namespace has "TechNetWiki.SchemaResolver." in it. Should I make the custompipeline component have the namespace of my main solution? I'm assuming this shouldn't matter because I should be able to use this component in other solutions as it is meant to be generic to the business rules that are associated with the biztalk application.
The other piece I don't have is Figure 15 under the "THEN Action" they have it equal the destination schema they would like to disassemble to but then they put #Src1 at the end of "http://TechNetWiki.SchemaResolver.Schemas.SRC1_FF#Src1". What is the #Src1 for?
In the sample you've linked to, the probe method of the pipeline component is pushing the first 4 characters from the filename into a typed message that is then passed into the rules engine. Its those 4 characters that match the "SRC1" in the example.
string srcFileName = pInMsg.Context.Read("ReceivedFileName", "http://schemas.microsoft.com/BizTalk/2003/file-properties This link is external to TechNet Wiki. It will open in a new window. ").ToString();
srcFileName = Path.GetFileName(srcFileName);
//Substring the first four digits to take source code to use to call BRE API
string customerCode = srcFileName.Substring(0, 4);
//create an instance of the XML object
XmlDocument xmlDoc = new XmlDocument();
xmlDoc.LoadXml(string.Format(#"<ns0:Root xmlns:ns0='http://TechNetWiki.SchemaResolver.Schemas.SchemaResolverBRE This link is external to TechNet Wiki. It will open in a new window. '>
<SrcCode>{0}</SrcCode>
<MessageType></MessageType>
</ns0:Root>", customerCode));
//retreive source code in case in our cache dictionary
if (cachedSources.ContainsKey(customerCode))
{
messageType = cachedSources[customerCode];
}
else
{
TypedXmlDocument typedXmlDocument = new TypedXmlDocument("TechNetWiki.SchemaResolver.Schemas.SchemaResolverBRE", xmlDoc);
Microsoft.RuleEngine.Policy policy = new Microsoft.RuleEngine.Policy("SchemaResolverPolicy");
policy.Execute(typedXmlDocument);
So the matching rule is based on the 1st 4 characters of the filename. If one isn't matched, the probe returns a false - i.e. unrecognised.
The final part is that the message type is pushed into the returned message - this is made up of the namespace and the root schema node with a # separator - so your #src1 is the root node.
You need to implement IProbeMessage near to class
I forgot to add IProbeMessage in the code of article. It is updated now.
but it is there in sample source code
Src1 is the the root node name of schema. I mentioned that in article that message type is TargetNamespace#Root
I recommend to download the sample code
I hope this will help you
My plan:
I'm trying to setup my C# project to communicate with Nodebox to call a certain function which populates a graph and draws it in a new window.
Current situation: [fixed... see Update2]
I have already included all python-modules needed, but im still getting a
Library 'GL' not found
it seems that the pyglet module needs a reference to GL/gl.h, but can't find it due to IronPython behaviour.
Requirement:
The project needs to stay as small as possible without installing new packages. Thats why i have copied all my modules into the project-folder and would like to keep it that or a similar way.
My question:
Is there a certain workaround for my problem or a fix for the library-folder missmatch.
Have read some articles about Tao-Opengl and OpenTK but can't find a good solution.
Update1:
Updated my sourcecode with a small pyglet window-rendering example. Problem is in pyglet and referenced c-Objects. How do i include them in my c# project to be called? No idea so far... experimenting alittle now. Keeping you updated.
SampleCode C#:
ScriptRuntimeSetup setup = Python.CreateRuntimeSetup(null);
ScriptRuntime runtime = new ScriptRuntime(setup);
ScriptEngine engine = Python.GetEngine(runtime);
ScriptSource source = engine.CreateScriptSourceFromFile("test.py");
ScriptScope scope = engine.CreateScope();
source.Execute(scope);
SampleCode Python (test.py):
from nodebox.graphics import *
from nodebox.graphics.physics import Vector, Boid, Flock, Obstacle
flock = Flock(50, x=-50, y=-50, width=700, height=400)
flock.sight(80)
def draw(canvas):
canvas.clear()
flock.update(separation=0.4, cohesion=0.6, alignment=0.1, teleport=True)
for boid in flock:
push()
translate(boid.x, boid.y)
scale(0.5 + boid.depth)
rotate(boid.heading)
arrow(0, 0, 15)
pop()
canvas.size = 600, 300
def main(canvas):
canvas.run(draw)
Update2:
Line 139 [pyglet/lib.py] sys.platform is not win32... there was the error. Fixed it by just using the line:
from pyglet.gl.lib_wgl import link_GL, link_GLU, link_WGL
Now the following Error:
'module' object has no attribute '_getframe'
Kind of a pain to fix it. Updating with results...
Update3:
Fixed by adding following line right after first line in C#-Code:
setup.Options["Frames"] = true;
Current Problem:
No module named unicodedata, but in Python26/DLLs is only a *.pyd file`. So.. how do i implement it now?!
Update4:
Fixed by surfing: link text and adding unicodedata.py and '.pyd to C# Projectfolder.
Current Problem:
'libGL.so not found'... guys.. im almost giving up on nodebox for C#.. to be continued
Update5:
i gave up :/ workaround: c# communicating with nodebox over xml and filesystemwatchers. Not optimal, but case solved.
-X:Frames enables the frames option as runtime (it slows code down a little to have access to the Python frames all the time).
To enable frames when hosting you just need to do:
ScriptRuntimeSetup setup = Python.CreateRuntimeSetup(new Dictionary<string, object>() {
{ "Frames", true }
});
Instead of the null that you're passing now. That's just creating a new dictionary for the options dictionary w/ the contents "Frames" set to true. You can set other options in there as well and in general the -X:Name option is the same here as it is for the command line.