Develop Reference

I am trying to call python using C# with the IronPython nuget

I am very new to this and require some help. I am trying to call something basic as per the section below using C# with the IronPython nuget.
Below is what I am trying:
Microsoft.Scripting.Hosting.ScriptEngine py = Python.CreateEngine();
Microsoft.Scripting.Hosting.ScriptScope s = py.CreateScope();
py.Execute("import numpy as np incomes = np.random.normal(27000, 15000, 10000) x = np.mean(incomes)", s);
I keep receiving the following error:
An exception of type 'Microsoft.Scripting.SyntaxErrorException' occurred in Microsoft.Scripting.dll but was not handled in user code
Any help would be appreciated thank you

Be aware of indentation when working with Python. This works:
using IronPython.Hosting;
namespace PythonFromCSharp
{
class Program
{
static void Main(string[] args)
{
Microsoft.Scripting.Hosting.ScriptEngine py = Python.CreateEngine();
Microsoft.Scripting.Hosting.ScriptScope s = py.CreateScope();
// add paths to where your libs are installed
var libs = new[] { #"c:\path\to\lib1", #"c:\path\to\lib2" };
py.SetSearchPaths(libs);
py.Execute(
#"import numpy as np
incomes = np.random.normal(27000, 15000, 10000)
x = np.mean(incomes)"
, s);
}
}
}

Your Python syntax is incorrect. Insert line breaks (\n)
py.Execute("import numpy as np\nincomes = np.random.normal(27000, 15000, 10000)\nx = np.mean(incomes)", s);

C# SharpDX compilefromfile with embedded resource

Hello I have few files which are now embedded resources, but problem is using:
CompilationResult result = SharpDX.D3DCompiler.ShaderBytecode.CompileFromFile(
fileName,
entryPoint,
profile,
shaderFlags,
include: FileIncludeHandler.Default,
defines: defines);
Put error:
System.IO.FileNotFoundException: 'Unable to find file'. I found that there is other function which Compiles from source:
CompilationResult result = SharpDX.D3DCompiler.ShaderBytecode.Compile(data,profile,shaderFlags);
To read from embedded resource file I using this little class: Link 1
Usage looks like this:
string data = ResourceHelper.GetEmbeddedResource(fileName);
CompilationResult result = SharpDX.D3DCompiler.ShaderBytecode.Compile(data,profile,shaderFlags);
But now I get this error:
System.ArgumentNullException: 'Value cannot be null.
Parameter name: entryPoint'
To check if app loads all embeded resource I used this small method and it returns all embedded resources
string[] zz = Assembly.GetExecutingAssembly().GetManifestResourceNames();
MessageBox.Show(string.Join("\n", zz));
What I'm missing here?
Edit 1:
This is full method of how I use it:
public static ShaderBytecode CompileShader(string fileName, string entryPoint, string profile, ShaderMacro[] defines = null)
{
var shaderFlags = ShaderFlags.None;
var assembly = Assembly.GetExecutingAssembly();
using (Stream stream = assembly.GetManifestResourceStream(fileName))
{
using (var reader = new StreamReader(stream))
{
CompilationResult result = SharpDX.D3DCompiler.ShaderBytecode.Compile(reader.ReadToEnd(),entrypoint,profile,shaderFlags);
/*CompilationResult result = SharpDX.D3DCompiler.ShaderBytecode.CompileFromFile(
fileName,
entryPoint,
profile,
shaderFlags,
include: FileIncludeHandler.Default,
defines: defines);*/
return new ShaderBytecode(result);
}
}
}
Real Usage:
string MainName = "my_project";
_shaders["standardVS"] = D3DUtility.CompileShader(MainName+".Shaders.Default.hlsl", "VS", "vs_5_1");
//50 other files...
Image 1
EDIT 2: Added entrypoint to Compile method, but now it throws error:
System.NullReferenceException: 'Object reference not set to an instance of an object.'
In:
return new ShaderBytecode(result);
Execption handler shows this:
Message="C\FileLocation: error X1505: No include handler specified, can't perform a #include. Use D3DX APIs or provide your own include handler.\n"
EDIT 3: Using #J. van Langen updated method it shows like this:
System.Exception: 'C:\Users\test\source\repos\myapp\Debug\Resources\DemoScene\unknown(14,10-30): error X1505: No include handler specified, can't perform a #include. Use D3DX APIs or provide your own include handler.'
result.Message = result.Message="error CS0452: The type 'ShaderBytecode' must be a reference type in order to use it as parameter 'T' in the generic type or method 'CompilationResultBase<T>'"
EDIT 4: I think I found why its throwing error since it has include inside it:
Pastebin

You shouldn't use CompileFromFile but just Compile also, you need the overload with the Compile(string shaderSource, string entryPoint, string profile, ShaderFlags shaderFlags, ..... the rest of the parameters uses default.
Here's an example from an old project i've done:
static GradientRenderer()
{
// This may be changed to GetType() (see 'The new reflection API')
var assembly = typeof(GradientRenderer).GetTypeInfo().Assembly;
//string[] resources = assembly.GetManifestResourceNames();
string code;
// use the full filename with namespace
using (var stream = assembly.GetManifestResourceStream("MirageDX11.Renderers.Gradient.Gradient.hlsl"))
using (var reader = new StreamReader(stream))
// read the whole content to a string.
code = reader.ReadToEnd();
var shaderFlags = ShaderFlags.None;
#if DEBUG
shaderFlags |= ShaderFlags.Debug;
shaderFlags |= ShaderFlags.SkipOptimization;
#endif
// Compile the vertex shader and the pixel shader "VS" & "PS" => entrypoint
_vertexShaderByteCode = ShaderBytecode.Compile(code, "VS", "vs_5_0", shaderFlags);
_pixelShaderByteCode = ShaderBytecode.Compile(code, "PS", "ps_5_0", shaderFlags);
}
Here's the shader
struct VertexIn
{
float3 PosL : POSITION;
float4 Color : COLOR;
};
struct VertexOut
{
float4 PosH : SV_POSITION;
float4 Color: COLOR;
};
VertexOut VS(VertexIn vin)
{
VertexOut vout;
vout.PosH = float4(vin.PosL, 1.0f);
vout.Color = vin.Color;
return vout;
}
float4 PS(VertexOut pin) : SV_Target
{
float4 value = pin.Color;
value.w = 1.0f;
return value;
}
Update about how to capture shader errors:
I've modified your method to capture the shader compile errors. It will raise an exception with a useful message.
public static ShaderBytecode CompileShader(string fileName, string entryPoint, string profile, ShaderMacro[] defines = null)
{
var shaderFlags = ShaderFlags.None;
var assembly = Assembly.GetExecutingAssembly();
using (Stream stream = assembly.GetManifestResourceStream(fileName))
{
using (var reader = new StreamReader(stream))
{
CompilationResult result = SharpDX.D3DCompiler.ShaderBytecode.Compile(reader.ReadToEnd(),entrypoint,profile,shaderFlags);
// when the Bytecode == null, means that an error has occurred
if (result.Bytecode == null)
throw new InvalidOperationException(result.Message);
// removed old code in comment...
return new ShaderBytecode(result);
}
}
}

How to use incompleted types in reflection?

I want to dynamically generate the assembly, which can have functions with different structures. To be more accurate, these functions can be recursive, they can call other functions within the same assembly etc. I found the System.Reflection module which theoretically provides tools to do this, but in practice I have encountered many drawbacks for this approach. For example - I cannot generate recursive functions via TypeBuilder and MethodBuilder classes, because exception will be thrown (usage of incomplete types). I learned that I can generate selfrecursive functions via IlGenerator - but it is too cumbersome - I hoped that there is an easier way to do this.
Here is my program which demonstrates the problem (at generation of method Fact the following Exception is thrown:
Exception thrown: 'System.NotSupportedException' in mscorlib.dll
Additional information: Specified method is not supported..
Code:
using System;
using System.Reflection;
using System.Reflection.Emit;
using System.Linq.Expressions;
namespace GenericFuncs
{
public class ProgramBuilder
{
public Type createMyProgram()
{
var assmName = new AssemblyName("DynamicAssemblyExample");
var assmBuilder = AppDomain.CurrentDomain.DefineDynamicAssembly(assmName, AssemblyBuilderAccess.RunAndSave);
var moduleBuilder = assmBuilder.DefineDynamicModule(assmName.Name, assmName.Name + ".dll");
var myProgramType = buildMyProgram(moduleBuilder, moduleBuilder.DefineType("MyProgram", TypeAttributes.Public));
assmBuilder.Save(assmName.Name + ".dll");
return myProgramType;
}
private Type buildMyProgram(ModuleBuilder mb, TypeBuilder programBuilder)
{
buildFactFunction2(mb, mb.GetType("MyProgram"), programBuilder.DefineMethod("InfLoop", MethodAttributes.Public | MethodAttributes.Static));
buildFactFunction(mb, mb.GetType("MyProgram"), programBuilder.DefineMethod("Fact", MethodAttributes.Public | MethodAttributes.Static));
return programBuilder.CreateType();
}
private void buildFactFunction(ModuleBuilder mb, Type program_type, MethodBuilder methodBuilder)
{
var param = Expression.Parameter(typeof(int), "n");
var lambda = Expression.Lambda(Expression.Call(methodBuilder, param), param);
lambda.CompileToMethod(methodBuilder);
}
static public void my_print(string s)
{
Console.WriteLine(s);
}
private void buildFactFunction2(ModuleBuilder mb, Type program_type, MethodBuilder methodBuilder)
{
var il = methodBuilder.GetILGenerator();
il.Emit(OpCodes.Ldstr, "a");
il.Emit(OpCodes.Call, typeof(ProgramBuilder).GetMethod("my_print"));
il.Emit(OpCodes.Call, methodBuilder);
il.Emit(OpCodes.Ret);
}
}
class Program
{
static void Main(string[] args)
{
var pbuilder = new ProgramBuilder();
var ptype = pbuilder.createMyProgram();
Console.ReadLine();
}
}
}
Any help is appreciated.

Expressions and reflection in C# have their limitations (albeit still very powerful for simple use cases). To the best of my knowledge, Emit is the way to go if you need the functionality which you describe (even the basic requirement of emitting assemblies).
Some years ago, I used RunSharp quite effectively in some minimal dynamic code generation use cases. It takes away most of the IL pains. For example, in this code I create a proxy wrapper at runtime.
You could also have a look at what Castle Project uses for their code generation, e.g., their DynamicProxy is quite popular.

This is probably not what you are after but have you thought about CodeDom ?
The downside is it wont be truly dynamic, as you have to load the assembly
Dynamic Source Code Generation and Compilation
As pointed out by Steven Jeuris There is a great question on the differences between CodeDom and Emit Reflection.Emit vs CodeDOM
CodeDom generates C# source code and is usually used when generating code to be included as part of a solution and compiled in the IDE (for
example, LINQ to SQL classes, WSDL, XSD all work this way). In this
scenario you can also use partial classes to customize the generated
code. It is less efficient, because it generates C# source and then
runs the compiler to parse it (again!) and compile it. You can
generate code using relatively high-level constructs (similar to C#
expressions & statements) such as loops.
Reflection.Emit generates an IL so it directly produces an assembly that can be also stored only in memory. As a result is a lot more
efficient.You have to generate low-level IL code (values are stored on
stack; looping has to be implemented using jumps), so generating any
more complicated logic is a bit difficult.
Simple CodeDom Exmaple from MSDN
public Sample()
{
targetUnit = new CodeCompileUnit();
CodeNamespace samples = new CodeNamespace("CodeDOMSample");
samples.Imports.Add(new CodeNamespaceImport("System"));
targetClass = new CodeTypeDeclaration("CodeDOMCreatedClass");
targetClass.IsClass = true;
targetClass.TypeAttributes =
TypeAttributes.Public | TypeAttributes.Sealed;
samples.Types.Add(targetClass);
targetUnit.Namespaces.Add(samples);
}
public void AddFields()
{
// Declare the widthValue field.
CodeMemberField widthValueField = new CodeMemberField();
widthValueField.Attributes = MemberAttributes.Private;
widthValueField.Name = "widthValue";
widthValueField.Type = new CodeTypeReference(typeof(System.Double));
widthValueField.Comments.Add(new CodeCommentStatement(
"The width of the object."));
targetClass.Members.Add(widthValueField);
// Declare the heightValue field
CodeMemberField heightValueField = new CodeMemberField();
heightValueField.Attributes = MemberAttributes.Private;
heightValueField.Name = "heightValue";
heightValueField.Type =
new CodeTypeReference(typeof(System.Double));
heightValueField.Comments.Add(new CodeCommentStatement(
"The height of the object."));
targetClass.Members.Add(heightValueField);
}
public void GenerateCSharpCode(string fileName)
{
CodeDomProvider provider = CodeDomProvider.CreateProvider("CSharp");
CodeGeneratorOptions options = new CodeGeneratorOptions();
options.BracingStyle = "C";
using (StreamWriter sourceWriter = new StreamWriter(fileName))
{
provider.GenerateCodeFromCompileUnit(
targetUnit, sourceWriter, options);
}
}
static void Main()
{
Sample sample = new Sample();
sample.AddFields();
//sample.AddProperties();
//sample.AddMethod();
//sample.AddConstructor();
//sample.AddEntryPoint();
sample.GenerateCSharpCode(outputFileName);
}
And you could use one of several methods to run your code
private static Assembly CompileSourceCodeDom(string sourceCode)
{
CodeDomProvider cpd = new CSharpCodeProvider();
var cp = new CompilerParameters();
cp.ReferencedAssemblies.Add("System.dll");
cp.GenerateExecutable = false;
CompilerResults cr = cpd.CompileAssemblyFromSource(cp, sourceCode);
return cr.CompiledAssembly;
}
private static void ExecuteFromAssembly(Assembly assembly)
{
Type fooType = assembly.GetType("Foo");
MethodInfo printMethod = fooType.GetMethod("Print");
object foo = assembly.CreateInstance("Foo");
printMethod.Invoke(foo, BindingFlags.InvokeMethod, null, null, CultureInfo.CurrentCulture);
}

DLR: Do i really need code generation here?

spent some time again with the scripting interface of my app.
i guess i now have an advanced dlr problem here.
I have a python script
I have an .NET object [o1]
I call a method on the python script from .NET via Iron Python.
The python code creates an object [o2]
The python code calls an method on object [o1] passing [o2] as an argument (Via subclassing DynamicMetaObject)
In the .NET code of the o1-method i want to dynamically call methods on o2
For example i could do that via
((dynamic)o2).FuncInPythonScript
so far so good thats all working.
.NET calls Python (step 3)
Python calls back .NET (step 5)
So i have a basic biderectional control flow between .NET and Python.
We go further:
In the [o1]-method I use LanguageContext.GetMemberNames on [o2]
I wanna call these members somehow via reflection or expressions.
Meaning i dont wanna use the dynamic keyword as in step 7.
Instead somehow call the methods via reflection.
Problem is:
a) I do not know how to get the RuntimeType of the Python-Type, meaning i have no System.Reflection.MethodInfo so i stuck here
b) I try to use LanguageContext.CreateCallBinder and MetaObject.BindInvokeMember so i should have the method 'FuncInPythonScript' bound
But then i'm stuck in how to finally call the bound method.
I see i could use code generation to just generate the code as in step 7, just with the member names from step 8.
But is that really necessary?
I do not see wether approach a) or b) might work or maybe there is somthing i did not think of.
Please do not answer with basic "How do i invoke a python method from .NET" hints.
That is done in steps 1-7 and i have no problem doing this. It's really an advanced problem.
namespace DynamicMetaObjectTest
{
using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.Threading.Tasks;
using System.IO;
using System.Dynamic;
using System.Linq.Expressions;
using Microsoft.Scripting.Hosting;
using Microsoft.Scripting.Hosting.Providers;
class Program
{
internal sealed class CDotNetObject : IDynamicMetaObjectProvider
{
DynamicMetaObject IDynamicMetaObjectProvider.GetMetaObject(Expression aExp)
{
return new CInvoker(this, aExp);
}
private sealed class CInvoker : DynamicMetaObject
{
internal CInvoker(CDotNetObject aGws, Expression aExp) : base(aExp, BindingRestrictions.Empty, aGws)
{
this.DotNetObject = aGws;
}
private readonly CDotNetObject DotNetObject;
public override DynamicMetaObject BindGetMember(GetMemberBinder binder)
{
var aMethodInfo = this.GetType().GetMethod("GetSetResultDelegate");
var aExp = Expression.Call(Expression.Constant(this), aMethodInfo);
var aRestrictions = BindingRestrictions.GetTypeRestriction(this.Expression, this.LimitType);
var aMetaObject = new DynamicMetaObject(aExp, aRestrictions);
return aMetaObject;
}
public Action<object> GetSetResultDelegate()
{
return this.DotNetObject.SetResultProvider;
}
}
public void SetResultProvider(object aPythonObject_O2)
{
var aResult = ((dynamic)aPythonObject_O2).GetResult(); // this is for noobs. ;-)
var aMetaObjectProvider = (IDynamicMetaObjectProvider)aPythonObject_O2;
var aMetaObject = aMetaObjectProvider.GetMetaObject(Expression.Constant(aPythonObject_O2));
var aLanguageContext = HostingHelpers.GetLanguageContext(gScriptEngine);
var aMemberNames = aLanguageContext.GetMemberNames(aPythonObject_O2);
var aNonSystemMembers = from aMemberName in aMemberNames where !aMemberName.StartsWith("__") select aMemberName;
foreach (var aMemberName in aNonSystemMembers)
{
Console.WriteLine("Getting function result from Python script: " + aMemberName);
// Now problem:
// P1) How to determine wether its an function or an member variable?
// P2) How to invoke the method respectively get the value of the member variable?
// Your turn ;-)
// some of my failures:
{ // does not work:
//var aVar1Binder = aLanguageContext.CreateGetMemberBinder("GetVar1", false);
//var aVar1Bound = aMetaObject.BindGetMember(aVar1Binder);
//var aCallInfo = new CallInfo(0 , new string[]{});
//var aInvokeBinder = aLanguageContext.CreateCallBinder("GetVar1", false, aCallInfo);
//var aInvokeBound = aMetaObject.BindInvokeMember(aInvokeBinder, new DynamicMetaObject[]{ aVar1Bound});
////var aInvokeExp = Expression.Invoke(Expression.Constant(aInvokeBound), new Expression[] { });
}
{ // does not work
//var aExpandable = (IronPython.Runtime.Binding.IPythonExpandable)aMetaObject;
}
}
}
}
static ScriptEngine gScriptEngine;
static void Main(string[] args)
{
var aScriptRuntime = IronPython.Hosting.Python.CreateRuntime();
// That's the python script from step 1:
var aCode = "class CustomView(object) :" + Environment.NewLine +
"\tdef GetResult(self) :" + Environment.NewLine +
"\t\treturn 42;" + Environment.NewLine + // cuz 42 is the answer to everything ;-)
"DotNetObject.SetResultProvider(CustomView())";
var aEngine = aScriptRuntime.GetEngine("py");
gScriptEngine = aEngine;
var aScope = aEngine.CreateScope();
var aDotNetObject = new CDotNetObject();
aScope.SetVariable("DotNetObject", aDotNetObject);
// That's the invoke to pything from step 3:
aEngine.Execute(aCode, aScope);
}
}
}

Embedding Python in C# via resources

I have been working on a problem for a while now which I cannot seem to resolve so I need some help! The problem is that I am writing a program in C# but I require a function from a Python file I created. This in itself is no problem:
...Usual Stuff
using IronPython.Hosting;
using IronPython.Runtime;
using Microsoft.Scripting;
using Microsoft.Scripting.Hosting;
namespace Program
{
public partial class Form1 : Form
{
Microsoft.Scripting.Hosting.ScriptEngine py;
Microsoft.Scripting.Hosting.ScriptScope s;
public Form1()
{
InitializeComponent();
py = Python.CreateEngine(); // allow us to run ironpython programs
s = py.CreateScope(); // you need this to get the variables
}
private void doPython()
{
//Step 1:
//Creating a new script runtime
var ironPythonRuntime = Python.CreateRuntime();
//Step 2:
//Load the Iron Python file/script into the memory
//Should be resolve at runtime
dynamic loadIPython = ironPythonRuntime.;
//Step 3:
//Invoke the method and print the result
double n = loadIPython.add(100, 200);
numericUpDown1.Value = (decimal)n;
}
}
}
However, this requires for the file 'first.py' to be wherever the program is once compiled. So if I wanted to share my program I would have to send both the executable and the python files which is very inconvenient. One way I thought to resolve this is by adding the 'first.py' file to the resources and running from there... but I don't know how to do this or even if it is possible.
Naturally the above code will not work for this as .UseFile method takes string arguments not byte[]. Does anyone know how I may progress?

Lets start with the simplest thing that could possibly work, you've got some code that looks a little like the following:
// ...
py = Python.CreateEngine(); // allow us to run ironpython programs
s = py.CreateScope(); // you need this to get the variables
var ironPythonRuntime = Python.CreateRuntime();
var x = py.CreateScriptSourceFromFile("SomeCode.py");
x.Execute(s);
var myFoo = s.GetVariable("myFoo");
var n = (double)myFoo.add(100, 200);
// ...
and we'd like to replace the line var x = py.CreateScriptSourceFromFile(... with something else; If we could get the embedded resource as a string, we could use ScriptingEngine.CreateScriptSourceFromString().
Cribbing this fine answer, we can get something that looks a bit like this:
string pySrc;
var resourceName = "ConsoleApplication1.SomeCode.py";
using (var stream = System.Reflection.Assembly.GetExecutingAssembly()
.GetManifestResourceStream(resourceName))
using (var reader = new System.IO.StreamReader(stream))
{
pySrc = reader.ReadToEnd();
}
var x = py.CreateScriptSourceFromString(pySrc);