Develop Reference

C# - Python interop using Python.Net [No module named '_ctypes']

I am trying to use Python.NET to perform interop between C# and Python on a Windows machine.
Specifically I have the following code.
PythonEngine.PythonHome = #"C:\Python\3_5_4";
PythonEngine.PythonPath = #"C:\Python\3_5_4\Lib;C:\Python\3_5_4\Lib\site-packages";
using (Python.Runtime.Py.GIL())
{
dynamic np = Py.Import("numpy");
dynamic sin = np.sin;
Console.WriteLine(sin);
}
While I can successfully execute a general python statement as such:
var res = PythonEngine.Eval("1 + 1");
Console.WriteLine(res);
//res = 2
Which indicates that the python engine itself is working successfully, I can also invoke something like this:
var html = Py.Import("html");
Console.WriteLine(html);
//html = <module 'html' from 'C:\\Python\\3_5_4\\Lib\\html\\__init__.py'>
Which further indicates that the module loading functionality is also working correctly.
However whenever I attempt to invoke the line:
dynamic np = Py.Import("numpy");
I receive the following error:
{"ImportError : No module named '_ctypes'"}.
[' File "C:\\Python\\3_5_4\\Lib\\site-packages\\numpy\\__init__.py", line 140, in <module>\n from . import _distributor_init\n', ' File "C:\\Python\\3_5_4\\Lib\\site-packages\\numpy\\_distributor_init.py", line 9, in <module>\n from ctypes import WinDLL\n', ' File "C:\\Python\\3_5_4\\Lib\\ctypes\\__init__.py", line 8, in <module>\n from _ctypes import Union, Structure, Array\n']
The file referenced is located # 'C:\Python\3_5_4\Lib\ctypes\__init__.py'.
I have verified that all the expected paths are set correctly and that the Ctypes folder exists in my python path.
I have tried everything from uninstalling and reinstalling Python to modifying the ctypes/__init__.py file to try and import Ctypes directly to no effect.
Having researched this topic online throughly I have found a number of suggestions which point to running yum install libffi-devel as detailed here. However this does not appear to be something that I can perform on windows given that yum appears to be a Linux only application.
Can anyone provide any guidance?

How do I run an external python script that's importing 3rd party library himself, in C#?

I was trying to combine a Python code with C#, to use all those cool libraries like Speech Recognition inside my C# application.
I made two different projects one for python (IronPython) where I've included the module I need (Speech Recognition) through the python environment in VS2017 and the other one is just a console application where I want to call this app.
I thought the point was to change the searchPaths of Ironpython and afterwards it'll work.
Maybe I'm doing something wrong or maybe it just shouldn't work anyway?
C# Code Main.cs:
using IronPython.Hosting;
using System.Collections.Generic;
private static void Main(string[] args)
{
//Using Iron python
var engine = IronPython.Hosting.Python.CreateEngine();
System.Console.WriteLine("Search paths:");
ICollection<string> searchPaths = engine.GetSearchPaths();
foreach (string path in searchPaths)
{
System.Console.WriteLine(path);
}
System.Console.WriteLine();
searchPaths.Add("..\\..");
///Trying to add a searchPath for the place with the module I need
searchPaths.Add(#"C:\Program Files (x86)\Microsoft Visual
Studio\Shared\Anaconda3_64\Lib\site-packages");
engine.SetSearchPaths(searchPaths);
var res = engine.CreateScriptSourceFromFile(
#"D:\Python\Projects\TestSpeechRecognition
\TestForNETinPython\TestForNETinPython.py"
);
engine.ImportModule("speech_recognition");
var result = res.Execute();
}
Python code
import speech_recognition as sr
r = sr.Recognizer()
with sr.Microphone() as source:
print ("Hello: ")
audio = r.listen(source)
try:
print ("I said: " + r.recognize_google(audio))
except sr.UnknownValueError:
print ("Cant't rec")
except sr.RequestError as e:
print ("Can't connect: (0)".format(e))
Exception:
Unhandled Exception:
IronPython.Runtime.Exceptions.ImportException: no module named speech_recognition
at IronPython.Hosting.PythonService.ImportModule(ScriptEngine engine, String name)
at IronPython.Hosting.Python.ImportModule(ScriptEngine engine, String moduleName)
at Test.Program.Main(String[] args) in D:\Python\Projects\TestSpeechRecognition\Test\Program.cs:line 26
I even read some literature about creating the modules like this one http://www.needfulsoftware.com/IronPython/IronPythonCS2 or this one it's for pythonnet and a little bit creepy, but I guess it may be useful (NoteBook).
Hope someone can use it to solve the problem.

C# app to call Python script that uses USB-to-CAN transceiver

Just wondering if someone out there might have some insight to what's going wrong here...
I have a python script that connects to a USB-to-CAN transceiver/dongle (made by PEAK System) to do some CAN communications. The script works pretty flawlessly. The script accepts command-line arguments and works fine when called from the Windows command-line.
I am trying to integrate this script into a C# Forms project. I have been successful at calling the Python script from the C# app, but things fall apart when it gets to the point at which the Python script tries to use the CAN transceiver.
It feels like the C# app front-end is not allowing the Python script to access the serial port.
Here is the error I get (Python script writing to StandardOut on the Visual Studio output):
line 86, in canSendRec
self.bus.send(canMessage, timeout=0.1)
AttributeError: 'Node' object has no attribute 'bus'
Unable to Connect to USB-CAN Device
Here is the line from canSendRec -- where the exception handler came from (which we wrote):
try:
self.bus = can.interface.Bus('PCAN_USBBUS1',bitrate=1000000)
self.bus.flush_tx_buffer()
except:
print("Unable to Connect to USB-CAN Device")
Here is my C# code calling the Python script:
public string pythonMakeCall(string script, string arg1){
ProcessStartInfo pyProcessStartInfo = new ProcessStartInfo(py_path);
pyProcessStartInfo.FileName = py_path;
pyProcessStartInfo.Arguments = string.Format("{0} {1}", script, arg1);
pyProcessStartInfo.CreateNoWindow = true;
pyProcessStartInfo.UseShellExecute = false;
pyProcessStartInfo.RedirectStandardOutput = true;
pyProcessStartInfo.RedirectStandardError = true;
Process pyProcess = new Process();
pyProcess.StartInfo = pyProcessStartInfo;
pyProcess.Start();
retString = pyProcess.StandardOutput.ReadToEnd();
pyProcess.WaitForExit();
return retString;}
Like I said, it feels like there is something going on with the C# app not allowing Python to access the USB ports, but I'm not sure where to begin with debugging that hunch, since, Disclaimer: this is my first time dabbling in C#/Visual Studio and I'm no Python expert either.

Communication between Python and C#

I have a Python backend running machine learning algorithms. I want to use the same backend for both an Excel plugin (C#) and a website. I want both interfaces to send my training data (thousands of lines of numbers in arrays) to the same Python application and retrieve the results in the form of another array up to a few thousand lines.
The website would fetch data from a SQL database and send that data to Python, while the Excel plugin would take the data that is in the current worksheet and send that data to Python. I need to be able to create numpy arrays in Python before continuing to process the data. Note that the website would be running on the same machine where the Python application resides. I still haven't decided what I will use to code the website, but I was leaning towards Node.js.
I have done some research and found a few options:
1- Named pipes
2- Sockets
3- RPC server such as gRPC or XML-RPC.
4- Writing the data to a file and reading it back in Python
5- Web Service
Note: I would need the Python "server" to be stateful and keep the session running between calls. So I would need to have a kind of daemon running, waiting for calls.
Which one would you experts recommend and why? I need flexibility to handle several parameters and also large arrays of numbers. Using IronPython is not an option because I am running Keras on Python, which apparently does not support IronPython.

I had the same problem recently.
I used a named pipe to transport data from python to my c# server, hope it helps you.
Python:
import win32pipe, win32file
class PipeServer():
def __init__(self, pipeName):
self.pipe = win32pipe.CreateNamedPipe(
r'\\.\pipe\\'+pipeName,
win32pipe.PIPE_ACCESS_OUTBOUND,
win32pipe.PIPE_TYPE_MESSAGE | win32pipe.PIPE_READMODE_MESSAGE | win32pipe.PIPE_WAIT,
1, 65536, 65536,
0,
None)
#Carefull, this blocks until a connection is established
def connect(self):
win32pipe.ConnectNamedPipe(self.pipe, None)
#Message without tailing '\n'
def write(self, message):
win32file.WriteFile(self.pipe, message.encode()+b'\n')
def close(self):
win32file.CloseHandle(self.pipe)
t = PipeServer("CSServer")
t.connect()
t.write("Hello from Python :)")
t.write("Closing now...")
t.close()
For this code to work you need to install pywin32 (best choice is from binarys): https://github.com/mhammond/pywin32
C#-Server:
using System;
using System.IO;
using System.IO.Pipes;
class PipeClient
{
static void Main(string[] args)
{
using (NamedPipeClientStream pipeClient =
new NamedPipeClientStream(".", "CSServer", PipeDirection.In))
{
// Connect to the pipe or wait until the pipe is available.
Console.Write("Attempting to connect to pipe...");
pipeClient.Connect();
Console.WriteLine("Connected to pipe.");
Console.WriteLine("There are currently {0} pipe server instances open.",
pipeClient.NumberOfServerInstances);
using (StreamReader sr = new StreamReader(pipeClient))
{
// Display the read text to the console
string temp;
while ((temp = sr.ReadLine()) != null)
{
Console.WriteLine("Received from server: {0}", temp);
}
}
}
Console.Write("Press Enter to continue...");
Console.ReadLine();
}
}

You can use Python for .NET (Python.NET). It may require some changes to your code, but then it should work very well, once everything is in good shape.
Python.NET allows two-way communication between CPython and CLR.

Let me give you a neat and quick recipe, in the form of example code.
There are basically two ways to tie python in the backend of C# (or a C# winform app or gui or something similar).
Method1: Iron Python. In this method you install a .net package in your visual studio called IronPython. I would not prefer this, because assuming your machine learning model uses keras or a lot of other libraries. It would be another quest to get you installations ready and working in IronPython. And most importantly, it is not as good as your common virtual env or conda environment.
Method2: (The Good Method): Create a Custom Process in your C# that takes arguments from your GUI, knows the path to your script and your python env. Using all these things, it calls your python code exactly the way you would call it in your terminal and pass arguments to it.
Now the tasty example code (I have used this simple trick and it always helps make my black screen python stuff look good with the cover of C# apps).
Python Part
import sys
a = sys.argv[1]
b = sys.argv[2]
print("The Sum = ", float(a)+float(b))
The C# Part
So here is the python process/function that you need to call on the click event of your sum button in the application
static void PythonProcess()
{
//1) Create Process Info
var psi = new ProcessStartInfo();
//Conda Env Path
psi.FileName = #"C:\Users\jd\.conda\pkgs\py\python.exe";
//2) Provide Script and the Arguments
var script = #"C:\Users\jd\Desktop\script.py";
var a = "15";
var b = "18";
psi.Arguments = $"\"{script}\" \"{a}\" \"{b}\"";
//3) Process Configuration
psi.UseShellExecute = false;
psi.CreateNoWindow = true;
psi.RedirectStandardOutput = true;
psi.RedirectStandardError = true;
//4) Execute Process and get Output.
var errors = "";
var results = "";
using(var process = Process.Start(psi))
{
errors = process.StandardError.ReadToEnd();
results = process.StandardOutput.ReadToEnd();
}
//5) Display Output
Console.WriteLine("ERRORS: ");
Console.WriteLine(errors);
Console.WriteLine();
Console.WriteLine("RESULTS: ");
Console.WriteLine(results);
}

Calling Python from C# is easily possible via Pyrolite where your Python code is running as a Pyro4 server. It should be fast enough to handle "large arrays of numbers" however you didn't specify any performance constraints.

I had the same issue and seem to end up with named pipes. Here is a nice example of how to set it up to talk C# => Python, assuming C# is the server.
It can use the same way to talk back or just Python.net to call directly through CLR as shown here. I use the latter.

IronPython and Nodebox in C#

My plan:
I'm trying to setup my C# project to communicate with Nodebox to call a certain function which populates a graph and draws it in a new window.
Current situation: [fixed... see Update2]
I have already included all python-modules needed, but im still getting a
Library 'GL' not found
it seems that the pyglet module needs a reference to GL/gl.h, but can't find it due to IronPython behaviour.
Requirement:
The project needs to stay as small as possible without installing new packages. Thats why i have copied all my modules into the project-folder and would like to keep it that or a similar way.
My question:
Is there a certain workaround for my problem or a fix for the library-folder missmatch.
Have read some articles about Tao-Opengl and OpenTK but can't find a good solution.
Update1:
Updated my sourcecode with a small pyglet window-rendering example. Problem is in pyglet and referenced c-Objects. How do i include them in my c# project to be called? No idea so far... experimenting alittle now. Keeping you updated.
SampleCode C#:
ScriptRuntimeSetup setup = Python.CreateRuntimeSetup(null);
ScriptRuntime runtime = new ScriptRuntime(setup);
ScriptEngine engine = Python.GetEngine(runtime);
ScriptSource source = engine.CreateScriptSourceFromFile("test.py");
ScriptScope scope = engine.CreateScope();
source.Execute(scope);
SampleCode Python (test.py):
from nodebox.graphics import *
from nodebox.graphics.physics import Vector, Boid, Flock, Obstacle
flock = Flock(50, x=-50, y=-50, width=700, height=400)
flock.sight(80)
def draw(canvas):
canvas.clear()
flock.update(separation=0.4, cohesion=0.6, alignment=0.1, teleport=True)
for boid in flock:
push()
translate(boid.x, boid.y)
scale(0.5 + boid.depth)
rotate(boid.heading)
arrow(0, 0, 15)
pop()
canvas.size = 600, 300
def main(canvas):
canvas.run(draw)
Update2:
Line 139 [pyglet/lib.py] sys.platform is not win32... there was the error. Fixed it by just using the line:
from pyglet.gl.lib_wgl import link_GL, link_GLU, link_WGL
Now the following Error:
'module' object has no attribute '_getframe'
Kind of a pain to fix it. Updating with results...
Update3:
Fixed by adding following line right after first line in C#-Code:
setup.Options["Frames"] = true;
Current Problem:
No module named unicodedata, but in Python26/DLLs is only a *.pyd file`. So.. how do i implement it now?!
Update4:
Fixed by surfing: link text and adding unicodedata.py and '.pyd to C# Projectfolder.
Current Problem:
'libGL.so not found'... guys.. im almost giving up on nodebox for C#.. to be continued
Update5:
i gave up :/ workaround: c# communicating with nodebox over xml and filesystemwatchers. Not optimal, but case solved.

-X:Frames enables the frames option as runtime (it slows code down a little to have access to the Python frames all the time).
To enable frames when hosting you just need to do:
ScriptRuntimeSetup setup = Python.CreateRuntimeSetup(new Dictionary<string, object>() {
{ "Frames", true }
});
Instead of the null that you're passing now. That's just creating a new dictionary for the options dictionary w/ the contents "Frames" set to true. You can set other options in there as well and in general the -X:Name option is the same here as it is for the command line.