Develop Reference

Calling Pyenv Python From C#

I'm trying to run Python from C# via a command line process.
System.Diagnostics.Process proc = new System.Diagnostics.Process {
StartInfo = new System.Diagnostics.ProcessStartInfo {
FileName = "/Users/username/.pyenv/shims/python",
Arguments = cmd+" "+args,
UseShellExecute = false,
RedirectStandardOutput = true,
RedirectStandardError = true,
CreateNoWindow = true
}
};
I derived the FileName from using which:
username$ which python
/Users/username/.pyenv/shims/python
However, running python --version from the terminal, and then again through the C# process, yield different results:
username$ python -V
Python 2.7.11
And C#:
Python 2.7.10
I understand the base problem - that it's calling another version of Python, probably the base one that came from Apple. But what I don't understand is why, because as I understand it I'm telling it to call python from pyenv directly. Is there a way to get C# to use the same python executable I'm using from the terminal?

My bet is that the python shim depends on something else in your environment, and you should probably figure out what it’s eventually running, then call that directly.
According to the pyenv documentation, it's probably at:
$(pyenv root)/versions/2.7.11/bin/python
You might also be able to find out by running the shim with set -x, and looking at the command trace:
(set -x; python --version)

C# app to call Python script that uses USB-to-CAN transceiver

Just wondering if someone out there might have some insight to what's going wrong here...
I have a python script that connects to a USB-to-CAN transceiver/dongle (made by PEAK System) to do some CAN communications. The script works pretty flawlessly. The script accepts command-line arguments and works fine when called from the Windows command-line.
I am trying to integrate this script into a C# Forms project. I have been successful at calling the Python script from the C# app, but things fall apart when it gets to the point at which the Python script tries to use the CAN transceiver.
It feels like the C# app front-end is not allowing the Python script to access the serial port.
Here is the error I get (Python script writing to StandardOut on the Visual Studio output):
line 86, in canSendRec
self.bus.send(canMessage, timeout=0.1)
AttributeError: 'Node' object has no attribute 'bus'
Unable to Connect to USB-CAN Device
Here is the line from canSendRec -- where the exception handler came from (which we wrote):
try:
self.bus = can.interface.Bus('PCAN_USBBUS1',bitrate=1000000)
self.bus.flush_tx_buffer()
except:
print("Unable to Connect to USB-CAN Device")
Here is my C# code calling the Python script:
public string pythonMakeCall(string script, string arg1){
ProcessStartInfo pyProcessStartInfo = new ProcessStartInfo(py_path);
pyProcessStartInfo.FileName = py_path;
pyProcessStartInfo.Arguments = string.Format("{0} {1}", script, arg1);
pyProcessStartInfo.CreateNoWindow = true;
pyProcessStartInfo.UseShellExecute = false;
pyProcessStartInfo.RedirectStandardOutput = true;
pyProcessStartInfo.RedirectStandardError = true;
Process pyProcess = new Process();
pyProcess.StartInfo = pyProcessStartInfo;
pyProcess.Start();
retString = pyProcess.StandardOutput.ReadToEnd();
pyProcess.WaitForExit();
return retString;}
Like I said, it feels like there is something going on with the C# app not allowing Python to access the USB ports, but I'm not sure where to begin with debugging that hunch, since, Disclaimer: this is my first time dabbling in C#/Visual Studio and I'm no Python expert either.

Communication between Python and C#

I have a Python backend running machine learning algorithms. I want to use the same backend for both an Excel plugin (C#) and a website. I want both interfaces to send my training data (thousands of lines of numbers in arrays) to the same Python application and retrieve the results in the form of another array up to a few thousand lines.
The website would fetch data from a SQL database and send that data to Python, while the Excel plugin would take the data that is in the current worksheet and send that data to Python. I need to be able to create numpy arrays in Python before continuing to process the data. Note that the website would be running on the same machine where the Python application resides. I still haven't decided what I will use to code the website, but I was leaning towards Node.js.
I have done some research and found a few options:
1- Named pipes
2- Sockets
3- RPC server such as gRPC or XML-RPC.
4- Writing the data to a file and reading it back in Python
5- Web Service
Note: I would need the Python "server" to be stateful and keep the session running between calls. So I would need to have a kind of daemon running, waiting for calls.
Which one would you experts recommend and why? I need flexibility to handle several parameters and also large arrays of numbers. Using IronPython is not an option because I am running Keras on Python, which apparently does not support IronPython.

I had the same problem recently.
I used a named pipe to transport data from python to my c# server, hope it helps you.
Python:
import win32pipe, win32file
class PipeServer():
def __init__(self, pipeName):
self.pipe = win32pipe.CreateNamedPipe(
r'\\.\pipe\\'+pipeName,
win32pipe.PIPE_ACCESS_OUTBOUND,
win32pipe.PIPE_TYPE_MESSAGE | win32pipe.PIPE_READMODE_MESSAGE | win32pipe.PIPE_WAIT,
1, 65536, 65536,
0,
None)
#Carefull, this blocks until a connection is established
def connect(self):
win32pipe.ConnectNamedPipe(self.pipe, None)
#Message without tailing '\n'
def write(self, message):
win32file.WriteFile(self.pipe, message.encode()+b'\n')
def close(self):
win32file.CloseHandle(self.pipe)
t = PipeServer("CSServer")
t.connect()
t.write("Hello from Python :)")
t.write("Closing now...")
t.close()
For this code to work you need to install pywin32 (best choice is from binarys): https://github.com/mhammond/pywin32
C#-Server:
using System;
using System.IO;
using System.IO.Pipes;
class PipeClient
{
static void Main(string[] args)
{
using (NamedPipeClientStream pipeClient =
new NamedPipeClientStream(".", "CSServer", PipeDirection.In))
{
// Connect to the pipe or wait until the pipe is available.
Console.Write("Attempting to connect to pipe...");
pipeClient.Connect();
Console.WriteLine("Connected to pipe.");
Console.WriteLine("There are currently {0} pipe server instances open.",
pipeClient.NumberOfServerInstances);
using (StreamReader sr = new StreamReader(pipeClient))
{
// Display the read text to the console
string temp;
while ((temp = sr.ReadLine()) != null)
{
Console.WriteLine("Received from server: {0}", temp);
}
}
}
Console.Write("Press Enter to continue...");
Console.ReadLine();
}
}

You can use Python for .NET (Python.NET). It may require some changes to your code, but then it should work very well, once everything is in good shape.
Python.NET allows two-way communication between CPython and CLR.

Let me give you a neat and quick recipe, in the form of example code.
There are basically two ways to tie python in the backend of C# (or a C# winform app or gui or something similar).
Method1: Iron Python. In this method you install a .net package in your visual studio called IronPython. I would not prefer this, because assuming your machine learning model uses keras or a lot of other libraries. It would be another quest to get you installations ready and working in IronPython. And most importantly, it is not as good as your common virtual env or conda environment.
Method2: (The Good Method): Create a Custom Process in your C# that takes arguments from your GUI, knows the path to your script and your python env. Using all these things, it calls your python code exactly the way you would call it in your terminal and pass arguments to it.
Now the tasty example code (I have used this simple trick and it always helps make my black screen python stuff look good with the cover of C# apps).
Python Part
import sys
a = sys.argv[1]
b = sys.argv[2]
print("The Sum = ", float(a)+float(b))
The C# Part
So here is the python process/function that you need to call on the click event of your sum button in the application
static void PythonProcess()
{
//1) Create Process Info
var psi = new ProcessStartInfo();
//Conda Env Path
psi.FileName = #"C:\Users\jd\.conda\pkgs\py\python.exe";
//2) Provide Script and the Arguments
var script = #"C:\Users\jd\Desktop\script.py";
var a = "15";
var b = "18";
psi.Arguments = $"\"{script}\" \"{a}\" \"{b}\"";
//3) Process Configuration
psi.UseShellExecute = false;
psi.CreateNoWindow = true;
psi.RedirectStandardOutput = true;
psi.RedirectStandardError = true;
//4) Execute Process and get Output.
var errors = "";
var results = "";
using(var process = Process.Start(psi))
{
errors = process.StandardError.ReadToEnd();
results = process.StandardOutput.ReadToEnd();
}
//5) Display Output
Console.WriteLine("ERRORS: ");
Console.WriteLine(errors);
Console.WriteLine();
Console.WriteLine("RESULTS: ");
Console.WriteLine(results);
}

Calling Python from C# is easily possible via Pyrolite where your Python code is running as a Pyro4 server. It should be fast enough to handle "large arrays of numbers" however you didn't specify any performance constraints.

I had the same issue and seem to end up with named pipes. Here is a nice example of how to set it up to talk C# => Python, assuming C# is the server.
It can use the same way to talk back or just Python.net to call directly through CLR as shown here. I use the latter.

Compile java code in run time using C#

I know it is possible to compile c# code at runtime using C#, with CSharpCodeProvider and CodeDom.
Is it possible to compile also Java? If it isn't, is there any alternative?
I want my application to be able to compile C# and Java code.

You will need to have JDK (or equivalent installed) on the system performing the compilation
You will need to invoke the Java compiler
You will presumably need to use the compiled code using the Java Runtime (or equivalent)
Method A:
The easiest way to both use the compiler and use the compiled code would be trough Process.Start as mentioned in Kieren's answer. This is easy provided you have the necessary components.
//add this either atusing System.Diagnostics;
static void CompileJava(string javacPathName, string javaFilePathName, string commandLineOptions = "")
{
var startInfo = new ProcessStartInfo();
startInfo.CreateNoWindow = true;
startInfo.UseShellExecute = false;
startInfo.FileName = javacPathName;
startInfo.WindowStyle = ProcessWindowStyle.Hidden;
startInfo.Arguments = commandLineOptions + " " + javaFilePathName;
try {
using (var javacProcess = Process.Start(startInfo))
{
javacProcess.WaitForExit();
}
}
catch(Exception e)
{
// do something e.g. throw a more appropriate exception
}
}
Method B:
If you require deeper integration, you could try the native linking method (i.e. use .NET and the Java native interfaces for the two to interoperate without invoking external processes). The pre-requisites are the same as with Method A. The investment required is much higher and you should only consider this if there are specific performance or other constraints Method A cannot meet.
You can find some information by following the links below:
From the C# side: http://blogs.msdn.com/b/texblog/archive/2007/04/05/linking-native-c-into-c-applications.aspx
From the Java side: http://docs.oracle.com/javase/6/docs/technotes/guides/jni/

To expand on what driis and ppeterka are saying, there's no built-in way since Java isn't a .NET language (J# was close but doesn't exist anymore). You'd need to use Process.Start to fire up the java compiler.

Your specific situation may require it, but Java isn't built for this, it is built to be compiled at compile time, not at runtime. While Miltiadis Kokkonidis's answer works, it's probably best to either use a different language that fits your problem better, or use the languages strengths for the situation at hand, rather then try to squeeze it to fit places it doesn't want to go.

Changing environment variables of the calling process

This one seems trivial but the answer has eluded me for a few days now.
I have a Windows batch file, that calls a C# program to do an extra verification that cannot be done in a batch file. After the verification is complete I need to return a status and a string back to the calling shell.
Now the return value is trivial and my C# console app simply sets a return value (exit code if you will). And I thought the string will also be a piece of cake. I attempted to define a new shell variable using the:
Environment.SetEnvironmentVariable("ERR", "Some text");
This call should (and does) define a shell variable within the current process - that is the very C# process that created the variable. The value is lost as soon as the C# app terminates and the shell that created the C# app knows nothing about the variable. So... A call with no particular use... At all... Unless perhaps if I created a child process from the C3 app, perhaps it would inherit my variables.
The EnvironmentVariableTarget.Machine and EnvironmentVariableTarget.User targets for the SetEnvironmentVariable call don't solve the problem either, as only a newly created process will get these new values from the registry.
So the only working solution I can think of is:
write to stdout
write to a file
encode extra meaning into the return value
The first two are a bit ugly and the last one has its limitations and problems.
Any other ideas (how to set a shell variable in the parent process)? Maybe such shell variable modifications are a security concern (think PATH)...
Thank-you for your time.

I had the same problem as Ryan and the only thing that came to my mind as a work-around was to write a batch in error out to set the variable and to call it from the batch.
ConsoleApplication1.exe:
'put some sensible code here
'put result in variable myResult
Dim myResult As String = Guid.NewGuid().ToString("D").ToUpperInvariant()
Console.WriteLine("Normal output from the consonle app")
Console.Error.WriteLine("#ECHO OFF")
Console.Error.WriteLine("SET zzzResult={0}", myResult)
Test.cmd (the calling batch):
#ECHO OFF
:Jump to folder of batch file
PUSHD %~d0%~p0
:Define a temp file
SET zzzTempFile=%TEMP%\TMP%Random%.CMD
:Call .NET console app
ConsoleApplication1.exe 2>%zzzTempFile%
:Call the generated batch file
CALL %zzzTempFile%
:Clean up temp file
DEL %zzzTempFile%
:Clean up variable
SET zzzTempFile=
:Do something with the result
ECHO Yeah, we finally got it!
ECHO:
ECHO The value is "%zzzResult%".
ECHO:
:Clean up result variable
SET zzzResult=
:Go back to original folder
POPD
That should do the trick. And yes, I do know this is an old post and Ryan is solving other issues by now, but there might be still somebody else out there having the same problem...

What you are asking is to be able to arbitrarily write to the memory space of a running process. For good reason, this is not possible without SeDebugPrivilege.
Any of the three solutions you list will work. Stdout is the standard way to communicate with a batch script.
By the way, you're writing a Windows batch file. I'm pretty sure the ship has already sailed on "a bit ugly".

If you want to put a value of some output into a variable in the batch you can use the following construct:
FOR /F "usebackq tokens=4 delims=\[\] " %i IN (`ver`) DO set VERSION=%i
ECHO %VERSION%
Output on my OS:
6.1.7601
'usebackq' means we are using back quotes which gives the ability to use a fileset in the command quoted with double quotes. You may not need this. 'tokens' means the index in the resulting string array to select (it can be a range M-N). If you need to skip lines use 'skip=X'). 'delims' are the string separators to use (like string-Split() in .Net).
You will put your console app instead of 'ver' and adapt the delimiters and tokens to match your specific output. If you have more variables to fill you will need to make the if a bit more complex but that should make a good start.

My BAT is a bit rusty, but I think it's possible to retrieve the 'exit' code from processes you've run externally, perhaps via %ERRORLEVEL%. If that's the case, make sure to exit your program via
Environment.Exit(123); // where 123 = error code
You can't add any messages, so you'll have to do that in the .bat file.
If this isn't the case, stdout is probably the best way.

After stumbling on this myself as well recently, I came up with this approach. What I did is run the bat file using the Process class, i.e.
// Spawn your process as you normally would... but also have it dump the environment varaibles
Process process = new Process();
process.StartInfo.FileName = mybatfile.bat;
process.StartInfo.Arguments = #"&&set>>envirodump.txt";
process.StartInfo.UseShellExecute = false;
process.StartInfo.RedirectStandardOutput = true;
process.StartInfo.RedirectStandardError = false;
process.Start();
string output = process.StandardOutput.ReadToEnd();
process.WaitForExit();
// Read the environment variable lines into a string array
string[] envirolines = File.ReadAllLines("envirodump.txt");
File.Delete("envirodump.txt");
// Now simply set the environment variables in the parent process
foreach(string line in a)
{
string var = line.Split('=')[0];
string val = line.Split('=')[1];
Environment.SetEnvironmentVariable(var, val);
}
This seems to have worked for me. It's not the cleanest approach, but will work in a bind. :)