Develop Reference

C# - Consume pipeline like 'more' does (windows)

This seems trivial, but google only gives me powershell-related stuff and how to implement pipelines within programs.
I'm trying to consume input arguments from the (standard command processor cmd.exe for Windows 7, not powershell) command line pipeline on windows like 'more' does.
I'm trying to get this to work with two self-created .exe files:
randgen (creates a random number and writes it to the console)
wordwrite (the c# program that I want to consume this random number from the pipeline)
And using two common windows executables to test and model the behaviour:
echo (writes input to console)
more (displays output one screen at a time, can consume input from the pipeline)
Here's the current content of my Main function in wordwrite:
static void Main(string[] args)
{
string ouname = args[0];
DocWrite(ouname);
}
and the behaviour trying to naively pipe output from randgen into wordwrite:
D:\code>randgen | wordwrite
Unhandled Exception: System.IndexOutOfRangeException: Index was outside the bounds of the array.
at ADDoc.ADOUDoc.Main(String[] args)
(plus a popup telling me wordwrite.exe has stopped working)
Which is the behaviour of trying to consume a null array of arguments as input:
D:\code>wordwrite
Unhandled Exception: System.IndexOutOfRangeException: Index was outside the bounds of the array.
at ADDoc.ADOUDoc.Main(String[] args)
So the output of randgen isn't being read from the pipeline correctly as an argument to the wordwrite executable's args array (thanks #Peter Duniho for explaining the difference between stdin and how argument assignation works). Contrast that with the behaviour of the 'more' executable:
D:\code>randgen | more
56929227
That said, this 'pipeline-awareness' seems to be inherent to the 'more' executable itself, as another common windows executable, echo.exe, is just as pipeline-blind as wordwrite.exe:
D:\code>randgen | echo
ECHO is on.
D:\code>echo blah
blah
D:\code>echo
ECHO is on.
Here we can see that the attempt to use the pipeline is effectively the same as providing no arguments for echo.exe (contrasted in the middle with behaviour when an argument is provided).
Which suggests there is something I can add the the body of my Main function in wordwrite.exe to enable pipeline-consuming behaviour like the 'more' executable does. Does anyone know how this is done?

You can read data 'from the pipeline' (effectively, the stdin stream*) by using the Console.ReadLine() method, like so:
static void Main()
{
string ouname = Console.ReadLine();
DocWrite(ouname);
}
*The pipeline is a tool to direct the output of one process (stdout) to the input (stdin) of another one. So when you think of 'consuming input from the pipeline', think 'assign stdin to a variable that you then do stuff with'.
Reference:
C# Console receive input with pipe

Communication between Python and C#

I have a Python backend running machine learning algorithms. I want to use the same backend for both an Excel plugin (C#) and a website. I want both interfaces to send my training data (thousands of lines of numbers in arrays) to the same Python application and retrieve the results in the form of another array up to a few thousand lines.
The website would fetch data from a SQL database and send that data to Python, while the Excel plugin would take the data that is in the current worksheet and send that data to Python. I need to be able to create numpy arrays in Python before continuing to process the data. Note that the website would be running on the same machine where the Python application resides. I still haven't decided what I will use to code the website, but I was leaning towards Node.js.
I have done some research and found a few options:
1- Named pipes
2- Sockets
3- RPC server such as gRPC or XML-RPC.
4- Writing the data to a file and reading it back in Python
5- Web Service
Note: I would need the Python "server" to be stateful and keep the session running between calls. So I would need to have a kind of daemon running, waiting for calls.
Which one would you experts recommend and why? I need flexibility to handle several parameters and also large arrays of numbers. Using IronPython is not an option because I am running Keras on Python, which apparently does not support IronPython.

I had the same problem recently.
I used a named pipe to transport data from python to my c# server, hope it helps you.
Python:
import win32pipe, win32file
class PipeServer():
def __init__(self, pipeName):
self.pipe = win32pipe.CreateNamedPipe(
r'\\.\pipe\\'+pipeName,
win32pipe.PIPE_ACCESS_OUTBOUND,
win32pipe.PIPE_TYPE_MESSAGE | win32pipe.PIPE_READMODE_MESSAGE | win32pipe.PIPE_WAIT,
1, 65536, 65536,
0,
None)
#Carefull, this blocks until a connection is established
def connect(self):
win32pipe.ConnectNamedPipe(self.pipe, None)
#Message without tailing '\n'
def write(self, message):
win32file.WriteFile(self.pipe, message.encode()+b'\n')
def close(self):
win32file.CloseHandle(self.pipe)
t = PipeServer("CSServer")
t.connect()
t.write("Hello from Python :)")
t.write("Closing now...")
t.close()
For this code to work you need to install pywin32 (best choice is from binarys): https://github.com/mhammond/pywin32
C#-Server:
using System;
using System.IO;
using System.IO.Pipes;
class PipeClient
{
static void Main(string[] args)
{
using (NamedPipeClientStream pipeClient =
new NamedPipeClientStream(".", "CSServer", PipeDirection.In))
{
// Connect to the pipe or wait until the pipe is available.
Console.Write("Attempting to connect to pipe...");
pipeClient.Connect();
Console.WriteLine("Connected to pipe.");
Console.WriteLine("There are currently {0} pipe server instances open.",
pipeClient.NumberOfServerInstances);
using (StreamReader sr = new StreamReader(pipeClient))
{
// Display the read text to the console
string temp;
while ((temp = sr.ReadLine()) != null)
{
Console.WriteLine("Received from server: {0}", temp);
}
}
}
Console.Write("Press Enter to continue...");
Console.ReadLine();
}
}

You can use Python for .NET (Python.NET). It may require some changes to your code, but then it should work very well, once everything is in good shape.
Python.NET allows two-way communication between CPython and CLR.

Let me give you a neat and quick recipe, in the form of example code.
There are basically two ways to tie python in the backend of C# (or a C# winform app or gui or something similar).
Method1: Iron Python. In this method you install a .net package in your visual studio called IronPython. I would not prefer this, because assuming your machine learning model uses keras or a lot of other libraries. It would be another quest to get you installations ready and working in IronPython. And most importantly, it is not as good as your common virtual env or conda environment.
Method2: (The Good Method): Create a Custom Process in your C# that takes arguments from your GUI, knows the path to your script and your python env. Using all these things, it calls your python code exactly the way you would call it in your terminal and pass arguments to it.
Now the tasty example code (I have used this simple trick and it always helps make my black screen python stuff look good with the cover of C# apps).
Python Part
import sys
a = sys.argv[1]
b = sys.argv[2]
print("The Sum = ", float(a)+float(b))
The C# Part
So here is the python process/function that you need to call on the click event of your sum button in the application
static void PythonProcess()
{
//1) Create Process Info
var psi = new ProcessStartInfo();
//Conda Env Path
psi.FileName = #"C:\Users\jd\.conda\pkgs\py\python.exe";
//2) Provide Script and the Arguments
var script = #"C:\Users\jd\Desktop\script.py";
var a = "15";
var b = "18";
psi.Arguments = $"\"{script}\" \"{a}\" \"{b}\"";
//3) Process Configuration
psi.UseShellExecute = false;
psi.CreateNoWindow = true;
psi.RedirectStandardOutput = true;
psi.RedirectStandardError = true;
//4) Execute Process and get Output.
var errors = "";
var results = "";
using(var process = Process.Start(psi))
{
errors = process.StandardError.ReadToEnd();
results = process.StandardOutput.ReadToEnd();
}
//5) Display Output
Console.WriteLine("ERRORS: ");
Console.WriteLine(errors);
Console.WriteLine();
Console.WriteLine("RESULTS: ");
Console.WriteLine(results);
}

Calling Python from C# is easily possible via Pyrolite where your Python code is running as a Pyro4 server. It should be fast enough to handle "large arrays of numbers" however you didn't specify any performance constraints.

I had the same issue and seem to end up with named pipes. Here is a nice example of how to set it up to talk C# => Python, assuming C# is the server.
It can use the same way to talk back or just Python.net to call directly through CLR as shown here. I use the latter.

Reading CI Frequency In C# With NI USB-6363

Working LabVIEW Code
Attached above is LabVIEW code that I have successfully used in the past to read frequency data from a device. I also usually use the Start Task VI between my property node and while loop.
I am trying to code this in C#. So far I have successfully been able to code analog Output's and analog Input's on my device, USB-6363, (so I know I am able to write and read data from the device successfully with C#).
I have also used multimeters (Grainger link at bottom of post) to read frequency data (Orange Hz mode that the device is set to in the picture).
However, my C# code seems to be having issues reading the frequency data. My C# code is attached. When I try running this program I get the following error. This is the same error that I get when using the example program called 'MeasDigFreqBuffCont_ExtClk_ArmStart.2013'. The code I show is just creating the task, I do call the code later in my program in a different section and that is how I am getting the error.
------------------------------------------------- Begin Error Code -------------------------------------------------
{Error=-200077 Message="Requested value is not a supported value for
this property. The property value may be invalid because it conflicts
with another property.\n\nProperty:
NationalInstruments.DAQmx.CIChannel.FrequencyDivisor\nRequested Value:
1\nPossible Values: 4 to 4294967295\nChannel Name: Digital
Frequency\n\nTask Name: _unnamedTask<0>\n\nStatus Code: -200077"}
------------------------------------------------- End Error Code --------------------------------------------------
In the example program it asks for a sample clock source (A PFI channel from the device). However in the LabVIEW code it does not ask for this. Is this example maybe more in detail than what I am trying to do?
Task frequencyInput = new Task();
frequencyInput.CIChannels.CreateFrequencyChannel(
"Dev1/ctr0",
"Digital Frequency",
200,
15000,
CIFrequencyStartingEdge.Rising,
CIFrequencyMeasurementMethod.DynamicAveraging,
0.001,
1,
CIFrequencyUnits.Hertz
);
frequencyInput.CIChannels["Digital Frequency"].FrequencyTerminal = "/Dev1/PFI0";
CounterSingleChannelReader counterFreq = new CounterSingleChannelReader(frequencyInput.Stream);
double counterFreqData = counterFreq.ReadSingleSampleDouble();
txtPFI0.Text = Convert.ToString(counterFreqData);
FLUKE (R) Fluke-115 Compact - Basic Features Digital Multimeter, 14° to 122°F Temp. Range

Formatting the error message:
Requested value is not a supported value for this property. The property value may be invalid because it conflicts with another property.
Property: NationalInstruments.DAQmx.CIChannel.FrequencyDivisor
Requested Value: 1
Possible Values: 4 to 4294967295
Task Name: _unnamedTask<0>
Status Code: -200077
According to the documentation, you are asking the device to use an invalid divisor. Change your 1 to a 4:
frequencyInput.CIChannels.CreateFrequencyChannel(
"Dev1/ctr0",
"Digital Frequency",
200,
15000,
CIFrequencyStartingEdge.Rising,
CIFrequencyMeasurementMethod.DynamicAveraging,
0.001,
/* here */ 4,
CIFrequencyUnits.Hertz
);
NI installs C# examples for DAQmx, and it includes one for measuring frequency:
C:\Users\Public\Documents\National Instruments\NI-DAQ\Examples\DotNET4.0\Counter\Measure Digital Frequency\MeasDigFrequency_LowFreq1Ctr\CS

Get time of speech in a video file (in code)

I'm looking for a way (in java, c#..) to get the time when people speak in a video file (even in a movie).
I don't need to know the accurate words, just the time.
Output example:
00:03 - 01:03 (someone spoke for a minute),
03:00 - 06:12 (someone spoke again),
.
.
.
I have found Sphinx (written in java): http://cmusphinx.sourceforge.net/
but couldn't get it to recognize properly.
Any ideas?
Thanks.
EDIT:
This is what I've tried in sphinx (very basic):
StreamSpeechRecognizer recognizer = new StreamSpeechRecognizer(configuration);
recognizer.startRecognition(somefile);
SpeechResult result;
while ((result = recognizer.getResult()) != null) {
System.out.println(result);
}
recognizer.stopRecognition();
There were only 3 results (there should be allot more).
EDIT2:
well, I tried this on a song in my computer:
https://www.assembla.com/code/sonido/subversion/nodes/12/sphinx4/src/sphinx4/edu/cmu/sphinx/tools/endpoint/Segmenter.java
This is the output:
DataStartSignal: creation time: 1399716763914
SpeechStartSignal
DoubleData: 44100Hz, first sample #: 8820, collect time: 200
DoubleData: 44100Hz, first sample #: 9261, collect time: 210
.....
DoubleData: 44100Hz, first sample #: 1745037, collect time: 39570
SpeechEndSignal
SpeechStartSignal
DoubleData: 44100Hz, first sample #: 1894536, collect time: 42960
......
Two Problems:
1. My goal is to be able to do it on movies. It works on audio files (.wav)
2. I'm not sure it works well. As you can see, the output says the speech started after 200 milliseconds, where actually it started after 3 seconds at least (the song is 'Bee Gees - How Deep Is Your Love').

I have found Sphinx (written in java): http://cmusphinx.sourceforge.net/ but couldn't get it to recognize properly.
Like you said, you do not need to recognize. To get only voice activity detection in Java with times see the segmenter class edu.cmu.sphinx.tools.endpoint.Segmenter

Changing environment variables of the calling process

This one seems trivial but the answer has eluded me for a few days now.
I have a Windows batch file, that calls a C# program to do an extra verification that cannot be done in a batch file. After the verification is complete I need to return a status and a string back to the calling shell.
Now the return value is trivial and my C# console app simply sets a return value (exit code if you will). And I thought the string will also be a piece of cake. I attempted to define a new shell variable using the:
Environment.SetEnvironmentVariable("ERR", "Some text");
This call should (and does) define a shell variable within the current process - that is the very C# process that created the variable. The value is lost as soon as the C# app terminates and the shell that created the C# app knows nothing about the variable. So... A call with no particular use... At all... Unless perhaps if I created a child process from the C3 app, perhaps it would inherit my variables.
The EnvironmentVariableTarget.Machine and EnvironmentVariableTarget.User targets for the SetEnvironmentVariable call don't solve the problem either, as only a newly created process will get these new values from the registry.
So the only working solution I can think of is:
write to stdout
write to a file
encode extra meaning into the return value
The first two are a bit ugly and the last one has its limitations and problems.
Any other ideas (how to set a shell variable in the parent process)? Maybe such shell variable modifications are a security concern (think PATH)...
Thank-you for your time.

I had the same problem as Ryan and the only thing that came to my mind as a work-around was to write a batch in error out to set the variable and to call it from the batch.
ConsoleApplication1.exe:
'put some sensible code here
'put result in variable myResult
Dim myResult As String = Guid.NewGuid().ToString("D").ToUpperInvariant()
Console.WriteLine("Normal output from the consonle app")
Console.Error.WriteLine("#ECHO OFF")
Console.Error.WriteLine("SET zzzResult={0}", myResult)
Test.cmd (the calling batch):
#ECHO OFF
:Jump to folder of batch file
PUSHD %~d0%~p0
:Define a temp file
SET zzzTempFile=%TEMP%\TMP%Random%.CMD
:Call .NET console app
ConsoleApplication1.exe 2>%zzzTempFile%
:Call the generated batch file
CALL %zzzTempFile%
:Clean up temp file
DEL %zzzTempFile%
:Clean up variable
SET zzzTempFile=
:Do something with the result
ECHO Yeah, we finally got it!
ECHO:
ECHO The value is "%zzzResult%".
ECHO:
:Clean up result variable
SET zzzResult=
:Go back to original folder
POPD
That should do the trick. And yes, I do know this is an old post and Ryan is solving other issues by now, but there might be still somebody else out there having the same problem...

What you are asking is to be able to arbitrarily write to the memory space of a running process. For good reason, this is not possible without SeDebugPrivilege.
Any of the three solutions you list will work. Stdout is the standard way to communicate with a batch script.
By the way, you're writing a Windows batch file. I'm pretty sure the ship has already sailed on "a bit ugly".

If you want to put a value of some output into a variable in the batch you can use the following construct:
FOR /F "usebackq tokens=4 delims=\[\] " %i IN (`ver`) DO set VERSION=%i
ECHO %VERSION%
Output on my OS:
6.1.7601
'usebackq' means we are using back quotes which gives the ability to use a fileset in the command quoted with double quotes. You may not need this. 'tokens' means the index in the resulting string array to select (it can be a range M-N). If you need to skip lines use 'skip=X'). 'delims' are the string separators to use (like string-Split() in .Net).
You will put your console app instead of 'ver' and adapt the delimiters and tokens to match your specific output. If you have more variables to fill you will need to make the if a bit more complex but that should make a good start.

My BAT is a bit rusty, but I think it's possible to retrieve the 'exit' code from processes you've run externally, perhaps via %ERRORLEVEL%. If that's the case, make sure to exit your program via
Environment.Exit(123); // where 123 = error code
You can't add any messages, so you'll have to do that in the .bat file.
If this isn't the case, stdout is probably the best way.

After stumbling on this myself as well recently, I came up with this approach. What I did is run the bat file using the Process class, i.e.
// Spawn your process as you normally would... but also have it dump the environment varaibles
Process process = new Process();
process.StartInfo.FileName = mybatfile.bat;
process.StartInfo.Arguments = #"&&set>>envirodump.txt";
process.StartInfo.UseShellExecute = false;
process.StartInfo.RedirectStandardOutput = true;
process.StartInfo.RedirectStandardError = false;
process.Start();
string output = process.StandardOutput.ReadToEnd();
process.WaitForExit();
// Read the environment variable lines into a string array
string[] envirolines = File.ReadAllLines("envirodump.txt");
File.Delete("envirodump.txt");
// Now simply set the environment variables in the parent process
foreach(string line in a)
{
string var = line.Split('=')[0];
string val = line.Split('=')[1];
Environment.SetEnvironmentVariable(var, val);
}
This seems to have worked for me. It's not the cleanest approach, but will work in a bind. :)