i want to create some service that will run as simple process and will give some other application the possibility to send him xml stream.
What i mean is to create simple process ( exe ) with Infinite loop - and any application will be able to send XML ( file / stream ) to this process => and this process will send the xml to some socket.
Is it possible to do it without pipe ?
I want to do something like COM - that can 'catch' instance of working process.
sure.
you can use Named Pipe classes in c# :
Server :
using (var s = new NamedPipeServerStream ("myPipe"))
{
s.WaitForConnection();
s.WriteByte (100);
Console.WriteLine (s.ReadByte());
}
client code:
using (var s = new NamedPipeClientStream ("myPipe"))
{
s.Connect();
Console.WriteLine (s.ReadByte());
s.WriteByte (200);
}
edit
you can do it by file. + systemfileWatcher Class
put a file in a folder.
the other process will audit this folder.
and now you can transfer info.
edit2
you can use memoryMappedFile
and open a view in each process to see the same mempry region - and transfer data.
I think its the best.
Process A :
static void Main(string[] args)
{
using (MemoryMappedFile mmf = MemoryMappedFile.CreateNew("testmap", 4000))
{
bool mutexCreated;
Mutex mutex = new Mutex(true, "testmapmutex", out mutexCreated);
using (MemoryMappedViewStream stream = mmf.CreateViewStream())
{
BinaryWriter writer = new BinaryWriter(stream);
string st = "Hellow";
int stringSize = Encoding.UTF8.GetByteCount(st); //6
writer.Write(st);
writer.Write(123); //6+4 bytes = 10 bytes
}
mutex.ReleaseMutex();
Console.WriteLine("Start Process B and press ENTER to continue.");
Console.ReadLine();
mutex.WaitOne();
using (MemoryMappedViewStream stream = mmf.CreateViewStream())
{
BinaryReader reader = new BinaryReader(stream);
Console.WriteLine("Process A says: {0}", reader.ReadString());
Console.WriteLine("Process A says: {0}", reader.ReadInt32());
Console.WriteLine("Process B says: {0}", reader.ReadInt32());
}
mutex.ReleaseMutex();
}
}
Process B writes to its region
static void Main(string[] args)
{
try
{
using (MemoryMappedFile mmf = MemoryMappedFile.OpenExisting("testmap"))
{
Mutex mutex = Mutex.OpenExisting("testmapmutex");
mutex.WaitOne();
using (MemoryMappedViewStream stream = mmf.CreateViewStream(11, 0)) // From the 11 byte....
{
BinaryWriter writer = new BinaryWriter(stream, Encoding.UTF8);
writer.Write(2);
}
mutex.ReleaseMutex();
}
}
catch (FileNotFoundException)
{
Console.WriteLine("Memory-mapped file does not exist. Run Process A first.");
}
}
Just use C# Sockets that listen for connections from the other process and write a custom XML file receiver.
Yes, of course you can use a TCP socket connection .If you want to avoid network connection as enlightened in a comment you can use a shared memory approach, for example with Memory-Mapped Files .
What you are looking for is some form of IPC (Inter-process communuication). There's a huge number of possibilities:
Regular file. Windows provides location specifically for temp files (%TEMP%)
For small data, you could use registry, although in most cases it's not a proper use
Memory-mapped file, it's similar to file but in RAM
As Royi properly mentioned, NamedPipeStream is a way to go if you decide to give pipes a try
You could create a WCF endpoint. It sounds like a drag, but Visual Studio will create you all the scaffolding, so it's not such an issue in the end
Window messages could be used if you are developing forms application, and sometimes even if not
You mentioned that the data is XML, so this methodology is not for you, but I'll mention it anyway: you could use named kernel objects, such as: mutexes, events, semaphores to pass signals from one program to another.
Related
I'm using SSH.NET to connect to my Raspberry Pi from a Console Application in C#.
I want to send text from my very own stream, writing to it through a StreamWriter.
The problem is that it does nothing. It's like the WriteLine("ls") doesn't produce any effect.
This is the code:
using System;
using System.IO;
using Renci.SshNet;
namespace SSHTest
{
class Program
{
static void Main(string[] args)
{
var ssh = new SshClient("raspberrypi", 22, "pi", "raspberry");
ssh.Connect();
var input = new MemoryStream();
var streamWriter = new StreamWriter(input) { AutoFlush = true };
var stdout = Console.OpenStandardOutput();
var shell = ssh.CreateShell(input, stdout, new MemoryStream());
shell.Start();
streamWriter.WriteLine("ls");
while (true)
{
}
}
}
}
What's the problem?
MemoryStream is not a good class for implementing an input stream.
When you write to MemoryStream, as with most stream implementations, its pointer is moved at the end of the written data.
So when SSH.NET channel tries to read data, it has nothing to read.
You can move the pointer back:
streamWriter.WriteLine("ls");
input.Position = 0;
But the right approach is to use PipeStream from SSH.NET, which has separate read and write pointers (just as a *nix pipe):
var input = new PipeStream();
Another option is to use SshClient.CreateShellStream (ShellStream class), which is designed for task like this. It gives you one Stream interface, that you can both write and read.
See also Is it possible to execute multiple SSH commands from a single login session with SSH.NET?
Though SshClient.CreateShell (SSH "shell" channel) is not the right method for automating command execution. Use "exec" channel. For simple cases, use SshClient.RunCommand. If you want to read a command output continuously, use SshClient.CreateCommand to retrieve the command output stream:
var command = ssh.CreateCommand("ls");
var asyncExecute = command.BeginExecute();
command.OutputStream.CopyTo(Console.OpenStandardOutput());
command.EndExecute(asyncExecute);
I have a class library that gets called from a windows service, the class library can be called many times at the same time.
I have an issue where i have to read file contents in my class, so i get the error that the file is being used by another process if the class is getting called by many processes.
This is how i read the file content:
File.ReadAllBytes("path");
What is the best solution in this case ?
Thank you
The following code demonstrates to access a file by setting its share permissions. The first using block creates and writes file, second and third using blocks access and read the file.
var fileName = "test.txt";
using (var fsWrite = new FileStream(fileName, FileMode.OpenOrCreate, FileAccess.Write, FileShare.ReadWrite))
{
var content = Encoding.UTF8.GetBytes("test");
fsWrite.Write(content, 0, content.Length);
fsWrite.Flush();
using (var fsRead_1 = new FileStream(fileName, FileMode.Open, FileAccess.Read, FileShare.ReadWrite))
{
var bufRead_1 = new byte[fsRead_1.Length];
fsRead_1.Read(bufRead_1, 0, bufRead_1.Length);
Console.WriteLine("fsRead_1:" + Encoding.UTF8.GetString(bufRead_1));
using (var fsRead_2 = new FileStream(fileName, FileMode.Open, FileAccess.Read, FileShare.ReadWrite))
{
var bufRead_2 = new byte[fsRead_2.Length];
fsRead_2.Read(bufRead_2, 0, bufRead_2.Length);
Console.WriteLine("fsRead_2:" + Encoding.UTF8.GetString(bufRead_2));
}
}
}
You need to synchronize the access to the whole file using standard thread synchronization approaches.
The simplest one is Monitor using lock statement:
public class A
{
private static readonly object _sync = new object();
public void DoStuff()
{
// All threads trying to enter this critical section will
// wait until the first to enter exits it
lock(_sync)
{
byte[] buffer = File.ReadAllBytes(#"C:\file.jpg");
}
}
}
Note
Firstly, I was understanding OP was accessing the file from different processes, but when I double-checked the statement:
I have a class library that gets called from a windows service, the
class library can be called many times at the same time.
...I realized OP is calling a method which reads all bytes from some file within the same Windows service instance.
Use Mutex for syncing among different processes. File.ReadAllBytes uses a FileAccess.Read and FileShare.Read when reads the file, so normally you don't need to use any locks here. So you get this exception because the file is being written somewhere (or at least is locked for writing).
Solution 1 - if you are the one who writes this file
private static Mutex mutex;
public void WriteFile(string path)
{
Mutex mutex = GetOrCreateMutex();
try
{
mutex.WaitOne();
// TODO: ... write file
}
finally
{
mutex.ReleaseMutex();
}
}
public byte[] ReadFile(string path)
{
// Note: If you just read the file, this lock is completely unnecessary
// because ReadAllFile uses Read access. This just protects the file
// being read and written at the same time
Mutex mutex = GetOrCreateMutex();
try
{
mutex.WaitOne();
return File.ReadAllBytes(path);
}
finally
{
mutex.ReleaseMutex();
}
}
private static Mutex GetOrCreateMutex()
{
try
{
mutex = Mutex.OpenExisting("MyMutex");
}
catch (WaitHandleCannotBeOpenedException)
{
mutex = new Mutex(false, "MyMutex");
}
}
Remark: A ReadWriteLock would be better here because you can read a file safely parallelly when it is not being written; however, there is no built-in inter-process read-write lock in .NET. Here is an example how you can implement one with Mutex and Semaphor types.
Solution 2 - if you just read the file
You must simply being prepared that the file can be locked when it is being written by a 3rd process:
public byte[] TryReadFile(string path, int maxTry)
{
Exception e;
for (int i = 0; i < maxTry; i++)
{
try
{
return File.ReadAllBytes(path);
}
catch (IOException io)
{
e = io;
Thread.Sleep(100);
}
}
throw e; // or just return null
}
Can I get a GZipStream for a file on disk without writing the entire compressed content to temporary storage? I'm currently using a temporary file on disk in order to avoid possible memory exhaustion using MemoryStream on very large files (this is working fine).
public void UploadFile(string filename)
{
using (var temporaryFileStream = File.Open("tempfile.tmp", FileMode.CreateNew, FileAccess.ReadWrite))
{
using (var fileStream = File.OpenRead(filename))
using (var compressedStream = new GZipStream(temporaryFileStream, CompressionMode.Compress, true))
{
fileStream.CopyTo(compressedStream);
}
temporaryFileStream.Position = 0;
Uploader.Upload(temporaryFileStream);
}
}
What I'd like to do is eliminate the temporary storage by creating GZipStream, and have it read from the original file only as the Uploader class requests bytes from it. Is such a thing possible? How might such an implementation be structured?
Note that Upload is a static method with signature static void Upload(Stream stream).
Edit: The full code is here if it's useful. I hope I've included all the relevant context in my sample above however.
Yes, this is possible, but not easily with any of the standard .NET stream classes. When I needed to do something like this, I created a new type of stream.
It's basically a circular buffer that allows one producer (writer) and one consumer (reader). It's pretty easy to use. Let me whip up an example. In the meantime, you can adapt the example in the article.
Later: Here's an example that should come close to what you're asking for.
using (var pcStream = new ProducerConsumerStream(BufferSize))
{
// start upload in a thread
var uploadThread = new Thread(UploadThreadProc(pcStream));
uploadThread.Start();
// Open the input file and attach the gzip stream to the pcStream
using (var inputFile = File.OpenRead("inputFilename"))
{
// create gzip stream
using (var gz = new GZipStream(pcStream, CompressionMode.Compress, true))
{
var bytesRead = 0;
var buff = new byte[65536]; // 64K buffer
while ((bytesRead = inputFile.Read(buff, 0, buff.Length)) != 0)
{
gz.Write(buff, 0, bytesRead);
}
}
}
// The entire file has been compressed and copied to the buffer.
// Mark the stream as "input complete".
pcStream.CompleteAdding();
// wait for the upload thread to complete.
uploadThread.Join();
// It's very important that you don't close the pcStream before
// the uploader is done!
}
The upload thread should be pretty simple:
void UploadThreadProc(object state)
{
var pcStream = (ProducerConsumerStream)state;
Uploader.Upload(pcStream);
}
You could, of course, put the producer on a background thread and have the upload be done on the main thread. Or have them both on background threads. I'm not familiar with the semantics of your uploader, so I'll leave that decision to you.
In C# (.NET 4.0 running under Mono 2.8 on SuSE) I would like to run an external batch command and capture its ouput in binary form. The external tool I use is called 'samtools' (samtools.sourceforge.net) and among other things it can return records from an indexed binary file format called BAM.
I use Process.Start to run the external command, and I know that I can capture its output by redirecting Process.StandardOutput. The problem is, that's a text stream with an encoding, so it doesn't give me access to the raw bytes of the output. The almost-working solution I found is to access the underlying stream.
Here's my code:
Process cmdProcess = new Process();
ProcessStartInfo cmdStartInfo = new ProcessStartInfo();
cmdStartInfo.FileName = "samtools";
cmdStartInfo.RedirectStandardError = true;
cmdStartInfo.RedirectStandardOutput = true;
cmdStartInfo.RedirectStandardInput = false;
cmdStartInfo.UseShellExecute = false;
cmdStartInfo.CreateNoWindow = true;
cmdStartInfo.Arguments = "view -u " + BamFileName + " " + chromosome + ":" + start + "-" + end;
cmdProcess.EnableRaisingEvents = true;
cmdProcess.StartInfo = cmdStartInfo;
cmdProcess.Start();
// Prepare to read each alignment (binary)
var br = new BinaryReader(cmdProcess.StandardOutput.BaseStream);
while (!cmdProcess.StandardOutput.EndOfStream)
{
// Consume the initial, undocumented BAM data
br.ReadBytes(23);
// ... more parsing follows
But when I run this, the first 23bytes that I read are not the first 23 bytes in the ouput, but rather somewhere several hundred or thousand bytes downstream. I assume that StreamReader does some buffering and so the underlying stream is already advanced say 4K into the output. The underlying stream does not support seeking back to the start.
And I'm stuck here. Does anyone have a working solution for running an external command and capturing its stdout in binary form? The ouput may be very large so I would like to stream it.
Any help appreciated.
By the way, my current workaround is to have samtools return the records in text format, then parse those, but this is pretty slow and I'm hoping to speed things up by using the binary format directly.
Using StandardOutput.BaseStream is the correct approach, but you must not use any other property or method of cmdProcess.StandardOutput. For example, accessing cmdProcess.StandardOutput.EndOfStream will cause the StreamReader for StandardOutput to read part of the stream, removing the data you want to access.
Instead, simply read and parse the data from br (assuming you know how to parse the data, and won't read past the end of stream, or are willing to catch an EndOfStreamException). Alternatively, if you don't know how big the data is, use Stream.CopyTo to copy the entire standard output stream to a new file or memory stream.
Since you explicitly specified running on Suse linux and mono, you can work around the problem by using native unix calls to create the redirection and read from the stream. Such as:
using System;
using System.Diagnostics;
using System.IO;
using Mono.Unix;
class Test
{
public static void Main()
{
int reading, writing;
Mono.Unix.Native.Syscall.pipe(out reading, out writing);
int stdout = Mono.Unix.Native.Syscall.dup(1);
Mono.Unix.Native.Syscall.dup2(writing, 1);
Mono.Unix.Native.Syscall.close(writing);
Process cmdProcess = new Process();
ProcessStartInfo cmdStartInfo = new ProcessStartInfo();
cmdStartInfo.FileName = "cat";
cmdStartInfo.CreateNoWindow = true;
cmdStartInfo.Arguments = "test.exe";
cmdProcess.StartInfo = cmdStartInfo;
cmdProcess.Start();
Mono.Unix.Native.Syscall.dup2(stdout, 1);
Mono.Unix.Native.Syscall.close(stdout);
Stream s = new UnixStream(reading);
byte[] buf = new byte[1024];
int bytes = 0;
int current;
while((current = s.Read(buf, 0, buf.Length)) > 0)
{
bytes += current;
}
Mono.Unix.Native.Syscall.close(reading);
Console.WriteLine("{0} bytes read", bytes);
}
}
Under unix, file descriptors are inherited by child processes unless marked otherwise (close on exec). So, to redirect stdout of a child, all you need to do is change the file descriptor #1 in the parent process before calling exec. Unix also provides a handy thing called a pipe which is a unidirectional communication channel, with two file descriptors representing the two endpoints. For duplicating file descriptors, you can use dup or dup2 both of which create an equivalent copy of a descriptor, but dup returns a new descriptor allocated by the system and dup2 places the copy in a specific target (closing it if necessary). What the above code does, then:
Creates a pipe with endpoints reading and writing
Saves a copy of the current stdout descriptor
Assigns the pipe's write endpoint to stdout and closes the original
Starts the child process so it inherits stdout connected to the write endpoint of the pipe
Restores the saved stdout
Reads from the reading endpoint of the pipe by wrapping it in a UnixStream
Note, in native code, a process is usually started by a fork+exec pair, so the file descriptors can be modified in the child process itself, but before the new program is loaded. This managed version is not thread-safe as it has to temporarily modify the stdout of the parent process.
Since the code starts the child process without managed redirection, the .NET runtime does not change any descriptors or create any streams. So, the only reader of the child's output will be the user code, which uses a UnixStream to work around the StreamReader's encoding issue,
I checked out what's happening with reflector. It seems to me that StreamReader doesn't read until you call read on it. But it's created with a buffer size of 0x1000, so maybe it does. But luckily, until you actually read from it, you can safely get the buffered data out of it: it has a private field byte[] byteBuffer, and two integer fields, byteLen and bytePos, the first means how many bytes are in the buffer, the second means how many have you consumed, should be zero. So first read this buffer with reflection, then create the BinaryReader.
Maybe you can try like this:
public class ThirdExe
{
private static TongueSvr _instance = null;
private Diagnostics.Process _process = null;
private Stream _messageStream;
private byte[] _recvBuff = new byte[65536];
private int _recvBuffLen;
private Queue<TonguePb.Msg> _msgQueue = new Queue<TonguePb.Msg>();
void StartProcess()
{
try
{
_process = new Diagnostics.Process();
_process.EnableRaisingEvents = false;
_process.StartInfo.FileName = "d:/code/boot/tongueerl_d.exe"; // Your exe
_process.StartInfo.UseShellExecute = false;
_process.StartInfo.CreateNoWindow = true;
_process.StartInfo.RedirectStandardOutput = true;
_process.StartInfo.RedirectStandardInput = true;
_process.StartInfo.RedirectStandardError = true;
_process.ErrorDataReceived += new Diagnostics.DataReceivedEventHandler(ErrorReceived);
_process.Exited += new EventHandler(OnProcessExit);
_process.Start();
_messageStream = _process.StandardInput.BaseStream;
_process.BeginErrorReadLine();
AsyncRead();
}
catch (Exception e)
{
Debug.LogError("Unable to launch app: " + e.Message);
}
private void AsyncRead()
{
_process.StandardOutput.BaseStream.BeginRead(_recvBuff, 0, _recvBuff.Length
, new AsyncCallback(DataReceived), null);
}
void DataReceived(IAsyncResult asyncResult)
{
int nread = _process.StandardOutput.BaseStream.EndRead(asyncResult);
if (nread == 0)
{
Debug.Log("process read finished"); // process exit
return;
}
_recvBuffLen += nread;
Debug.LogFormat("recv data size.{0} remain.{1}", nread, _recvBuffLen);
ParseMsg();
AsyncRead();
}
void ParseMsg()
{
if (_recvBuffLen < 4)
{
return;
}
int len = IPAddress.NetworkToHostOrder(BitConverter.ToInt32(_recvBuff, 0));
if (len > _recvBuffLen - 4)
{
Debug.LogFormat("current call can't parse the NetMsg for data incomplete");
return;
}
TonguePb.Msg msg = TonguePb.Msg.Parser.ParseFrom(_recvBuff, 4, len);
Debug.LogFormat("recv msg count.{1}:\n {0} ", msg.ToString(), _msgQueue.Count + 1);
_recvBuffLen -= len + 4;
_msgQueue.Enqueue(msg);
}
The key is _process.StandardOutput.BaseStream.BeginRead(_recvBuff, 0, _recvBuff.Length, new AsyncCallback(DataReceived), null); and the very very important is that convert to asynchronous reads event like Process.OutputDataReceived.
I have a proxy process which a number of client applications will connect to using .Net remoting. I would like to manage this proxy's existence by using some reference counting scheme. To do this I thought I would use a small Memory Mapped File in which I will store a key-value (string-int) pair which will look something like this:
ref_count 2
However, I would like to update the ref_count value from time to time but I'm having problems doing so. Here is my code:
using System;
using System.IO;
using System.IO.MemoryMappedFiles;
using System.Threading;
class Program
{
static void Main(string[] args)
{
using (MemoryMappedFile mmf = MemoryMappedFile.CreateNew("testmap", 100))
{
bool mutexCreated;
Mutex mutex = new Mutex(true, "testmapmutex", out mutexCreated);
using (MemoryMappedViewStream stream = mmf.CreateViewStream())
{
BinaryWriter writer = new BinaryWriter(stream);
writer.Write("count:");
writer.Write(3);
}
mutex.ReleaseMutex();
mutex.WaitOne();
using (MemoryMappedViewStream stream = mmf.CreateViewStream())
{
BinaryReader reader = new BinaryReader(stream);
BinaryWriter writer = new BinaryWriter(stream);
Console.WriteLine("String value is: {0}", reader.ReadString());
Console.WriteLine("UInt32 value is: {0}", reader.ReadUInt32());
// Update mmf data
writer.Write("count:");
writer.Write(30);
// empty string where "count" was expected
Console.WriteLine("String value is: {0}", reader.ReadString());
// 0 where 30 was expected
Console.WriteLine("UInt32 value is: {0}", reader.ReadUInt32());
}
mutex.ReleaseMutex();
}
}
}
So, my question is, what is the best way to update my reference count and also, why does my second write not work in the above code example?
Thanks,
It is a stream. So reading from it advances the Position. You'll need to add
stream.Position = 0;
both before the code that updates the count and the code that reads it again. Further improve it by not writing the string, you don't need it.
The second Write calls work - but you're writing/reading from the "wrong" location.
After all this is done, you should have in memory:
5 (length of "code:")
"code:" (the actual characters for "code:")
3 (value you wrote/read)
5 (length of "code:")
"code:" (the actual characters for "code:")
30 (value you wrote)
0 (length of the empty string you read)
0 (value you read)
Each call to any of the BinaryReader/BinaryWriter methods is advancing the underlying Stream. It works the first time because you constructed a new stream on the view of the memory mapped file (thus resetting the Stream). Try calling stream.Seek(0L, SeekOrigin.Begin) between the write and the read and seeing what happens.