I've written a very small application to get a feeling for TCP latencies on our local network. That's where I encountered a very strange delay for certain package sizes that I could up to now not relate to a bug of mine.
Using the code below, I get sub-millisecond ping/pong times for all DataLengths up to 100 kiB, with one strange exception: if DataLength is between about 1010 and 2400 bytes, there seems to be a fixed 400ms delay added to that for no reason I can see.
Where does this long delay come from?
Client code
using (var client = new TcpClient())
{
client.Connect(Server); // Server is set somewhere else
var stream = client.GetStream();
var writer = new StreamWriter(stream, Encoding.UTF8) { AutoFlush = true };
var reader = new StreamReader(stream, Encoding.UTF8);
Stopwatch stopwatch = new Stopwatch();
string data = "".PadLeft(DataLength, 'T');
for (int i = 0; i < NumberOfPings; i++)
{
stopwatch.Restart();
writer.WriteLine(data);
reader.ReadLine();
// record elapsed time
}
}
Server code
while (true)
{
var client = tcpListener.AcceptTcpClient();
var stream = client.GetStream();
var reader = new StreamReader(stream, Encoding.UTF8);
var writer = new StreamWriter(stream, Encoding.UTF8) { AutoFlush = true };
while (client.Connected)
{
var request = reader.ReadLine();
if (request == null)
break;
writer.WriteLine(request);
}
}
This delay seems to be coming from something like Nagle's Algorithm which is used to collect multiple small packages before sending.
The problem I experienced is described in this knowledgebase article.
TcpClient has a NoDelay property that, when set to true, avoids this behaviour.
Related
I have this C# Script that is supposed to download a zip archive from GitHub, unpack it and put it in a specific folder:
using (var client = new HttpClient())
{
var filePath = Path.GetFullPath("runtime");#"https://github.com/BlackBirdTV/tank/releases/latest/download/runtime.zip?raw=true";
ConsoleUtilities.UpdateProgress("Downloading Runtime...", 0);
var request = await client.GetStreamAsync(url);
var buffer = new byte[(int)bufferSize];
var totalBytesRead = 0;
int bytes = 0;
while ((bytes = await request.ReadAsync(buffer, 0, buffer.Length)) != 0)
{
totalBytesRead += bytes;
ConsoleUtilities.UpdateProgress($"Downloading Runtime... ({totalBytesRead} of {bufferSize} bytes read) ", (int)(totalBytesRead / bufferSize * 100));
}
}
Decompress(buffer, filePath);
When I now run this, the download starts and it seems like it finishes, yet at a sporadic place it just stops. Somehow, It downloads the bytes as my Console shows, but they are zeroed out. It seems like either my computer receives zeros (which I doubt) or the bytes don't get written to the buffer.
Weirdly enough, downloading the file over the browser works just fine.
Any help is greatly appreciated
As I state in the comments, your problem is that each iteration of your while loop is overwriting your buffer, and you are not accumulating the data anywhere. So your last iteration doesn't completely fill the buffer and all you're left with is whatever data you got in the last iteration.
You could fix that bug by storing the accumulated buffer somewhere, but a far better solution is to not fuss with buffers and such and just use the built-in CopyToAsync method of Stream:
using var client = new HttpClient();
using var stream = await client.GetStreamAsync("https://github.com/BlackBirdTV/tank/releases/latest/download/runtime.zip?raw=true");
using var file = new FileStream(#"c:\temp\runtime.zip", FileMode.Create);
await stream.CopyToAsync(file);
Here I'm saving it to a local file at c:\temp\runtime.zip, but obviously change that to suit your needs. I suppose you're avoiding this method so you can track progress, which is fair enough. So if that's really important to you, read on for a fix to your original solution.
For completeness, here's your original code fixed up to work by writing the buffer to a FileStream:
var bufferSize = 1024 * 10;
var url = #"https://github.com/BlackBirdTV/tank/releases/latest/download/runtime.zip?raw=true";
using var client = new HttpClient();
using var stream = await client.GetStreamAsync(url);
using var file = new FileStream(#"c:\temp\runtime.zip", FileMode.Create);
var filePath = Path.GetFullPath("runtime");
var request = await client.GetStreamAsync(url);
var buffer = new byte[(int)bufferSize];
var totalBytesRead = 0;
int bytes = 0;
while ((bytes = await request.ReadAsync(buffer, 0, buffer.Length)) != 0)
{
await file.WriteAsync(buffer, 0, bytes);
totalBytesRead += bytes;
ConsoleUtilities.UpdateProgress($"Downloading Runtime... ({totalBytesRead} of {bufferSize} bytes read) ", (int)(totalBytesRead / bufferSize * 100));
}
What i would like to do is, constantly listen the microphone, have the microphone writer that writes into a stream, when the stream has x lenght, yield result, clear the stream and keep doing it it in a loop.
Then from the caller analyse all bytes received, i am a bit new to microphone recording, i don't event know if the code below register something, I am using NAudio library:
this is the caller:
var buffers = service.StreamHearing();
foreach (var buffer in buffers)
{
//analyse
}
and then is the actual service:
public IEnumerable<byte[]> StreamHearing()
{
var buffer = new byte[512];
using(var stream = new MemoryStream(buffer))
using(var writer = new WaveFileWriter(new IgnoreDisposeStream(stream), new WaveFormat(44100, 1)))
{
var recorder = new WaveInEvent
{
WaveFormat = new WaveFormat(44100, 1),
BufferMilliseconds = 100
};
recorder.StartRecording();
while (true)
{
yield return buffer;
buffer = new byte[512];
stream.SetLength(0);
}
}
}
this doesn't work, please give me a hand on how to do it.
thanks
You need to understand a couple of things which you miss.
First of all, WaveInEvent will write the data from the microphone asynchronously. Here is a correct snippet for writing the data from the microphone to some stream:
var buffer = new byte[512];
var recorder = new WaveInEvent
{
WaveFormat = new WaveFormat(44100, 1),
BufferMilliseconds = 100
};
var stream = new MemoryStream(buffer);
var writer = new WaveFileWriter(new IgnoreDisposeStream(stream), recorder.WaveFormat);
recorder.DataAvailable += (source, eventArgs) =>
{
var data = eventArgs.Buffer;
var bytesRead = eventArgs.BytesRecorded;
//here is a place where data from the microphone will be available
//you can add your processing right here in case you don't need to record and save the data
writer.Write(data, 0, bytesRead);
};
waveIn.StartRecording();
There are multiple different ways on how to proceed with the data you receive, but I don't know your main goal, so it is hard to say what is better way you choose the processing.
I have a StreamReader listening to an IP port and running in a thread / loop, this is working correctly (C#, Visual Studio) -
ns = tc.GetStream();
sr = new System.IO.StreamReader(ns);
private void ReceiveData()
{
while (true)
{
messageId = sr.ReadLine();
bool flag = messageId.Length <= 31;
if (!flag)
{
messageId = messageId.Substring(20, 12);
m_search.m_queue.add(messageId);
}
System.Threading.Thread.Sleep(250);
flag = true;
}
}
I wish to send back some characters and so I create a StreamWriter (in a separate .cs and so I have to reference the underlying stream variable in the first .cs)
var xs = Socketclient.ns;
using (System.IO.StreamWriter sw = new System.IO.StreamWriter(xs))
{
sw.Write("TH1\r\n");
sw.Flush();
//Wait 3 secs then deactivate relay
System.Threading.Thread.Sleep(3000);
sw.Write("TH0\r\n");
sw.Flush();
}
}
Which works until it 'returns' (after the last sw.Flush();) and reports Unable to read data from the transport connection here -
messageId = sr.ReadLine();
I assume I need to run my StreamWriter in a different thread or I need some way of suspending / restarting StreamReader whilst StreamWriter runs.
According to various sites StreamWriter should not affect StreamReader but it does and seems to close the underlying stream (ns) when finished.
Thoughts appreciated.
Regards
I have a Xamarin Android app that captures h264 video frames from an Android 4.4 (KitKat) device (x86 hardware) and sends them via TCP to a Windows 10 client (via WiFi). I am using protobuf.net to package the frames with SerializeWithLengthPrefix (Fixed32). This works well most of the time but randomly (between 20 seconds and 10 minutes) the data on the receive side gets corrupted. You can see that I also save the data to the device for debugging. Reading this data using the client app does not produce any errors (it's not corrupted). I am at a loss as to where the issue is. It seems like a platform bug with Xamarin's TCP client, but I have a hard time believing that I would be the only one having this issue. Note: The TCP comm is working in it's own thread.
private static void ProcessFrameQueue(TcpClient client)
{
//debug log for comparing TCP socket sent data with client recieved
_tempDumpFile = StreamControl.GetOutputTempFilePath(DateTime.Now.Ticks.ToString() + "-probuf-dump.bin");
var sentFrameCount = 0;
try
{
while (client.Client.IsConnected())
{
var data = _packetQueue.Take();
try
{
using (var stream = new MemoryStream())
{
Serializer.SerializeWithLengthPrefix(stream, data, PrefixStyle.Fixed32);
var protoBufData = stream.ToArray();
client.Client.Send(protoBufData);
//for debugging -- save the TCP data for comparison to what is recieved
//todo: delete as this is debuggng
using (var filestream = new FileStream(_tempDumpFile, FileMode.Append))
{
filestream.Write(protoBufData, 0, protoBufData.Length);
filestream.Flush(true);
}
}
sentFrameCount++;
}
catch (Exception ex)
{
//log error
}
} //end while
}
catch (Exception ex)
{
//log error
}
}
this is a simple client I wrote for debugging (I am manually calculating the packet size to make sure it's not an error in protobuf.net -- it's not a protobuf.net issue). Eventually the size packet will contain bad data leading to an overflow.
var client = new TcpClient("x.x.x.x", 19901);
client.ReceiveTimeout = 100000000;
byte[] bytes = new byte[client.ReceiveBufferSize];
var netStream = client.GetStream();
var sizezReadBtyes = 0;
var sizeBytes = new byte[4];
var packetCount = 0;
while (true)
{
//reads until it gets 4 bytes to calculate the packet size
var sizeOffset = 0;
var sizeLength = 4;
while ((sizezReadBtyes = netStream.Read(sizeBytes, sizeOffset, sizeLength)) > 0)
{
sizeOffset += sizezReadBtyes;
sizeLength -= sizezReadBtyes;
}
//read the remaining data...
var offset = 0;
var packetBytes = 0;
int packetlength = BitConverter.ToInt32(sizeBytes, 0);
var buffer = new byte[packetlength];
while (packetlength > 0 && (packetBytes = netStream.Read(buffer, offset, packetlength)) > 0)
{
offset += packetBytes;
packetlength -= packetBytes;
}
using (var ms = new MemoryStream(buffer))
{
var obj = ProtoBuf.Serializer.Deserialize<NetworkMediaPacket>(ms);
Console.WriteLine($"packet found {packetCount++} {obj.Data.Length}");
}
if (packetlength > 0) throw new EndOfStreamException();
}
I was able to fix by removing my Blocking Queue (_packetQueue). I just used a writeasync on a TCPClient NetStream. This looks to me like a bug in the Xamarin/Mono/.net code.
I'm using a C# NetworkStream to read/write to an IP address (not a DNS address).
My program is replacing a very old assembly language program that was on a mainframe. Now it's on Windows.
I'm writing/reading less than 200 bytes. The strings end with a LineFeed character so I'm using a StreamReader.Readline() to read a response, after my Stream.Write(). On the IBM a write/read cycle took 300ms.
Now about after every 2nd or 3 read, it takes 15 seconds for the read. When I read the log of the sender it is sending the data in less than a second. For some reason I get these 15 second delays.
I'm clueless on what's happening.
p.s.
One weird thing I noticed if I set the stream read timeout to 4 seconds, it times out around 4 seconds. If I set the timeout to 10 seconds or no timeout, it times out after 15 seconds.
TcpClient tcpc = null;
NetworkStream stream = null;
StreamReader sr = null;
tcpc = new TcpClient();
tcpc.NoDelay = true;
tcpc.ExclusiveAddressUse = false;
tcpc.Connect("172.18.10.100", 4004);
stream = tcpc.GetStream();
sr = new StreamReader(stream, Encoding.ASCII);
sr.Peek();
string Message = null;
Message = "IX3543543" + '\r';
stream.Write(Encoding.ASCII.GetBytes(Message), 0, Message.Length);
string readmsg = null;
for (int i = 0; i < 4; i++)
readmsg = sr.ReadLine();
Your connection stays open as your code never free your IDisposable resources.
I think that your code should run faster if you add the using constructure.
Also you can merge declaration and assignment, like this (this is only a style comment, the main concern is `IDisposable usage)
And you can ReadToEnd your message in return, and examine it by youself after releasing the resources.
So your code could look something like this:
string response = null;
using(var tcpc = new TcpClient())
{
tcpc.NoDelay = true;
tcpc.ExclusiveAddressUse = false;
tcpc.Connect("172.18.10.100", 4004);
using (var stream = tcpc.GetStream())
using (var sr = new StreamReader(stream, Encoding.ASCII))
{
sr.Peek();
var Message = "IX3543543" + '\r';
stream.Write(Encoding.ASCII.GetBytes(Message), 0, Message.Length);
response = sr.ReadToEnd();
}
}
// examine message here
var lines = response.Split(new string[] { Environment.NewLine }, StringSplitOptions.RemoveEmptyEntries);