I'm trying creating a program that allows users to upload JAR Files for some third-party code they've written to an online server and then receive a String message back in response.
The online server is coded using Java with standard TCP Socket networking. The client uses a piece of additional software, which means I have to use C# for the File uploader. The code I've written is included below. In this program the File uploader works fine, but for some reason the client hangs when it reaches input.ReadLine() where it is supposed to receive the String message response from the server.
public static string sendFile(string filepath) {
String response = "";
// Get the file
Stream fileStream = File.OpenRead(filepath);
byte[] buffer = new byte[fileStream.Length];
fileStream.Read(buffer, 0, buffer.Length);
try {
// Create the connection
TCPClient client = new TCPClient("127.0.0.1", 21000);
NetworkStream stream = client.GetStream();
// Send the file to the server
stream.Write(buffer, 0, buffer.Length);
stream.Flush();
// Receive a single string response from the server
if (stream.CanRead) {
StreamReader input = new StreamReader(stream);
inputString = input.ReadLine();
}
input.Close();
stream.Close();
client.Close();
}
catch (IOException e) {
print ("Error: " + e);
}
// Return the response message string
return inputString;
}
I have also tried implementing the code above using a StreamWriter rather than writing directly from the NetworkStream itself. Unfortunately the StreamWriter class doesn't have a method for sending a byte array, (only a char array). I'm wondering whether the problem is being caused by the fact I'm calling the Write method of the NetworkStream directly rather than using a StreamWriter.
If anybody has any idea why the code above isn't working then please let me know. Alternatively, if you have a different solution that would allow me to send a file (byte array) and receive back a string message using the same TCPClient connection then please also feel free to mention it.
Regards,
Midavi.
The readline hangs because it will only return when it has successfully read a line from the server, this is the disadvantage of using blocking sockets. Please make sure your server is accually sending a line( string ending with "\n"
Is your stream terminated with an end of line?
Readline will block until the stream ends or receives the end of line character. If your uploader doesn't terminate the string it could act like you're saying.
Checkout networkComms.net, an open source C# network communication library. A short example demonstrating the most basic functionality here, hopefully not overly complex! Most of the problems you might come across will already have been solved and it might save you some time.
Readline wait for end of line \r\n but in java the end of line is by default \n only. This can be the difference that can block your comunication, but to be sure you need to snoop the network activity or you can use a different method to read data.
You can try to use the read method:
byte[] myReadBuffer = new byte[512];
int numberOfBytesRead = 0;
// Incoming message may be larger than the buffer size.
do{
numberOfBytesRead = stream.Read(myReadBuffer, 0, myReadBuffer.Length);
// do something with data in myReadBuffer
}
while(stream.DataAvailable);
Related
I've been working on a project recently that involves a lot of FileStreaming, something which I've not really touched on before.
To try and better acquaint myself with the principles of such methods, I've written some code that (theoretically) downloads a file from one dir to another, and gone through it step by step, commenting in my understanding of what each step achieves, like so...
Get fileinfo object from DownloadRequest Object
RemoteFileInfo fileInfo = svr.DownloadFile(request);
DownloadFile method in WCF Service
public RemoteFileInfo DownloadFile(DownloadRequest request)
{
RemoteFileInfo result = new RemoteFileInfo(); // create empty fileinfo object
try
{
// set filepath
string filePath = System.IO.Path.Combine(request.FilePath , #"\" , request.FileName);
System.IO.FileInfo fileInfo = new System.IO.FileInfo(filePath); // get fileinfo from path
// check if exists
if (!fileInfo.Exists)
throw new System.IO.FileNotFoundException("File not found",
request.FileName);
// open stream
System.IO.FileStream stream = new System.IO.FileStream(filePath,
System.IO.FileMode.Open, System.IO.FileAccess.Read);
// return result
result.FileName = request.FileName;
result.Length = fileInfo.Length;
result.FileByteStream = stream;
}
catch (Exception ex)
{
// do something
}
return result;
}
Use returned FileStream from fileinfo to read into a new write stream
// set new location for downloaded file
string basePath = System.IO.Path.Combine(#"C:\SST Software\DSC\Compilations\" , compName, #"\");
string serverFileName = System.IO.Path.Combine(basePath, file);
double totalBytesRead = 0.0;
if (!Directory.Exists(basePath))
Directory.CreateDirectory(basePath);
int chunkSize = 2048;
byte[] buffer = new byte[chunkSize];
// create new write file stream
using (System.IO.FileStream writeStream = new System.IO.FileStream(serverFileName, FileMode.OpenOrCreate, FileAccess.ReadWrite))
{
do
{
// read bytes from fileinfo stream
int bytesRead = fileInfo.FileByteStream.Read(buffer, 0, chunkSize);
totalBytesRead += (double)bytesRead;
if (bytesRead == 0) break;
// write bytes to output stream
writeStream.Write(buffer, 0, bytesRead);
} while (true);
// report end
Console.WriteLine(fileInfo.FileName + " has been written to " + basePath + " - Done!");
writeStream.Close();
}
What I was hoping for is any clarification or expansion on what exactly happens when using a FileStream.
I can achieve the download, and now I know what code I need to write in order to perform such a download, but I would like to know more about why it works. I can find no 'beginner-friendly' or step by step explanations on the web.
What is happening here behind the scenes?
A stream is just an abstraction, fundamentally it works like a pointer within a collection of data.
Take the example string of "Hello World!" for example, it is just a collection of characters, which are fundamentally just bytes.
As a stream, it could be represented to have:
A length of 12 (possibly more including termination characters etc)
A position in the stream.
You read a stream by moving the position around and requesting data.
So reading the text above could be (in pseudocode) seen to be like this:
do
get next byte
add gotten byte to collection
while not the end of the stream
the entire data is now in the collection
Streams are really useful when it comes to accessing data from sources such as the file system or remote machines.
Imagine a file that is several gigabytes in size, if the OS loaded all of that into memory any time a program wanted to read it (say a video player), there would be a lot of problems.
Instead, what happens is the program requests access to the file, and the OS returns a stream; the stream tells the program how much data there is, and allows it to access that data.
Depending on implementation, the OS may load a certain amount of data into memory ahead of the program accessing it, this is known as a buffer.
Fundamentally though, the program just requests the next bit of data, and the OS either gets it from the buffer, or from the source (e.g. the file on disk).
The same principle applies to streams between different computers, except requesting the next bit of data may very well involve a trip to the remote machine to request it.
The .NET FileStream class and the Stream base class, all just defer to the windows systems for working with streams in the end, there's nothing particularly special about them, it's just what you can do with the abstraction that makes them so powerful.
Writing to a stream is just the same, but it just puts data into the buffer, ready for the requester to access.
Infinite Data
As a user pointed out, streams can be used for data of indeterminate length.
All stream operations take time, so reading a stream is typically a blocking operation that will wait until data is available.
So you could loop forever while the stream is still open, and just wait for data to come in - an example of this in practice would be a live video broadcast.
I've since located a book - C# 5.0 All-In-One For Dummies - It explains everything about all Stream classes, how they work, which one is most appropriate and more.
Only been reading about 30 minutes, already have such a better understanding. Excellent guide!
Working on some socket layers using SslStream. reference
Using the reference, I implemented a simple client. The awkward part is when you run the application, it seems the server is not replying to the client.
Going into the debug screen and setting some breakpoints, I realized it was this function that was on an endless loop.
static string ReadMessage(SslStream sslStream)
{
// Read the message sent by the server.
// The end of the message is signaled using the
// "<EOF>" marker.
byte [] buffer = new byte[2048];
StringBuilder messageData = new StringBuilder();
int bytes = -1;
do
{
bytes = sslStream.Read(buffer, 0, buffer.Length);
// Use Decoder class to convert from bytes to UTF8
// in case a character spans two buffers.
Decoder decoder = Encoding.UTF8.GetDecoder();
char[] chars = new char[decoder.GetCharCount(buffer,0,bytes)];
decoder.GetChars(buffer, 0, bytes, chars,0);
messageData.Append (chars);
// Check for EOF.
if (messageData.ToString().IndexOf("<EOF>") != -1)
{
break;
}
} while (bytes != 0);
return messageData.ToString();
}
Further investigation pointed out the real criminal here:
bytes = sslStream.Read(buffer, 0, buffer.Length);
Seems like, SslStream.Read() is not returning. Checking the byte[] buffer in debug screen reveals that the response has been written to the buffer till crlf. The function has done it's job, still it's not returning with success?!
What could be the reason for this? What steps should I take to ignore with this issue?
Also, for the skeptical ones: I used openssl to see if the server is behaving as it should, and everything is fine on the server side.
Note: I'm already aware of the SslStream.ReadTimeout property. Though it does the job by raising an exception it is not the right answer for every scenario, specially when server is responding with a large stream of data which can only be read efficiently using a while loop and a buffer.
I was struggling with the same problem with me, the program was infinite loop.
I solved through the following information
İf you using http,https protocol ? you add to header , "Connection: close\r\n"
Http Standart Article
14.10 Connection
The Connection general-header field allows the sender to specify options that are desired for that particular connection and MUST NOT be communicated by proxies over further connections.
The Connection header has the following grammar:
Connection = "Connection" ":" 1#(connection-token)
connection-token = token
HTTP/1.1 proxies MUST parse the Connection header field before a message is forwarded and, for each connection-token in this field, remove any header field(s) from the message with the same name as the connection-token. Connection options are signaled by the presence of a connection-token in the Connection header field, not by any corresponding additional header field(s), since the additional header field may not be sent if there are no parameters associated with that connection option.
Message headers listed in the Connection header MUST NOT include end-to-end headers, such as Cache-Control.
HTTP/1.1 defines the "close" connection option for the sender to signal that the connection will be closed after completion of the response. For example,
**Connection: close**
in either the request or the response header fields indicates that the connection SHOULD NOT be considered `persistent' (section 8.1) after the current request/response is complete.
HTTP/1.1 applications that do not support persistent connections MUST include the "close" connection option in every message.
A system receiving an HTTP/1.0 (or lower-version) message that includes a Connection header MUST, for each connection-token in this field, remove and ignore any header field(s) from the message with the same name as the connection-token. This protects against mistaken forwarding of such header fields by pre-HTTP/1.1 proxies. See section 19.6.2.
If the connection is still open and the server hasn't written <EOF> then it absolutely makes sense that it's just "hung". It's waiting for more data. The only way it could know there's no more data coming is for the server to have closed the connection.
Has it already managed to read all the data which the server has actually sent? What does messageData look like on the iteration before the call which isn't returning?
After some thinking I figured out a workaround.
sslStream.ReadTimeout = 100; //How much time does it takes for a processor to read and write 2048bytes to the buffer?
Had to use timeout, nothing else seemed to work. Modified the Reader to deal with exceptions.
static string ReadMessage(SslStream sslStream)
{
// Read the message sent by the server.
// The end of the message is signaled using the
// "<EOF>" marker.
byte[] buffer = new byte[2048];
StringBuilder messageData = new StringBuilder();
int bytes = -1;
do
{
try
{
bytes = sslStream.Read(buffer, 0, buffer.Length);
}
catch (Exception ex)
{
}
// Use Decoder class to convert from bytes to UTF8
// in case a character spans two buffers.
Decoder decoder = Encoding.ASCII.GetDecoder();
char[] chars = new char[decoder.GetCharCount(buffer, 0, bytes)];
decoder.GetChars(buffer, 0, bytes, chars, 0);
messageData.Append(chars);
// Check for EOF.
if (messageData.ToString().IndexOf("\r\n") != -1)
{
break;
}
}
while (bytes != -1);
return messageData.ToString();
}
Though this works, doesn't mean it's a good answer. If someone can provide a better answer, it would be great.
So i am trying to create something like a syncronized paint program by using sockets.I have a server side..and a client side and i am trying to send the inkCollection from the server to the client.This works for simple text, but i cant seem to send the inkCollection.Or it would be even cooler if you could help me send the last stroke so that the data transfers faster.Here is some code of what i've been trying:
sending strokes:
byte[] data;
using (MemoryStream ms = new MemoryStream())
{
inkcanvas.Strokes.Save(ms);
data = ms.ToArray();
}
svr.SendToAll("u getting some strokes");
svr.SendStrokes(data);
svr.SendStrokes(byte[] data):
public void SendStrokes(byte[] data)
{
for (int i = 0; i < no; i++)
{
byte[] dt = data;
accepted[i].Send(dt);
}
MessageBox.Show("dONE");
}
and this is on the clientside:
byte[] buffer=new byte[1024];
MessageBox.Show("receiving strokes");
int rec = conn.Receive(buffer, 0, buffer.Length, 0);
if (rec <= 0)
throw new SocketException();
MessageBox.Show("strokes received");
//Array.Resize(ref buffer, rec);
using (MemoryStream ms = new MemoryStream(buffer))
{
inkcanvas.Strokes = new System.Windows.Ink.StrokeCollection(ms);
ms.Close();
}
MessageBox.Show("strokes added");
these exact same methods work perfectly for string, but when i try to do it with the strokeCollection, it fails.Nothing shows up on the client and i get the following SocketException ont the serverside: An existing connection was forcibly closed by the remote host.
But if you guys got a better way on how to do this it would be great...is it something i am missing? i mean..if it works for text transformed into a byte array...why wouldint it work for this strokecollection?
Thanks!
EDIT: You think you could help me out with some sample code? i cant really seem to implement it;
You forgot to either design or implement a protocol! You can't just send a bunch of bytes over TCP and assume the receiver will be able to make sense out of it.
You have an application message that consists of a bunch of strokes which you are trying to send over TCP. But TCP is a byte stream service, not an application message service. So you need some kind of application message protocol to package the message for transport and unpackage it on receipt. But you have not written any such code. So you're basically expecting it to work by magic.
Define a protocol. For example, it might say:
Canvas strokes shall be sent by a '1' byte indicating canvas strokes, followed by 4 bytes indicating the number of bytes in the strokes object in network byte order, followed by the stroke data. The receiver will read the first byte and identify that it's a canvas strokes object. Then the receiver will read the next four bytes to determine the length. The receiver shall accumulate that number of bytes (using multiple reads if necessary) and then process the reconstructed canvas strokes object.
Do not skip the step of creating a written protocol definition.
Then, when you have problems, follow this handy troubleshooting guide:
Does the sender follow the specification? If not, stop, the sender is broken.
Does the receiver follow the specification? If not, stop, the receiver is broken.
Stop, the specification is broken.
If you want simple, you can convert the data into base64 and encode each message as a line of text. That will allow you to use a ReadLine function to grab exactly one message. Then you can just use a message format like an "S" (for "strokes") followed by the data in base64 format. Use a WriteLine function to send the text message followed by a newline.
i think you forgot to set the memorystream position. you should set the memory stream position to 0 before you send out the stream, cause after strokes.save, the position of the stream is at the end.
I'm having trouble sending the folowing 140kb image file over the network using sockets:
The beginning of the file gets transfered correctly, then the rest gets messed up leading to the following result:
I have a server, that waits for files and a client that connects to server and starts the file tranfer. When client closes the connection, server knows that the file has been done sending and saves it.
All the data on the server side is being collected in byte list (GLOBAL.FILE_BUFFER), before written to a file:
public void OnDataReceived(IAsyncResult asyn)
{
try
{
SocketPacket socketData = (SocketPacket)asyn.AsyncState;
int iRx = socketData.m_currentSocket.EndReceive(asyn);
for (int f = 0; f < iRx; f++)
{
GLOBAL.FILE_BUFFER.Add(socketData.dataBuffer[f]); // Collect all the incoming data into a buffer
}
WaitForData(socketData.m_currentSocket, socketData.socket_id, REQUEST_INDEX_ID);
}
catch (ObjectDisposedException)
{
}
catch (SocketException se)
{
}
}
This is the code I'm using to write to a file:
public static void ByteArrayToFile(string _FileName, byte[] _ByteArray, int BytesRead)
{
System.IO.FileStream _FileStream = new System.IO.FileStream(_FileName, System.IO.FileMode.Create, System.IO.FileAccess.Write);
_FileStream.Write(_ByteArray, 0, BytesRead);
_FileStream.Close();
}
I used the same ByteArrayToFile function on cient side and it saved the image correctly. It appears that client is sending the correct data, but when server recieves it, it gets messed up. I saved file on both ends, by client and by server and then compared them:
Notice the purple line/bar in the bottom center. It indicates that data doesn't match there, but notice that it matches perfectly at the beginning..
I don't understand how can this happen since I'm using list and simply keep adding to the byte list until there's nothing left to read!? If you don't have solution can you give me any tips please?! This problem has been bugging me for weeks, would love to get it figured out finaly. Thanks!
Can you tell from your data comparison what the problem is? Is the server simply dropping bytes or is the data corrupted? How much data is transferred correctly?
Consider using something like Wireshark to examine the bytes as they come off the wire. That should help you determine whether it's a network error or something in your server code.
When making a request using HttpWebRequest object, I need to call the method GetResponse() to send the request and get the response back.
The problem with this method is that it doesn't return the response object until all data has been received. Say I am downloading a 100 MB file, I won't be able to read it until the response finish and all the 100 MB is downloaded.
What I want is to be able to read the response stream bytes as soon as they arrive, without waiting for the response to complete.
I know I can use the Range Http header, but it won't work on my situation.
I think this is very close to what #Zachary suggests. And it (seems to) work(s); actually I think using using as #Zachary does is even "nicer".
My main point being I cannot see the blocking behaviour of GetResponse() you (seem to) describe.
In addition the following code only roughly shows how everything works; it will not read the stream to the end for example (unless by coincidence :)). But it should work if you copy-n-paste it into an empty "Console Application"-project in Visual Studio.
You can try using some "shorter" URL for a test. The example here starts downloading an ISO of the debian distribution (a bit more than 600 MByte). Sorry debian, I did not mean to steal your bandwidth. -> Btw: is there something sensible one can use to test such a scenario?
The Code is strongly inspired by C# - How to read a continuous stream of XML over HTTP.
namespace StreamReadWebRequest
{
using System;
using System.Collections.Generic;
using System.Text;
using System.Net;
using System.IO;
class Program
{
static void Main(string[] args)
{
HttpWebRequest req;
HttpWebResponse res = null;
try
{
req = (HttpWebRequest)WebRequest.Create(
"http://cdimage.debian.org/debian-cd/5.0.4/i386/iso-cd/debian-504-i386-CD-1.iso");
res = (HttpWebResponse)req.GetResponse();
Stream stream = res.GetResponseStream();
byte[] data = new byte[4096];
int read;
while ((read = stream.Read(data, 0, data.Length)) > 0)
{
Process(data, read);
}
}
finally
{
if (res != null)
res.Close();
}
Console.In.Read();
}
private static void Process(byte[] data, int read)
{
Console.Out.Write(ASCIIEncoding.ASCII.GetString(data));
}
}
}
I was looking for the same thing: server streams chunked XML data and I needed a C# client that could access this data while server is streaming. I tried many different ways to access the source (WebChannelFactory, WebClient, HttpWebRequest/Response, TcpClient) but failed so far. Finding this thread I focused on HttpWebRequest/Response where I have the same problem that following line is blocking:
HttpWebResponse resp = (HttpWebResponse)request.GetResponse();
As Artiom Chilaru stated, if it's blocking: something is wrong, because it should not. Now focusing on trying to replicate default behavior with downloading large .ISO files I found out that Fiddler was blocking the GetResponse() method!
However there is no problem to open Fiddler once the stream has been set up (i.e. GetResponse() has already been called), but during the HTTP GET if you find GetResponse() is blocking try to close Fiddler and see if your application now continuous it's normal flow (i.e. reading the stream).
If you set the buffer size on your read, you can read in the data in chunks... example...
// Get the response stream
using(Stream resStream = response.GetResponseStream())
{
string parseString = null;
int count = 0;
do
{
// Read a chunk of data
count = resStream.Read(buf, 0, buf.Length);
if (count != 0)
{
// Convert to ASCII
parseString = Encoding.ASCII.GetString(buf, 0, count);
// Append string to results
sb.Append(tempString);
}
}
while (count > 0);
}
I'm not sure what you have on your side, but I know for a fact (and I'm sure many people will agree here) that GetResponse() will NOT download the whole file back. It will send the request, wait for the response, and get the response headers.
After you have the response, you can easily get the response stream with GetResponseStream(), which is the actual data stream that's downloading from the server. And you can easily access the response stream BEFORE the whole file is downloaded. This is 100% true and tested.
If you're not getting the same behaviour (which is really strange, and shouldn't happen) could you add a code example that is not working as I explained above?
Also, do test the example posted by scherand. It just proves once again that it works just fine, without any special hacks.