I'm working in a remote administration tool to my own use and the project is stopped because a error on receiver function on Server side that is this.:
Arithmetic operation resulted in an overflow. (Adding integers)
in this line:
var imageData = new byte[length];
void server_Received(Listener l, Info i, string received) {
string[] cmd = received.Split('|');
switch (cmd[1])
{
case "PRINT":
//MessageBox.Show("print");
Thread threadPrint;
threadPrint = new Thread(() =>
{
var lengthData = new byte[4];
var lengthBytesRead = 0;
while (lengthBytesRead < lengthData.Length)
{
var read = i.sock.Receive(lengthData, lengthBytesRead, lengthData.Length - lengthBytesRead, SocketFlags.None);
if (read == 0) return;
lengthBytesRead += read;
}
var length = BitConverter.ToInt32(lengthData, 0);
var imageData = new byte[length];
var imageBytesRead = 0;
while (imageBytesRead < imageData.Length)
{
var read = i.sock.Receive(imageData, imageBytesRead, imageData.Length - imageBytesRead, SocketFlags.None);
if (read == 0) return;
imageBytesRead += read;
}
using (var stream = new MemoryStream(imageData))
{
var bitmap = new Bitmap(stream);
Invoke(new ImageCompleteDelegate(ImageComplete), new object[] { bitmap });
}
});
threadPrint.Start();
break;
}
}
the code to receiver i taked from this reference (see Client code) and was adapted to my need like you can see above.
Here is how i'm sending the bitmap on android (Java):
try {
dos = new DataOutputStream(SocketBackgroundService.xclientSocket.getOutputStream());
PrintWriter out = new PrintWriter(
new BufferedWriter(
new OutputStreamWriter(dos)
)
);
out.println("|PRINT|");
out.flush();
dos.writeInt(array.length);
dos.write(array, 0, array.length);
dos.flush();
} catch (IOException e) {
e.printStackTrace();
}
Then, i want know what can be done to solve this trouble?
Thanks in advance.
The solution was insert:
Array.Reverse(lengthData);
before of:
var length = BitConverter.ToInt32(lengthData, 0);
Related
I'm working on a multiplayer game and I'm having an issue with the the way I parse the packets from the connection. When I'm debugging the game it runs at a lower performance and the packets are received, when I'm not, packets aren't fully received and the ParsePacket method isn't called.
My packet structure is this:
2 Bytes Short Command, 2 Bytes Short Payload Size, (Optional) Payload Bytes
IInputStream inputStream = null;
DataReader dataReader = null;
byte[] data = new byte[1024];
IBuffer buffer = data.AsBuffer();
try
{
inputStream = StreamSocket.InputStream;
dataReader = DataReader.FromBuffer(buffer);
dataReader.InputStreamOptions = InputStreamOptions.Partial;
dataReader.ByteOrder = ByteOrder.LittleEndian;
while (connected)
{
await inputStream.ReadAsync(buffer, 1024, InputStreamOptions.Partial);
Debug.WriteLine("Buffer " + buffer.Length);
if (buffer.Length >= PacketHeaderSize)
{
short command = dataReader.ReadInt16();
short payloadSize = dataReader.ReadInt16();
byte[] payload = null;
if (payloadSize == 0)
{
UpdateBuffer(buffer, (uint)(PacketHeaderSize + payloadSize));
Packet packet = new Packet(command, payloadSize, payload);
ParsePacket(packet);
}
else if (payloadSize > 0)
{
if (buffer.Length >= (PacketHeaderSize + payloadSize))
{
payload = new byte[payloadSize];
dataReader.ReadBytes(payload);
UpdateBuffer(buffer, (uint)(PacketHeaderSize + payloadSize));
Packet packet = new Packet(command, payloadSize, payload);
ParsePacket(packet);
}
}
}
}
}
catch (Exception e) {
// ...
}
private void UpdateBuffer(IBuffer buffer, uint bytesRead)
{
if (buffer.Length > bytesRead)
{
byte[] bufferBytes = new byte[buffer.Length - bytesRead];
System.Buffer.BlockCopy(buffer.ToArray(), (int)bytesRead, bufferBytes, 0, (int)(buffer.Length - bytesRead));
buffer = bufferBytes.AsBuffer();
}
else
{
byte[] bufferBytes = new byte[1024];
buffer = bufferBytes.AsBuffer();
}
}
What I'm doing wrong?
Things fixed to make this work:
Take important variables outside the parsing packets loop as we need the value next time we need to parse another packet.
Read the packet header once if a incomplete app or game packet is received.
Load only the bytes we need
Read the header and payload once it has been fully received using UnconsumedBufferLength.
Code:
short command = 0;
short payloadSize = 0;
byte[] payload = null;
bool packetHeaderRead = false;
while (connected)
{
if (!packetHeaderRead)
{
if (dataReader.UnconsumedBufferLength < PacketHeaderSize)
{
int headerBytesLeft = PacketHeaderSize - (int)dataReader.UnconsumedBufferLength;
if (headerBytesLeft > 0)
{
await dataReader.LoadAsync((uint)headerBytesLeft);
continue;
}
}
else
{
command = dataReader.ReadInt16();
payloadSize = dataReader.ReadInt16();
packetHeaderRead = true;
continue;
}
}
else
{
int payloadBytesLeft = payloadSize - (int)dataReader.UnconsumedBufferLength;
if (payloadBytesLeft > 0)
{
await dataReader.LoadAsync((uint)payloadBytesLeft);
}
if (payloadSize == 0)
{
Packet packet = new Packet(command, payloadSize, payload);
ParsePacket(packet);
packetHeaderRead = false;
}
else if (dataReader.UnconsumedBufferLength >= payloadSize)
{
payload = new byte[payloadSize];
dataReader.ReadBytes(payload);
Packet packet = new Packet(command, payloadSize, payload);
ParsePacket(packet);
packetHeaderRead = false;
}
}
}
I have encountered a problem with TcpClient running on an Android device. The issue is that reading from NetworkStream takes a lot of time. I use it to send bitmaps from server application (also written in C#) to my Android device (Nexus 9). I push 6000 bitmaps each of them around 1.4KB in size. That takes around 30 seconds to complete (5ms per bitmap). On the other hand I also implemented the same in Java and it takes 5 seconds to do the same (less than 1ms per bitmap)!
Are you aware of any issues regarding TcpClient in Xamarin.Android? I have to admit that this is very surprising as I have never had similar problems with Xamarin.
I pasted just a little code below so you can take a look if it is not me that messed up something. At the very bottom I also pasted Java code and as you can see it is very similar. Do you have any suggestions regarding this? I appreciate any feedback.
Here is the method that I use to receive bitmaps (this function runs asynchronously and is started through Task.Factory.StartNew()):
private void HandleRequestBitmaps(TcpClient client, CancellationToken token)
{
if (client == null)
{
throw new ArgumentNullException("client");
}
var encoder = new UTF8Encoding();
using (var stream = client.GetStream())
{
stream.WriteMessageWithLength(ServerCommand.PushBitmaps.ServerCommandToBytes(encoder));
var initialMetadataBytes = stream.ReadMessageWithLength(client.ReceiveBufferSize);
int[] initialMetadata;
if (!TryParseMetadataMessage(initialMetadataBytes, encoder, 3, out initialMetadata))
{
BitmapReceiveEnd(this, new BitmapReceiveEndEventArgs(false));
return;
}
BitmapsReceiveBegin(this,
new BitmapsReceiveBeginEventArgs(initialMetadata[0], initialMetadata[1], initialMetadata[2]));
while (!token.IsCancellationRequested)
{
var messageBytes = stream.ReadMessageWithLength(client.ReceiveBufferSize);
if (ServerCommand.PushBitmapsDone.Equals(messageBytes.BytesToServerCommand(encoder)))
{
BitmapReceiveEnd(this, new BitmapReceiveEndEventArgs(true));
break;
}
int[] metadata;
if (!TryParseMetadataMessage(messageBytes, encoder, 3, out metadata))
{
BitmapReceiveEnd(this, new BitmapReceiveEndEventArgs(false));
return;
}
var bitmapBytes = stream.ReadMessageWithLength(client.ReceiveBufferSize);
BitmapReceived(this,
new BitmapReceivedEventArgs(metadata[0], new Point(metadata[1], metadata[2]), bitmapBytes));
}
}
}
Where TryParseMetadaMessage method looks like this:
private static bool TryParseMetadataMessage(byte[] message, Encoding encoder, int expectedLength,
out int[] metadata)
{
var messageString = encoder.GetString(message, 0, message.Length);
var messageStringSeparated = messageString.Split('x');
if (messageStringSeparated.Length != expectedLength)
{
metadata = new int[0];
return false;
}
var parameteres = new List<int>();
foreach (var messageStringPart in messageStringSeparated)
{
int value;
if (!int.TryParse(messageStringPart, out value))
{
metadata = new int[0];
return false;
}
parameteres.Add(value);
}
metadata = parameteres.ToArray();
return true;
}
}
Also ReadMessage with length looks like this:
public static byte[] ReadMessageWithLength(this Stream stream, int bufferSize = 2048)
{
if (stream == null)
{
throw new ArgumentNullException("stream");
}
var messageSizeBuffer = new byte[4];
if (stream.Read(messageSizeBuffer, 0, messageSizeBuffer.Length) < 1)
{
return new byte[0];
}
if (BitConverter.IsLittleEndian)
{
Array.Reverse(messageSizeBuffer);
}
var totalBytesToRead = BitConverter.ToInt32(messageSizeBuffer, 0);
if (totalBytesToRead < 0)
{
throw new Exception("Number of bytes to read cannot be negative!");
}
using (var memoryStream = new MemoryStream())
{
var buffer = new byte[bufferSize];
int chunkBytesRead, totalBytesRead = 0;
while (
(chunkBytesRead = stream.Read(buffer, 0, Math.Min(totalBytesToRead - totalBytesRead, buffer.Length))) >
0)
{
memoryStream.Write(buffer, 0, chunkBytesRead);
totalBytesRead += chunkBytesRead;
if (totalBytesRead >= totalBytesToRead)
{
break;
}
}
return memoryStream.ToArray();
}
}
Now time for Java code. The function used to receive bitmaps:
public void requestBitmaps() {
if (socket == null || !socket.isConnected()) {
throw new IllegalStateException("Socket has to be initialized and connected to call this method!");
}
requestBitmapsThread = new Thread(new Runnable() {
#Override
public void run() {
try {
InputStream sis = new BufferedInputStream(socket.getInputStream());
OutputStream sos = new BufferedOutputStream(socket.getOutputStream());
writeMessageWithLength(sos, ServerCommand.PushBitmaps.getValueBytes());
byte[] initialMetadataBytes = readMessageWithLength(sis, socket.getReceiveBufferSize());
int[] initialMetadata = parseMetadataMessage(initialMetadataBytes, 3);
if (initialMetadata == null) {
listener.onBitmapReceivedEnd(false);
return;
}
listener.onBitmapReceivedBegin(initialMetadata[0], initialMetadata[1], initialMetadata[2]);
while (!Thread.currentThread().isInterrupted()) {
byte[] messageBytes = readMessageWithLength(sis, socket.getReceiveBufferSize());
if (ServerCommand.PushBitmapsDone.equals(ServerCommand.fromValueBytes(messageBytes))) {
listener.onBitmapReceivedEnd(true);
break;
}
int[] metadata = parseMetadataMessage(messageBytes, 3);
if (metadata == null) {
listener.onBitmapReceivedEnd(false);
break;
}
byte[] bitmapBytes = readMessageWithLength(sis, socket.getReceiveBufferSize());
listener.onBitmapReceived(metadata[0], metadata[1], metadata[2], bitmapBytes);
}
} catch (IOException e) {
e.printStackTrace();
}
}
});
requestBitmapsThread.start();
}
And also parseMetadataMessage method:
private int[] parseMetadataMessage(byte[] metadataBytes, int expectedLength) {
String metadataString = new String(metadataBytes, Charset.forName("UTF-8"));
String[] metadataStringSeparated = metadataString.split("x");
if (metadataStringSeparated.length != expectedLength) {
return null;
}
try {
int[] parameters = new int[metadataStringSeparated.length];
for (int i = 0; i < metadataStringSeparated.length; i++) {
parameters[i] = Integer.parseInt(metadataStringSeparated[i]);
}
return parameters;
} catch (NumberFormatException e) {
e.printStackTrace();
return null;
}
}
And readMessageWithLength:
private byte[] readMessageWithLength(InputStream is, int bufferLength) throws IOException {
byte[] messageLengthBuffer = new byte[4];
if (is.read(messageLengthBuffer) < 0) {
return null;
}
ByteBuffer messageLengthByteBuffer = ByteBuffer.wrap(messageLengthBuffer).order(ByteOrder.BIG_ENDIAN);
int totalBytesToRead = messageLengthByteBuffer.getInt();
if (totalBytesToRead < 0) {
throw new IllegalStateException("Total bytes to read cannot be less than 0!");
}
ByteArrayOutputStream bitmapBytesBuffer = null;
try {
bitmapBytesBuffer = new ByteArrayOutputStream();
byte[] buffer = new byte[bufferLength];
int chunkBytesRead, totalBytesRead = 0;
while ((chunkBytesRead = is.read(buffer, 0, Math.min(totalBytesToRead - totalBytesRead, buffer.length))) > 0) {
bitmapBytesBuffer.write(buffer, 0, chunkBytesRead);
totalBytesRead += chunkBytesRead;
if (totalBytesRead >= totalBytesToRead) {
break;
}
}
return bitmapBytesBuffer.toByteArray();
} finally {
if (bitmapBytesBuffer != null) {
bitmapBytesBuffer.close();
}
}
}
If you have come this far, thanks for attention. I appreciate any feedback and I hope it will be possible to speed up the code.
Best regards,
Bartosz
I have the following class:
class MultiWebProgram
{
static void Main(string[] args)
{
string[] websites = new string[]
{
"https://www.google.com/search?q=one",
"https://www.google.com/search?q=two",
"https://www.google.com/search?q=three",
"https://www.google.com/search?q=four"
};
Task<int>[] taskList = new Task<int>[websites.Length];
int i = 0;
foreach(var website in websites)
{
taskList[i] = Task<int>.Run(() => GetSiteBytes(website));
i++;
}
Task.WaitAll(taskList);
}
public static int GetSiteBytes(string website)
{
WebClient client = new WebClient();
var stream = client.OpenRead(new Uri(website));
byte[] buffer = new byte[4096];
int totalBytes = 0;
int bytesRead = 0;
do
{
bytesRead = stream.Read(buffer, 0, 4096);
totalBytes += bytesRead;
}
while (bytesRead >= 4096);
Console.WriteLine("Got {0} bytes from {1}", totalBytes, website);
return totalBytes;
}
}
When I run this, the code blocks on the WaitAll command.
I tried changing the program to this:
class MultiWebProgram
{
static void Main(string[] args)
{
string[] websites = new string[]
{
"https://www.google.com/search?q=one",
"https://www.google.com/search?q=two",
"https://www.google.com/search?q=three",
"https://www.google.com/search?q=four"
};
Task<int>[] taskList = new Task<int>[websites.Length];
int i = 0;
foreach(var website in websites)
{
taskList[i] = GetSiteBytesAsync(website);
i++;
}
Task.WaitAll(taskList);
}
public async static Task<int> GetSiteBytesAsync(string website)
{
WebClient client = new WebClient();
var stream = await client.OpenReadTaskAsync(new Uri(website));
byte[] buffer = new byte[4096];
int totalBytes = 0;
int bytesRead = 0;
do
{
bytesRead = await stream.ReadAsync(buffer, 0, 4096);
totalBytes += bytesRead;
}
while (bytesRead >= 4096);
Console.WriteLine("Got {0} bytes from {1}", totalBytes, website);
return totalBytes;
}
}
In other words, I made the GetSiteBytes method async - but it made no difference.
One thing I did notice; when I started up Fiddler and ran either version of the program, there was no blocking.
What could be going on here?
** EDIT: **
The problem seemed to be the OpenRead method. When I used DownloadData/DownloadDataAsync, the blocking went away.
You forgot about closing the stream after reading all data. Try this and it will work:
using (var stream = client.OpenRead(new Uri(website)))
{
...
}
I'm not sure what OpenRead method does under the hood but I assume that it allocates some resources which are limited and you cannot use more than is available. Personally I always assume that if a method returns a stream, then I have to close it.
Whenever I try to upload a large video via Direct Upload using the YouTube API. I get an OutOfMemory Exception. Is there anything I can do to get rid of this? The YouTube API does not say anything about video size limit using direct upload.
I gave up on the Direct Upload. Now I trying the resumable upload way. My code is below.
YouTubeRequest request;
YouTubeRequestSettings settings = new YouTubeRequestSettings("YouTube Upload", Client Key, "Username", "Password");
request = new YouTubeRequest(settings);
Video newVideo = new Video();
ResumableUploader m_ResumableUploader = null;
Authenticator YouTubeAuthenticator;
m_ResumableUploader = new ResumableUploader(256); //chunksize 256 kilobyte
m_ResumableUploader.AsyncOperationCompleted += new AsyncOperationCompletedEventHandler(m_ResumableUploader_AsyncOperationCompleted);
m_ResumableUploader.AsyncOperationProgress += new AsyncOperationProgressEventHandler(m_ResumableUploader_AsyncOperationProgress);
YouTubeAuthenticator = new ClientLoginAuthenticator("YouTubeUploader", ServiceNames.YouTube, "kjohnson#resoluteinnovations.com", "password");
//AtomLink link = new AtomLink("http://uploads.gdata.youtube.com/resumable/feeds/api/users/uploads");
//link.Rel = ResumableUploader.CreateMediaRelation;
//newVideo.YouTubeEntry.Links.Add(link);
System.IO.FileStream stream = new System.IO.FileStream(filePath, System.IO.FileMode.Open, System.IO.FileAccess.Read);
byte[] chunk = new byte[256000];int count = 1;
while (true) {
int index = 0;
while (index < chunk.Length) {
int bytesRead = stream.Read(chunk, index, chunk.Length - index);
if (bytesRead == 0) {
break;
}
index += bytesRead;
}
if (index != 0) { // Our previous chunk may have been the last one
newVideo.MediaSource = new MediaFileSource(new MemoryStream(chunk), filePath, "video/quicktime");
if (count == 1) {
m_ResumableUploader.InsertAsync(YouTubeAuthenticator, newVideo.YouTubeEntry, new MemoryStream(chunk));
count++;
}
else
m_ResumableUploader.ResumeAsync(YouTubeAuthenticator, new Uri("http://uploads.gdata.youtube.com/resumable/feeds/api/users/uploads"), "POST", new MemoryStream(chunk), "video/quicktime", new object());
}
if (index != chunk.Length) { // We didn't read a full chunk: we're done
break;
}
}
Can anyone tell me what is wrong? My 2 GB video not uploading.
The reason I was getting a 403 Forbidden error was due to the fact that I was not passing in:
Username & Password
A developer key
The request variable in the code above is not being used/sent in the upload. Therefore I was doing an unauthorized upload.
Chances are that you are not disposing your objects. Ensure all disposable objects are within a using statement..
For example, this code will upload a large zip file to a server:
try
{
using (Stream ftpStream = FTPRequest.GetRequestStream())
{
using (FileStream file = File.OpenRead(ImagesZipFile))
{
// set up variables we'll use to read the file
int length = 1024;
byte[] buffer = new byte[length];
int bytesRead = 0;
// write the file to the request stream
do
{
bytesRead = file.Read(buffer, 0, length);
ftpStream.Write(buffer, 0, bytesRead);
}
while (bytesRead != 0);
}
}
}
catch (Exception e)
{
// throw the exception
throw e;
}
I'm trying to securely transfer files between 2 devices, so I'm using an SslStream attached to a TcpClient. Documents and text files come across just fine, but image files don't show up correctly. The following is the server code:
listener = new TcpListener(IPAddress.Any, 1337);
listener.Start();
while (true)
{
TcpClient client = listener.AcceptTcpClient();
SslStream sslStream = new SslStream(client.GetStream(), false, new RemoteCertificateValidationCallback(CertificateValidationCallback), new LocalCertificateSelectionCallback(CertificateSelectionCallback));
var certificate = Connection.GetClientCertificate(((IPEndPoint)client.Client.RemoteEndPoint).Address.ToString());
try
{
sslStream.AuthenticateAsServer(certificate, true, SslProtocols.Default, true);
sslStream.ReadTimeout = 5000;
sslStream.WriteTimeout = 5000;
var messageData = ReadMessage(sslStream);
var mode = messageData[0];
var tokenBytes = messageData.Splice(1, 16);
var fileNameBytes = messageData.Splice(17, 128);
var fileBytes = messageData.Splice(146);
var fileName = Encoding.ASCII.GetString(fileNameBytes).TrimEnd('\0');
using (var tempFile = new FileStream(fileName, FileMode.OpenOrCreate, FileAccess.Write))
{
tempFile.Write(fileBytes, 0, fileBytes.Length);
tempFile.Flush();
}
if (mode == 0)
tempFiles.Add(fileName);
Process.Start(fileName);
}
catch (AuthenticationException e)
{
MessageBox.Show("The other side failed to authenticate.");
}
finally
{
sslStream.Close();
client.Close();
}
}
And ReadMessage is defined as follows:
private static byte[] ReadMessage(SslStream sslStream)
{
byte[] buffer = new byte[2048];
MemoryStream stream = new MemoryStream();
int bytes = -1;
while (bytes != 0)
{
bytes = sslStream.Read(buffer, 0, buffer.Length);
stream.Write(buffer, 0, bytes);
}
return stream.ToArray();
}
And then the client code is this:
TcpClient client = new TcpClient();
client.Connect(new IPEndPoint(IPAddress.Parse(ip), 1337));
SslStream sslStream = new SslStream(client.GetStream(), false, new RemoteCertificateValidationCallback(CertificateValidationCallback), new LocalCertificateSelectionCallback(CertificateSelectionCallback));
var certificate = Connection.GetClientCertificate(ip);
try
{
sslStream.AuthenticateAsClient(ip, new X509CertificateCollection() { certificate }, SslProtocols.Default, false);
sslStream.ReadTimeout = 5000;
sslStream.WriteTimeout = 5000;
sslStream.Write(data);
}
catch (AuthenticationException e)
{
MessageBox.Show("The other side failed to authenticate.");
}
finally
{
sslStream.Close();
client.Close();
}
And the code that calls into it just does:
var fileBytes = File.ReadAllBytes(file);
var tokenBytes = Encoding.UTF8.GetBytes(token);
var fileNameBytes = Encoding.UTF8.GetBytes(Path.GetFileName(file));
var buffer = new byte[145 + fileBytes.Length];
buffer[0] = 1;
for (int i = 0; i < 16; i++)
{
buffer[i + 1] = tokenBytes[i];
}
for (int i = 0; i < fileNameBytes.Length; i++)
{
buffer[i + 17] = fileNameBytes[i];
}
for (int i = 0; i < fileBytes.Length; i++)
{
buffer[i + 145] = fileBytes[i];
}
SocketConnection.Send(ip, buffer);
Is there anything inherently wrong with what I'm doing, or do I need to do something different for images?
EDIT: I have changed it to reflect the current code, and also, after doing a dump of the raw bytes on both ends, it looks like for some reason the bytes are getting rearranged when they come over the wire. Is there any way to ensure that the bytes come across in the original order?
In ReadMessage you're writing bytes.Length bytes to the stream, regardless of the number of bytes that were actually read. Try:
private static byte[] ReadMessage(SslStream sslStream)
{
byte[] buffer = new byte[2048];
MemoryStream stream = new MemoryStream();
int bytes = -1;
while (bytes != 0)
{
bytes = sslStream.Read(buffer, 0, buffer.Length);
// Use "bytes" instead of "buffer.Length" here
stream.Write(buffer, 0, bytes);
}
return stream.ToArray();
}
Based on your follow-up, you're also taking the file data from the wrong point in the buffer, and so you're losing the first byte of the file.
Your code should be:
var fileBytes = messageData.Splice(145); // File data starts at 145, not 146
Is this possibly a conflict between endianness? If the bytes from the server are ABCDEF and the client is seeing the image bytes as BADCFE then that's the issue.
I haven't worked with image files, but when I read a short or an int instead of a String from the bytes coming in over the wire, I do something like this:
int intFromStream = IPAddress.NetworkToHostOrder(BitConverter.ToInt32(byteArrayWithLength4, 0));