I'm trying to set up a very basic ZeroMQ-based socket link between Python server and C# client using simplejson and Json.NET.
I try to send a dict from Python and read it into an object in C#. Python code:
message = {'MessageType':"None", 'ContentType':"None", 'Content':"OK"}
message_blob = simplejson.dumps(message).encode(encoding = "UTF-8")
alive_socket.send(message_blob)
The message is sent as normal UTF-8 string or, if I use UTF-16, as "'\xff\xfe{\x00"\x00..." etc.
Code in C# is where my problem is:
string reply = client.Receive(Encoding.UTF8);
The UTF-8 message is received as "≻潃瑮湥≴›..." etc.
I tried to use UTF-16 and the message comes through OK, but the first symbols are still the little-endian \xFF \xFE BOM so when I try to feed it to the deserializer,
PythonMessage replyMessage = JsonConvert.DeserializeObject<PythonMessage>(reply);
//PythonMessage is just a very simple class with properties,
//not relevant to the problem
I get an error (obviously occurring at the first symbol, \xFF):
Unexpected character encountered while parsing value: .
Something is obviously wrong in the way I'm using encoding. Can you please show me the right way to do this?
The byte-order-mark is obligatory in UTF-16. You can use UTF-16LE or UTF-16BE to assume a particular byte order and the BOM will not be generated. That is, use:
message_blob = simplejson.dumps(message).encode(encoding = "UTF-16le")
Related
I am trying to use the FileReader to obtain the base-64 representation of an image and submit that to a .net WebApi service for image uploading.
My problem is that the contents of fileReader.result are not valid as a base-64 encoded image, at least according to .net.
I am just using a very simple method, and testing with fiddler to post to the service. If I post the complete result string from filereader.result, I get an error "Invalid length for a Base-64 char array or string" when I try and read the string using FromBase64String.
public void Post([FromBody]string imgString)
{
var myString = imgString.Split(new char[]{','});
byte[] bytes = Convert.FromBase64String(myString[1]);
using (MemoryStream ms = new MemoryStream(bytes))
{
Image image = Image.FromStream(ms);
image.Save("myBlargyImage.jpg");
}
}
Is cut+paste into fiddler doing something to the string that I need to account for here, or is there something else I am not doing correctly? This seems like it should be straightforward: Encode the image as a string, send the string, decode the string, save the image.
For example, using filereader to display a preview of the image on the client, I get the following in filereader.result:
src="data:image/jpeg;base64,/9j/4AAQSkZJRgABAgEAyADIAAD/...oBUA00AqYL/AMCg3//Z"
I have tried both sending the entire string ("data...Z"), and just the Base64 string. Currently, I am splitting the string server side to get the Base64 string. Doing this, I always get the invalid length error.
Alternatively, I have tried sending just the base64 string. Not knowing if the leading / was actually part of the string or not, I deleted it in the post body. Doing THIS, I can actually read the value into a byte array, but then I get an error using Image.FromStream that the array is not a valid image.
So, either I get an error that the entire string as provided by filereader is an invalid length, or, I hack it up and get an error that even if it is a valid length, it is not a valid image (when reading the bytearray). That is what makes me wonder if there is some issue of translation or formatting between the filereader.read, dev tools in chrome, then cutting and pasting into fiddler.
UPDATE:
I tried a more realistic implementation by just taking the filereader.result and putting it in a $.post() call, and it works as expected.
It appears I was right, that I, or notepad++, or fiddler, are doing something to the data when I touch it to cut and paste filereader.result into a service call.
If someone knows exactly what that might be, or how one can verify they are sending a valid base-64 encoding of an image to a service, it might help others who are attempting the same thing in the future.
Again, if in the browser filereader.result yielded 'data:image/jpeg;base64,somestring', I was simply copying that string from the developer tools panel, creating a fiddler call and in the request body including the copied string: "=data:image/jpeg;base64,somestring". Somehow, the base-64 'somestring' was getting munched in the cut+paste.
function readURL(input) {
if (input.files && input.files[0]) {
reader = new FileReader();
reader.onload = function (e) {
$('#imgPreview').attr('src', e.target.result);
$.post('/api/testy/t/4',
{'':e.target.result}
);
};
reader.readAsDataURL(input.files[0]);
reader.onloadend = function (e) {
console.log(e.target.result);
};
}
}
$("#imgUpload").change(function () {
readURL(this);
});
Don't forget to remove the 'noise' from a dataUrl,
For example in
data:image/png;base64,B64_DATA_HERE
you have to remove the data:image/png;base64, part before, so you process only the base 64 portion.
With js, it would be
var b64 = dataUrl.split("base64,")[1];
Hope this helps. Cheers
A data uri is not a base64 encode string, it may contain a base64 encoded string at the end of it. In this case it does, so you need to only send the base64 encoded string part.
var imagestr = datauri.split(',')[1];
sendToWebService(imagestr);
Make sure fiddler is not truncating the Base 64 String
I'm try to getting JSON data but getting Error.
Unexpected character encountered while parsing value: . Path '', line
0, position 0.
I'm using Net 4.5 and JSON.Net Framework
Here my code
WebClient net = new WebClient();
string str = await net.DownloadStringTaskAsync(url);
JObject o = JObject.Parse(str); // ERROR Here
And my code JSON Data View on Webservice
http://sv1.volcanosoft.com/test/index.php?area=ho-chi-minh
this site format index.php UTF-8 and header of php file
header('Content-Type:application/json; charset=utf-8');
echo '{"item":';
echo json_encode($data);
echo '}';
The downloaded string starts with two byte order marks (U+FEFF), which JSON.NET parser (correctly) doesn't understand.
The reason why the downloaded string contains two BOMs is because the data your service is sending contains 3 of them. The first one is removed automatically by UTF-8 encoding, but the two other remain.
BOM can be useful with files, where you can't store the charset used. But you are sending the charset used in a header, so you don't need to send BOM at all. And sending three of them is certainly incorrect.
I believe this is caused by BOMs in your PHP files, so you should probably remove them from there.
The Client sends a 1481 bytes array.
The server can read all the 1481 bytes message without any problems but by parsing the given messsage from the received binary array i get this exeption:
com.google.protobuf.InvalidProtocolBufferException: Protocol message contained an invalid tag (zero).
The binary data is the same. I checked that I am using the right version of the proto files. I am a bit at a loss tbh. Any help appreciated.
Code
byte [] data= IOUtils.toByteArray(br1, "ASCII");
System.out.println("SIZE:" + data.length);
AddressBook adb1 = AddressBook.parseFrom(data); System.out.println("Server: Addressbook:" + adb1.getPersonCount()); System.out.println("Server: Addressbook:" + adb1.getPerson(0).getName());
Question:
I need to find a way to correctly parse the received Adressbook msg from the read 1481 bytes arry.
Thanks.
This is the problem:
br1 = new InputStreamReader(s.getInputStream());
That's trying to treat opaque binary data as text. It's not text, it's binary data. So when you convert that Reader into a byte array, you've lost a load of the original data - no wonder it's an invalid protocol buffer.
Just use:
AddressBook adb1 = AddressBook.parseFrom(s.getInputStream());
and avoid the lossy text conversion. That's assuming you haven't got something equally broken on the C# side, of course.
If you must go via text, you should use base64 encoding on both sides.
Now it works I had same mistake by Serializing and Sending the Protocol Buffers Message
I have been trying to troubleshoot an issue and I really don't have any thoughts as to go about resolving the following problem:
I have a software application that is able to listen for a device that is capable of uploading data to a computer. Once this data is captured, it is written to a text file and stored for later use.
void DataRecieved(object sender, EventArgs e)
{
while ((bufferedData = comport.ReadLine()) != null)
{
uploadedData += bufferedData + Environment.NewLine;
}
comport.Close();
System.IO.StreamWriter writeUploadedPro = new System.IO.StreamWriter(uploadFilePath);
writeUploadedPro.Write(uploadedData);
writeUploadedPro.Close();
isUploadComplete = true;
}
I can establish, recieve, and verify a connection, and what I have programmed does indeed produce a text file of the uploaded data, however the data is contains is not complete.
Example:
%
N0?77??.5???3
G0? X3.??? Z4.5??6
Z5.??
?3.5?76
G01 Z5.??
Z4.9471
X?.?3 Z4.???9
Z?.???6
?3.?? Z?.???
Z4.????
X3.7??4
G?? X3.???? ?4.5??6
M30
?
It has numerous '?' which either should be a letter or digit. I have reaffirmed that the settings I have for the comport (Baud, Data, Stop, Parity, Handshake, and COM name are correctly specified). I have also tried setting the ReadBufferSize, RecievedBytesThreshold, Encoding, and NewLine settings. I am not at all familiar with these properties and I didn't find MSDN to be to helpful on explaining them either.
If you have any ideas or suggestions as to why I am getting incomplete lines of data in my upload, it would be most appreciated. Thank you for your assistance.
The problem is in your encoding. Try changing the Encoding property to UTF8. The default is ASCII. In ASCII encoding, any character greater than 0x7F gets converted into a ? by default because ASCII only goes up to 0x7F (127 decimal).
Though that might fix your problem, the better way to go would be to read the data into a byte array, then use one of these encoding classes to convert it into a proper string. The reason why that works is because you no longer converting the received bytes into a string. You're using the Read(byte[], int, int) overload which doesn't do a string conversion. Its pure bytes.
I think the problem lies in:
while ((bufferedData = comport.ReadLine()) != null)
Try changing the statement to something like:
while(comport.BytesToRead>0)
{
uploadedData += comport.ReadExisting() + Environment.NewLine;
}
Let me know if that helped.
I've recently started using the google translate API inside a c# project. I am trying to translate some text from english to french. I am having issues with some special characters though.
For example the word Company comes thru as Société instead of Société as it should. Is there some way in code I can convert these to the correct special characters? ie (é to é)
Thanks
If you need anymore info let me know.
I ran into this same exact issue. If you're using the WebClient class to download the json response from google, try setting the Encoding property to UTF8.
using(var webClient = new WebClient { Encoding = Encoding.UTF8 })
{
string json = webClient.DownloadString(someUri);
...
}
I have reproduced your problem, and it looks like you are using the UTF7 encoding. UTF8 is the way you need to go.
I use Google's API by creating a WebRequest to get an HTTP response from the server, then I read the response stream with a StreamReader. StreamReader defaults to UTF8, but to reproduce your problem, I passed Encoding.UTF7 into the StreamReader's constructor.