Closed. This question needs debugging details. It is not currently accepting answers.
Edit the question to include desired behavior, a specific problem or error, and the shortest code necessary to reproduce the problem. This will help others answer the question.
Closed 3 years ago.
Improve this question
If it helps, I'm using the CampBX API to get funds in my account. I wrote the following code to make the API Call:
using (var wb = new WebClient())
{
String url = "https://CampBX.com/api/myfunds.php";
var data = new NameValueCollection();
data["user"] = "USERNAME";
data["pass"] = "PASSWORD";
var response = wb.UploadValues(url, "POST", data);
}
WebClient.UploadValues() returns a byte[] and I have no idea how to correctly parse that.
Here is the CampBX info, under Account Balances.
Simply put, you need to use a JSON parser. Personally I like Newtonsoft.Json and it is what I will use in this example.
The first step is converting the byte[] to a sequence of characters, either a string object or a TextReader. The second step is to pass this information to the parser. So, in your case, the code would look something like this:
JToken parsedToken;
using (var responseReader = new StreamReader(new MemoryStream(response))) {
parsedToken = JToken.ReadFrom(responseReader);
}
The parsedToken object can then be used to extract whatever data you need. (See the documentation for information on extracting data from a JToken object.)
Note that WebClient.UploadValues() discards the information regarding the response entity's character encoding. StreamReader will use UTF-8 encoding by default, which is sufficient to parse UTF-8 or ASCII. Depending on the JSON encoder used by the server, the response may always be ASCII-compatible, so you might not have to worry about it. Nevertheless, it's something you should investigate.
DataContractJsonSerializer built-in object will be your friend here, provided you know what is the internal structure of the returned object (or can at least guess it, from the JSON).
The steps are:
Define a contract class to hold the deserialized JSON object
namespace AppNameSpace
{
[DataContract] /* Place this inside your app namespace */
internal class iResponse /*Name this class appropriately */
{
[DataMember]
internal string field1;
[DataMember]
internal string field2;
[DataMember]
internal Int32 field3;
}
...
}
The actual parsing itself is about three lines
DataContractJsonSerializer ser = new DataContractJsonSerializer(typeof(iOpenParams));
MemoryStream stream1 = new MemoryStream(response);
iResponse resp_json = (iResponse)ser.ReadObject(stream1);
For more details and examples, refer to: http://msdn.microsoft.com/en-us/library/bb412179.aspx
My solution is simpler:
Object retorno;
var response = wb.UploadValues(url, "POST", data);
using (var responseReader = new StreamReader(new MemoryStream(response)))
{
retorno = JsonConvert.DeserializeObject<Object>(responseReader.ReadToEnd());
}
Related
Suppose you need to read a large string from a stream and you want to put that string into a flatbuffer.
Currently what I do is read the stream into a string and then use the FlatbufferBuilder.CreateString(string s) function.
This works fine but it does have as a drawback that the string is copied and loaded into memory twice: once by reading it from the stream into the string; and then a second time the string is copied into the flatbuffer.
I was wondering if there is a way to fill the flatbuffer string directly from a stream?
For a more concrete example:
Suppose your flatbuffer schema looks like:
table Message
{
_Data: string;
}
root_type Message;
We can then create a flatbuffer like this (with myData a string)
var fbb = new FlatBufferBuilder(myData.Length);
var dataOffset = fbb.CreateString(myData);
var message = Message.CreateMessage(fbb, dataOffset);
Message.FinishMessageBuffer(fbb, message);
So the question is can we somehow do the same thing, where myData is a System.IO.Stream?
Obviously the following works, but I'd like to avoid first reading the Stream into memory.
using (var reader = new StreamReader(myStream)
{
var myData = reader.ReadToEnd();
var fbb = new FlatBufferBuilder(myData.Length);
var dataOffset = fbb.CreateString(myData);
var message = Message.CreateMessage(fbb, dataOffset);
Message.FinishMessageBuffer(fbb, message);
}
There is currently no way to avoid that copy twice, afaik.. it should be relatively simple to implement a version of CreateString that takes a stream and reduces it to one copy. You could have a go at that and open a PR on github with the result.
I searched for an answer to this question but I didn't see many specific answers to it. My question is relatively simple: given data serialized in the protobuf format, is it possible to deserialize it without knowing the data type?
It seems that I can serialize any type without knowing what it is by doing:
public static byte[] SerializeData<T>(T arg)
{
try
{
using (var testStream = new MemoryStream())
{
Serializer.Serialize(testStream, arg);
....
Is there a way to do this such that Serializer.Deserialize(stream) (or something of that sort)?
In the following code i want to use a predefined protobuf message in c#. I found that I was able to write and use the method to take a method that has been created and make a byte[]:
ContainerMessage containerMessage = new ContainerMessage();
containerMessage.Type = CommandType.Connect;
containerMessage.Connect = new Connect();
containerMessage.Connect.ClientName = "TemporaryClientName";
byte[] stream = new byte[containerMessage.CalculateSize()];
using (Google.Protobuf.CodedOutputStream outstream = new Google.Protobuf.CodedOutputStream(stream))
{
containerMessage.WriteTo(outstream);
}
This works as expected and i can inspect the message and the values are as expected as are the values in the byte[]. But if I try to Deserialize even this simple byte[] that i have just created:
using (Google.Protobuf.CodedInputStream instream = new Google.Protobuf.CodedInputStream(stream))
{
instream.ReadMessage(containerMessage);
}
It fails with:
An unhandled exception of type 'Google.Protobuf.InvalidProtocolBufferException' occurred in Google.Protobuf.dll
Additional information: Protocol message contained an invalid tag (zero).
Is this way of deserializing from a byte[] correct for protobuf?
The Protobuf Definition is:
message ContainerMessage {
CommandType type = 1;
bool commandSuccess = 2;
oneof message {
Connect connect = 3;
}
}
enum CommandType {
START = 0;
CONNECT = 2;
}
message Connect {
string ClientName = 1;
uint32 PushPullPort = 2;
}
And the CS file is generated with the command line:
protoc.exe -I=%CD% --csharp_out=..\GeneratedCsMessaging\ Connect.proto
The CodedOutputStream and CodedInputStream are mainly intended to be used by the compiled proto classes. The API for CodedOutputStream states such and mentions that if you want to have manually-written code calling either of both classes you need to use their WriteTag method before each value.
However, since you want to use the Google Protobuf for serializing and parsing any System.IO.Stream will do the job just like intended. This is very well documented and described in the Parsing and serialization section of the Protocol Buffer Basics for C#. The examples which can be found in Google Protobuf's Github can be quite helpful for getting the hang of Google Protobuf quickly. There you can see that a MemoryStream is used to serialize the object while the Parse.ParseFrom method can be used to parse an object out of the serialized data.
As you've mentioned in the comments to your question using Google.Protobuf; is an essential part to be able to use Google Protobuf's features.
EDIT: A sample usage in your case could look something like this
byte[] serializedBytes;
ContainerMessage containerMessage = new ContainerMessage()
{
Connect = new Connect()
{
ClientName = "TemporaryClientName",
},
Type = CommandType.Connect,
};
using( MemoryStream stream = new MemoryStream())
{
containerMessage.WriteTo(stream);
serializedBytes = stream.ToArray();
}
ContainerMessage parsedCopy = ContainerMessage.Parser.ParseFrom(serializedBytes);
This is related to my question HTTPClient Buffer Exceeded 2G; Cannot write more bytes to the buffer but is different enough that IMO it warrants a separate question.
In the other question, I'm trying to figure out how to deal with breaking the 2G request buffer. The idea was to use streaming, but I need to deserialize. In talking to Professor Google, I found that I have to use TextReader to stream/deserialize. so my code for that is:
public async Task<API_Json_Special_Feeds.RootObject> walMart_Special_Feed_Lookup(string url)
{
special_Feed_Lookup_Working = true;
HttpClientHandler handler = new HttpClientHandler()
{
AutomaticDecompression = DecompressionMethods.GZip | DecompressionMethods.Deflate
};
using (HttpClient http = new HttpClient(handler))
{
http.DefaultRequestHeaders.AcceptEncoding.Add(new System.Net.Http.Headers.StringWithQualityHeaderValue("gzip"));
http.Timeout = TimeSpan.FromMilliseconds(Timeout.Infinite);
url = String.Format(url);
using (var response = await http.GetAsync(url, HttpCompletionOption.ResponseHeadersRead))
{
Console.WriteLine(response);
var serializer = new JsonSerializer();
using (StreamReader sr = new StreamReader(await response.Content.ReadAsStreamAsync()))
{
using (var jsonTextReader = new JsonTextReader(sr))
{
API_Json_Special_Feeds.RootObject root = (API_Json_Special_Feeds.RootObject)serializer.Deserialize(jsonTextReader);
return root;
}
}
}
}
}
Now, as you can see, the return type is strongly typed. The return type of the method matches. Now, I go to the calling line:
API_Json_Special_Feeds.RootObject Items = await net.walMart_Special_Feed_Lookup(specialFeedsURLs[i].Replace("{apiKey}", Properties.Resources.API_Key_Walmart));
So, we have matching types API_Json_Special_Feeds.RootMethod all the way around.
When run, the calling line throws an InvalidCastException:
Undesired Result:
Unable to cast object of type 'Newtonsoft.Json.Linq.JObject' to type 'RootObject'
I've checked at the end of the method before return, and the result is indeed cast from an object to API_Json_Special_Feeds.RootMethod before being returned.
Question: somewhere between the return statement and the calling line, the object being returned is being converted from an API_Json_Special_Feeds.RootMethod to a Newtonsoft.Json.Linq.JObject. I can't debug it since there's no code in between. If I cast again in the calling line, I get a "Cannot cast" error. How can I prevent the degradation/changing of this object type?
Many, many thinks for your time, consideration, and for any thoughts or suggestions you can provide!
You need to use the generic overload JsonSerializer.Deserialize<T>()
var root = serializer.Deserialize<API_Json_Special_Feeds.RootObject>(jsonTextReader);
Unlike files generated by BinaryFormatter, JSON files generally do not include c# type information, so it's necessary for the receiving system to specify the expected type.
(There are extensions to the JSON standard in which c# type information can be included in a JSON file -- e.g. Json.NET's TypeNameHandling -- but it is still necessary to deserialize the JSON to an appropriate explicit base class.)
See Deserialize JSON from a file for another example of deserializing a strongly typed c# object from a stream.
I have a legacy app that uses SoapFormatter to persist a graph of objects (maybe 50 different classes). I want to move away from using this as it is deprecated, and increasingly hard to continue to support deserializing from old files as the classes change.
I want to use DataContractSerializer going forward. Does anyone have any suggestions as to a good strategy for migration? I need to continue to be able to deserializing old files written by SoapFormatter...
Thanks
I don't think you want to be limited to a backward-compatible format.
So you will need to distinguish old and new content. And easy method would be :
Old Format: <soapdata>
New Format: <header> <newdata>
And in your new Load() method:
(Try to) Read the header
If a header is found, continue to read the new format
else re-position to begin and use the SOAP formatter to read.
The simplest code would be to try deserialize with DataContractSerializer and fallback to SoapFormatter if it fails.
The save part will always use the DataContractSerializer, so that your new objects or the updated ones will use your new supported version.
public MyContract Deserialize(string file)
{
try
{
using (var stream = loadFile())
{
return loadWithDataContractSerializer(stream);
}
}
catch (SerializationException)
{
using (var stream = openForRead(file))
{
return convertToContract(loadWithSoapFormatter(stream));
}
}
}
private MyContract loadWithDataContractSerializer(Stream s);
private MyOldObject loadWithSoapFormatter(Stream s);
private MyContract convertToContract(MyOldObject obj);
public void Serialize(string file, MyContract data)
{
using (var stream = openForWrite(file))
{
writeWithDataContractSerializer(stream, data);
}
}
Of course, it might be possible to implement a custom logic to allow DataContractSerializer to understant the SoapFormatter structure, but you will have to provide a lot more work.