How to fix OutOfMemoryException when serializing to JSON string? - c#

I have a Class object which contains a property as byte array in it. This class is a Data Contract to my REST service. The byte array property will take any document whose Max Size is limited to 500MB. When I was trying to consume this service and serializing the object I am getting the Memory Out of exception error. Please find the below image
Below is the code snippet
public static string SerializeJSon<T>(T t)
{
MemoryStream stream = new MemoryStream();
DataContractJsonSerializer ds = new DataContractJsonSerializer(typeof(T));
DataContractJsonSerializerSettings s = new DataContractJsonSerializerSettings();
DateTimeFormat dt = new DateTimeFormat("MM/dd/yyyy");
s.DateTimeFormat = dt;
**ds.WriteObject(stream, t);**
string jsonString = Encoding.UTF8.GetString(stream.ToArray());
stream.Close();
return jsonString;
}

Try "do not use 500mb documents in web service calls". THs is the core problem - you try to use a method call mechanism to transport half a gigabyte of data that likely turns into some gigabyte of in memory objects. This is not what web services are designed to do.

Related

Deserialize JSON from Stream depending on first ~16 characters in .NET Core

I am getting a Gziped JSON which I check for a specific start string and if it's correct I deserialize into my object:
await using var decompressionStream = new GZipStream(externalStream, CompressionMode.Decompress);
var streamReader = new StreamReader(decompressionStream);
string eventString = streamReader.ReadToEnd();
if (!eventString.StartsWith("SomeString"))
{
// Drop it because it can't be deserialized
return;
}
MyObject item = Utf8Json.JsonSerializer.Deserialize<MyObject>(Encoding.UTF8.GetBytes(eventString));
This approach works but it seems to be wasteful to turn the stream into a string and back into an bytes array when Utf8Json can deserialize directly from a stream.
Because 3/4 of incoming streams contain data that is of a different type and can't be deserialized I don't want to put a try/catch block around it because that many exceptions would be too expensive.
Is there a way to peek into the first ~16 chars without consuming them?

DataContractJsonSerializer: serialize object or array (can be both)

I'm accessing the tomtom json api, and the api either returns me an array of objects, or a single object, when an error has happen.
Example:
[{"driverno": "...
Error Example:
{"errorCode": "8011","errorMsg": "request quota reached, error code: 8011"}
The data is accessed WebRequest, WebResponse and they return a stream, which can then be passed to a DataContractJsonSerializer. However, I can't create a serialization class, which accepts both forms of JSON, and the stream can't be passed twice, because the seek function is not supported.
Is there a way, to create a serialization class which supports both types of JSON input?
I found a workaround, where I copy the Stream to a MemoryStream, which enables seeking. I'm not completly settisfied with th solution, becuase it does a Stream copying and the DataContractJsonSerializer twice.
Sample:
string text = File.ReadAllText(PAHT);
text = Regex.Replace(text, "\\{[\\n\\r ]*\"__type", "{\"__type");
// copy to MemoryStream
using (MemoryStream dataStream = new MemoryStream(Encoding.UTF8.GetBytes(text)))
{
DataContractJsonSerializer errorDeserializer = new DataContractJsonSerializer(typeof(RequestError));
RequestError errorSerilaized = (RequestError)errorDeserializer.ReadObject(dataStream);
// check if an error happened
if (errorSerilaized.errorCode == null)
{
// seek the stream to position 0
dataStream.Position = 0;
DataContractJsonSerializer _deserializer = new DataContractJsonSerializer(typeof(NoneErrorSerializationClass));
NoneErrorSerializationClass tripReportsSerialized = (NoneErrorSerializationClass)_deserializer.ReadObject(dataStream);
// ...
}
else
{
MessageBox.Show(errorSerilaized.errorMsg);
}
}

c# - deserializing large object

I have a chat that has a file sharing system that I built by slightly modifying monotorrent.
When a user shares a file the client serializes the Monotorrent.common.torrent object (represents a .torrent file) and sends it to the server inside of another object and the server deserialize it. This works only when the file that the user shares is small(about less than 1 MB). When its larger the server gives the following exeption:
Binary stream '0' does not contain a valid BinaryHeader. Possible
causes are invalid stream or object version change between
serialization and deserialization.
This is my deserialization code:
public CommendData ByteArrayToCommendData(byte[] arrBytes)
{
using (MemoryStream memStream = new MemoryStream(arrBytes))
{
BinaryFormatter binForm = new BinaryFormatter();
memStream.Seek(0, SeekOrigin.Begin);
CommendData obj = (CommendData)binForm.Deserialize(memStream);
return obj;
}
}
(CommendData contains the Monotorrent.common.torrent object in this instance)

Deserializing from SQL Server Compact throws a "There is an unclosed literal string." error

I am trying to store some objects in an SQL Server Compact (SQL CE) database by serializing them with a SOAP formatter. Serializing seems to work just fine, but when I try to deserialize the object I get an error saying
There is an unclosed literal string. Line 53, position 72.
Furthermore, after restarting the application on attempting to fill the dataset I get the following error:
Failed to enable constraints. One or more rows contain values violating non-null, unique, or foreign-key constraints.
All my columns (except the ID) allow for null values and are non-unique, so I have no idea where this comes from. Here is the code of my serializer:
public static class Serializer
{
static public string Serialize(AssessmentReport theObject)
{
MemoryStream mStream = new MemoryStream();
SoapFormatter formatter = new SoapFormatter();
formatter.Serialize(mStream, theObject);
byte[] buffer = mStream.ToArray();
mStream.Close();
string value = Encoding.UTF8.GetString(buffer);
return value;
}
static public AssessmentReport Deserialize(string value)
{
byte[] buffer = Encoding.UTF8.GetBytes(value);
MemoryStream mStream = new MemoryStream(buffer);
SoapFormatter formatter = new SoapFormatter();
mStream.Position = 0;
AssessmentReport theReport = (AssessmentReport)formatter.Deserialize(mStream);
mStream.Close();
return theReport;
}
}
Here is how I call the serializer (theReport is an instance of the object to be serialized):
examTableAdapter.UpdateAsmFile(Serializer.Serialize(theReport), examID);
And here is how I am calling the deserializing method:
string value = Convert.ToString(examTableAdapter.GetAsmFile(2));
AsmReport theReport = Serializer.Deserialize(value)
The field in the SQL Server Compact database where the string is saved is of type nvarchar with a limit of 3500.
I tried using a binary formatter, but when serializing it seems to always return an empty byte[] buffer. I really need deep serializing, that's why the XML serializer is out of question.
Ok so this one I have been trying to figure out for more than two months now.
While I could not find a logical solution to the problem, it seems that changing the encoding to utf7 fixed the problem. I can't think of a reason why this would be the issue, but it seems to be specific to my machine (I finally had the opportunity to run the code on another computer and it worked perfectly with utf8).

How to send a XmlSerializer Class to a WebService and then Deserialize it?

I want to be able to send a XmlSerializer class (which is generated obvious in remote C# application) over a WebService that will then deserialize it into a class. (I didnt know it its possible either)
My class is:
SystemInfo
I'm serializing it this way:
XmlSerializer mySerializer = new XmlSerializer(typeof(SystemInfo));
StreamWriter myWriter = new StreamWriter(textBox1.Text);
mySerializer.Serialize(myWriter, sysinfo);
How can i build the WebService?
[WebMethod]
public void Reports(XmlSerializer xmlSerializer)
{
.
.
.
}
Can anyone help me out?
Regards
First I assume you want to pass arbitrary types to a single web method, where the types are shared by the client and the server.
There is not much point in sending the XmlSerializer, it only has the logic to serialize/deserialize. It does not have the actual data, that is either read/written to a stream. What you should do is pass either a string or and XmlNode.
The caller of the web service can then a client side instance of XmlSerializer and serialize the object to a string, then call the web method passing the string as an argument. The web method it self can then create an instance of a XmlSerializer and deserialize the string back into an object. Of course to create the server size instance of the serializer you will need to know the root type to create the serializer for, you can pass this as a type name and use Type.GetType() to get the correct type to pass to the XmlSerializer.
If you know upfront which types you are going to be passing then you could also declare your web method more strongly typed and explicitly create methods for the types you expect to recieve.
If the wire format is not too much of a concern, you could also user SoapFormatter or a BinaryFormatter to handle the serialization/deserialization. In the later case of the BinaryFormatter you would declare your web method to take a byte[] argument, the advantage of these formatters (serializers) is that they do not need additional info on the type when you create the instance of the formatter, but they can be slower than an XmlSerializer
Update: Added some simple examples (Untested)
Example using an XmlSerializer, here you will need to pass the type name from the client side, so I made it an additional argument.
[WebMethod]
public void Reports(string xml, string typeName)
{
XmlSerializer xs = new XmlSerializer(Type.GetType(typeName));
object obj = xs.Deserialize(new StringReader(xml));
// use the deserialize object
}
Example using a BinaryFormatter, no type name needed but you the class types will need to be serializable
[WebMethod]
public void Reports(byte[] data)
{
BinaryFormatter bf = new BinaryFormatter();
object obj = bf.Deserialize(new MemoryStream(data));
// use the deserialized object
}
On the client side you would use something like the following to serialize using the BinaryFormatter.
// initialize the SystemInfo instance that you want to pass to the server
SystemInfo si = new SystemInfo() { SystemName = "My System" };
// Serialize to a memory stream
BinaryFormatter bf = new BinaryFormatter();
MemoryStream ms = new MemoryStream();
bf.Serialize(ms, si);
// Call the service, passing the array from the memory stream
ws.Reports(ms.ToArray());
Chris, thanks for helping me out. It was a major step forward.
I solved the problem sending the xml string:
SystemInfo sysinfo = new SystemInfo();
sysinfo.RUN();
XmlSerializer mySerializer = new XmlSerializer(typeof(SystemInfo));
MemoryStream memStream = new MemoryStream();
mySerializer.Serialize(memStream, sysinfo);
memStream.Seek(0, System.IO.SeekOrigin.Begin);
XmlDocument doc = new XmlDocument();
doc.Load(memStream);
memStream.Close();
localhost.WS_Agente dasdsa = new localhost.WS_Agente();
dasdsa.Reports(doc.InnerXml);
And the WebService:
[WebMethod]
public void Reports(string xml)
{
XmlSerializer mySerializer = new XmlSerializer(typeof(SystemInfo));
SystemInfo obj = (SystemInfo)mySerializer.Deserialize(new StringReader(xml));
}
Its working like a charm now :)
My question is: Can i improve the code?
Thanks

Categories

Resources