Develop Reference

protobuf-net - Serialize class and save it to object

I have been using a Xml serializer to serialize an class and saved it into an object which i later will send to a server. Due the amount of messages i send to the server i decided to change the serialization method into something that will result into something smaller in size.
I found protobuf-net but i only did find documentation about how to serialize a class into a file stream. It seems to me that saving to a file then send it to the server would not be very effective if you send over 100 packages every second.
So my question is , how can i serialize a class and save it into an object?

protobuf-net can write to (or read from) any Stream implementation. FileStream is just an example. In the case of communications between machines, this could be a NetworkStream. If you just want to get an in-memory form, then use MemoryStream. For example:
byte[] chunk;
using(var ms = new MemoryStream())
{
Serializer.Serialize(ms, obj);
chunk = ms.ToArray();
}
// now do something interesting with 'chunk'

Cannot Serialize Neurotec NTemplate using c# - protobuf-net

I have a Neurotec NTemplate with one Finger record. Now i want to Serialize it with c# - protobuf-net. I dont getting any exeption but my MemoryStream is emplty.
what might be the problem?
code Is below:(where tenPrintTemplate is a NTemplate)
tenPrintTemplate.AddFingers(fingerPrintTemplate.Save());
//start Proto Buffer serialization
MemoryStream stream = new MemoryStream();
RuntimeTypeModel.Default.InferTagFromNameDefault = true;
RuntimeTypeModel.Default.Add(typeof(NTemplate), false);
ProtoBuf.Serializer.Serialize<NTemplate>(stream, tenPrintTemplate);

Here you've told it not to apply any standard pattern / configuration logic:
RuntimeTypeModel.Default.Add(typeof(NTemplate), false);
so you have basically told it "serialize nothing". If you specify false, it expects you to tell it how you want it to work, for example by using Add on the MetaType that is returned. I suspect you could also just specify true if it has suitable attributes.
Note that 0 is a perfectly reasonable length for protobuf-net and an object that doesn't have anything interesting to mention on the wire.

I have got the solution of serializing Neurotec's NTemplate using C# Protobuf-net. i'm adding the solution code below. if anyone face the same problem please use it as your solution.
//Its a NTemplate of TenPrint
tenPrintTemplate.AddFingers(fingerPrintTemplate.Save());
//start Proto Buffer serialization
MemoryStream stream = new MemoryStream();
int tenpritnTemplateSize = tenPrintTemplate.GetSize();
NBuffer buffer = new NBuffer(tenpritnTemplateSize);
// saving fingers template to buffer.
tenPrintTemplate.Save(buffer);
ProtoBuf.Serializer.Serialize<byte[]>(stream, buffer.ToArray());

File Stream Handling in C#

I have a Web Application in which I am using asp:FileUpload Control to upload user XSD and reading its content(actually not uploading the XML Schema) using following Statement in a Click button function
using (StreamReader reader = new StreamReader(FileUploadControlName.FileContent))
this reader object is used in Click button function to read XMLSchema and thus I generate collection on basis of that.
I just wanted to know is there any way to use this StreamReader object again so that I can handle the manipulation on other control action in Web-Application.
Means can there be a way to write the reader into Memorystream and reuse the MemoryStream.
Sorry I am new to this.

You can reuse MemoryStream by setting the Position property to 0.
Example:
Stream s = new MemoryStream();
StreamReader sr = new StreamReader(s);
// later... after we read stuff
s.Position = 0;
And if you want to reuse the same object trough the application then you can use a Session variable. Then when you need to reuse it just set position to 0 again and read it with StreamReader.

By calling using the SteamReader will be disposed after the block ends, but not the stream itself. You can store the stream in a Session variable and reuse it like that, but I suggest you keep in mind to clear the Session variable. You can use MemoryStream or you can move the file to e TEMP location and store the file location in a Session variable. I would go with that option.

Memory stream is empty

I need to generate a huge xml file from different sources (functions). I decide to use XmlTextWriter since it uses less memory than XmlDocument.
First, initiate an XmlWriter with underlying MemoryStream
MemoryStream ms = new MemoryStream();
XmlTextWriter xmlWriter = new XmlTextWriter(ms, new UTF8Encoding(false, false));
xmlWriter.Formatting = Formatting.Indented;
Then I pass the XmlWriter (note xml writer is kept open until the very end) to a function to generate the beginning of the XML file:
xmlWriter.WriteStartDocument();
xmlWriter.WriteStartElement();
// xmlWriter.WriteEndElement(); // Do not write the end of root element in first function, to add more xml elements in following functions
xmlWriter.WriteEndDocument();
xmlWriter.Flush();
But I found that underlying memory stream is empty (by converting byte array to string and output string). Any ideas why?
Also, I have a general question about how to generate a huge xml file from different sources (functions). What I do now is keeping the XmlWriter open (I assume the underlying memory stream should open as well) to each function and write. In the first function, I do not write the end of root element. After the last function, I manually add the end of root element by:
string endRoot = "</Root>";
byte[] byteEndRoot = Encoding.ASCII.GetBytes(endRoot);
ms.Write(byteEndRoot, 0, byteEndRoot.Length);
Not sure if this works or not.
Thanks a lot!

Technically you should only ask one question per question, so I'm only going to answer the first one because this is just a quick visit to SO for me at the moment.
You need to call Flush before attempting to read from the Stream I think.
Edit
Just bubbling up my second hunch from the comments below to justify the accepted answer here.
In addition to the call to Flush, if reading from the Stream is done using the Read method and its brethren, then the position in the stream must first be reset back to the start. Otherwise no bytes will be read.
ms.Position = 0; /*reset Position to start*/
StreamReader reader = new StreamReader(ms);
string text = reader.ReadToEnd();
Console.WriteLine(text);

Perhaps you need to call Flush() on the xml stream before checking the memory streazm.

Make sure you call Flush on the XmlTextWriter before checking the memory stream.

very large string in memory

I am writing a program for formatting 100s of MB String data (nearing a gig) into xml == And I am required to return it as a response to an HTTP (GET) request .
I am using a StringWriter/XmlWriter to build an XML of the records in a loop and returning the
using (StringWriter writer = new StringWriter())
using (writer = XmlWriter.Create(writer, settings)) //where settings are the xml props
writer.ToString()
during testing I saw a few --out of memory exceptions-- and quite clueless on how to find a solution? do you guys have any suggestions for a memory optimized delivery of the response?
is there a memory efficient way of encoding the data? or maybe chunking the data --
I just can not think of how to return it without building the whole thing into one HUGE string object
thanks
--
a few clarifications --
this is an asp .net webservices app over a gigabit ethernet link as josh noted. I am not very familiar with it so still a bit of a learning curve.
I am using XMLWriter to create the XML and create a string out of it using String
some stats --
response xml size = about 385 megs (my data size will grow very quickly to way more than this)
string object size as calculated by a memory profiler = peaked at 605MB
and thanks to everyone who responded...

Use XmlTextWriter wrapped around Reponse.OutputStream to send the XML to the client and periodically flush the response. This way you never have to have more than a few mb in memory at any one time (at least for sending to the client).

Can't you just stream the response to the client? XmlWriter doesn't require its underlying stream to be buffered in memory. If it's ASP.NET you can use the Response.OutputStream or if it's WCF, you can use response streaming.

HTTP get for 1 gig? that's a lot! Perhaps you should reconsider.
At least gziping the output could help.

You should not create XML using string manipulation.
Instead, you should use the XmlTextWriter, XmlDocument, or (in .Net 3.5) XElement classes to build an XML tree in memory, then write it directly to Response.OutputStream using an XmlTextWriter.
Writing directly to an XmlTextWriter that wraps Response.OutputStream wil be most efficient (you'll never have an entire element tree in memory at once), but will be somewhat more complicated.
By doing it this way, you will never have a single string (or array) containing the entire object, and should thus avoid OutOfMemoryExceptions.

Had a similar problem, hope this will help someone. My initial code was:
var serializer = new XmlSerializer(type);
string xmlString;
using (var writer = new StringWriter())
{
serializer.Serialize(writer, objectData, sn); // OutOfMemoryException here
xmlString = writer.ToString();
}
I ended up replaceing StringWriter with MemoryStream and this solved my problem
using (var mem = new MemoryStream())
{
serializer.Serialize(mem, objectData, sn);
xmlString = Encoding.UTF8.GetString(mem.ToArray());
}

You'll have to return each record (or a small group of records) on their own individual GETs.