I have a Streaming WCF service. It receives a stream of a serialized class called ContentObjectData. The bytes I receive in from the stream I have temporarily placed in an ArrayList as I don't know how big the Stream is and to be honest I don't know what to do with them anyway.
The ContentObjectData Class:
[Serializable]
public class ContentObjectData
{
string Hash { get; set; }
string Data { get; set; }
string FileName { get; set; }
string DisplayImage { get; set; }
}
This is the Service's Method that receives the stream from the client.
[OperationContract]
public void SendContentObject(Stream data)
{
ArrayList alBytes = new ArrayList();
Console.WriteLine("Client connected");
int count;
byte[] buffer = new byte[4096];
while ((count = data.Read(buffer, 0, buffer.Length)) > 0)
{
alBytes.AddRange(buffer);
}
long i = alBytes.Count;
data.Close();
}
At this moment in time I am sending an Image for testing using the following methods:
private void btnLoadImage_Click(object sender, EventArgs e)
{
DialogResult dr = OFD.ShowDialog();
if (dr == DialogResult.OK)
{
foreach (string filename in OFD.FileNames)
{
try
{
ContentObject co = new ContentObject();
co.Data = LoadFile(filename);
co.Hash = Hashing.HashString(co.Data);
co.DisplayImage = co.Data;
co.FileName = co.Hash;
Stream stream = SerializeToStream(co);
SendContentObject(stream);
}
catch (Exception ex)
{
throw ex;
}
}
}
}
private void SendContentObject(Stream stream)
{
NetTcpBinding binding = new NetTcpBinding(SecurityMode.None, false);
// TimeSpan.MaxValue is interpreted with a rounding error, so use 24 days instead
binding.SendTimeout = TimeSpan.FromDays(24);
binding.TransferMode = TransferMode.Streamed;
ChannelFactory<RaptorStreamingHost> factory = new ChannelFactory<RaptorStreamingHost>(
binding, new EndpointAddress("net.tcp://ccs-labs.com:804/"));
RaptorStreamingHost service = factory.CreateChannel();
service.SendContentObject(stream);
((IClientChannel)service).Close();
}
private string LoadFile(string filename)
{
return Hashing.BytesToString(File.ReadAllBytes(filename));
}
public static Stream SerializeToStream(object objectType)
{
MemoryStream stream = new MemoryStream();
IFormatter formatter = new BinaryFormatter();
formatter.Serialize(stream, objectType);
stream.Position = 0L; // REMEMBER to reset stream or WCF will just send the stream from the end resulting in an empty stream!
return (Stream)stream;
}
I have this to DeSerialize but it doesn't make much sense to me:
public static object DeserializeFromStream(MemoryStream stream)
{
IFormatter formatter = new BinaryFormatter();
stream.Seek(0, SeekOrigin.Begin);
object objectType = formatter.Deserialize(stream);
return objectType;
}
How do I convert the ArrayList of Bytes (I guess DeSerialize them) into a New ContentObject?
Update One
Ah So close!
Ok in this method
public static ContentObjectData DeserializeFromStream(MemoryStream stream)
{
IFormatter formatter = new BinaryFormatter();
stream.Seek(0, SeekOrigin.Begin);
ContentObjectData objectType = (ContentObjectData)formatter.Deserialize(stream);
return objectType;
}
I have a problem. The Deserializer can not find the ContentObjectData because it is looking in the client's Namespace for ContentObjectData and not this Host's namespace.
Firstly, don't use an ArrayList to store your bytes. Since ArrayList is non-generic each individual byte will be boxed and a pointer to the byte saved in the array, which will use 5 (32 bit) or 9 (64 bit) times more memory than necessary.
Instead, you can copy the Stream to a local MemoryStream, then save the underlying byte [] array:
public static void CopyStream(Stream input, Stream output)
{
byte[] buffer = new byte[32768];
int read;
while ((read = input.Read(buffer, 0, buffer.Length)) > 0)
{
output.Write(buffer, 0, read);
}
}
[OperationContract]
public void SendContentObject(Stream data)
{
Console.WriteLine("Client connected");
var memoryStream = new MemoryStream();
using (data)
CopyStream(data, memoryStream);
byte [] alBytes = memoryStream.ToArray();
}
Then later you can turn the byte [] array back to a MemoryStream for deserialization:
public static object DeserializeFromStream(byte [] allBytes)
{
using (var stream = new MemoryStream(allBytes))
return DeserializeFromStream(stream);
}
public static object DeserializeFromStream(MemoryStream stream)
{
IFormatter formatter = new BinaryFormatter();
stream.Seek(0, SeekOrigin.Begin);
object objectType = formatter.Deserialize(stream);
return objectType;
}
(Or, just keep the original MemoryStream and pass it around.)
Update
BinaryFormatter serializes full .Net type information (i.e. the fully qualified type name) to and from the serialization stream. If the receiving system doesn't have a type with exactly the same name, in exactly the same namespace, in exactly the same assembly, then deserialization fails.
If this is your situation, you have the following workarounds:
Extract a shared DLL containing the type in question and link it into both client and server; problem solved.
Write a SerializationBinder to map the type assemblies and names. See here and also here or here for instructions how to do it.
Consider a different, more contract-oriented binary format. Bson is one option. protobuf-net is another, albeit one I have not used. More here.
Related
I have googled my *** off and cannot find a reason why this is happening.
I have a WCF service (file) where I upload a PDF that is only in a memorystream. The memorystream shows I have data before it is sent to the service. When it gets to the service the stream is empty. I have a MessageContract for the upload that has a MessageBodyMember of Stream. I call the upload in the service and pass the parameters (the same ones that are in the MessageContract. When I look at the upload request object I can see all the values except Stream.
Here is my UploadRequest class:
[MessageContract]
public class UploadRequest
{
[MessageHeader(MustUnderstand = true)]
public long EntityID { get; set; }
[MessageHeader(MustUnderstand = true)]
public string FileName { get; set; }
[MessageHeader(MustUnderstand = false)]
public bool IsPrivileged { get; set; }
[MessageHeader(MustUnderstand = false)]
public string Folder { get; set; }
[MessageHeader(MustUnderstand = false)]
public Guid stream_id { get; set; }
[MessageBodyMember(Order = 1)]
public Stream Stream { get; set; }
}
Here is my call to the upload method in the service:
private void SaveAs()
{
var UserID = Convert.ToInt64(Application.Current.Properties["VetID"]);
var ms = new MemoryStream();
Control.SaveDocument(ms);
ms.Position = 0;
if (!svc.Upload(UserID, null, null, true, gVar.stream_id, ms, out string errormsg))
_ = errormsg;
}
Note: I did set the MemoryStream position to 0 and the object ms has data.
Here is FileService upload method:
public UploadResponse Upload(UploadRequest request)
{
DB.dbFileMgr fm = new DB.dbFileMgr();
return fm.UploadFile(request);
}
And here is the UploadFile method:
if (request.stream_id != null && request.stream_id != new Guid())
{
var stream = SharedCommon.ConvertStreamToMemoryStream(request.Stream);
dbConn.UpdateDocument(request.EntityID, request.IsPrivileged, request.stream_id, stream);
return new UploadResponse { UploadSucceeded = true };
}
else
{
dbConn.CreateDocumentFolder(request.Folder, request.EntityID);
var stream = SharedCommon.ConvertStreamToMemoryStream(request.Stream);
dbConn.InsertDocument(request.EntityID, stream, request.FileName, request.Folder, request.IsPrivileged);
return new UploadResponse { UploadSucceeded = true };
}
Here is what the Stream is when I put a breakpoint at the Upload:
Debug info of the Stream
I have a method to convert a Stream to a MemoryStream, thinking that would help, but when it reaches the while loop in the ReadFully method it skips over it. That tells me the Stream is empty.
Here is the code for the Convert and ReadFully methods:
public static MemoryStream ConvertStreamToMemoryStream(Stream stream)
{
MemoryStream memoryStream = new MemoryStream();
if (stream != null)
{
byte[] buffer = ReadFully(stream);
if (buffer != null)
{
var binaryWriter = new BinaryWriter(memoryStream);
binaryWriter.Write(buffer);
}
}
return memoryStream;
}
public static byte[] ReadFully(this Stream input)
{
byte[] buffer = new byte[48 * 1024];
using (MemoryStream ms = new MemoryStream())
{
int read;
while ((read = input.Read(buffer, 0, buffer.Length)) > 0)
{
ms.Write(buffer, 0, read);
}
return ms.ToArray();
}
}
I have read all the Microsoft, C-SharpCorner & StackOverflow articles and I am at a loss as to what is wrong. I would appreciate any ideas as to what is going on and how to fix this.
Edit
Maybe I did not make this clear, I am sorry. I can upload a file from the file system using the FileUpload.InputStream but when I try and send a dynamically created PDF that is only in a MemoryStream to the same service as the File Upload it fails. Before anyone says that the file might be bad, I can save the MemoryStream to my desktop and the file opens fine in Adobe. The issue seems to be sending a MemoryStream through MessageContract. It seems to have no problem with an InputStream (FileUpload) but not a MemoryStream.
Stream.Length only works on Stream implementations where seeking is available. You can usually check to see if Stream.CanSeek is true. Many streams, since they're being streamed, are of a nature where it's impossible to know the length in advance.
If you must know the length, you may need to actually buffer the entire stream, loading it into memory in advance.
You can consider copying the uploaded file to MemoryStream:
public void Upload(UploadRequest request)
{
MemoryStream outstream = new MemoryStream();
const int bufferLen = 4096;
byte[] buffer = new byte[bufferLen];
int count = 0;
while ((count = request.Stream.Read(buffer, 0, bufferLen)) > 0)
{
outstream.Write(buffer, 0, count);
}
byte[] b = outstream.ToArray();
string s = System.Text.Encoding.UTF8.GetString(b, 0, b.Length);
Console.WriteLine(s);
}
UPDATE:
You need to modify your ConvertStreamToMemoryStream method:
public static MemoryStream ConvertStreamToMemoryStream(Stream stream)
{
MemoryStream outstream = new MemoryStream();
const int bufferLen = 48 * 1024;
byte[] buffer = new byte[bufferLen];
int count = 0;
while ((count = stream.Read(buffer, 0, bufferLen)) > 0)
{
outstream.Write(buffer, 0, count);
}
return outstream;
}
private byte[] ToByteArray(Message msg)
{
if(msg == null)
{
return null;
}
BinaryFormatter bf = new BinaryFormatter();
using(MemoryStream ms = new MemoryStream())
{
bf.Serialize(ms, msg);
return ms.ToArray();
}
}
private Message ToMessageObject(Byte[] bytes)
{
if(bytes == null)
{
return null;
}
BinaryFormatter bf = new BinaryFormatter();
using (MemoryStream ms = new MemoryStream(bytes))
{
ms.Write(bytes, 0, bytes.Length);
ms.Position = 0;
Message msg = (Message)bf.Deserialize(ms);
return msg;
}
}
I am using the two above methods for serialization and deserialization, but it keeps on throwing the error during deserializaion.
system.runtime.serialization.serializationexception: 'end of stream encountered before parsing was completed.'
My message class has the "Serializable" attribute.
One of the things I noticed is when the text is small, like two words, it deserializes fine but when it contains a lot of characters, close to 100, that's when I get that error. I have been checking other solutions()frome here and other places to this problem but none seems to work for me.
From the comments, I was able to figure out that the problem was caused by the initialization of the byte array,
var responseStream = client.GetStream()
var bytes = new byte[1024];
if (responseStream.DataAvailable)
{
await responseStream.ReadAsync(bytes, 0, bytes.Length);
var responseMessage =ToMessageObject(bytes);
messages.Add(responseMessage);
}
Initially, I set the bytes array length to 256. The error was thrown anytime I had to deserialize a byte array of length greater than 256. So, I am wondering if it's possible to use a dynamic array when reading from a stream.
I'm trying to serialize my MMF to file and here is the code:
class MMF {
private const string filename = #"c:\NFS";
private long offset = 0;
private long length = 194;
private byte[] buffer;
public MMF() {
using (var mmf =
MemoryMappedFile.CreateFromFile(filename, FileMode.OpenOrCreate, null, offset + length, MemoryMappedFileAccess.ReadWriteExecute)) {
using (var accessor = mmf.CreateViewAccessor(offset, length, MemoryMappedFileAccess.ReadWriteExecute)) {
buffer = new byte[194];
/*FS fs = new FS();
fs.Files[0].Path = "test";
accessor.WriteArray<byte>(0, buffer, 0, (int)length);*/
accessor.ReadArray<byte>(0, buffer, 0, (int)length);
FS fs = (FS)ToObject(buffer);
Console.WriteLine(fs.Files[0].Path);
}
}
}
private byte[] ToByteArray(object source) {
var formatter = new BinaryFormatter();
using (var stream = new MemoryStream()) {
formatter.Serialize(stream, source);
return stream.ToArray();
}
}
private object ToObject(byte[] source) {
var formatter = new BinaryFormatter();
using (var stream = new MemoryStream(source)) {
formatter.Deserialize(stream);
return stream;
}
}
}
On deserialization part I'm getting error:
An unhandled exception of type
'System.Runtime.Serialization.SerializationException' occurred in
mscorlib.dll
Additional information: A binary stream "0" does not contain a valid
binary header BinaryHeader. Possible causes: invalid stream or object
version change between serialization and deserialization.
How to Deserialize the file properly? Where is my mistake?
thank you
You might want to let MMF implement ISerializable and implement the GetObjectData method.
I am currently trying to serialize a List, it serializes (I think fine), but when it deserialize,
Sorry for the amount of code, but I am really stuck and have no idea why this is happening, i also tried to changed the struct into a class and no help.
THANKS.
i get the following error UPDATED
There was an error deserializing the object of type There was an error deserializing the object of type
`System.Collections.Generic.List`1[[A.B.C.DataValues, A.V, Version=1.0.0.0, Culture=neutral, PublicKeyToken=null]]. Unexpected end of file. Following elements are not closed: Time, DataValues, ArrayOfDataValues.`
I am serializing like this UPDATED
public void SerializeDataValue(List<DataValues> values)
{
DataContractSerializer serializer = new DataContractSerializer(typeof(List<DataValues>));
using (MemoryStream stream = new MemoryStream())
{
using (GZipStream compress = new GZipStream(stream, CompressionMode.Compress))
{
XmlDictionaryWriter w = XmlDictionaryWriter.CreateBinaryWriter(compress);
serializer.WriteObject(w, values);
}
_serializedData = stream.ToArray();
}
}
I am deserializing like this UPDATED
public List<DataValues> DeserializeDataValue()
{
if (SerializedData == null || SerializedData.Length == 0)
{
return new List<DataValues> ();
}
else
{
DataContractSerializer serializer = new DataContractSerializer(typeof(List<DataValues>));
using (MemoryStream stream = new MemoryStream(SerializedData))
{
using (GZipStream decompress = new GZipStream(stream, CompressionMode.Decompress))
{
XmlDictionaryReader r = XmlDictionaryReader.CreateBinaryReader(decompress, XmlDictionaryReaderQuotas.Max);
return serializer.ReadObject(r, true) as List<DataValues>;
}
}
}
}
Properties
private byte[] _serializedData;
[DataMember]
[Browsable(false)]
public byte[] SerializedData
{
get { return _serializedData; }
set { _serializedData = value; }
}
helper Methods
public static byte[] ReadFully(Stream input)
{
byte[] buffer = new byte[16 * 1024];
input.Position = 0;
using (MemoryStream ms = new MemoryStream())
{
int read;
while ((read = input.Read(buffer, 0, buffer.Length)) > 0)
{
ms.Write(buffer, 0, read);
}
return ms.ToArray();
}
}
Struct
[DataContract(Name = "DataValues", Namespace = "A.B.C")]
public struct DataValues
{
[DataMember]
public DateTime Time { get; set; }
[DataMember]
public Single Value { get; set; }
public DataValues(DateTime dateTime, Single value)
{
Time = dateTime;
Value = value;
}
}
It’s because you are not serialising the object(s) completely. You need to close the stream(s) after writing, especially when using gzip. Recommended practice is to use using:
public void SerializeDataValue(List<DataValues> values)
{
DataContractSerializer serializer = new DataContractSerializer(typeof(List<DataValues>));
using (MemoryStream stream = new MemoryStream())
{
using (GZipStream compress = new GZipStream(stream, CompressionMode.Compress))
{
XmlDictionaryWriter w = XmlDictionaryWriter.CreateBinaryWriter(compress);
serializer.WriteObject(w, values);
}
_serializedData = stream.ToArray();
}
}
Sorry to be late to this question.
The problem with the initial approach was simply that you weren't flushing (read: disposing) the XmlDictionaryWriter.
This should work (note the 2nd using clause):
using (GZipStream compress = new GZipStream(stream, CompressionMode.Compress))
using (XmlDictionaryWriter w = XmlDictionaryWriter.CreateBinaryWriter(compress))
{
serializer.WriteObject(w, values);
}
Hope this helps someone.
I can get the sample to work by removing the XmlDictionaryReader and instead directly feeding the input/output stream into the DataContractSerializer. It may be a defect in the XmlDictionaryReader for large compressed collections but I'm not sure.
Hope this helps:
public void SerializeDataValue(List<DataValues> values)
{
DataContractSerializer serializer = new DataContractSerializer(typeof(List<DataValues>));
using (MemoryStream stream = new MemoryStream())
{
using (GZipStream compress = new GZipStream(stream, CompressionMode.Compress))
{
serializer.WriteObject(compress , values);
}
_serializedData = stream.ToArray();
}
}
public List<DataValues> DeserializeDataValue()
{
if (SerializedData == null || SerializedData.Length == 0)
{
return new List<DataValues> ();
}
else
{
DataContractSerializer serializer = new DataContractSerializer(typeof(List<DataValues>));
using (MemoryStream stream = new MemoryStream(SerializedData))
{
using (GZipStream decompress = new GZipStream(stream, CompressionMode.Decompress))
{
return serializer.ReadObject(decompress , true) as List<DataValues>;
}
}
}
}
I ran exactly into the same problem and I finally found the solution : the XmlDictionaryWriter needs to be disposed/closed before the Stream you are writing into is itself closed. I discovered that thanks to the thorough example found at http://www.albahari.com/nutshell/ch15.aspx whiche are more complete than the MSDN ones.
In your sample code, that would be :
using (XmlDictionaryWriter w = XmlDictionaryWriter.CreateBinaryWriter(compress))
{
serializer.WriteObject(w, values);
}
On my own example, using the XmlDictionaryWriter instead of the plain and by default Xml writer only gave me a ~25% decrease in file size but a factor 3 when reading back the object.
I am created a post before " Object to byte not working " .
I fixed problems that users said me , but there is still problem .
Error Message : The constructor to deserialize an object of type 'WindowsFormsApplication1.Form1+Item' was not found.;
void start()
{
Item item = new Item();
item.files.Add(#"test");
byte[] b = ObjectToByteArray(item);
Item k = Desriles(b);
}
[Serializable]
public class Item : ISerializable
{
public Item()
{
files = new List<string>();
Exclude = false;
CleanEmptyFolder = false;
}
public List<string> files;
public string MusicProfileName;
public bool Exclude;
#region ISerializable Members
public void GetObjectData(SerializationInfo info, StreamingContext context)
{
info.AddValue("files", files);
info.AddValue("MusicProfileName", MusicProfileName);
info.AddValue("Exclude", Exclude);
}
#endregion
}
public byte[] ObjectToByteArray(object _Object)
{
using (var stream = new MemoryStream())
{
// serialize object
var formatter = new BinaryFormatter();
formatter.Serialize(stream, _Object);
// get a byte array
var bytes = new byte[stream.Length];
using (BinaryReader br = new BinaryReader(stream))
{
bytes = br.ReadBytes(Convert.ToInt32(stream.Length));
}
return bytes;
}
}
public Item Desriles(byte[] items)
{
using (MemoryStream stream = new MemoryStream())
{
stream.SetLength(items.LongLength);
stream.write(items, 0, items.Length);
var formatter = new BinaryFormatter();
stream.Seek(0, SeekOrigin.Begin);
object item = formatter.Deserialize(stream); // Here I will get error
return (Item)item;
}
}
The serialization code can't work properly, you forgot to reset the stream back to the beginning. The better mouse trap:
public byte[] ObjectToByteArray(object _Object) {
using (var stream = new MemoryStream()) {
var formatter = new BinaryFormatter();
formatter.Serialize(stream, _Object);
return stream.ToArray();
}
}
The deserialization code can similarly be simplified:
public Item Desriles(byte[] items) {
using (MemoryStream stream = new MemoryStream(items)) {
var formatter = new BinaryFormatter();
return (Item)formatter.Deserialize(stream);
}
}
And you don't need GetObjectData().
In this section:
using (MemoryStream stream = new MemoryStream())
{
stream.SetLength(items.LongLength);
stream.Read(items, 0, items.Length);
[...]
object item = formatter.Deserialize(stream);
it seems you are creating a new, empty memory stream, then attempting to read from it, and Deserialize from it.
Of course it fails. The stream is empty.
abelenky makes a good point, but I don't think your:
public byte[] ObjectToByteArray(object _Object)
works either.
C# Code Snippet - Object to byte array
C# Code Snippet - Byte array to object
Thanks to All :
I found problem : I should base that to ISerializable