Protobuf-Net InvalidOperationException w/out error message - c#

I am using the following class to serialize individual objects (T) with a prefix, and then attempting to deserialize back to a list w/ the DeserializeItems() method:
public class ProtobufSerializationProvider<T> : ISerializationProvider<T>
{
readonly Type type;
readonly TypeModel model;
public ProtobufSerializationProvider()
{
this.type = typeof(T);
this.model = createModel(this.type);
}
public byte[] Serialize(T instance)
{
byte[] buffer;
using (MemoryStream strm = new MemoryStream())
{
model.SerializeWithLengthPrefix
(strm, instance, type, PrefixStyle.Base128, 1);
buffer = strm.GetBuffer();
}
return buffer;
}
// here is the problem method
public IEnumerable<T> DeserializeAll(MemoryStream stream)
{
return model.DeserializeItems<T>(stream, PrefixStyle.Base128, 1);
}
TypeModel createModel(Type type)
{
try
{
RuntimeTypeModel runtimeModel = TypeModel.Create();
this.addTypeToModel(runtimeModel, type);
this.addTypePropertiesToModel(runtimeModel, type);
return runtimeModel.Compile();
}
catch (Exception e)
{
throw e.InnerException;
}
}
void addTypePropertiesToModel(RuntimeTypeModel typeModel, Type type)
{
PropertyInfo[] properties = type.GetProperties();
for (int i = 0; i < properties.Length; i++)
{
Type innerType = properties[i].PropertyType;
if (!innerType.IsPrimitive && !(innerType == typeof(string)))
{
addTypeToModel(typeModel, properties[i].PropertyType);
}
}
}
MetaType addTypeToModel(RuntimeTypeModel typeModel, Type t)
{
var properties = t.GetProperties()
.Select(p => p.Name)
.OrderBy(name => name);
return typeModel
.Add(t, true)
.Add(properties.ToArray());
}
}
I am getting a default InvalidOperationException "Operation is not valid due to the current state of the object" when I try to enumerate the IEnumerable, whether by casting ToList(), or counting it Count(), etc. Specifically the MoveNext() method will throw the error:
enumerator.MoveNext()
Also the stream has to be open until DeserializeItems(stream) returns, and I ensured that was the case. But once the IEnumerable successfully returns I cannot use it.
Not sure if there is a problem with the serialized items or not. Also I noticed that each of my items 256 bytes, although a large portion of those bytes are just trailing nulls.
This is the class I am serializing as a test. Note that I am not using attributes since I create the model manually:
public class NestedFoo
{
public string NestedFooStr { get; set; }
}
public class Foo
{
public string Foo1 { get; set; }
public string Foo2 { get; set; }
public string Foo3 { get; set; }
public string Foo4 { get; set; }
public NestedFoo NestedFoo { get; set; }
}
Thanks.
UPDATE
After switching from MemoryStream.GetBuffer() to MemoryStream.ToArray(), the message is shortened but trying to cast the IEnumerable .ToList() or any other operation produces the same error.
However, I have noticed that after switching to .ToArray() the 'Current' property of the IEnumerable after enumeration reaches the last element in the IEnumerable before throwing the error, whereas when using .GetBuffer() the 'Current' property on the first element when the error is thrown.
I suspect the problem could be that the IEnumerable<> doesn't know when its out of elements because its reached the end of the stream? I append several items serialized with SerializeWithLengthPrefix() to a single byte[], and then there may be nulls left over trailing the array. If I read that entire array into the DeserializeItems method, does it need to know how to terminate or does it only continue if it detects a length prefix (as I would expect?).
Also one other question I had was to what extent using the SerializeWithLengthPrefix() method is equivalent to just serializing a List in one operation, using the DataFormat.Group enumeration, like:
[ProtoContract]
public class ProtoList<T>
{
[ProtoMember(1, DataFormat = DataFormat.Group)]
public List<T> List { get; set; }
public ProtoList() { }
public ProtoList(IEnumerable<T> items)
{
this.List = new List<T>(items);
}
}
Using a container such as this seems to perform very well, so I may just go with this route in cases where I want to serialize a list in one shot.
However for serializing items individually as they are added I am currently using my own prefixing function, so if I can get the Protobuf prefixing methods to work that would be good too.

The 256/trailing nulls is because you are using MemoryStream.GetBuffer(). This method is fine as long as it is used in combination with the Length, as the buffer is oversized. If you want a right-sized array, use ToArray() instead. Alternatively use ArraySegment<byte> to avoid copying any data.
It is possible this also fixes the exception (0 is not a valid value for a field-header). If not, leave a comment so I know to look in more detail. If it does fix it, please also let me know, and I'll try to make the error message (for the 0 field-header case) more helpful.

Just to confirm, ran into same problem.
Do not use tags / packet ids as 0. Solved everything.

Related

RedisSessionStateProvider with ProtoBuf serialization causing errors

I am trying to get protobuf serialization working with the RedisSessionStateProvider. I have specified the redisSerializerType as a custom class which implements Microsoft.Web.Redis.ISerializer - here is the deserialization code:
public object Deserialize(byte[] data)
{
return DeserializeDirect(data);
}
private object DeserializeDirect(byte[] data)
{
using (var memoryStream = new MemoryStream(data))
{
return Serializer.Deserialize<object>(memoryStream);
}
return null;
}
As I need to implement Microsoft.Web.Redis.ISerializer the signature for deserialize uses a return type of object and there is no way to pass in the actual type being returned. So when DeserializeDirect tries to use the Protobuf.Serializer to deserialize it (as expected) says "Type is not expected, and no contract can be inferred: System.Object". I am using a web app with .NET framework 4.6.1 and I was hoping somebody could point out what I am doing wrong.
Thanks!
Normally, protobuf-net really wants to know the exact type. You can, however, cheat using DynamicType. This tells protobuf-net to include additional type metadata - something it doesn't usually include.
Note that this can make you code brittle - it may fail if the type changes in you code!
I will be implementing Any soon (as part of 2.3.0), which is another option here.
public static void Main()
{
// the actual object we care about
object obj = new Foo { X = 1 };
// serialize and deserialize via stub
var stub = new Stub { Data = obj };
var clone = Serializer.DeepClone(stub);
// prove it worked
Console.WriteLine(clone.Data);
// prove it is a different instance
Console.WriteLine(ReferenceEquals(obj, clone.Data));
}
[ProtoContract]
public class Foo
{
[ProtoMember(1)]
public int X { get; set; }
public override string ToString() => $"X={X}";
}
[ProtoContract]
public sealed class Stub
{
[ProtoMember(1, DynamicType = true)]
public object Data { get; set; }
}

Cast generic parameter

I get an object back from a method. The object is of type object and I can't change this because of backwards compatibility. If it is of a certain type (Response<T> bellow) then I need to access the property Payload which is of type T so that I can serialize it as part of a another object and send it off. The problem is that since I don't know the type of T I can't cast the object to Response<T> to access Payload even though I don't care about its type.
Here is my object with the generic type:
public class Response
{
public int Status { get; set; }
public string Message { get; set; }
}
public class Response<T> : Response
{
public T Payload { get; set; }
}
Here is what I would like to do:
// Will sometimes be of type Response<T>
object data = LegacyCode();
if (data.GetType().IsGenericType && data.GetType().GetGenericTypeDefinition() == typeof(Response<>)) {
var payload = ((Response<object>)data).Payload; // Unable to cast object of type...
}
But the only way I could find to do this is by using dynamics.
// Will sometimes be of type Response<T>
object data = LegacyCode();
if (data.GetType().IsGenericType && data.GetType().GetGenericTypeDefinition() == typeof(Response<>)) {
var payload = ((dynamice)data).Payload;
}
Don't ask why things are the way they are (I'm wondering that myself). I have to do code gymnastics to keep backwards compatibility in this system. I just want compile time checking on the name of the property.
Here is a fiddle: https://dotnetfiddle.net/dXxHbD
UPDATE:
I need to be able to serialize and deserialize this object. Originally Response had a property Payload of type object. This caused serialization issues when Response<T> was deserialized because the Payload property was of type Newtonsoft.Json.Linq.JObject which could not be cast to T. Here is an example: https://dotnetfiddle.net/uc15HD
The problem was that I was going the wrong direction and the deserialization works if I cast T to object rather than try to cast object to T. When I store the value as its specific type T then the serializer knows what to deserialize the string to.
Here is an example using Jon's answer below: https://dotnetfiddle.net/KwudAx
Here is a similar example using Matias' solution of using covariance: https://dotnetfiddle.net/kCjZr4
To get compile-time checking of the name of the property, you can keep the dynamic typing, but get the runtime "mini-compiler" to do the hard work:
object data = LegacyCode();
object payload = GetPayload(data);
// Use payload
...
private static object GetPayload<T>(Response<T> response)
{
return response.Payload;
}
public static object GetPayload(object data)
{
// Fallback method. You could return null here and use
// that to indicate in the calling code that it wasn't a
// Response<T>...
}
A much better solution would be to add an non-generic interface or an extra base class though. For example:
public class Response
{
public int Status { get; set; }
public string Message { get; set; }
}
public interface IPayloadHolder
{
public object Payload { get; }
}
public class Response<T> : Response, IPayloadHolder
{
public T Payload { get; set; }
// By using explicit interface implementation, this
// doesn't get in the way for normal usage.
IPayloadHolder.Payload { get { return Payload; } }
}
Then you can use:
var payloadHolder = data as IPayloadHolder;
if (payloadHolder != null)
{
var payload = payloadHolder.Payload;
}
I believe that you need to use covariance.
Design an interface IResponse<out T>:
public interface IResponse<out T>
{
public T Payload { get; }
}
And implement it on Response<T>. Now you can cast it to IResponse<object>:
Response<string> x = new Response<string>();
IResponse<object> y = x; // implicit cast

Add instance of Generic List to object using reflection

I'm trying to attach a Generic list to a instance of a class using reflection, unlike when attaching simple objects method the PropertyInfo.SetValue(obj, value, index) is returning the exception {"Parameter count mismatch."} when value is an instance of List<SomeType> (as opposed to a string, int, bool or even a custom class).
That summery might not make much sense; The following may help to explain what I'm trying to do.
Say we are trying to populate the following class with reflection:
public class Foo
{
public virtual int someInt {get; set;}
public virtual IList<SomeClass> list {get; set;}
}
The method may look something like this:
public static T Parse<T>(HttpRequest request) where T : new()
{
returnObj = new T();
PropertyInfo[] properties = typeof(T).GetProperties();
foreach (PropertyInfo p in properties)
{
// Get a meaningful property name
string ins = System.Text.RegularExpressions.Regex.Replace(p.PropertyType.FullName, "([^,]*),.*$", "$1");
switch(ins)
{
// populate int
case "System.Int32":
p.SetValue(returnObj, Int32.Parse(request[p.Name]) , null);
break;
// populate list
case "System.Collections.Generic.IList`1[[SomeNamespace.Domain.SomeClass":
IList<SomeClass> list = new List<SomeClass>();
foreach (string s in request[p.Name].Split(','))
{
list.Add(new SomeClass(s));
}
// This will throw the exception 'Parameter count mismatch.'
p.SetValue(returnObj, list, null);
break;
}
}
return returnObj;
}
However when trying to add an instance of List (IList) in this way an exception is thrown.
Edit: To clarify, If been over this method with a fine tooth comb(Breakpoints) (well, the one in the application, not exactly this one) and all the variables are populated as expected; until SetValue throws an exception; if anyone needs some more information, do ask.
Edit2: So I built a smaller application to test this (In order to upload it as an example); and I'm having trouble recreating my own issue; As many of you have suggested this works. I'll update this question with the issue when I manage to track it down. Its probably something trivial, as these things so often are (The original codebase is massive and therefore not appropriate for me to post). Thanks for all your help so far and my apologies for wasting your time.
Im running the code from your question with some minor changes to make it compile and it seems to work fine:
void Main()
{
Parse<Foo>();
}
public static T Parse<T>() where T : new()
{
var returnObj = new T();
PropertyInfo[] properties = typeof(T).GetProperties();
foreach (PropertyInfo p in properties)
{
// Get a meaningful property name
string ins = p.PropertyType.Name;
switch(ins)
{
// populate int
case "Int32":
p.SetValue(returnObj, 1 , null);
break;
// populate list
case "IList`1":
var list = new List<string>();
// This will throw the exception 'Parameter count mismatch.'
p.SetValue(returnObj, list, null);
break;
}
}
return returnObj;
}
public class Foo
{
public virtual int someInt {get; set;}
public virtual IList<string> list {get; set;}
}
If you change the Foo to have an indexer property that returns IList on the other hand you get the exception in your question:
public class Foo
{
public virtual int someInt {get; set;}
public virtual IList<string> this[int key]
{
get{ return null; }
set
{
}
}
}
Generates:
TargetParameterCountException: Parameter count mismatch.
Have you verified that the case
"System.Collections.Generic.IList`1[[SomeNamespace.Domain.SomeClass"
is being hit? When I pass in a List, even one created as an IList, GetType() gives me
System.Collections.Generic.List`1[SomeNamespace.Domain.SomeClass]
Would it potentially be more reliable to use:
typeOf(System.Collections.Generic.IList).isAssignableFrom(p.GetType())

How do I deal with arrays using reflection

I am writing some validation code. The code will take data passed into a web service and decide whether it can do the action, or return a message to the caller that they have missed out some fields etc.
I have it mostly working except for arrays. I am marking up the properties with a [RequiredField] attribute to represent fields that are required. So if this is some of my data,
public enum EnumTest
{
Value1,
Value2
}
[DataContract]
public class DummyWebserviceData
{
[DataMember]
[RequiredField]
public EnumTest[] EnumTest{ get; set; }
[DataMember]
[RequiredField]
public DummyWebserviceData2[] ArrayOfData { get; set; }
}
[DataContract]
public class DummyWebserviceData2
{
[DataMember]
[RequiredField]
public string FirstName { get; set;}
[DataMember]
[RequiredField]
public string LastName { get; set;}
[DataMember]
public string Description { get; set;}
}
So what do I have working? I have validation of dates, and strings working. It uses recursion to go any level deep required on the data.
But ... so what about the two arrays there. The first is an array of enums. I want to check in that case that the array is not empty.
The second is the array of DummyWebserviceData2 values. I need to pull each value out and have a look at it individually.
To simplify the code I have written it looks something like this,
foreach (PropertyInfo propertyInfo in data.GetType().GetProperties())
{
if (propertyInfo.PropertyType.IsArray)
{
// this craps out
object[] array = (object[])propertyInfo.GetValue(data, new object[] { 0 });
}
}
So it seems to me that the first thing is that I can tell it is an array. But how then can I tell how many items are in the array?
At runtime the object will have been dynamically subclassed from the Array data type (this MSDN topic details that), therefore you don't need to reflect into the array, you can cast the object to Array, and then use the Array.GetValue instance method:
Array a = (Array)propertyInfo.GetValue(data);
for(int i = 0; i< a.Length; i++)
{
object o = a.GetValue(i);
}
You can also iterate over an array as well - since from .Net 2.0 onwards:
In the .NET Framework version 2.0, the Array class implements the System.Collections.Generic::IList, System.Collections.Generic::ICollection, and System.Collections.Generic::IEnumerable generic interfaces.
You don't need to know the T, since from these you can get an IEnumerable; which you can then use a Cast() operation on, or indeed just work at the object level.
Incidentally, the reason why your code isn't working is because you can't cast an array of MyType[] to object[] because object[] is not a base type of MyType[] - only object is.
This method works rather well, and it's simple code.
var array = ((IEnumerable)propertyInfo.GetValue(instance)).Cast<object>().ToArray();
foreach (PropertyInfo propertyInfo in data.GetType().GetProperties())
{
if (propertyInfo.PropertyType.IsArray)
{
// first get the array
object[] array = (object[])propertyInfo.GetValue(data)
// then find the length
int arrayLength = array.GetLength(0);
// now check if the length is > 0
}
}
The answer with the array is nice, but as mentioned it does not work for some other collection types. If you don't know what type your collection is of try something like:
IEnumerable<object> a = (IEnumerable<object>)myPropInfo.GetValue(myResourceObject);
// at least foreach now is available
foreach (object o in a)
{
// get the value
string valueAsString = o.ToString();
}

c# copy constructor generator

I want to copy values from one object to another object. Something similar to pass by value but with assignment.
For example:
PushPin newValPushPin = oldPushPin; //I want to break the reference here.
I was told to write a copy constructor for this. But this class has a lot of properties, it will probably take an hour to write a copy constructor by hand.
Is there a better way to assign an object to another object by value?
If not, is there a copy constructor generator?
Note: ICloneable is not available in Silverlight.
If you can mark the object that is to be cloned as Serializable then you can use in-memory serialization to create a copy. Check the following code, it has the advantage that it will work on other kinds of objects as well and that you don't have to change your copy constructor or copy code each time an property is added, removed or changed:
class Program
{
static void Main(string[] args)
{
var foo = new Foo(10, "test", new Bar("Detail 1"), new Bar("Detail 2"));
var clonedFoo = foo.Clone();
Console.WriteLine("Id {0} Bar count {1}", clonedFoo.Id, clonedFoo.Bars.Count());
}
}
public static class ClonerExtensions
{
public static TObject Clone<TObject>(this TObject toClone)
{
var formatter = new BinaryFormatter();
using (var memoryStream = new MemoryStream())
{
formatter.Serialize(memoryStream, toClone);
memoryStream.Position = 0;
return (TObject) formatter.Deserialize(memoryStream);
}
}
}
[Serializable]
public class Foo
{
public int Id { get; private set; }
public string Name { get; private set; }
public IEnumerable<Bar> Bars { get; private set; }
public Foo(int id, string name, params Bar[] bars)
{
Id = id;
Name = name;
Bars = bars;
}
}
[Serializable]
public class Bar
{
public string Detail { get; private set; }
public Bar(string detail)
{
Detail = detail;
}
}
There is a protected member called "MemberwiseClone", you can write this in your class...
public MyClass Clone(){
return (MyClass)this.MemberwiseClone();
}
then you can access..
MyClass newObject = oldObject.Clone();
The only way (that I'm aware of) to do this, and do it correctly, is to implement the copy yourself. Take for example:
public class FrobAndState
{
public Frob Frobber { get; set;}
public bool State { get; set; }
}
public class Frob
{
public List<int> Values { get; private set; }
public Frob(int[] values)
{
Values = new List<int>(values);
}
}
In this example you'd need to know how Frob was implemented, i.e. the fact that you need to call the constructor to create a copy of it as Values is read-only, to be able to make a copy of a given instance of FrobAndState.
Also - you couldn't just implement FrobAndState.Copy thusly:
public class FrobAndState
{
// ... Properties
public FrobAndState Copy()
{
var new = new FrobAndState();
new.State = this.State;
new.Frobber = this.Frobber;
}
}
Because both the instance of FrobAndState that you called .Copy() on, and the new instance would both have a reference to the same instance of Frobber.
In short, copying things is hard and any Copy implementation is difficult to get right.
C# does not have a copy constructor. There are different ways to tackle this. At the OOP level you could use inheritance or aggregation. AutoMapper might also be worth a try.
I want to copy values from one object
to another object. Something similiar
to pass by value but with assignment.
What do you mean by "with assignment"? If you mean that you want people to be able to say:
a = b;
And for you to define what = means, the only way you can do that in C# is if b is a different type to a and you've defined an implicit conversion (or more tenuously, if a stands for something of the form x.Y where Y is a property with a setter). You can't override = for a simple assignment between identical types in C#.
I was told to write a copy constructor
for this. But this class has alot of
properties, it will probably take an
hour to write a copy constructor by
hand.
If that's really true, then I would guess that you have a different problem. Your class is too big.
If you make your class Serializable you could Serialize it to a MemoryStream and Deserialize to a new instance.
If you want copy-on-assignment you should be using a struct instead of a class. But be careful, it is easy to make subtle mistakes. It is highly recommended that all stucts be immmutable to reduce the chance for error.
Though, this may not answer your question directly, but to add a cent; usually the term Clone is linked with shallow copy(referenced objects). To have a deep copy, I believe you will need to look into the some creational pattern(prototype?). The answer to this question might help.
You implement Justin Angel's method of cloning objects in Silverlight
using System;
using System.Reflection;
using System.Windows;
namespace JustinAngelNet.Silverlight.Framework
{
public static class SilverlightExtensions
{
public static T Clone<T>(T source)
{
T cloned = (T) Activator.CreateInstance(source.GetType());
foreach (PropertyInfo curPropInfo in source.GetType().GetProperties())
{
if (curPropInfo.GetGetMethod() != null
&& (curPropInfo.GetSetMethod() != null))
{
// Handle Non-indexer properties
if (curPropInfo.Name != "Item")
{
// get property from source
object getValue = curPropInfo.GetGetMethod().Invoke(source, new object[] {});
// clone if needed
if (getValue != null && getValue is DependencyObject)
getValue = Clone((DependencyObject) getValue);
// set property on cloned
if (getValue != null)
curPropInfo.GetSetMethod().Invoke(cloned, new object[] {getValue});
}
// handle indexer
else
{
// get count for indexer
int numberofItemInColleciton =
(int)
curPropInfo.ReflectedType.GetProperty("Count").GetGetMethod().Invoke(source, new object[] {});
// run on indexer
for (int i = 0; i < numberofItemInColleciton; i++)
{
// get item through Indexer
object getValue = curPropInfo.GetGetMethod().Invoke(source, new object[] {i});
// clone if needed
if (getValue != null && getValue is DependencyObject)
getValue = Clone((DependencyObject) getValue);
// add item to collection
curPropInfo.ReflectedType.GetMethod("Add").Invoke(cloned, new object[] {getValue});
}
}
}
}
return cloned;
}
}
}
Then you can do this
MyClass newObject = SilverlightExtensions.Clone(oldObject);

Categories

Resources