protobuf-net implicit contracts - c#

Does anyone have a working example where protobuf-net can be used as a drop-in replacement for BinaryFormatter? Is that even possible?
Actually I just need a serializer for one message type which looks like
public class A {
public B[] Bs { get;set; }
public C[] Cs { get;set; }
}
All types are defined in a different assembly and have a lot of properties.
Is there an option to automatically generate proto contracts with all public properties included, for class A and other used types (B, C), so something like
var formatter = ProtoBuf.Serializer.CreateFormatter<A>()
just works?

Firstly, protobuf-net is not intended to be, and does not claim to be, a 100% drop in replacement for BinaryFormatter. It has slightly different features.
Would you be content to do a little reflection? Basically, there is support for ImplicitFields, but currently this is only available via [ProtoContract], and cannot be done conveniently via RuntimeTypeModel, which is a bit of a pain, and is on my list. Although, I should point out that I consider implicit-fields to be a bit risky, and it should only be done if you know the DTO innards won't change! But: to answer your question, you could iterate over the types you expect, and add them to the model manually:
static void Main()
{
Prepare(typeof(A), typeof(B), typeof(C));
// if you really want to use IFormatter...
var formatter = RuntimeTypeModel.Default.CreateFormatter(typeof (A));
var obj = new A {Bs = new B[] {new B()}};
using (var ms = new MemoryStream())
{
formatter.Serialize(ms, obj);
ms.Position = 0;
var clone = formatter.Deserialize(ms);
}
}
static void Prepare(params Type[] types)
{
if(types != null) foreach(var type in types) Prepare(type);
}
static void Prepare(Type type)
{
if(type != null && !RuntimeTypeModel.Default.IsDefined(type))
{
Debug.WriteLine("Preparing: " + type.FullName);
// note this has no defined sort, so invent one
var props = type.GetProperties();
Array.Sort(props, (x, y) => string.Compare(
x.Name, y.Name, StringComparison.Ordinal));
var meta = RuntimeTypeModel.Default.Add(type, false);
int fieldNum = 1;
for(int i = 0 ; i < props.Length ; i++)
{
meta.Add(fieldNum++, props[i].Name);
}
}
}
Note that the use of IFormatter here is entirely unnecessary; you could also use RuntimeTypeModel.Default.Serialize(...) or Serializer.Serialize<T>(...).
As a footnote: I would advise defining the models more ... repeatably. For example:
RuntimeTypeModel.Default.Add(typeof(A)).Add("Bs", "Cs");
RuntimeTypeModel.Default.Add(typeof(B)).Add("Foo");
RuntimeTypeModel.Default.Add(typeof(C)).Add("Bar", "Blap", "Blop");

This is the final version of Prepare function I used:
static void Prepare(params Type[] types)
{
foreach (var type in types)
{
if (type != null && !RuntimeTypeModel.Default.IsDefined(type))
{
if (type.Namespace.StartsWith("System"))
return;
Debug.WriteLine("Preparing: " + type.FullName);
// note this has no defined sort, so invent one
var props = type.GetProperties();
Array.Sort(props, (x, y) => string.Compare(
x.Name, y.Name, StringComparison.Ordinal));
var meta = RuntimeTypeModel.Default.Add(type, false);
int fieldNum = 1;
for (int i = 0; i < props.Length; i++)
if (props[i].CanWrite)
{
meta.Add(fieldNum++, props[i].Name);
if (!RuntimeTypeModel.Default.IsDefined(props[i].PropertyType))
if (props[i].PropertyType.HasElementType)
Prepare(props[i].PropertyType.GetElementType()); //T[]
else if (props[i].PropertyType.IsGenericType)
Prepare(props[i].PropertyType.GetGenericArguments()); //List<T>
else
Prepare(props[i].PropertyType);
}
}
}
}
If somebody is interested, this is the result of my benchmark (serialize + deserialize):
BinaryFormatter 10000 messages: 2.131s 5028 bytes/msg
Json.NET Bson 10000 messages: 1.679s 1071 bytes/msg
MetSys Bson 10000 messages: 1.581s 1035 bytes/msg
Protobuf 10000 messages: 0.145s 109 bytes/msg
MsgPack 10000 messages: 0.844s 106 bytes/msg
Notes:
MetSys.Bson library does not handle decimal type well.
MsgPack library has problems with read-only properties
Thanks Marc for your help and such a great library.

Related

Deep copy .NET class object

I want to write a generic method that returns a deep copy for a given object:
TTEntity CreateDeepCopy<TTEntity>(TTEntity obj) where TTEntity : MyBaseEntity
There are some constraints:
I cannot implement copy constructors or custom interfaces for each new class that derives from MyBaseEntity (or anything that involves writing code in each new class added in project)
I cannot use Binary Formatter because is obsolete
I tried XmlSerializer but MyBaseEntity has a Dictionary and it throws an error:
Cannot serialize member {...} of type System.Collections.Generic.Dictionary`2[[System.String, System.Private.CoreLib, Version=6.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e],[System.Boolean, System.Private.CoreLib, Version=6.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e]], because it implements IDictionary.
Usually I prefer to use a simple Json serialization to deep clone an object because by default you don't have to decorate classes with attribute to make serialization and deserialization working.
Protobuf and MessagePack are faster and have a smaller memory footprint but the serialization result is less readable (also MessagePack can be used without decorating classes but with this options it serializes the message in Json format).
Here's an example using System.Text.Json:
TTEntity CreateDeepCopy<TTEntity>(TTEntity obj) where TTEntity : MyBaseEntity
{
if (obj == null)
{
return null;
}
var serialized = System.Text.Json.JsonSerializer.Serialize(obj, new System.Text.Json.JsonSerializerOptions()
{
DefaultIgnoreCondition = System.Text.Json.Serialization.JsonIgnoreCondition.WhenWritingNull,
WriteIndented = false,
Converters = {
new System.Text.Json.Serialization.JsonStringEnumConverter()
},
IncludeFields = true
});
return System.Text.Json.JsonSerializer.Deserialize<TTEntity>(serialized);
}
Here using Newtonsoft.Json:
TTEntity CreateDeepCopy<TTEntity>(TTEntity obj) where TTEntity : MyBaseEntity
{
if (obj == null)
{
return null;
}
var jsonSettings = new Newtonsoft.Json.JsonSerializerSettings()
{
NullValueHandling = Newtonsoft.Json.NullValueHandling.Ignore,
Formatting = Newtonsoft.Json.Formatting.None,
};
jsonSettings.Converters.Add(new Newtonsoft.Json.Converters.StringEnumConverter());
var serialized = Newtonsoft.Json.JsonConvert.SerializeObject(obj, jsonSettings);
return System.Text.Json.JsonSerializer.Deserialize<TTEntity>(serialized);
}
Note: Newtonsoft.Json.JsonSerializerSettings and System.Text.Json.JsonSerializerOptions are best defined only once: create a singleton class or register your class as a singleton within dependency injection container.
As an alternative to Ben's answer and ale91's answer (the concept of which, i.e using a production ready package, I highly suggest) is handrolling your own using reflection.
Basically, we need to
Get every property we want to clone
Create a copy of that property
Assign it to the (new) object
Step 2 is a pain, since not all types can easily be clone, i.e collections like List and Dictionary are harder to clone as well as reference types having to go through the cloning process as well until we reach only primitives/ value types. Another problem is circular references, i.e a Company has many Employees, and an Employee has one Company leads to Company1 -> Employee -> Company1 -> Employee ... which will result in a stack overflow exception if we don't mitigate against that, I will just use a naïve hashset to keep track of already cloned objects and not clone them again. So here's an implementation of this
static TEntity CreateDeepCopy<TEntity>(TEntity obj, HashSet<object>? seenObjects = default)
{
if (seenObjects == default)
seenObjects = new HashSet<object>();
if (seenObjects.TryGetValue(obj, out _))
return obj;
seenObjects.Add(obj);
var type = obj.GetType();
if (type.IsCopyType())
return obj;
var clone = Activator.CreateInstance<TEntity>();
var properties = type.GetProperties(
BindingFlags.Public | BindingFlags.Instance
);
foreach (var property in properties)
{
var value = property.GetValue(obj);
if (value is null)
continue;
// This is the easy case
if (property.PropertyType.IsCopyType())
{
property.SetValue(clone, value);
seenObjects.Add(value);
}
else if (typeof(IList).IsAssignableFrom(property.PropertyType))
{
var list = (IList) Activator.CreateInstance(property.PropertyType)!;
var listValue = (IEnumerable)value;
foreach (var element in listValue)
{
var targetMethod = typeof(Program).GetMethod(nameof(CreateDeepCopy), BindingFlags.NonPublic | BindingFlags.Static);
var genericTarget = targetMethod.MakeGenericMethod(element.GetType());
var subClone = genericTarget.Invoke(null, new[] { element, seenObjects });
list.Add(subClone);
seenObjects.Add(element);
}
property.SetValue(clone, list);
seenObjects.Add(value);
}
else if (typeof(IEnumerable).IsAssignableFrom(property.PropertyType))
{
throw new Exception($"Only supports cloning collection types which implement 'IList', {property.PropertyType} is not supported");
}
else
{
var targetMethod = typeof(Program).GetMethod(nameof(CreateDeepCopy), BindingFlags.NonPublic | BindingFlags.Static);
var genericTarget = targetMethod.MakeGenericMethod(value.GetType());
var subClone = genericTarget.Invoke(null, new[] { value, seenObjects });
property.SetValue(clone, subClone);
seenObjects.Add(value);
}
}
return clone;
}
static bool IsCopyType(this Type type)
{
return type.IsPrimitive ||
type.IsValueType ||
type == typeof(string);
}
I also wrote some tests as a dotnetfiddle, but again, I highly encourage you to test this implementation much more thoroughly or use some form of serialization.
Plus, this code is quite messy, I would clean up a few places if you were to use this in production
P.S this currently doesn't work on types with collection properties other than IList, if you want to support dictionary and others you'll have to implement that on your own
In preference to BinaryFormatter, I would suggest you try MessagePack.
https://github.com/neuecc/MessagePack-CSharp
Here is a simple unit test I created to verify that an object replicated via MessagePack serialize/deserialize matched the original object.
[Fact]
public void TestMessagePack()
{
var options = MessagePack.Resolvers.ContractlessStandardResolver.Options;
var ms = new MemoryStream();
MessagePackSerializer.Serialize(ms, _testObject, options);
ms.Position = 0;
var result = MessagePackSerializer.Deserialize<TestObject>(ms, options);
result.Should().BeEquivalentTo(_testObject);
ms.Close();
}
In the above example _testObject was a complex object with multiple arrays, collections and nested objects.

Compiling methods for optimal runtime performance

I'm trying to determine what the best way to get fully optimized delegates for various objects to improve the speed of serialization. Simply put: I'd like to remove various different checks, and compile more efficient serialize functions one time at the start of my app.
Let's take a look at this simple example:
public class GamePacket
{
[Length(10)]
[ReadBackwards]
public string Id { get; set; }
}
Now, I'd likely create a serializer, and for performance reasons store the attributes in a cached field. Everytime I want to deserialize a GamePacket from a stream (or byte array), I'd call something like:
Deserialize(byte[] stream)
{
var header = stream.ReadByte();
var packet = cachedDeserializers[header];
var instance = packet.DelegateForCreateInstance();
foreach (var field in packet.Fields)
{
if (field.Type != TypeCode.String) continue;
var str = stream.ReadBytes(field.LengthAttribute.Length);
if (field.HasReadBackwardsAttribute)
str = str.Reverse();
field.DelegateForSetValue(instance, str);
}
}
The problem now lies in the fact that EVERY time I'm calling Deserialize on that stream, I need to loop through and check various things like attributes, and other checks. In the example, these things can potentially be omitted (And maybe more):
if (field.Type != TypeCode.String) continue;
if (field.HasReadBackwardsAttribute)
If I know the field has a read backwards attribute, I'd like to compile a simplified delegate on app start that omits these checks, and simply reads it backwards. Is it possible to create a delegate that can remove unneeded logic? For example:
Deserialize(byte[] stream)
{
var header = stream.ReadByte();
var packet = cachedDeserializers[header];
var instance = packet.CallCachedCompile(stream);
}
// CallCachedCompile for GamePacket would look something like this:
CallCachedCompile(byte[] stream)
{
var instance = this.DelegateForCreateInstance();
var str = stream.ReadBytes(10);
str = str.Reverse();
this.DelegateForSetValue(instance, "Id", str);
return instance;
}
I've looked briefly into expression trees. Would something like this be doable in expression Trees? What would be the most efficient way?
Yes, using code generation approach you can generate delegates for the particular type. So instead of this generic reflection-like code:
foreach (var field in packet.Fields)
{
if (field.Type != TypeCode.String) continue;
var str = stream.ReadBytes(field.LengthAttribute.Length);
if (field.HasReadBackwardsAttribute)
str = str.Reverse();
field.DelegateForSetValue(instance, str);
}
You can generate code for a specific type:
gamePacketInstance.Id = SomeConvertionToString(stream.ReadBytes(field.LengthAttribute.Length).Revers());
The code generation topic is quite big and I don't know what exactly you do inside your delegates. You can generate specific delegates in runtime (emit il or expression trees) or in compile time (source generators). I suggest you to read my article Dotnet code generation overview by example. It will give good overview with examples.
Without using any code generation you can still do this pretty efficiently.
You basically need to use generics and polymorphism to cache all code that you want done for each type that you encounter.
Bearing in mind that properties have underlying methods, we can create delegates to set properties without using code generation.
abstract class DeserializerBase
{
object DeserializePacket(Stream stream);
}
class Deserializer<T> : DeserializerBase where T : new()
{
FieldAction<T>[] fieldActions =
typeof(T).GetProperties()
.Where(p => p.Type == TypeCode.String)
.Select(p => IsReverseAttribute(p)
? new FieldActionReverse<T>
{
Setter = p.SetMethod.CreateDelegate<Action<T, string>>(),
Length = GetLengthAttribute(p),
}
: new FieldAction<T>
{
Setter = p.SetMethod.CreateDelegate<Action<T, string>>(),
Length = GetLengthAttribute(p),
})
.ToArray();
object DeserializePacket(Stream stream);
{
var packet = new T();
foreach (var action in fieldActions)
action.Deserialize(packet);
}
}
class FieldAction<T>
{
public Action<T, string> Setter;
public int Length;
void Deserialize(Stream stream, T instance)
{
var str = ReadString(stream);
Setter(instance, str);
}
virtual string GetString(Stream stream)
{
return stream.ReadBytes(Length);
}
}
class FieldActionReverse<T> : FieldAction<T>
{
override string GetString(Stream stream)
{
return stream.ReadBytes(Length).Reverse();
}
}
Your final entry code becomes this.
Dictionary<int, DeserializerBase> cachedDeserializers = new Dictionary<int, DeserializerBase>
{
{5, new Deserializer<GamePacket>()}
};
Deserialize(Stream stream)
{
var header = stream.ReadByte();
var packet = cachedDeserializers[header].DeserializePacket(stream);
}
You can even place generic constraints on T to ensure it is a Packet then you can return a base Packet type from this entry function.

dynamic type does not contain a definition for 'property', error? [duplicate]

My situation is very simple. Somewhere in my code I have this:
dynamic myVariable = GetDataThatLooksVerySimilarButNotTheSame();
//How to do this?
if (myVariable.MyProperty.Exists)
//Do stuff
So, basically my question is how to check (without throwing an exception) that a certain property is available on my dynamic variable. I could do GetType() but I'd rather avoid that since I don't really need to know the type of the object. All that I really want to know is whether a property (or method, if that makes life easier) is available. Any pointers?
I think there is no way to find out whether a dynamic variable has a certain member without trying to access it, unless you re-implemented the way dynamic binding is handled in the C# compiler. Which would probably include a lot of guessing, because it is implementation-defined, according to the C# specification.
So you should actually try to access the member and catch an exception, if it fails:
dynamic myVariable = GetDataThatLooksVerySimilarButNotTheSame();
try
{
var x = myVariable.MyProperty;
// do stuff with x
}
catch (RuntimeBinderException)
{
// MyProperty doesn't exist
}
I thought I'd do a comparison of Martijn's answer and svick's answer...
The following program returns the following results:
Testing with exception: 2430985 ticks
Testing with reflection: 155570 ticks
void Main()
{
var random = new Random(Environment.TickCount);
dynamic test = new Test();
var sw = new Stopwatch();
sw.Start();
for (int i = 0; i < 100000; i++)
{
TestWithException(test, FlipCoin(random));
}
sw.Stop();
Console.WriteLine("Testing with exception: " + sw.ElapsedTicks.ToString() + " ticks");
sw.Restart();
for (int i = 0; i < 100000; i++)
{
TestWithReflection(test, FlipCoin(random));
}
sw.Stop();
Console.WriteLine("Testing with reflection: " + sw.ElapsedTicks.ToString() + " ticks");
}
class Test
{
public bool Exists { get { return true; } }
}
bool FlipCoin(Random random)
{
return random.Next(2) == 0;
}
bool TestWithException(dynamic d, bool useExisting)
{
try
{
bool result = useExisting ? d.Exists : d.DoesntExist;
return true;
}
catch (Exception)
{
return false;
}
}
bool TestWithReflection(dynamic d, bool useExisting)
{
Type type = d.GetType();
return type.GetProperties().Any(p => p.Name.Equals(useExisting ? "Exists" : "DoesntExist"));
}
As a result I'd suggest using reflection. See below.
Responding to bland's comment:
Ratios are reflection:exception ticks for 100000 iterations:
Fails 1/1: - 1:43 ticks
Fails 1/2: - 1:22 ticks
Fails 1/3: - 1:14 ticks
Fails 1/5: - 1:9 ticks
Fails 1/7: - 1:7 ticks
Fails 1/13: - 1:4 ticks
Fails 1/17: - 1:3 ticks
Fails 1/23: - 1:2 ticks
...
Fails 1/43: - 1:2 ticks
Fails 1/47: - 1:1 ticks
...fair enough - if you expect it to fail with a probability with less than ~1/47, then go for exception.
The above assumes that you're running GetProperties() each time. You may be able to speed up the process by caching the result of GetProperties() for each type in a dictionary or similar. This may help if you're checking against the same set of types over and again.
Maybe use reflection?
dynamic myVar = GetDataThatLooksVerySimilarButNotTheSame();
Type typeOfDynamic = myVar.GetType();
bool exist = typeOfDynamic.GetProperties().Where(p => p.Name.Equals("PropertyName")).Any();
Just in case it helps someone:
If the method GetDataThatLooksVerySimilarButNotTheSame() returns an ExpandoObject you can also cast to a IDictionary before checking.
dynamic test = new System.Dynamic.ExpandoObject();
test.foo = "bar";
if (((IDictionary<string, object>)test).ContainsKey("foo"))
{
Console.WriteLine(test.foo);
}
The two common solutions to this include making the call and catching the RuntimeBinderException, using reflection to check for the call, or serialising to a text format and parsing from there. The problem with exceptions is that they are very slow, because when one is constructed, the current call stack is serialised. Serialising to JSON or something analogous incurs a similar penalty. This leaves us with reflection but it only works if the underlying object is actually a POCO with real members on it. If it's a dynamic wrapper around a dictionary, a COM object, or an external web service, then reflection won't help.
Another solution is to use IDynamicMetaObjectProvider to get the member names as the DLR sees them. In the example below, I use a static class (Dynamic) to test for the Age field and display it.
class Program
{
static void Main()
{
dynamic x = new ExpandoObject();
x.Name = "Damian Powell";
x.Age = "21 (probably)";
if (Dynamic.HasMember(x, "Age"))
{
Console.WriteLine("Age={0}", x.Age);
}
}
}
public static class Dynamic
{
public static bool HasMember(object dynObj, string memberName)
{
return GetMemberNames(dynObj).Contains(memberName);
}
public static IEnumerable<string> GetMemberNames(object dynObj)
{
var metaObjProvider = dynObj as IDynamicMetaObjectProvider;
if (null == metaObjProvider) throw new InvalidOperationException(
"The supplied object must be a dynamic object " +
"(i.e. it must implement IDynamicMetaObjectProvider)"
);
var metaObj = metaObjProvider.GetMetaObject(
Expression.Constant(metaObjProvider)
);
var memberNames = metaObj.GetDynamicMemberNames();
return memberNames;
}
}
Denis's answer made me think to another solution using JsonObjects,
a header property checker:
Predicate<object> hasHeader = jsonObject =>
((JObject)jsonObject).OfType<JProperty>()
.Any(prop => prop.Name == "header");
or maybe better:
Predicate<object> hasHeader = jsonObject =>
((JObject)jsonObject).Property("header") != null;
for example:
dynamic json = JsonConvert.DeserializeObject(data);
string header = hasHeader(json) ? json.header : null;
Well, I faced a similar problem but on unit tests.
Using SharpTestsEx you can check if a property existis. I use this testing my controllers, because since the JSON object is dynamic, someone can change the name and forget to change it in the javascript or something, so testing for all properties when writing the controller should increase my safety.
Example:
dynamic testedObject = new ExpandoObject();
testedObject.MyName = "I am a testing object";
Now, using SharTestsEx:
Executing.This(delegate {var unused = testedObject.MyName; }).Should().NotThrow();
Executing.This(delegate {var unused = testedObject.NotExistingProperty; }).Should().Throw();
Using this, i test all existing properties using "Should().NotThrow()".
It's probably out of topic, but can be usefull for someone.
Following on from the answer by #karask, you could wrap the function as a helper like so:
public static bool HasProperty(ExpandoObject expandoObj,
string name)
{
return ((IDictionary<string, object>)expandoObj).ContainsKey(name);
}
For me this works:
if (IsProperty(() => DynamicObject.MyProperty))
; // do stuff
delegate string GetValueDelegate();
private bool IsProperty(GetValueDelegate getValueMethod)
{
try
{
//we're not interesting in the return value.
//What we need to know is whether an exception occurred or not
var v = getValueMethod();
return v != null;
}
catch (RuntimeBinderException)
{
return false;
}
catch
{
return true;
}
}
If you control the type being used as dynamic, couldn't you return a tuple instead of a value for every property access? Something like...
public class DynamicValue<T>
{
internal DynamicValue(T value, bool exists)
{
Value = value;
Exists = exists;
}
T Value { get; private set; }
bool Exists { get; private set; }
}
Possibly a naive implementation, but if you construct one of these internally each time and return that instead of the actual value, you can check Exists on every property access and then hit Value if it does with value being default(T) (and irrelevant) if it doesn't.
That said, I might be missing some knowledge on how dynamic works and this might not be a workable suggestion.
If your use case is to convert an api response, carrying about only a few fields, you can use this:
var template = new { address = new { street = "" } };
var response = JsonConvert.DeserializeAnonymousType(await result.Content.ReadAsStringAsync(), template);
string street = response?.address?.street;
Here is the other way:
using Newtonsoft.Json.Linq;
internal class DymanicTest
{
public static string Json = #"{
""AED"": 3.672825,
""AFN"": 56.982875,
""ALL"": 110.252599,
""AMD"": 408.222002,
""ANG"": 1.78704,
""AOA"": 98.192249,
""ARS"": 8.44469
}";
public static void Run()
{
dynamic dynamicObject = JObject.Parse(Json);
foreach (JProperty variable in dynamicObject)
{
if (variable.Name == "AMD")
{
var value = variable.Value;
}
}
}
}
In my case, I needed to check for the existence of a method with a specific name, so I used an interface for that
var plugin = this.pluginFinder.GetPluginIfInstalled<IPlugin>(pluginName) as dynamic;
if (plugin != null && plugin is ICustomPluginAction)
{
plugin.CustomPluginAction(action);
}
Also, interfaces can contain more than just methods:
Interfaces can contain methods, properties, events, indexers, or any
combination of those four member types.
From: Interfaces (C# Programming Guide)
Elegant and no need to trap exceptions or play with reflexion...
I know this is really old post but here is a simple solution to work with dynamic type in c#.
can use simple reflection to enumerate direct properties
or can use the object extention method
or use GetAsOrDefault<int> method to get a new strongly typed object with value if exists or default if not exists.
public static class DynamicHelper
{
private static void Test( )
{
dynamic myobj = new
{
myInt = 1,
myArray = new[ ]
{
1, 2.3
},
myDict = new
{
myInt = 1
}
};
var myIntOrZero = myobj.GetAsOrDefault< int >( ( Func< int > )( ( ) => myobj.noExist ) );
int? myNullableInt = GetAs< int >( myobj, ( Func< int > )( ( ) => myobj.myInt ) );
if( default( int ) != myIntOrZero )
Console.WriteLine( $"myInt: '{myIntOrZero}'" );
if( default( int? ) != myNullableInt )
Console.WriteLine( $"myInt: '{myNullableInt}'" );
if( DoesPropertyExist( myobj, "myInt" ) )
Console.WriteLine( $"myInt exists and it is: '{( int )myobj.myInt}'" );
}
public static bool DoesPropertyExist( dynamic dyn, string property )
{
var t = ( Type )dyn.GetType( );
var props = t.GetProperties( );
return props.Any( p => p.Name.Equals( property ) );
}
public static object GetAs< T >( dynamic obj, Func< T > lookup )
{
try
{
var val = lookup( );
return ( T )val;
}
catch( RuntimeBinderException ) { }
return null;
}
public static T GetAsOrDefault< T >( this object obj, Func< T > test )
{
try
{
var val = test( );
return ( T )val;
}
catch( RuntimeBinderException ) { }
return default( T );
}
}
As ExpandoObject inherits the IDictionary<string, object> you can use the following check
dynamic myVariable = GetDataThatLooksVerySimilarButNotTheSame();
if (((IDictionary<string, object>)myVariable).ContainsKey("MyProperty"))
//Do stuff
You can make a utility method to perform this check, that will make the code much cleaner and re-usable

Persist an object that is not marked as serializable

I need to persist an object that is not marked with the serializable attribute. The object is from a 3rd party library which I cannot change.
I need to store it in a persist place, like for example the file system, so the optimal solution would be to serialize the object to a file, but since it isn't marked as serializable, that is not a straight forward solution.
It's a pretty complex object, which also holds a collection of other objects.
Do you guys have any input on how to solve this? The code will never run in a production environment, so I'm ok with almost any solution and performance.
XmlSerializer may be a useful first thing to try, if the types are public etc
If that fails, v2 of protobuf-net (in progress, you'd need to build from source, but I can help) works with unattributed objects, so ideal for types outside your control - you just need to tell it what to include (via a DSL). The v2 code isn't complete, but it covers most common scenarios, including collections etc (the incomplete work is mainly callbacks and enums).
You could write a recursive method that would run down the object graph using reflection to persist the object... Putting it back could be much more difficult. Who knows if any of those objects are holding references to unmanaged or system resources. If i was to do anything this nuts, I would go for the .GetFields(...) method on the type.
Another idea...
If you are only doing this to speed up development why not wrap their clases with your own adapter classes. This would allow you to replace the third party libraries with your own simplifed mock classes and allow for better chance for replacement and reuse later.
Sick as it is... This was easier then I thought it would be. (While this works... please consider wrapping the third party classes.)
public static class Tools
{
public static XElement AsXml(this object input)
{
return input.AsXml(string.Empty);
}
public static XElement AsXml(this object input, string name)
{
if (string.IsNullOrEmpty(name))
name = input.GetType().Name;
var xname = XmlConvert.EncodeName(name);
if (input == null)
return new XElement(xname);
if (input is string || input is int || input is float /* others */)
return new XElement(xname, input);
var type = input.GetType();
var fields = type.GetFields(BindingFlags.Instance |
BindingFlags.NonPublic)
.Union(type.GetFields(BindingFlags.Instance |
BindingFlags.Public));
var elems = fields.Select(f => f.GetValue(input)
.AsXml(f.Name));
return new XElement(xname, elems);
}
public static void ToObject(this XElement input, object result)
{
if (input == null || result == null)
throw new ArgumentNullException();
var type = result.GetType();
var fields = type.GetFields(BindingFlags.Instance |
BindingFlags.NonPublic)
.Union(type.GetFields(BindingFlags.Instance |
BindingFlags.Public));
var values = from elm in input.Elements()
let name = XmlConvert.DecodeName(elm.Name.LocalName)
join field in fields on name equals field.Name
let backType = field.FieldType
let val = elm.Value
let parsed = backType.AsValue(val, elm)
select new
{
field,
parsed
};
foreach (var item in values)
item.field.SetValue(result, item.parsed);
}
public static object AsValue(this Type backType,
string val,
XElement elm)
{
if (backType == typeof(string))
return (object)val;
if (backType == typeof(int))
return (object)int.Parse(val);
if (backType == typeof(float))
return (float)int.Parse(val);
object ret = FormatterServices.GetUninitializedObject(backType);
elm.ToObject(ret);
return ret;
}
}
public class Program
{
public static void Main(string[] args)
{
var obj = new { Matt = "hi", Other = new { ID = 1 } };
var other = new { Matt = "zzz", Other = new { ID = 5 } };
var ret = obj.AsXml();
ret.ToObject(other);
Console.WriteLine(obj); //{ Matt = hi, Other = { ID = 1 } }
Console.WriteLine(other); //{ Matt = hi, Other = { ID = 1 } }
}
}
This is one way you could do it:
http://www.codeproject.com/KB/dotnet/Surrogate_Serialization.aspx
here is the msdn link showing it:
http://msdn.microsoft.com/en-us/magazine/cc188950.aspx
I don't know if it is overkill for your usage, but I have been playing around with db4o lately. It will persist any object, just call IObjectContainer.Store(object), and it is lightweight and file-based. Does not require any installation.
I haven't had any problems with it yet.
///Here OBJECT is Class name and Object_to_write is instance
XmlSerializer serializer = new XmlSerializer(typeof(OBJECT));
using (TextWriter writer = new StreamWriter(#"C:\Xml.xml"))
{
serializer.Serialize(writer, OBJECT_to_Write);
}

What is the best way to dump entire objects to a log in C#?

So for viewing a current object's state at runtime, I really like what the Visual Studio Immediate window gives me. Just doing a simple
? objectname
Will give me a nicely formatted 'dump' of the object.
Is there an easy way to do this in code, so I can do something similar when logging?
For a larger object graph, I second the use of Json but with a slightly different strategy. First I have a static class that is easy to call and with a static method that wraps the Json conversion (note: could make this an extension method).
using Newtonsoft.Json;
public static class F
{
public static string Dump(object obj)
{
return JsonConvert.SerializeObject(obj);
}
}
Then in your Immediate Window,
var lookHere = F.Dump(myobj);
lookHere will auto-show up in the Locals window prepended with a $ or you can add a watch to it. On the right hand side of the Value column in the inspector, there is a magnifying glass with a dropdown caret beside it. Choose the dropdown caret and choose Json visualizer.
I am using Visual Studio 2013.
You could base something on the ObjectDumper code that ships with the Linq samples.
Have also a look at the answer of this related question to get a sample.
You could use Visual Studio Immediate Window
Just paste this (change actual to your object name obviously):
Newtonsoft.Json.JsonConvert.SerializeObject(actual);
It should print object in JSON
You should be able to copy it over textmechanic text tool or notepad++ and replace escaped quotes (\") with " and newlines (\r\n) with empty space, then remove double quotes (") from beginning and end and paste it to jsbeautifier to make it more readable.
UPDATE to OP's comment
public static class Dumper
{
public static void Dump(this object obj)
{
Console.WriteLine(Newtonsoft.Json.JsonConvert.SerializeObject(obj)); // your logger
}
}
this should allow you to dump any object.
Hope this saves you some time.
I'm certain there are better ways of doing this, but I have in the past used a method something like the following to serialize an object into a string that I can log:
private string ObjectToXml(object output)
{
string objectAsXmlString;
System.Xml.Serialization.XmlSerializer xs = new System.Xml.Serialization.XmlSerializer(output.GetType());
using (System.IO.StringWriter sw = new System.IO.StringWriter())
{
try
{
xs.Serialize(sw, output);
objectAsXmlString = sw.ToString();
}
catch (Exception ex)
{
objectAsXmlString = ex.ToString();
}
}
return objectAsXmlString;
}
You'll see that the method might also return the exception rather than the serialized object, so you'll want to ensure that the objects you want to log are serializable.
ServiceStack.Text has a T.Dump() extension method that does exactly this, recursively dumps all properties of any type in a nice readable format.
Example usage:
var model = new TestModel();
Console.WriteLine(model.Dump());
and output:
{
Int: 1,
String: One,
DateTime: 2010-04-11,
Guid: c050437f6fcd46be9b2d0806a0860b3e,
EmptyIntList: [],
IntList:
[
1,
2,
3
],
StringList:
[
one,
two,
three
],
StringIntMap:
{
a: 1,
b: 2,
c: 3
}
}
Here is a stupidly simple way to write a flat object, nicely formatted:
using Newtonsoft.Json.Linq;
Debug.WriteLine("The object is " + JObject.FromObject(theObjectToDump).ToString());
What's going on is that the object is first converted to a JSON internal representation by JObject.FromObject, and then converted to JSON string by ToString. (And of course a JSON string is a very nice representation of a simple object, especially since ToString will include newlines and indents.) The "ToString" is of course extraneous (as it's implied by using + to concat a string and an object), but I kinda like to specify it here.
You could use reflection and loop through all the object properties, then get their values and save them to the log. The formatting is really trivial (you could use \t to indent an objects properties and its values):
MyObject
Property1 = value
Property2 = value2
OtherObject
OtherProperty = value ...
What I like doing is overriding ToString() so that I get more useful output beyond the type name. This is handy in the debugger, you can see the information you want about an object without needing to expand it.
Following is another version that does the same thing (and handle nested properties), which I think is simpler (no dependencies on external libraries and can be modified easily to do things other than logging):
public class ObjectDumper
{
public static string Dump(object obj)
{
return new ObjectDumper().DumpObject(obj);
}
StringBuilder _dumpBuilder = new StringBuilder();
string DumpObject(object obj)
{
DumpObject(obj, 0);
return _dumpBuilder.ToString();
}
void DumpObject(object obj, int nestingLevel = 0)
{
var nestingSpaces = "".PadLeft(nestingLevel * 4);
if (obj == null)
{
_dumpBuilder.AppendFormat("{0}null\n", nestingSpaces);
}
else if (obj is string || obj.GetType().IsPrimitive)
{
_dumpBuilder.AppendFormat("{0}{1}\n", nestingSpaces, obj);
}
else if (ImplementsDictionary(obj.GetType()))
{
using (var e = ((dynamic)obj).GetEnumerator())
{
var enumerator = (IEnumerator)e;
while (enumerator.MoveNext())
{
dynamic p = enumerator.Current;
var key = p.Key;
var value = p.Value;
_dumpBuilder.AppendFormat("{0}{1} ({2})\n", nestingSpaces, key, value != null ? value.GetType().ToString() : "<null>");
DumpObject(value, nestingLevel + 1);
}
}
}
else if (obj is IEnumerable)
{
foreach (dynamic p in obj as IEnumerable)
{
DumpObject(p, nestingLevel);
}
}
else
{
foreach (PropertyDescriptor descriptor in TypeDescriptor.GetProperties(obj))
{
string name = descriptor.Name;
object value = descriptor.GetValue(obj);
_dumpBuilder.AppendFormat("{0}{1} ({2})\n", nestingSpaces, name, value != null ? value.GetType().ToString() : "<null>");
DumpObject(value, nestingLevel + 1);
}
}
}
bool ImplementsDictionary(Type t)
{
return t.GetInterfaces().Any(i => i.Name.Contains("IDictionary"));
}
}
I found a library called ObjectPrinter which allows to easily dump objects and collections to strings (and more). It does exactly what I needed.
You can write your own WriteLine method-
public static void WriteLine<T>(T obj)
{
var t = typeof(T);
var props = t.GetProperties();
StringBuilder sb = new StringBuilder();
foreach (var item in props)
{
sb.Append($"{item.Name}:{item.GetValue(obj,null)}; ");
}
sb.AppendLine();
Console.WriteLine(sb.ToString());
}
Use it like-
WriteLine(myObject);
To write a collection we can use-
var ifaces = t.GetInterfaces();
if (ifaces.Any(o => o.Name.StartsWith("ICollection")))
{
dynamic lst = t.GetMethod("GetEnumerator").Invoke(obj, null);
while (lst.MoveNext())
{
WriteLine(lst.Current);
}
}
The method may look like-
public static void WriteLine<T>(T obj)
{
var t = typeof(T);
var ifaces = t.GetInterfaces();
if (ifaces.Any(o => o.Name.StartsWith("ICollection")))
{
dynamic lst = t.GetMethod("GetEnumerator").Invoke(obj, null);
while (lst.MoveNext())
{
WriteLine(lst.Current);
}
}
else if (t.GetProperties().Any())
{
var props = t.GetProperties();
StringBuilder sb = new StringBuilder();
foreach (var item in props)
{
sb.Append($"{item.Name}:{item.GetValue(obj, null)}; ");
}
sb.AppendLine();
Console.WriteLine(sb.ToString());
}
}
Using if, else if and checking interfaces, attributes, base type, etc. and recursion (as this is a recursive method) in this way we may achieve an object dumper, but it is tedious for sure. Using the object dumper from Microsoft's LINQ Sample would save your time.
All of the paths above assume that your objects are serializable to XML or JSON,
or you must implement your own solution.
But in the end you still get to the point where you have to solve problems like
recursion in objects
non-serializable objects
exceptions
...
Plus log you want more information:
when the event happened
callstack
which threead
what was in the web session
which ip address
url
...
There is the best solution that solves all of this and much more.
Use this Nuget package: Desharp.
For all types of applications - both web and desktop applications.
See it's Desharp Github documentation. It has many configuration options.
Just call anywhere:
Desharp.Debug.Log(anyException);
Desharp.Debug.Log(anyCustomValueObject);
Desharp.Debug.Log(anyNonserializableObject);
Desharp.Debug.Log(anyFunc);
Desharp.Debug.Log(anyFunc, Desharp.Level.EMERGENCY); // you can store into different files
it can save the log in nice HTML (or in TEXT format, configurable)
it's possible to write optionally in background thread (configurable)
it has options for max objects depth and max strings length (configurable)
it uses loops for iteratable objects and backward reflection for everything else,
indeed for anything you can find in .NET environment.
I believe it will help.
So far a simplest and tidiest way for me is a serializer from YamlDotNet package.
using YamlDotNet.Serialization;
List<string> strings=new List<string>{"a","b","c"};
new Serializer().Serialize(strings)
will give you
- a
- b
- c
A more comprehensive example is here https://dotnetfiddle.net/KuV63n
Today you don't even need an external dependency. You can just use the built-in Microsoft Json Serializer.
using System;
using System.Text.Json;
namespace MyCompany.Core.Extensions
{
public static class ObjectExtensions
{
public static string Dump(this object obj)
{
try
{
return JsonSerializer.Serialize(obj);
}
catch(Exception)
{
return string.Empty;
}
}
}
}
Notice that you can pass a JsonSerializerOptions parameter to further customize the serialization to your liking:
Let's say you want to write the JSON indented for easy reading... we'd use:
new JsonSerializerOptions { WriteIndented = true }
#######
Here's a good guide if you wish to migrate from NewtonSoft.Json to System.Text.Json:
Compare Newtonsoft.Json to System.Text.Json, and migrate to System.Text.Json
Based on #engineforce answer, I made this class that I'm using in a PCL project of a Xamarin Solution:
/// <summary>
/// Based on: https://stackoverflow.com/a/42264037/6155481
/// </summary>
public class ObjectDumper
{
public static string Dump(object obj)
{
return new ObjectDumper().DumpObject(obj);
}
StringBuilder _dumpBuilder = new StringBuilder();
string DumpObject(object obj)
{
DumpObject(obj, 0);
return _dumpBuilder.ToString();
}
void DumpObject(object obj, int nestingLevel)
{
var nestingSpaces = "".PadLeft(nestingLevel * 4);
if (obj == null)
{
_dumpBuilder.AppendFormat("{0}null\n", nestingSpaces);
}
else if (obj is string || obj.GetType().GetTypeInfo().IsPrimitive || obj.GetType().GetTypeInfo().IsEnum)
{
_dumpBuilder.AppendFormat("{0}{1}\n", nestingSpaces, obj);
}
else if (ImplementsDictionary(obj.GetType()))
{
using (var e = ((dynamic)obj).GetEnumerator())
{
var enumerator = (IEnumerator)e;
while (enumerator.MoveNext())
{
dynamic p = enumerator.Current;
var key = p.Key;
var value = p.Value;
_dumpBuilder.AppendFormat("{0}{1} ({2})\n", nestingSpaces, key, value != null ? value.GetType().ToString() : "<null>");
DumpObject(value, nestingLevel + 1);
}
}
}
else if (obj is IEnumerable)
{
foreach (dynamic p in obj as IEnumerable)
{
DumpObject(p, nestingLevel);
}
}
else
{
foreach (PropertyInfo descriptor in obj.GetType().GetRuntimeProperties())
{
string name = descriptor.Name;
object value = descriptor.GetValue(obj);
_dumpBuilder.AppendFormat("{0}{1} ({2})\n", nestingSpaces, name, value != null ? value.GetType().ToString() : "<null>");
// TODO: Prevent recursion due to circular reference
if (name == "Self" && HasBaseType(obj.GetType(), "NSObject"))
{
// In ObjC I need to break the recursion when I find the Self property
// otherwise it will be an infinite recursion
Console.WriteLine($"Found Self! {obj.GetType()}");
}
else
{
DumpObject(value, nestingLevel + 1);
}
}
}
}
bool HasBaseType(Type type, string baseTypeName)
{
if (type == null) return false;
string typeName = type.Name;
if (baseTypeName == typeName) return true;
return HasBaseType(type.GetTypeInfo().BaseType, baseTypeName);
}
bool ImplementsDictionary(Type t)
{
return t is IDictionary;
}
}
Option 1: CSharpObjectFormatter from the Microsoft.CodeAnalysis.CSharp.Scripting.
C# REPL Command-Line Interface (CSI.EXE) and CSharpRepl use exactly this formatter.
using Microsoft.CodeAnalysis.CSharp.Scripting.Hosting;
var students = new[]
{
new Student
{
Id = 1,
Name = "Michael Hilus"
},
new Student
{
Id = 2,
Name = "Alicia Keys"
}
};
var formattedStudents = CSharpObjectFormatter.Instance.FormatObject(students);
Console.WriteLine(formattedStudents);
Output:
Student[2] { Student { Id=1, Name="Michael Hilus" }, Student { Id=2, Name="Alicia Keys" } }
But it's suitable for relatively simple objects, for example it can't dump the instance of System.Data.DataTable.
Option 2: vm.Aspects.Diagnostics.ObjectTextDumper - see documentation.
There is also very popular library:
ObjectDumper.NET, it's an open source, but not free for commercial use.

Categories

Resources