How to assign to referenced object - c#

I have two lists of object
[Serializable]
private class MemorySet
{
public Dictionary<string, object> _Map;
public List<object> _Results;
public List<object> _Storage;
}
MemorySet Memory = new MemorySet();
I can have keys assigned for an object, for example
_Map.Add("someKey", _Results[_Results.Count - 1]);
I have a method
private object Mapper(string key)
{
if (Memory._Map.ContainsKey(key))
{
return Memory._Map[key];
}
else if (key.ToLower() == "result")
{
return Memory._Results[Memory._Results.Count - 1];
}
else if (key.ToLower() == "storage")
{
return Memory._Storage[Memory._Storage.Count - 1];
}
else if (key.ToLower().Contains("result"))
{
int n = Convert.ToInt32(key.ToLower().Split(new string[] { "result" }, StringSplitOptions.None)[1]);
return Memory._Results[n];
}
else if (key.ToLower().Contains("storage"))
{
int n = Convert.ToInt32(key.ToLower().Split(new string[] { "storage" }, StringSplitOptions.None)[1]);
return Memory._Storage[n];
}
else return null;
}
Now I must assign to an object from _Storage or _Results like that:
object obj = key != "" ? Mapper(key) : Memory._Storage[Memory._Storage.Count - 1];
if(obj is string) obj as string = "test";
this will change obj to reference some new string in memory. But I want to change the object that obj references to instead.
In other words obj will become "test", but the underlying object won't be changed.
I understand why that happends, though I didn't imagine it that way when writing the whole engine, and now I have big trouble with that one. In C++ we have pointers, but in C# I don't want to use GCHandles or unmanaged code for that trivial stuff, would be extremely ugly.
So, how to assign to the object that object points to, instead assigning to the object itself?

Try this
[Serializable]
private class MemorySet
{
public Dictionary<string, object> _Map = new Dictionary<string,object>();
public List<object> _Results = new List<object>();
public List<object> _Storage = new List<object>)_;
}

If you don't want to mess with your current design, you could just add a method that would update your data structures on the same template as your Mapper method. This is what it would look like:
private void Update(string key, object value)
{
if (Memory._Map.ContainsKey(key))
{
Memory._Map[key] = value;
}
else if (key.ToLower() == "result")
{
Memory._Results[Memory._Results.Count - 1] = value;
}
else if (key.ToLower() == "storage")
{
Memory._Storage[Memory._Storage.Count - 1] = value;
}
else if (key.ToLower().Contains("result"))
{
int n = Convert.ToInt32(key.ToLower().Split(new string[] { "result" }, StringSplitOptions.None)[1]);
Memory._Results[n] = value;
}
else if (key.ToLower().Contains("storage"))
{
int n = Convert.ToInt32(key.ToLower().Split(new string[] { "storage" }, StringSplitOptions.None)[1]);
Memory._Storage[n] = value;
}
else
{
throw new ArgumentException("Failed to compute valid mapping", nameof(key));
}
}
Maybe you could also add the key == "" pattern in there, I'm not sure to understand how this would be used exactly, but hopefully you get the idea.
EDIT: OK, so references to the same object are used in different structures. You should consider to design a MemorySet that avoids this. If you still think this is the proper design considering your needs, you have a simple solution: wrap your target objects in other objects.
public class ObjectWrapper
{
public object ObjectOfInterest { get; set; }
}
Now you store ObjectWrapper objects. Then you can update the property ObjectOfInterest and this change will be reflected to all structures that contain this ObjectWrapper:
ObjectWrapper wrapper = Mapper(key);
wrapper.ObjectOfInterest = "test";

Related

Posting a dictionary to .net core 3.1. API

I want to post the following DTO to a .NET Core API:
{
"Name": "Foo",
"Street": "Bar",
"DynamicInfo": {
"MetadataAsString": "23423",
"MetadataAsInt": 2,
"MetadataAsBool": true,
"SomeOtherValues": "blub"
}
}
The class I want to map this in C# looks like this:
public class Foo
{
public string Name { get; set; }
public string Street { get; set; }
public Dictionary<string, object> DynamicInfo { get; set; }
}
You can see that I am using two static properties (Name and Street), but I also want to post some dynamic data.
I expect the dynamic data to be written in a dictionary, but unfortunately this does not work.
The result I am getting in my debugger is a little confusing:
So the values arrive successful, but are in a strange format... I dont know how to even access the values.
How can I convert this to just a normal dictionary, containing objects?
For a solution that would not depend on your use case, I would use 2 objects. 1 for client input and 1 that has validated input. The client input object would contain a Dictionary<string,string>. And the validated input can contain your Dictionary<string,object> if its still something you intend to use.
If you use this approach. During validation of the client input, you could use bool.TryParse(DynamicInfo["MetadataAsBool"], out YourBoolean). Then simply add the YourBoolean to your new Dictionary<string,object> objectDictionary like objectDictionary.Add("BoolMetadata", YourBoolean)
I found a solution. The resulting object (ValueKind=String: 23423) is nothing else than a JSONElement. I did not understand this before.
This JSONElement has an enum that tells me what datatype I have, so I can use this to map my Dictionary of JSONElements to another dictionary of "real" objects.
var newDic = new Dictionary<string, object>();
foreach (var d in obj.DynamicInfo)
{
object obt;
string key = d.Key;
var a = enterprise.BasicInformation.TryGetValue(key, out obt);
if (obt == null) continue;
var doc = (JsonElement)obt;
string myString = null;
bool? myBool = null;
int? myInteger = null;
double? myFloatNumber = null;
if (doc.ValueKind == JsonValueKind.String)
{
myString = doc.ToString();
}
if (doc.ValueKind == JsonValueKind.True)
{
myBool = true;
}
if (doc.ValueKind == JsonValueKind.False)
{
myBool = false;
}
if (doc.ValueKind == JsonValueKind.Number)
{
double floatNumber;
doc.TryGetDouble(out floatNumber);
if ((floatNumber % 1) == 0)
{
myInteger = (int)floatNumber;
}
else
{
myFloatNumber = floatNumber;
}
}
if (myString != null)
{
newDic.Add(key, myString);
}
if (myBool != null)
{
newDic.Add(key, myBool);
}
if (myInteger != null)
{
newDic.Add(key, myInteger);
}
if (myFloatNumber != null)
{
newDic.Add(key, myFloatNumber);
}
}
The code might not be perfect - I will try to optimize it. But it does what it should.

Returning a reference of a struct instead of a copy on C# 3.0?

I have this code:
using System;
using System.Collections.Generic;
using UnityEngine;
public interface HaveId
{
int id { get; }
}
public struct BusinessData : HaveId
{
// business type data
public int graphic_asset_id;
public string name;
public int id { get; set; }
}
public class LookupHelper<T> where T: HaveId
{
private T[] _list;
public T[] list
{
get { return _list; }
set { _list = value; _mapToDictionary(); }
}
private Dictionary<int, int> idxById = new Dictionary<int, int>();
public LookupHelper(){}
private void _mapToDictionary()
{
if(idxById.Count > 0) idxById = new Dictionary<int, int>();
for(var z =0 ; z < list.Length; ++z)
{
idxById[list[z].id] = z;
}
}
public bool IsIdExists(int id)
{
return idxById.ContainsKey(id);
}
public T ById(int id) // is this a reference?
{
var idx = idxById[id];
if (idx >= list.Length) throw new Exception(
String.Format("Invalid Index: {0} >= {1} on {2}",idx.ToString(),list.Length.ToString(), typeof(T).Name)
);
return list[idx];
}
}
And the test code:
LookupHelper<BusinessData> bd = new LookupHelper<BusinessData>();
bd.list = new BusinessData[]
{
new BusinessData{id = 1, name = "test"},
new BusinessData{id = 2, name = "test2"},
};
bd.ById(1).name = "foo";
This give an error: "Cannot modify struct member when accessed struct is not classified as a variable"
How can I change the value of first BusinessData and keep the array still allocated on a contiguous memory (array of struct, needed for cache locality)?
This should be a simple matter of splitting it up into a few lines. Extract the object to get a copy, modify the copy, then overwrite it in the array:
BusinessData bsd = bd.ById(1);
bsd.name = "foo";
bd.SetById(1, bsd);
Of course, you'll need to write that SetById method to reinsert things into the array:
public void SetById(int id, T obj)
{
Int32 idx = idxById[id];
list[idx] = obj;
}
As you know C# borrowed something’s from C and Java. But not everything.
In C, you can create a place for struct on the stack or the heap. On the heap, I can then pass a pointer around and change the content. Very powerful.
But C# emphasizes ease of memory management via garbage collection. To make it easy, C# has the concept of boxing value types into System.Object. Additional details, can be found on Microsoft C# Programming Guide on Boxing and unboxing.
So when you access the value type in your list, you have to explicitly unbox the value. Therefore it’s a copy of the item in the list. You can do what #Nyerguds suggested.
But to make life easy, why not turn your BusinessData into a class?

How to maintain instance number in list of objects

I have a class baseClass, and a list of objects of the baseClass. What i want to achieve is that i have to dynamically assign the instance number to each object in the list. for that what am doing is that use a constructor to do this.
Following is the class definition:
public class baseClass
{
private int _InstanceNumber;
private int _MyIntVal;
private string _MyString;
public string MyString
{
get { return _MyString; }
set { _MyString = value; }
}
public int MyIntVal
{
get { return _MyIntVal; }
set { _MyIntVal = value; }
}
public int MyProperty
{
get { return _InstanceNumber; }
}
public baseClass(int instance)
{
_InstanceNumber = instance;
}
}
The creation of the List of objects is as follows:
int instanceNumber = 0;
List<baseClass> classList = new List<baseClass>();
classList.Add(new baseClass(instanceNumber++) { MyString = "sample1", MyIntVal = 10 });
classList.Add(new baseClass(instanceNumber++) { MyString = "sample2", MyIntVal = 11 });
I know it is not the actual way for creating this. it does not give the index number actually. how can i calculate the instance number?
Consider the following scenario, that am creating another list of objects then it hard to maintain the instance number. or if i create another object(this also be an instance) external to the list.
int instanceNumber = 0;
List<baseClass> anotherClassList = new List<baseClass>();
classList.Add(new baseClass(instanceNumber++) { MyString = "sample1", MyIntVal = 10 });
classList.Add(new baseClass(instanceNumber++) { MyString = "sample2", MyIntVal = 11 });
Updates:
This is my temporary solution for this. i need proper way/ method to maintain instance number
If you want to find the index of item in the list, you should ask it from the list, not the item like:
var index = list.IndexOf(item);
But it seems that you expect the item to be aware of its position in the list. In order to do this, you should pass the list to the item so it can use it to find its own place in it:
public class Item
{
private List<Item> _containerList;
public Item(List<Item> containerList)
{
_containerList = containerList;
}
public int InstanceNumber
{
get { return _containerList.IndexOf(this); }
}
}
and change your code to:
List<Item> classList = new List<Item>();
classList.Add(new Item(classList ) { ... });
classList.Add(new Item(classList ) { ... });

On string interning and alternatives

I have a large file which, in essence contains data like:
Netherlands,Noord-holland,Amsterdam,FooStreet,1,...,...
Netherlands,Noord-holland,Amsterdam,FooStreet,2,...,...
Netherlands,Noord-holland,Amsterdam,FooStreet,3,...,...
Netherlands,Noord-holland,Amsterdam,FooStreet,4,...,...
Netherlands,Noord-holland,Amsterdam,FooStreet,5,...,...
Netherlands,Noord-holland,Amsterdam,BarRoad,1,...,...
Netherlands,Noord-holland,Amsterdam,BarRoad,2,...,...
Netherlands,Noord-holland,Amsterdam,BarRoad,3,...,...
Netherlands,Noord-holland,Amsterdam,BarRoad,4,...,...
Netherlands,Noord-holland,Amstelveen,BazDrive,1,...,...
Netherlands,Noord-holland,Amstelveen,BazDrive,2,...,...
Netherlands,Noord-holland,Amstelveen,BazDrive,3,...,...
Netherlands,Zuid-holland,Rotterdam,LoremAve,1,...,...
Netherlands,Zuid-holland,Rotterdam,LoremAve,2,...,...
Netherlands,Zuid-holland,Rotterdam,LoremAve,3,...,...
...
This is a multi-gigabyte file. I have a class that reads this file and exposes these lines (records) as an IEnumerable<MyObject>. This MyObject has several properties (Country,Province,City, ...) etc.
As you can see there is a LOT of duplication of data. I want to keep exposing the underlying data as an IEnumerable<MyObject>. However, some other class might (and probably will) make some hierarchical view/structure of this data like:
Netherlands
Noord-holland
Amsterdam
FooStreet [1, 2, 3, 4, 5]
BarRoad [1, 2, 3, 4]
...
Amstelveen
BazDrive [1, 2, 3]
...
...
Zuid-holland
Rotterdam
LoremAve [1, 2, 3]
...
...
...
...
When reading this file, I do, essentially, this:
foreach (line in myfile) {
fields = line.split(",");
yield return new MyObject {
Country = fields[0],
Province = fields[1],
City = fields[2],
Street = fields[3],
//...other fields
};
}
Now, to the actual question at hand: I could use string.Intern() to intern the Country, Province, City, and Street strings (those are the main 'vilains', the MyObject has several other properties not relevant to the question).
foreach (line in myfile) {
fields = line.split(",");
yield return new MyObject {
Country = string.Intern(fields[0]),
Province = string.Intern(fields[1]),
City = string.Intern(fields[2]),
Street = string.Intern(fields[3]),
//...other fields
};
}
This will save about 42% of memory (tested and measured) when holding the entire dataset in memory since all duplicate strings will be a reference to the same string. Also, when creating the hierarchical structure with a lot of LINQ's .ToDictionary() method the keys (Country, Province etc.) of the resp. dictionaries will be much more efficient.
However, one of the drawbacks (aside a slight loss of performance, which is not problem) of using string.Intern() is that the strings won't be garbage collected anymore. But when I'm done with my data I do want all that stuff garbage collected (eventually).
I could use a Dictionary<string, string> to 'intern' this data but I don't like the "overhead" of having a key and value where I am, actually, only interested in the key. I could set the value to null or the use the same string as value (which will result in the same reference in key and value). It's only a small price of a few bytes to pay, but it's still a price.
Something like a HashSet<string> makes more sense to me. However, I cannot get a reference to a string in the HashSet; I can see if the HashSet contains a specific string, but not get a reference to that specific instance of the located string in the HashSet. I could implement my own HashSet for this, but I am wondering what other solutions you kind StackOverflowers may come up with.
Requirements:
My "FileReader" class needs to keep exposing an IEnumerable<MyObject>
My "FileReader" class may do stuff (like string.Intern()) to optimize memory usage
The MyObject class cannot change; I won't make a City class, Country class etc. and have MyObject expose those as properties instead of simple string properties
Goal is to be (more) memory efficient by de-duplicating most of the duplicate strings in Country, Province, City etc.; how this is achieved (e.g. string interning, internal hashset / collection / structure of something) is not important. However:
I know I can stuff the data in a database or use other solutions in such direction; I am not interested in these kind of solutions.
Speed is only of secondary concern; the quicker the better ofcourse but a (slight) loss in performance while reading/iterating the objects is no problem
Since this is a long-running process (as in: windows service running 24/7/365) that, occasionally, processes a bulk of this data I want the data to be garbage-collected when I'm done with it; string interning works great but will, in the long run, result in a huge string pool with lots of unused data
I would like any solutions to be "simple"; adding 15 classes with P/Invokes and inline assembly (exaggerated) is not worth the effort. Code maintainability is high on my list.
This is more of a 'theoretical' question; it's purely out of curiosity / interest that I'm asking. There is no "real" problem, but I can see that in similar situations this might be a problem to someone.
For example: I could do something like this:
public class StringInterningObject
{
private HashSet<string> _items;
public StringInterningObject()
{
_items = new HashSet<string>();
}
public string Add(string value)
{
if (_items.Add(value))
return value; //New item added; return value since it wasn't in the HashSet
//MEH... this will quickly go O(n)
return _items.First(i => i.Equals(value)); //Find (and return) actual item from the HashSet and return it
}
}
But with a large set of (to be de-duplicated) strings this will quickly bog down. I could have a peek at the reference source for HashSet or Dictionary or... and build a similar class that doesn't return bool for the Add() method but the actual string found in the internals/bucket.
The best I could come up with until now is something like:
public class StringInterningObject
{
private ConcurrentDictionary<string, string> _items;
public StringInterningObject()
{
_items = new ConcurrentDictionary<string, string>();
}
public string Add(string value)
{
return _items.AddOrUpdate(value, value, (v, i) => i);
}
}
Which has the "penalty" of having a Key and a Value where I'm actually only interested in the Key. Just a few bytes though, small price to pay. Coincidally this also yields 42% less memory usage; the same result as when using string.Intern() yields.
tolanj came up with System.Xml.NameTable:
public class StringInterningObject
{
private System.Xml.NameTable nt = new System.Xml.NameTable();
public string Add(string value)
{
return nt.Add(value);
}
}
(I removed the lock and string.Empty check (the latter since the NameTable already does that))
xanatos came up with a CachingEqualityComparer:
public class StringInterningObject
{
private class CachingEqualityComparer<T> : IEqualityComparer<T> where T : class
{
public System.WeakReference X { get; private set; }
public System.WeakReference Y { get; private set; }
private readonly IEqualityComparer<T> Comparer;
public CachingEqualityComparer()
{
Comparer = EqualityComparer<T>.Default;
}
public CachingEqualityComparer(IEqualityComparer<T> comparer)
{
Comparer = comparer;
}
public bool Equals(T x, T y)
{
bool result = Comparer.Equals(x, y);
if (result)
{
X = new System.WeakReference(x);
Y = new System.WeakReference(y);
}
return result;
}
public int GetHashCode(T obj)
{
return Comparer.GetHashCode(obj);
}
public T Other(T one)
{
if (object.ReferenceEquals(one, null))
{
return null;
}
object x = X.Target;
object y = Y.Target;
if (x != null && y != null)
{
if (object.ReferenceEquals(one, x))
{
return (T)y;
}
else if (object.ReferenceEquals(one, y))
{
return (T)x;
}
}
return one;
}
}
private CachingEqualityComparer<string> _cmp;
private HashSet<string> _hs;
public StringInterningObject()
{
_cmp = new CachingEqualityComparer<string>();
_hs = new HashSet<string>(_cmp);
}
public string Add(string item)
{
if (!_hs.Add(item))
item = _cmp.Other(item);
return item;
}
}
(Modified slightly to "fit" my "Add() interface")
As per Henk Holterman's request:
public class StringInterningObject
{
private Dictionary<string, string> _items;
public StringInterningObject()
{
_items = new Dictionary<string, string>();
}
public string Add(string value)
{
string result;
if (!_items.TryGetValue(value, out result))
{
_items.Add(value, value);
return value;
}
return result;
}
}
I'm just wondering if there's maybe a neater/better/cooler way to 'solve' my (not so much of an actual) problem. By now I have enough options I guess
Here are some numbers I came up with for some simple, short, preliminary tests:
Non optimizedMemory: ~4,5GbLoad time: ~52s
StringInterningObject (see above, the ConcurrentDictionary variant)Memory: ~2,6GbLoad time: ~49s
string.Intern()Memory: ~2,3GbLoad time: ~45s
System.Xml.NameTableMemory: ~2,3GbLoad time: ~41s
CachingEqualityComparerMemory: ~2,3GbLoad time: ~58s
StringInterningObject (see above, the (non-concurrent) Dictionary variant) as per Henk Holterman's request:Memory: ~2,3GbLoad time: ~39s
Although the numbers aren't very definitive, it seems that the many memory-allocations for the non-optimized version actually slow down more than using either string.Intern() or the above StringInterningObjects which results in (slightly) longer load times. Also, string.Intern() seems to 'win' from StringInterningObject but not by a large margin; << See updates.
I've had exactly this requirement and indeed asked on SO, but with nothing like the detail of your question, no useful responses. One option that is built in is a (System.Xml).NameTable, which is basically a string atomization object, which is what you are looking for, we had (we've actually move to Intern because we do keep these strings for App-life).
if (name == null) return null;
if (name == "") return string.Empty;
lock (m_nameTable)
{
return m_nameTable.Add(name);
}
on a private NameTable
http://referencesource.microsoft.com/#System.Xml/System/Xml/NameTable.cs,c71b9d3a7bc2d2af shows its implemented as a Simple hashtable, ie only storing one reference per string.
Downside? is its completely string specific. If you do cross-test for memory / speed I'd be interested to see the results. We were already using System.Xml heavily, might of course not seem so natural if you where not.
When in doubt, cheat! :-)
public class CachingEqualityComparer<T> : IEqualityComparer<T> where T : class
{
public T X { get; private set; }
public T Y { get; private set; }
public IEqualityComparer<T> DefaultComparer = EqualityComparer<T>.Default;
public bool Equals(T x, T y)
{
bool result = DefaultComparer.Equals(x, y);
if (result)
{
X = x;
Y = y;
}
return result;
}
public int GetHashCode(T obj)
{
return DefaultComparer.GetHashCode(obj);
}
public T Other(T one)
{
if (object.ReferenceEquals(one, X))
{
return Y;
}
if (object.ReferenceEquals(one, Y))
{
return X;
}
throw new ArgumentException("one");
}
public void Reset()
{
X = default(T);
Y = default(T);
}
}
Example of use:
var comparer = new CachingEqualityComparer<string>();
var hs = new HashSet<string>(comparer);
string str = "Hello";
string st1 = str.Substring(2);
hs.Add(st1);
string st2 = str.Substring(2);
// st1 and st2 are distinct strings!
if (object.ReferenceEquals(st1, st2))
{
throw new Exception();
}
comparer.Reset();
if (hs.Contains(st2))
{
string cached = comparer.Other(st2);
Console.WriteLine("Found!");
// cached is st1
if (!object.ReferenceEquals(cached, st1))
{
throw new Exception();
}
}
I've created an equality comparer that "caches" the last Equal terms it analyzed :-)
Everything could then be encapsulated in a subclass of HashSet<T>
/// <summary>
/// An HashSet<T;gt; that, thorough a clever use of an internal
/// comparer, can have a AddOrGet and a TryGet
/// </summary>
/// <typeparam name="T"></typeparam>
public class HashSetEx<T> : HashSet<T> where T : class
{
public HashSetEx()
: base(new CachingEqualityComparer<T>())
{
}
public HashSetEx(IEqualityComparer<T> comparer)
: base(new CachingEqualityComparer<T>(comparer))
{
}
public T AddOrGet(T item)
{
if (!Add(item))
{
var comparer = (CachingEqualityComparer<T>)Comparer;
item = comparer.Other(item);
}
return item;
}
public bool TryGet(T item, out T item2)
{
if (Contains(item))
{
var comparer = (CachingEqualityComparer<T>)Comparer;
item2 = comparer.Other(item);
return true;
}
item2 = default(T);
return false;
}
private class CachingEqualityComparer<T> : IEqualityComparer<T> where T : class
{
public WeakReference X { get; private set; }
public WeakReference Y { get; private set; }
private readonly IEqualityComparer<T> Comparer;
public CachingEqualityComparer()
{
Comparer = EqualityComparer<T>.Default;
}
public CachingEqualityComparer(IEqualityComparer<T> comparer)
{
Comparer = comparer;
}
public bool Equals(T x, T y)
{
bool result = Comparer.Equals(x, y);
if (result)
{
X = new WeakReference(x);
Y = new WeakReference(y);
}
return result;
}
public int GetHashCode(T obj)
{
return Comparer.GetHashCode(obj);
}
public T Other(T one)
{
if (object.ReferenceEquals(one, null))
{
return null;
}
object x = X.Target;
object y = Y.Target;
if (x != null && y != null)
{
if (object.ReferenceEquals(one, x))
{
return (T)y;
}
else if (object.ReferenceEquals(one, y))
{
return (T)x;
}
}
return one;
}
}
}
Note the use of WeakReference so that there aren't useless references to objects that could prevent garbage collection.
Example of use:
var hs = new HashSetEx<string>();
string str = "Hello";
string st1 = str.Substring(2);
hs.Add(st1);
string st2 = str.Substring(2);
// st1 and st2 are distinct strings!
if (object.ReferenceEquals(st1, st2))
{
throw new Exception();
}
string stFinal = hs.AddOrGet(st2);
if (!object.ReferenceEquals(stFinal, st1))
{
throw new Exception();
}
string stFinal2;
bool result = hs.TryGet(st1, out stFinal2);
if (!object.ReferenceEquals(stFinal2, st1))
{
throw new Exception();
}
if (!result)
{
throw new Exception();
}
edit3:
instead of indexing strings, putting them in non-duplicate lists will save much more ram.
we have int indexes in class MyObjectOptimized. access is instant.
if list is short(like 1000 item) speed of setting values wont be noticable.
i assumed every string will have 5 character .
this will reduce memory usage
percentage : 110 byte /16byte = 9x gain
total : 5gb/9 = 0.7 gb + sizeof(Country_li , Province_li etc )
with int16 index (will further halve ram usage )
*note:* int16 capacity is -32768 to +32767 ,
make sure your list is not bigger than 32 767
usage is same but will use the class MyObjectOptimized
main()
{
// you can use same code
foreach (line in myfile) {
fields = line.split(",");
yield
return
new MyObjectOptimized {
Country = fields[0],
Province = fields[1],
City = fields[2],
Street = fields[3],
//...other fields
};
}
}
required classes
// single string size : 18 bytes (empty string size) + 2 bytes per char allocated
//1 class instance ram cost : 4 * (18 + 2* charCount )
// ie charcounts are at least 5
// cost: 4*(18+2*5) = 110 byte
class MyObject
{
string Country ;
string Province ;
string City ;
string Street ;
}
public static class Exts
{
public static int AddDistinct_and_GetIndex(this List<string> list ,string value)
{
if( !list.Contains(value) ) {
list.Add(value);
}
return list.IndexOf(value);
}
}
// 1 class instance ram cost : 4*4 byte = 16 byte
class MyObjectOptimized
{
//those int's could be int16 depends on your distinct item counts
int Country_index ;
int Province_index ;
int City_index ;
int Street_index ;
// manuallly implemented properties will not increase memory size
// whereas field WILL increase
public string Country{
get {return Country_li[Country_index]; }
set { Country_index = Country_li.AddDistinct_and_GetIndex(value); }
}
public string Province{
get {return Province_li[Province_index]; }
set { Province_index = Province_li.AddDistinct_and_GetIndex(value); }
}
public string City{
get {return City_li[City_index]; }
set { City_index = City_li.AddDistinct_and_GetIndex(value); }
}
public string Street{
get {return Street_li[Street_index]; }
set { Street_index = Street_li.AddDistinct_and_GetIndex(value); }
}
//beware they are static.
static List<string> Country_li ;
static List<string> Province_li ;
static List<string> City_li ;
static List<string> Street_li ;
}

Serialize a class that uses reflection to fill its properties

I have the following code:
[Serializable]
public class CustomClass
{
public CustomClass()
{
this.Init();
}
public void Init()
{
foreach (PropertyInfo p in this.GetType().GetProperties())
{
DescriptionAttribute da = null;
DefaultValueAttribute dv = null;
foreach (Attribute attr in p.GetCustomAttributes(true))
{
if (attr is DescriptionAttribute)
{
da = (DescriptionAttribute) attr;
}
if (attr is DefaultValueAttribute)
{
dv = (DefaultValueAttribute) attr;
}
}
UInt32 value = 0;
if (da != null && !String.IsNullOrEmpty(da.Description))
{
value = Factory.Instance.SelectByCode(da.Description, 3);
}
if (dv != null && value == 0)
{
value = (UInt32) dv.Value;
}
p.SetValue(this, value, null);
}
}
private UInt32 name;
[Description("name")]
[DefaultValue(41)]
public UInt32 Name
{
get { return this.name; }
set { this.name = value; }
}
(30 more properties)
}
Now the weird thing is: when I try to serialize this class I will get an empty node CustomClass!
<CustomClass />
And when I remove Init from the constructor it works as expected! I will get the full xml representation of the class but ofcourse without values (all with value 0).
<CustomClass>
<Name>0</Name>
...
</CustomClass>
Also, when I comment out the body of Init, I will get the same as above (the one with default values)
I've tried it with a public method, with a Helper class everything, but it does not work. That is, instead of the expected:
<CustomClass>
<Name>15</Name>
...
</CustomClass>
I will get
<CustomClass />
It seems when I use reflection in this class, serialization is not possible.
Or to summarize: when I call Init or when I fill my properties with reflection -> Serialization fails, when I remove this code part -> Serialization works but of course without my values.
Is this true? And does somebody know an alternative for my solution?
It should automatically get something from the database based on the Description and when this returns nothing it falls back to the DefaultValue...
PS1: I am using the XmlSerializer
PS2: When I set a breakpoint before the serialization, I can see that all the properties are filled with the good values (like 71, 72 etc).
Now the weird thing is: when I try to serialize this class I will get an empty node CustomClass!
XmlSerializer uses DefaultValue to decide which values to serialize - if it matches the default value, it doesn't store it. This approach is consistent with similar models such as data-binding / model-binding.
Frankly, I would say that in this case both DefaultValueAttribute and DescriptionAttribute are poor choices. Write your own - perhaps EavInitAttribute - then use something like:
[EavInit(41, "name")]
public uint Name {get;set;}
Note that there are other ways of controlling this conditional serialization - you could write a method like:
public bool ShouldSerializeName() { return true; }
which will also work to convince it to write the value (this is another pattern recognised by various serialization and data-binding APIs) - but frankly this is even more work (it is per-property, and needs to be public, so it makes a mess of the API).
Finally, I would say that hitting the database multiple times (once per property) for every new object construction is very expensive - especially since many of those values are likely to be assigned values in a moment anyway (so looking them up is wasted effort). I would put a lot of thought into making this both "lazy" and "cached" if it was me.
An example of a lazy and "sparse" implementation:
using System;
using System.Collections.Generic;
using System.Runtime.CompilerServices;
using System.Xml.Serialization;
static class Program
{
static void Main()
{
var obj = new CustomClass();
Console.WriteLine(obj.Name);
// show it working via XmlSerializer
new XmlSerializer(obj.GetType()).Serialize(Console.Out, obj);
}
}
public class CustomClass : EavBase
{
[EavInit(42, "name")]
public uint Name
{
get { return GetEav(); }
set { SetEav(value); }
}
}
public abstract class EavBase
{
private Dictionary<string, uint> values;
protected uint GetEav([CallerMemberName] string propertyName = null)
{
if (values == null) values = new Dictionary<string, uint>();
uint value;
if (!values.TryGetValue(propertyName, out value))
{
value = 0;
var prop = GetType().GetProperty(propertyName);
if (prop != null)
{
var attrib = (EavInitAttribute)Attribute.GetCustomAttribute(
prop, typeof(EavInitAttribute));
if (attrib != null)
{
value = attrib.DefaultValue;
if (!string.IsNullOrEmpty(attrib.Key))
{
value = LookupDefaultValueFromDatabase(attrib.Key);
}
}
}
values.Add(propertyName, value);
}
return value;
}
protected void SetEav(uint value, [CallerMemberName] string propertyName = null)
{
(values ?? (values = new Dictionary<string, uint>()))[propertyName] = value;
}
private static uint LookupDefaultValueFromDatabase(string key)
{
// TODO: real code here
switch (key)
{
case "name":
return 7;
default:
return 234;
}
}
[AttributeUsage(AttributeTargets.Property, AllowMultiple = false, Inherited = true)]
protected class EavInitAttribute : Attribute
{
public uint DefaultValue { get; private set; }
public string Key { get; private set; }
public EavInitAttribute(uint defaultValue) : this(defaultValue, "") { }
public EavInitAttribute(string key) : this(0, key) { }
public EavInitAttribute(uint defaultValue, string key)
{
DefaultValue = defaultValue;
Key = key ?? "";
}
}
}

Categories

Resources