I need to transfer .NET objects (with hierarchy) over network (multiplayer game). To save bandwidth, I'd like to transfer only fields (and/or properties) that changes, so fields that won't change won't transfer.
I also need some mechanism to match proper objects on the other client side (global object identifier...something like object ID?)
I need some suggestions how to do it.
Would you use reflection? (performance is critical)
I also need mechanism to transfer IList deltas (added objects, removed objects).
How is MMO networking done, do they transfer whole objects?
(maybe my idea of per field transfer is stupid)
EDIT:
To make it clear: I've already got mechanism to track changes (lets say every field has property, setter adds field to some sort of list or dictionary, which contains changes - structure is not final now).
I don't know how to serialize this list and then deserialize it on other client. And mostly how to do it effectively and how to update proper objects.
There's about one hundred of objects, so I'm trying avoid situation when I would write special function for each object. Decorating fields or properties with attributes would be ok (for example to specify serializer, field id or something similar).
More about objects: Each object has 5 fields in average. Some object are inherited from other.
Thank you for all answeres.
Another approach; don't try to serialize complex data changes: instead, send just the actual commands to apply (in a terse form), for example:
move 12432 134, 146
remove 25727
(which would move 1 object and remove another).
You would then apply the commands at the receiver, allowing for a full resync if they get out of sync.
I don't propose you would actually use text for this - that is just to make the example clearer.
One nice thing about this: it also provides "replay" functionality for free.
The cheapest way to track dirty fields is to have it as a key feature of your object model, I.e. with a "fooDirty" field for every data field "foo", that you set to true in the "set" (if the value differs). This could also be twinned with conditional serialization, perhaps the "ShouldSerializeFoo()" pattern observed by a few serializers. I'm not aware of any libraries that match exactly what you describe (unless we include DataTable, but ... think of the kittens!)
Perhaps another issue is the need to track all the objects for merge during deserialization; that by itself doesn't come for free.
All things considered, though, I think you could do something alon the above lines (fooDirty/ShouldSerializeFoo) and use protobuf-net as the serializer, because (importantly) that supports both conditional serialization and merge. I would also suggest an interface like:
ISomeName {
int Key {get;}
bool IsDirty {get;}
}
The IsDrty would allow you to quickly check all your objects for those with changes, then add the key to a stream, then the (conditional) serialization. The caller would read the key, obtain the object needed (or allocate a new one with that key), and then use the merge-enabled deserialize (passing in the existing/new object).
Not a full walk-through, but if it was me, that is the approach I would be looking at. Note: the addition/removal/ordering of objects in child-collections is a tricky area, that might need thought.
I'll just say up front that Marc Gravell's suggestion is really the correct approach. He glosses over some minor details, like conflict resolution (you might want to read up on Leslie Lamport's work. He's basically spent his whole career describing different approaches to dealing with conflict resolution in distributed systems), but the idea is sound.
If you do want to transmit state snapshots, instead of procedural descriptions of state changes, then I suggest you look into building snapshot diffs as prefix trees. The basic idea is that you construct a hierarchy of objects and fields. When you change a group of fields, any common prefix they have is only included once. This might look like:
world -> player 1 -> lives: 1
... -> points: 1337
... -> location -> X: 100
... -> Y: 32
... -> player 2 -> lives: 3
(everything in a "..." is only transmitted once).
It is not logical to transfer only changed fields because you would be wasting your time on detecting which fields changed and which didn't and how to reconstruct on the receiver's side which will add a lot of latency to your game and make it unplayable online.
My proposed solution is for you to decompose your objects to the minimum and sending these small objects which is fast. Also, you can use compression to reduce bandwidth usage.
For the Object ID, you can use a static ID which increases when you construct a new Object.
Hope this answer helps.
You will need to do this by hand. Automatically keeping track of property and instance changes in a hierarchy of objects is going to be very slow compared to anything crafted by hand.
If you decide to try it out anyway, I would try to map your objects to a DataSet and use its built in modification tracking mechanisms.
I still think you should do this by hand, though.
Related
Background: I have a list of objects that are directly linked to the UI in WPF. I need to do some work with those objects, but while I am working with them (asynchronously) I do not want any refreshes on the UI (performance and aesthetic reasons).
So I thought, I might copy the items (using Collection<T>.CopyTo(T[] array, int index)), work on the copy, then override my original list with the copied one. The problem is, that even then the reference are kept and the UI is continuously refreshed.
Code example of what I did:
MyUIObject[] myCopiedList = new MyUIObject[MyObjectsLinkedToTheUI.Count];
MyObjectsLinkedToTheUI.CopyTo(myCopiedList);
foreach (MyUIObject myCopiedItem in myCopiedList)
{
//while I do this, the UI is still updated
PerformLongAndWearyOperationAsync(myCopiedItem);
}
MyObjectsLinkedToTheUI.Clear();
foreach (var myCopiedItem in myCopiedList)
{
MyObjectsLinkedToTheUI.Add(myCopiedItem);
}
Is there a possibility to copy my items without keeping a reference to the original object?
UPDATE 1
Thank you for your contributions so far. One thing I forgot to mention: This is for Windows Phone 8.1, so ICloneable is not available.
You need to clone them somehow. Either implement ICloneable interface and do all rewriting manually or you can use some hacks/tricks like serializing and deserializing object.
Flow with serialization is something like this:
Take your object
Serialize it to, for example, JSON, binary format etc.
Now deserialize what you got in step 2 into new object
You'll have a copy of your object that way but it costs more processing power and is prone to some hard to catch errors. But it's an easy way to go. Thing with implementing ICloneable is more reliable but you need to write all that mapping by yourself.
Also, consider using structs instead of classes. Structs are always copied by value not reference. It has some drawbacks so it's up to you if they suit your usage scenario.
i am building a sort of program that generates a random list of word according to a database.
I Made a class that deals with the word selecting and handling (a random select function, a connect to the database function etc..)
I have 3 variables that indicate the last 3 words chosen.
how do I use a funcion on the form1 (button 1 press), to manipulate the same 3 variables, without creating them from scratch everytime (what happens now...)
To make myself clearer:
accualy what I need is to know how to keep track of a variable between multiple classes.
I might be using the whole classes thing wrong... I am now triyng to get the grasp of it.
Thank you very much,
Barak.
Your two options as I see it are:
1) an instance of a class that holds those variables that can be passed around
You may want to use the singleton pattern for this class if you want to make sure there is only ever one of them.
2) A static class with static members holding this information.
It may be that your entire random word class could be static. In this case you'd just call the methods and properties on that class to generate and access your words.
Also I would suggest that you may want to consider a collection to hold your words rather than three separate variables. It will of course depend on your implementation so I will mention it just inc ase you haven't thought of it and I'm not saying you definitely should. :)
I would avoid static or Singletons just for this purpose - they're not good habits to pick up for simple object oriented scenarios.
Encapsulate the state variables in a class, which you instantiate first, then pass by reference into the form and/or data fetch logic.
Key to this is understanding the concept of reference - your form and fetch logic will see the same instance of your state class, effectively sharing it.
If you implement the "variables" as properties on the state class, you can use events to notify other parts of your code when the word states change.
Consider also clearly defining the possible interactions (interfaces) on the state class. One aspect seems to be to add a word, another to pull out statistics based on the added words. The state class can accommodate all this, and provide a nice place for future extensions.
Try to think in terms of public interface methods/properties, while keeping "variables" (i.e. fields like counters or collections) private.
I also agree that your post should be improved with snippets of actual code - help us helping you.
And I hope your code is not being used to generate spam mails/posts... :-)
Why do we need reference types in .NET?
I can think of only 1 cases, that it support sharing data between different functions and hence gives storage optimization.
Other than that I could not enumerate any reason, why reference types are needed?
Why do we need reference types in .NET? I can think of only one reason: that it support sharing of data and hence gives storage optimization.
You've answered your own question. Do you need a better reason than that?
Suppose every time you wanted to refer to the book The Hobbit, you had to instead make a copy of the entire text. That is, instead of saying "When I was reading The Hobbit the other day...", you'd have to say "When I was reading In a hole in the ground there lived a hobbit... [all the text] ... Well thank goodness for that, said Bilbo, handing him the tobacco jar. the other day..."
Now suppose every time you used a database in a program, instead of referring to the database, you simply made a full copy of the entire database, every single time you used any of it in any way. How fast do you think such a program would be?
References allow you to write sentences that talk about books by use of their titles instead of their contents. Reference types allow you to write programs that manipulate objects by using small references rather that enormous quantities of data.
class Node {
Node parent;
}
Try implementing that without a reference type. How big would it be? How big would a string be? An array? How much space would you need to reserve on the stack for:
string s = GetSomeString();
How would any data be used in a method that wasn't specific to one call-path? Multi-threaded code, for example.
Three reasons that I can think of off the top of my head.
You don't want to continually copy objects every time you need to pass them to a Method or Collection Type.
When iterating through collections, you may want to modify the original object with new values.
Limited Stack Space.
If you look at value types like int, long, float you can see that the biggest type store 8 bytes or 64 bits.
However, think about a list or an array of long values, in that case, if we have a list of 1000 values then the worst case will take 8000 bytes.
Now, to pass by value 8000 bytes will make our program to run super slow, because the function that took the list as a parameter will now have to copy all these values into a new list and by that we loose time and space.
That's why we have reference types, because if we pass that list then we don't lose time and space to copy that list because we pass the address of the list in the memory.
The reference type in the function will work on the same address as the list you passed, and if you want to copy that list you can do that manually.
By using reference types we save time and space for our program because we don't bother to copy and save the argument we passed.
I have a class that basically represents the parameters of a model, and encapsulates the logic to calculate values of the model with those parameters. I'm trying to decide if this class should be immutable. In practice, instances of the model will be generated by fitting to some data set, so in that sense it makes sense (to me at least) for that instance to be immutable since it's tied to external data.
On the other hand, there will be a GUI to let a user do a "what-if" wherein they can change the parameters to see how it changes model values. So I could make the model mutable to make this easy, or create new copies every time a parameter is changed. The latter seems awkward, especially if there are e.g. 5 parameters that could be ticked up and down individually...seems like I would have to implement a SetX() method for each parameter which returns a copy, right?
Am I overthinking this, or is there a proper pattern to use here? (This is C# code, though I guess not really language-specific)
Consider carefully how the object is going to actually be used for your speculative analysis. For straightforward, one-off, let's-mutate-the-field-try-something-and-change-it-back scenarios, sure, just make it mutable. But suppose you want to pull out the big guns; then it becomes much nicer to have an immutable model.
Data d = whatever;
// What if we mutate X and Y? Which one maximizes the value of Foo(d) ?
var query = from x in Range(0, 100)
from y in Range(0, 100)
let mutated = data.MutateX(x).MutateY(y)
orderby Foo(mutated)
select mutated;
var max = query.First();
And so on. With an immutable pattern it becomes much easier to write speculative queries, it becomes much easier to parallelize those queries across multiple cores, and so on.
The latter seems awkward
Well that's your answer. The goal of API design is to make writing code as easy as possible. If a particular pattern makes it more difficult or awkward than the alternative, the alternative is probably correct.
I think you're probably overthinking this a little bit. While there is probably a very elegant design pattern for this that uses eight classes and four interfaces, I think the simplest route forward would be to make it a normal, mutable class. Think about your intention: you want a Model that can be loaded from external data (perhaps a static method returning a Model instance) with parameters that can change according to user input. This seems like a use case for your everyday, garden variety Class.
You may also choose to separate your classes into a Data class and Strategy class, the second which contains the changeable parameters and uses something like a Strategy pattern to calculate the results.
What is the best way to deep clone an interconnected set of objects? Example:
class A {
B theB; // optional
// ...
}
class B {
A theA; // optional
// ...
}
class Container {
A[] a;
B[] b;
}
The obvious thing to do is walk the objects and deep clone everything as I come to it. This creates a problem however -- if I clone an A that contains a B, and that B is also in the Container, that B will be cloned twice after I clone the Container.
The next logical step is to create a Dictionary and look up every object before I clone it. This seems like it could be a slow and ungraceful solution, however.
Any thoughts?
Its not an elegant solution for sure, but it isn't uncommon to use a dictionary (or hashmap). One of the benefits is that a hashmap has a constant lookup time, so speed does not really suffer here.
Not that I am familiar with C#, but typically any type of crawling of a graph for some sort of processing will require a lookup table to stop processing an object due to cyclic references. So I would think you will need to do the same here.
The dictionary solution you suggested is the best I know of. To optimize further, you could use object.GetHashCode() to get a hash for the object, and use that as the dictionary key. Should be fast unless you're talking about huge object trees (10s to 100s of thousands of objects).
maybe create a bit flag to indicate whether this object has been cloned before.
Another possible solution you could investigate is serializing the objects into a stream, and then reconstructing them from that same stream into new instances. This often works wonders when everything else seems awfully convoluted and messy.
Marc
One of the practical ways to do deep cloning is serializing and then deserializing a source graph. Some serializers in .NET like DataContractSerializer are even capable of processing cycles within graphs. You can choose which serializer is the best choice for your scenario by looking at the feature comparison chart.