C# Large object in medium size collection - c#

I'm pretty new to the memory problem. Hope you don't think this is a stupid question to ask.
I know that memory larger than 85,000 Bytes would be put into LOH in C#
i.e.
Byte[] hugeByteCollection = new Byte[85000];
I'm wondering if a collection with size 10000 - 20000 with an object that contains 10 member variables (byte type) will be put into LOH or SOH ?

The size of an array of objects is the number of objects times the pointer size. This is because only value types is stored in the array itself, reference types (objects) will be stored somewhere else and will not count towards the size of the array. So 85000/4=21250 objects, and 85000/8=10625 objects can be stored in an array on the SOH in 32bit and 64bit mode, respectively.
Edit:
Thanks to Hans Passant for pointing out that this assumes that the collection type used is an array and not a list. Lists resize themselves to be bigger than the content to avoid too many allocations. See this link for details

Related

What are exaclty Large Objects in C#?

Can I call an array of 1 million integers as large object? Or one instance of that object should be > 85 KB to be considered as large object?
If I make an array like int[1000000]. This whole object with each member is treated as one object with size > 85 KB. right?
If I have a class X {int i; string j} .. then List having count > 100000. Will this be saved to LOH?
Basically what I mean is if the size of an object like Class X is say 8.6KB and I make a datastructure like List myList. Then if the list count is 9 then it is not LO but if it has count 10 then it is?
I want a last answer(almost go all the answers):
Now I know that array is a collection of pointers of 8 bytes. So an array to be Large object it should have 85000/8 number of elements or more. Is that correct?
Any object larger than 85,000 bytes is considered to a large object and is treated differently during garbage collection. An array can itself be over 85,000 bytes large if all its references (aka pointers) make up that amount.
In case of arrays the actual count is not made up of the size of the objects but their references. Let's say you have an array of Customer, and let's assume that a Customer has 3 integers, each 8 bytes in size, and each reference is also 8 bytes in size. Then an array of 10 Customers actually takes up 80 bytes, not 240. The 24 bytes of each Customer are separate objects.
See this article for further information, and refer also to the Large Object Heap Improvements in .NET 4.5.
If you are asking in context of garbage collection then yes 85 KB would be a big object since big objects are immediately marked as a generation 2 objects.

Does C# Array resize take longer the bigger the array?

In my application I do System.Array.Resize once per frame. Initially I set my arrays to a maximum possible size, and then Resize them to something smaller. In some cases it may be a lot smaller, in others it may be just a little smaller. It appears to me though that the more elements there are to resize, the longer it takes. Perhaps my observations are wrong, and that is why I am asking here.
It should do yes, resizing involves allocating new memory to the size you want and copying the old array into the new one. The larger the array, the more to copy.
From MSDN:
This method allocates a new array with the specified size, copies
elements from the old array to the new one, and then replaces the old
array with the new one.
Without knowing too much about the code, try using List<T> to manage the list and the resizing you need to do and when you need to provide it to Unity, call list.ToArray();.
This will still create the array and copy it, but only once per frame.
As other answers note, "resizing" an array requires copying all the elements, which is an O(N) operation when N gets large. Note that there are a number of approaches that can be used for copying arrays, with differing "setup" and "per-item" costs. A small array-copy operation may be processed 4 bytes at a time (or in some cases, one byte at a time), while a larger array operation would use special 16-byte operations to do most of the copying. These operations are limited to writing aligned 16-byte chunks of memory at a time. Depending upon source and destination alignment, a large array operation might require copying four groups of four bytes (the last byte of which will overlap the next group), many groups of 16 bytes, and four more groups of four bytes (the first byte of which will overlap the previous group). Determining how to subdivide the groups is a little tricky, so for smaller block-copy requests it's more efficient to use one- or four-byte operations.
Note that the real key to minimizing the expense of array resizing is to do it as seldom as possible. Whenever the List<T> type has to expand the size of its array, it doubles it. If its array starts at 16 items, then at the time it doubles the array to 256 elements, 128 will be empty, 64 will have been copied once, 32 will be copied twice, and 16 will have been copied three times. Note that while some elements will end up being copied lg(N) times, the total number of element copy operations in the process of building a list of size N will always be less than 2N.
There's no way to access the backing array of a List<T> as an array, but it's fairly easy to re-implement the class in such a way as to expose the array, and make sure any methods that accept an array as a parameter allow one to specify the length of the portion to be used (instead of just accessing the Length property of the array).
Yes. Array resizing is an O(n) operation. It has to copy each element into the new array.
Maybe it would be better however if you did not use arrays? What are the arrays used for? There might be a better data structure suitable for you application.

GC.GetTotalMemory use and its return value

I have a binary file save on disk of size 15KB but why its memory size is always 4 bytes only
long mem1=GC.GetTotalMemory(false);
Object[] array= new Object[1000000];
array[1]=obj; // obj is the object content of the file before it is saved on disk
long mem2=GC.GetTotalMemory(false);
long sizeOfOneElementInArray=(mem2-mem1)/1000000;
I am wrong about something somewhere. I think it is incorrect because 4 bytes is not enough to store even a hello world string, but why is it incorrect.
Thanks for any help.
Is the assumption that by assigning obj to the index [1] in the array o that it would take a substantial number of bytes? All you are doing is assigning a reference. Not only that, but all that new Object[1000000] did was create an array (space to associate 1,000,000 Object's and the memory required by Object[].), not allocate 1,000,000 Object's. I am sure someone can elaborate even more about the internal data structures being used and why 4 bytes shows up.
The key thing to realize is that assigned obj to o[1] is not allocating additional memory for obj. If you are trying to determine an approximation call GC.GetTotalMemory before obj is allocated, then after. In your test obj is already allocated before you call the first GC.GetTotalMemory
In general, when MSDN documentation says things like A number that is the best available approximation of the number of bytes currently allocated in managed memory it is a bad idea to rely on it for accurate values :-)
All kidding aside, if your object 'o' is not used in the function after line #3 in your example, it is possible that it is being collected between lines 3 and 4. Just a guess.
First, if I use the code as written, x becomes 0 for me, because o is not used after the assignment, so the GC can collect it. The following assumes o is not collected (e.g. by using GC.KeepAlive(o) at the end of the method).
Let's look carefully what each of the two important lines of your code do (assuming 32-bit architecture):
Object[] o = new Object[1000000];
This line allocates 1000000 * 4 + 16 bytes. Each element in the array takes 4 bytes, because it's a reference (pointer) to an object. 16 bytes is the overhead of an array.
o[1] = obj;
This line changes one of the references in o to reference to obj. This line allocates exactly 0 bytes.
I'm not sure why are you confused about the result. It has to be 4, unless there are some unreferenced objects from earlier part of the code. In that case, it could be less than 4 (even a negative number). Of course, if this were a multi-threaded application, the result could be pretty much anything, depending on what other threads do.
And this all assumes that GetTotalMemory() is precise, which it doesn't have to be.
In my experience, marshal.sizeof() is generally a better method of getting an object's true size than asking the garbage collector.
http://msdn.microsoft.com/en-us/library/y3ybkfb3.aspx

How much memory array of objects in c# consumes?

Suppose that we have previously instantiated three objects A, B, C from class D
now an array defines as below:
D[] arr = new D[3];
arr[0]=A;
arr[1]=B;
arr[2]=C;
does array contains references to objects or has separate copy?
C# distinguishes reference types and value types.
A reference type is declared using the word class. Variables of these types contain references, so an array will be an array of references to the objects. Each reference is 4 bytes (on a 32-bit system) or 8 bytes (on a 64-bit system) large.
A value type is declared using the word struct. Values of this type are copied every time you assign them. An array of a value type contains copies of the values, so the size of the array is the size of the struct times the number of elements.
Normally when we say “object”, we refer to instances of a reference type, so the answer to your question is “yes”, but remember the difference and make sure that you don’t accidentally create a large array of a large struct.
An array of reference types only contains references.
In a 32 bit application references are 32 bits (4 bytes), and in a 64 bit application references are 64 bits (8 bytes). So, you can calculate the approximate size by multiplying the array length with the reference size. (There are also a few extra bytes for internal variables for the array class, and some extra bytes are used for memory management.)
You can look at the memory occupied by an array using WinDBG + SOS (or PSSCOR2). IIRC, an array of reference types is represented in memory by its length, followed by references to its elements, i.e. it's exact size is PLATFORM_POINTER_SIZE * (array.Length + 1)
The array is made out of pointers (32bit or 64bit) that points to the objects. An object is a reference type, only value types are copied to the array itself.
As #Yves said it has references to the objects. The array is a block of memory as it as in C.
So it size is sizeof(element) * count + the amount of memory needed by oop.

sizeof() equivalent for reference types?

I'm looking for a way to get the size of an instance of a reference type. sizeof is only for value types. Is this possible?
You need Marshal.SizeOf
Edit: This is for unsafe code, but then, so is sizeof().
If you don't mind it being a little less accurate than perfect, and for comparative purposes, you could serialize the object/s and measure that (in bytes for example)
EDIT (I kept thinking after posting): Because it's a little more complicated than sizeof for valuetypes, for example: reference types can have references to other objects and so on... there's not an exact and easy way to do it that I know of...
I had a similar question recently and wanted to know the size of Object and LinkedListNode in C#. To solve the problem, I developed a program that would:
Measure the program's "Working Set"
Allocate a lot of objects.
Measure the "Working Set" again.
Divide the difference by the number of allocated objects.
On my computer (64-bit), I got the following data:
Measuring Object:
iter working set size estimate
-1 11190272
1000000 85995520 74.805248
2000000 159186944 73.998336
3000000 231473152 73.4276266666667
4000000 306401280 73.802752
5000000 379092992 73.580544
6000000 451387392 73.3661866666667
7000000 524378112 73.3125485714286
8000000 600096768 73.613312
9000000 676405248 73.9127751111111
Average size: 73.7577032239859
Measuring LinkedListNode<Object>:
iter working set size estimate
-1 34168832
1000000 147959808 113.790976
2000000 268963840 117.397504
3000000 387796992 117.876053333333
4000000 507973632 118.4512
5000000 628379648 118.8421632
6000000 748834816 119.110997333333
7000000 869265408 119.299510857143
8000000 993509376 119.917568
9000000 1114038272 119.985493333333
Average size: 118.296829561905
Estimated Object size: 29.218576886067
Estimated LinkedListNode<reference type> size: 44.5391263379189
Based on the data, the average size of allocating millions of Objects is approximately 29.2 bytes. A LinkedListNode object is approximately 44.5 bytes. This data illustrates two things:
It's very unlikely that the system is allocating a partial byte. The fractional measure of bytes indicates the overhead the CLR requires to allocate and track millions of reference types.
If we simply round-down the number of bytes, we're still unlikely to have the proper byte count for reference types. This is clear from the measure of Objects. If we round down, we assume the size is 29 bytes which, while theoretically possible, is unlikely because of padding. In order to improve performance, object allocations are usually padded for alignment purposes. I would guess that CLR objects will be 4 byte aligned.
Assuming CLR overhead and 4-byte alignment, I'd estimate an Object in C# is 28 bytes and a LinkedListNode is 44 bytes.
BTW Jon Skeet had the idea for the method above before I did and stated it in this answer to a similar question.
Beware that Marshal.SizeOf is for unsafe code...
I don't think it's possible for managed code though, maybe you can explain your problem, there may be another way to solve it
If you can - Serialize it!
Dim myObjectSize As Long
Dim ms As New IO.MemoryStream
Dim bf As New Runtime.Serialization.Formatters.Binary.BinaryFormatter()
bf.Serialize(ms, myObject)
myObjectSize = ms.Position
Please refer my answer in the below link.
It is possible via .sos.dll debugger extension
Find out the size of a .net object

Categories

Resources