I was considering using UnmanagedMemoryStream rather than MemoryStream for dealing with incoming (and perhaps outgoing?) packets in a network server. What I hope to achieve is less copying of values, and if possible, avoid copying to the heap (too much).
For example, for an incoming packet, one could do:
fixed (byte* p = &data) // where data comes from a socket receive
{
using (var stream = new UnmanagedMemoryStream(p, data.Length))
{
// do handling here...
}
}
Still, I'm not quite sure if there is any realistic benefit in doing this. Could anyone come with some feedback as to whether or not there would be any value in doing this, rather than using the good old managed MemoryStream?
Thanks in advance.
This sounds like premature optimization to me. Have you used MemoryStream and done some profiling to show that it caused you tangible, measurable performance problems?
I would stick with the conventional MemoryStream and save myself the headache until it became obvious (through profiling) that it was necessary to change.
Nope, you didn't improve what is already there. A byte[] is a reference type. You can simply pass it to the MemoryStream(byte[]) constructor and no data is getting copied. MS simply stores a reference to the same array.
In fact, you made it worse because you pinned the array. Getting a garbage collection to run inside the body of your snippet isn't unlikely, you are reading stuff from the array and are probably creating objects from the data, strings and what-not. The garbage collector needs to work around the pinned array, making its life considerably more difficult. This can actually affect the perf of your program for a while, compacting the heap is important to make the CPU cache efficient.
Related
To reduce the memory footprint in my C# app to below the limit (around 1-2GB), I would love to see a list of all variables in realtime (during runtime), along with how much memory they eat up (and maybe even the contents).
From what I can see, this seemingly simple request seems to have escaped the attention of the memory profilers out there. .NET Memory Profiler for instance shows the memory for each given type (e.g. Int32[] or String), but doesn't seem to allow finer granularity to show the memory for each named variable.
Although I haven't tried dotTrace or ANTS Memory Profiler, scanning the FAQ, videos and screenshots draws a blank too.
Apart from my own variables, the desired profiler would probably include 'overhead' memory usage typical for any .NET app, though to me, that's less important.
Is there any program (preferably free or under $100) which can do this?
------------- EDIT
For variables which reference each other (as shown by Jon Skeet), or for variables passed by reference to a method, the profiler could maybe either group them to show that they're really the same object (and therefore 'share' the same memory), or just show the original variable name and omit the references.
but doesn't seem to allow finer granularity to show the memory for each named variable.
That's probably because it doesn't make much sense. Variables themselves don't generally take up much memory - it's objects which take up memory, and variables just prevent those objects from being garbage collected.
So for example, consider the following code:
byte[] array = new byte[1024 * 1024]; // 1MB
byte[] array2 = array;
byte[] array3 = array;
Here we have three variables, all referring to the same array. How much memory would your desired tool show each of them taking? 1MB, because each one refers to a 1MB array? That would be confusing, as the total memory shown would be 3MB despite only 1MB actually being used. 0.3333MB? Surely more confusing. 1MB? Sort of accurate, but unhelpful.
You should concentrate on which objects stay alive longer than you want them to, then work out what's keeping those objects alive.
Usually, if a "seemingly simple request" seems to have "escaped the attention" of people who specialize in the area, it's because it's not a simple request after all.
Like GuyFawkes, I would like to use MemoryStream to store a large amount of data, but keep encountering the 'out of memory' exceptions.
TomTom's answer is what I would like to do - use an implementation that does not require a contiguous block - but I'm wondering if there is already a free implementation available, to save me writing one myself?
Does anyone know of a good, free re-implementation of MemoryStream that can work with large streams?
EDIT:
The MemoryMappedFile solution is very interesting and I will be remembering it for other projects, however as Henk says, it strays too far from the abstraction that MemoryStream is aiming for. Specifically, the requirement of a known capacity.
The data that the replacement shall handle will in some cases be very large, but in others relatively small (and no we don't know which it will be until its too late ;)); further, many instances of the class will be in existence at the same time.
Ultimately the work required to use MemoryMappedFiles (to determine an appropriate size for each one) would be equivalent to that of implementing TomTom's solution.
Here is my implementation in case anyone needs it; I will leave this question open for a bit in case someone still responds with anything better.
http://www.codeproject.com/Articles/348590/A-replacement-for-MemoryStream
You yould create a MemoryMappedFile without a file, i.e. one that lives in system memory.
The DelayAllocatePages option delays allocations until the memory is actually needed. You need to specify a maximum capacity in advance though. Use the CreateViewStream Method to create a stream.
Not exactly a re-implementation of MemoryStream, but consider whether you can use a Memory Mapped File for your requirement.
Memory Mapped Files can solve many of the classes of problems that large memory buffers can solve, are very efficient, and are supported directly by .NET.
for creating my own memory management in C# I need to have a possibility to intercept the new command before it returns a null or fires an exception. When using the new command I want to call the original handler first. If this handler fails to return a block of memory, I want to inform all my mappable objects to be written to disk and to free memory.
In C++ there has been a possibility to intercept the new command by assigned a different new handler. In C# I couldn't find anything which shows the same behaviour.
Has anyone seen a possibility to do this.
Thanks
Martin
You can't do what you're after in C#, or in any managed language. Nor should you try. The .NET runtime manages allocations and garbage collection. It's impossible for you to instruct your objects to free memory, as you have no guarantee when (or, technically, even if) a particular object will be collected once it's no longer rooted. Even eliminating all references and manually calling GC.Invoke() is not an absolute guarantee. If you're looking for granular memory management, you need to be using a lower-level environment.
As an important point, it is not possible for the new operator to return a null reference. It can only return either a reference to the specified type or throw an exception.
If you want to do your own management of how and when objects are allocated, you'll have to use something along the lines of a factory pattern.
I think you're approaching this from the wrong angle; the whole point of using a runtime with managed memory is so that you don't have to worry about memory. The tradeoff is that you can't do this type of low-level trickery.
As an aside, you can 'override new' for a limited class of objects (those descending from ContextBoundObject) by creating a custom ProxyAttribute, though this likely does not address what you're intending.
I believe that you are not understanding the side-effects of what you're asking for. Even in C++, you can't really do what you think you can do. The reason is simple, if you have run out of memory, you can't even make your objects serialize to disk because you have no memory to accomplish that. By the time memory is exhausted, the only real thing you can do is either discard memory (without saving or doing anything else first) or abend the program.
Now, what you're talking about will still work 95% of the time because your memory allocation will likely be sufficiently large that when it fails, you have a little room to play with, but you can't guarantee that this will be the case.
Example: If you have only 2MB of memory left, and you try to allocate 10MB, then it will fail, and you still have 2MB to play with to try and free up some memory, which will allow you to allocate small chunks of memory needed to serialize objects to disk.
But, if you only have 10 bytes of memory left, then you don't even have enough memory to create a new exception object (unless it comes from a reserved pool). So, in essence, you're creating a very poor situation that will likely crash at some point.
Even in C++ low memory conditions are almost impossible to get right, and it's almost impossible to recover from every case unless you have very carefully planned, and pre-allocated memory for your recovery routines.
Now, when you're talking about a garbage collected OS, you have no control over how memory is allocated or freed. At best, all you can do is give hints. There is very little you can reliably do here by the nature of garbage collection. It's non-deterministic.
Good afternoon,
I have some text files containing a list of (2-gram, count) pairs collected by analysing a corpus of newspaper articles which I need to load into memory when I start a given application I am developing. To store those pairs, I am using a structure like the following one:
private static Dictionary<String, Int64>[] ListaDigramas = new Dictionary<String, Int64>[27];
The ideia of having an array of dictionaries is due to efficiency questions, since I read somewhere that a long dictionary has a negative impact on performance. That said, every 2-gram goes into the dictionary that corresponds to it's first character's ASCII code minus 97 (or 26 if the first character is not a character in the range from 'a' to 'z').
When I load the (2-gram, count) pairs into memory, the application takes an overall 800Mb of RAM, and stays like this until I use a program called Memory Cleaner to free up memory. After this, the memory taken by the program goes down to the range 7Mb-100Mb, without losing functionality (I think).
Is there any way I can free up memory this way but without using an external application? I tried to use GC.Collect() but it doesn't work in this case.
Thank you very much.
You are using a static field so chances are once it is loaded it never gets garbage collected, so unless you call the .Clear() method on this dictionary it probably won't be subject to garbage collection.
It is fairly mysterious to me how utilities like that ever make it onto somebody's machine. All they do is call EmptyWorkingSet(). Maybe it looks good in Taskmgr.exe, but it is otherwise just a way to keep the hard drive busy unnecessarily. You'll get the exact same thing by minimizing the main window of your app.
I don't know the details of how memory cleaner works, but given that it's unlikely to know the inner workings of a programs memory allocations, the best it can probably do is just cause pages to be swapped out to disk reducing the apparent memory usage of the program.
Garbage collection won't help unless you actually have objects you aren't using any more. If you are using your dictionaries, which the GC considers that you are since it is a static field, then all the objects in them are considered in use and must belong to the active memory of the program. There's no way around this.
What you are seeing is the total usage of the application. This is 800MB and will stay that way. As the comments say, memory cleaner makes it look like the application uses less memory. What you can try to do is access all values in the dictionary after you've run the memory cleaner. You'll see that the memory usage goes up again (it's read from swap).
What you probably want is to not load all this data into memory. Is there a way you can get the same results using an algorithm?
Alternatively, and this would probably be the best option if you are actually storing information here, you could use a database. If it's cumbersome to use a normal database like SQLExpress, you could always go for SQLite.
About the only other idea I could come up with, if you really want to keep your memory usage down, would be store the dictionary in a stream and compress it. Factors to consider would be how often you're accessing/inflating this data, and how compressible the data is. Text from newspaper articles would compress extremely well, and the performance hit might be less than you'd think.
Using an open-source library like SharpZipLib ( http://www.icsharpcode.net/opensource/sharpziplib/ ), your code would look something like:
MemoryStream stream = new MemoryStream();
BinaryFormatter formatter = new BinaryFormatter();
formatter.Serialize(stream, ListaDigramas);
byte[] dictBytes = stream.ToArray();
Stream zipStream = new DeflaterOutputStream(new MemoryStream());
zipStream.Write(dictBytes, 0, dictBytes.Length);
Inflating requires an InflaterInputStream and a loop to inflate the stream in chunks, but is fairly straightforward.
You'd have to play with the app to see if performance was acceptable. Keeping in mind, of course, that you'll still need enough memory to hold the dictionary when you inflate it for use (unless someone has a clever idea to work with the object in its compressed state).
Honestly, though, keeping it as-is in memory and letting Windows swap it to the page file is probably your best/fastest option.
Edit
I've never tried it, but you might be able to serialize directly to the compression stream, meaning the compression overhead is minimal (you'd still have the serialization overhead):
MemoryStream stream = new MemoryStream();
BinaryFormatter formatter = new BinaryFormatter();
Stream zipStream = new DeflaterOutputStream(new MemoryStream());
formatter.Serialize(zipStream, ListaDigramas);
Thank you very much for all the answers. The data actually needs to be loaded during the whole running time of the application, so based on your answers I think there is nothing better to do... I could perhaps try an external database, but since I already need to deal with two other databases at the same time, I think it is not a good idea.
Do you think it is possible to be dealing with three databases at the same time and do not lose on performance?
If you are disposing of your applications resources correctly then the actual used memory may not be what you are seeing (if verifying through Task Manager).
The Garbage Collector will free up the unused memory at the best possible time. It usually isn't really a good idea to force collection either...see this post
"data actually needs to be loaded during the whole running time of the application" - why?
My application does a good deal of binary serialization and compression of large objects. Uncompressed the serialized dataset is about 14 MB. Compressed it is arround 1.5 MB. I find that whenever I call the serialize method on my dataset my large object heap performance counter jumps up from under 1 MB to about 90 MB. I also know that under a relatively heavy loaded system, usually after a while of running (days) in which this serialization process happens a few time, the application has been known to throw out of memory excpetions when this serialization method is called even though there seems to be plenty of memory. I'm guessing that fragmentation is the issue (though i can't say i'm 100% sure, i'm pretty close)
The simplest short term fix (i guess i'm looking for both a short term and a long term answer) i can think of is to call GC.Collect right after i'm done the serialization process. This, in my opinion, will garbage collect the object from the LOH and will do so likely BEFORE other objects can be added to it. This will allow other objects to fit tightly tightly against the remaining objects in the heap without causing much fragmentation.
Other than this ridiculous 90MB allocation i don't think i have anything else that uses a lost of the LOH. This 90 MB allocation is also relatively rare (arround every 4 hours). We of course will still have the 1.5 MB array in there and maybe some other smaller serialized objects.
Any ideas?
Update as a result of good responses
Here is my code which does the work. I've actually tried changing this to compress WHILE serializing so that serialization serializes to a stream at the same time and i don't get much better result. I've also tried preallocating the memory stream to 100 MB and trying to use the same stream twice in a row, the LOH goes up to 180 MB anyways. I'm using Process Explorer to monitor it. It's insane. I think i'm going to try the UnmanagedMemoryStream idea next.
I would encourage you guys to try it out if you wont. It doesn't have to be this exact code. Just serialize a large dataset and you will get surprising results (mine has lots of tables, arround 15 and lots of strings and columns)
byte[] bytes;
System.Runtime.Serialization.Formatters.Binary.BinaryFormatter serializer =
new System.Runtime.Serialization.Formatters.Binary.BinaryFormatter();
System.IO.MemoryStream memStream = new System.IO.MemoryStream();
serializer.Serialize(memStream, obj);
bytes = CompressionHelper.CompressBytes(memStream.ToArray());
memStream.Dispose();
return bytes;
Update after trying binary serialization with UnmanagedMemoryStream
Even if I serialize to an UnmanagedMemoryStream the LOH jumps up to the same size. It seems that no matter what i do, called the BinaryFormatter to serialize this large object will use the LOH. As for pre-allocating, it doesn't seem to help much. Say i pre-allocate say i preallocate 100MB, then i serialize, it will use 170 MB. Here is the code for that. Even simpler than the above code
BinaryFormatter serializer = new BinaryFormatter();
MemoryStream memoryStream = new MemoryStream(1024*1024*100);
GC.Collect();
serializer.Serialize(memoryStream, assetDS);
The GC.Collect() in the middle there is just to update the LOH performance counter. You will see that it will allocate the correct 100 MB. But then when you call the serialize, you will notice that it seems to add that on top of the 100 that you have already allocated.
Beware of the way collection classes and streams like MemoryStream work in .NET. They have an underlying buffer, a simple array. Whenever the collection or stream buffer grows beyond the allocated size of the array, the array gets re-allocated, now at double the previous size.
This can cause many copies of the array in the LOH. Your 14MB dataset will start using the LOH at 128KB, then take another 256KB, then another 512KB, etcetera. The last one, the one actually used, will be around 16MB. The LOH contains the sum of these, around 30MB, only one of which is in actual use.
Do this three times without a gen2 collection and your LOH has grown to 90MB.
Avoid this by pre-allocating the buffer to the expected size. MemoryStream has a constructor that takes an initial capacity. So do all collection classes. Calling GC.Collect() after you've nulled all references can help unclog the LOH and purge those intermediate buffers, at the cost of clogging the gen1 and gen2 heaps too soon.
Unfortunately, the only way I could fix this was to break up the data in chunks so as not to allocate large chunks on the LOH. All the proposed answers here were good and were expected to work but they did not. It seems that the binary serialization in .NET (using .NET 2.0 SP2) does its own little magic under the hood which prevents users from having control over memory allocation.
Answer then to the question would be "this is not likely to work". When it comes to using .NET serialization, your best bet is to serialize the large objects in smaller chunks. For all other scenarios, the answers mentioned above are great.
90MB of RAM is not much.
Avoid calling GC.Collect unless you have a problem. If you have a problem, and no better fix, try calling GC.Collect and seeing if your problem is solved.
Don't worry about LOH size jumping up. Worry about allocating/deallocating LOH. .Net very dumb about LOH -- rather than allocating LOH objects far away from regular heap, it allocates at next available VM page. I have a 3D app that does much allocate/deallocate of both LOH and regular objects -- the result (as seen in DebugDiag dump report) is that pages of small heap and large heap end up alternating throughout RAM, until there are no large chunks of the applications 2 GB VM space left. The solution when possible is to allocate once what you need, and then don't release it -- re-use it next time.
Use DebugDiag to analyze your process. See how the VM addresses gradually creep up towards 2 GB address mark. Then make a change that keeps that from happening.
I agree with some of the other posters here that you might want to try and use tricks to work with the .NET Framework instead of trying to force it to work with you via GC.Collect.
You may find this Channel 9 video helpful which discusses ways to ease pressure on the Garbage collector.
If you really need to use the LOH for something like a service or something that needs to be running for a long time, you need to use buffer pools that are never deallocated and that you can ideally allocate on start-up. This means you'll have to do your 'memory management' yourself for this, of course.
Depending on what you're doing with this memory, you might also have to p/Invoke over to native code for selected parts to avoid having to call some .NET API that forces you to put the data on newly allocated space in the LOH.
This is a good starting point article about the issues: https://devblogs.microsoft.com/dotnet/using-gc-efficiently-part-3/
I'd consider you very lucky if you GC trick would work, and it would really only work if there isn't much going on at the same time in the system. If you have work going on in parallel, this will just slightly delay the unevitable.
Also read up on the documentation about GC.Collect.IIRC, GC.Collect(n) only says that it collects no further than the generation n -- not that it actually ever GETS to generation n.