I am trying to write a tcp reconstruction program in c# , by using SharpPcap. So far I am doing a pretty good job, and the reconstruction is working fine. My only problem is, that in order to reconstruct big Pcap files by myself, I need to load them by parts/chunks to the memory, because sharppcap only let's me load the whole file( I think). Any suggestions?
Thanks
The pcap file format is really simple, see here: http://wiki.wireshark.org/FileFormatReference/libpcap
Why not load the file yourself, possibly a packet at a time, and then you can do what you want as you go along rather than having a library dictate your memory usage patterns?
Given that SharpPcap PcapDevices have a GetNextPacket method, from which both LibPcapLiveDevice and CaptureFileReaderDevice inherit that method, I don't see anything that would require you to load the whole file - you might have to read the entire file, but you can just ignore packets you don't want.
Related
I'm making an XNA game, which uses a lot (currently ~2800) of small resource files. It has become a problem to move them around from place to place unarchived, so I thought maybe I could just zip them and make the game unzip them automatically, into memory, preferably. I don't need the writing capability yet, right now only reading.
Is there an easy way to unzip a folder into memory and access those files just like, or as simple as the regular files on disk?
I've been reading some similar questions and I see many people say that the OS (Windows in my case) can handle file caching better than a ram drive. I'm just going for unzipping and reading files for now, but in future I might need to modify or create new files, and I'd like it to be quick and seamless for the user. Maybe I should take a different approach at solving my current problem, taking in account my future goal?
I haven't personally tried this but if you want to be able to zip/unzip in memory, you could just use a MemoryStream and pass that into a library (eg https://github.com/icsharpcode/SharpZipLib). Things you'll probably need to remember, are you just moving a bottleneck to a different bottleneck?
You could also try something like the approach with Sprites in HTML. You combine all your Zip's into 1 with an Index to where in the file they are. Then you move your FileStream.Position to the location for the resource you are looking for, read (the amount you need) then do what you need with it. You'd need to make sure that if you rebuild any you make something rebuild all your combining indexes etc. Then you just would be copying 1 file around, it just so happens that inside that file you have ~2800 smaller segments of interest.
Given the path of a string i want to wipe out the contents of a file. The natural way I thought (which maybe incorrect) was to open a FileStream to the file and write gibberish (random data perhaps taken from a RNGCryptoServiceProvider) to it. And then perhaps do this several times and then delete the file.
My problem is that while this may look logically correct, i read up on another blog that Windows might actually choose to write the file to a different place in the hard disk.
Is that the case in Windows Mobile? Will this actually be a problem? Does this writing to a different location in the hard disk apply to even flash based (SD etc) cards ?
I've not personally done this, but you will probably need to use the low-level FLASH driver IOCTLs to do this correctly.
http://msdn.microsoft.com/en-us/library/aa927166.aspx
I think IOCTL_FMD_RAW_WRITE_BLOCKS looks particularly useful.
-PaulH
Another possibility that may work would be to erase the file normally, then use the defragment APIs to wipe ALL of the freespace on your flash. Since you're wiping everything, it won't be necessary to know exactly where on the disk your file was. But, this will wear out your flash drive more quickly. The C# method is detailed in this blog post: http://blogs.msdn.com/b/jeffrey_wall/archive/2004/09/13/229137.aspx
In my web application I am working with files. Some files are very large. I use Response.Write() to write the file to the browser. This goes well for the smaller files, but for large files this can take a while and the bandwidth is fully used.
Is it possible to split large documents and send it piece by piece to the browser? Are there other ways to send the document quicker to the browser?
I hold the document as a property of an object.
Why don't you compress the file and store it in the DB and decompress it will extracting it?
You can do a lot of things depending on this questions:
How often does the file change?
Do I really need the files in the DB?
Why not store the File path in the
DB and the File on disk?
Anyhow, since your files are extremely high bandwidth and you would want your app to respond appropriately you might want to use AJAX load the files Asynchronously. You can have a WebHandler .ashx for this.
Here's a few examples:
http://www.dotnetcurry.com/ShowArticle.aspx?ID=193&AspxAutoDetectCookieSupport=1
http://www.viawindowslive.com/Articles/VirtualEarth/InvokingserversidecodeusingAJAX.aspx
My question is, is it possible to
split large documents and send it
piece by piece to the browser?
It depends on the file type, but in general no. If you are sending something like an excel file or a word doc etc. the receiving application will need all of the information (bytes) to fully form the document. You could physically separate the document into multiple ones, and that would allow you to do so.
If the bandwidth is fully used, then there is nothing you can do to "speed it up" short of compressing the document prior to send. In other words, zip it up.
Depending on the document (I know you said .mht, but we're talking content here) you will see the size go down by some amount. Maybe it's enough, maybe not.
Either way, this is entirely a function of the amount of content you want to send versus the size of the pipe available to send it. One of those is more difficult to change than the other.
Try setting IIS's dynamic compression. By default, it's set fairly low, but you can try setting it for a higher compression level and see how much that helps.
I'm not up to speed with ASP.NET but you might be able to buffer from a FileStream to some sort of output stream.
You can use the Flush method to send the currently buffered data to the client (the browser).
Note that this has some implications, as is described aptly here.
I've considered using it myself, a project sent documents that became fairly large and I was cautious about storing the whole data in memory. In the end I decided the data was not large enough to be a problem though.
Sadly the MSDN documentation is very, very vague on what Flush implies and you will probably have to use Google to troubleshoot.
I have a memory stream that contains a PDF file.
Is it possible to view the PDF without saving it to the hard disk ? Process.Start() only takes a path and not a stream.
Thank you
Only by implementing your own pseudo-file system in C#, somehow mounting this as a disk in Windows, and having it intercept the file open and stream the contents of your MemoryStream. Absolutely 100% certainly not worth the effort.
You can create a RAM drive and write the stream to it, this way you are still keeping it all in ram (assuming the disk operations are what worries you).
Sure, this is certainly possible. Just not via Process Start and Adobe Reader (I assume you are invoking Adobe or something similar)
If you are using .NET or Java you simply need to find a PDF viewer component, there are lots to choose from, google will give you plenty of links, Gnostice has a good one, but its expensive. Once you find a suitable control, view the PDF directly from your app.
If there is, process.Start won't be the way, but I'd risk guessing that there isn't.
Unless there's a specific PDF API that allows that somehow (I doubt) I'd save it to disk.
I am writing a client windows app which will allow files and respective metadata to be uploaded to a server. For example gear.stl (original file) and gear.stl.xml (metadata). I am trying to figure out the correct protcol to use to transfer the files.
I was thinking about using ftp since it is widely used and a proven method to transfer files, except that I would have to transfer 2 files for every actual file (.stl and .stl.xml). However, another thought had also crossed my mind ... What if I create an object and wrap the file, metadata and the directory I needed to tranfer it to, serialize the object and then submit a request to a webservice, to transfer the file.
Original file size would range from 100k to 10MB. Metadata size would probably be less than 200k
The webservice call seems like an easier process to me to deserialize the object and distribute the file and respective metadata accordingly. However I'm not sure if this is a sound idea or if there is a better way to transfer this data other than the two methods I have mentioned.
If someone can point me in the right direction it would be much appreciated.
You could wrap it in a zip file like the "new" office document format does. You might even be able to use their classes to package it all up.
Edit:
Take a look at the System.IO.Packaging.Package class. It seems to be what you need. This class resides in the WindowsBase.dll assembly and became available in .NET 3.0.
PS: Remember that even though it is a zip file, it doesn't need to be compressed. If you have very large files, it may be better to keep them uncompressed. It all depends on how they're going to be used and if the transport size is an issue.