I have about 50.000 text file (~5KB each). I need make data file one time, then my app read (not write) to use.
I'm finding a way to keep all these file to one (or several) file. Current I store it in a .zip file, then when run app, I read zip file and get the entry I need. This way is very slow to read data (about 2 seconds).
Is any way I can store data that both fast to use and convenient to tranfer app between computer? Thank!
[Edit: I'm not well work with data before, and my app is portable. Data is one-time create, no modify after create, and is plain text but have structure]
Data structure:
Section abcd
Item 1234
Item klmn
Section def
Item ...
Item ...
...
...
as a suggestion:
use the zip as primary source
before the app starts check if a specific folder exists
if not - unzip the files there,
if exists - check files count in the directory (as a dirty check that you do have data in the folder)
read files from the folder
Related
I want to have multiple network stream threads writing/downloading into one file simultaneosly.
So e.G you have one File and download the ranges:
0-1000
1001-2002
2003-3004...
And I want them all to write their receiving bytes into one File as efficient as possible.
Right now I am downloading each range part into one File and combine them later when they are all finished into the final File.
I would like them to, if it is possible to all write into one File to reduce disk usage and I feel like this could all be done better.
You could use persisted memory mapped files, see https://learn.microsoft.com/en-us/dotnet/standard/io/memory-mapped-files
Persisted files are memory-mapped files that are associated with a source file on a disk. When the last process has finished working with the file, the data is saved to the source file on the disk. These memory-mapped files are suitable for working with extremely large source files.
Suppose that I would like to add extra information about a file, without writing that information as content of that file. How would I do this? A couple of good examples are:
With Word documents, you can add Author tag to a document. And,
MP3 files have lots of info stored inside of them but when you play the file, you don't see that info (unless the program playing the file has been programmed to display that information).
How does Windows do this?
This information is stored in the file system (on windows - NTFS).
In NTFS, you can actually store another file, as part of this information, and it stores much more information about each file than you may expected.
NTFS file streams
Exapmle in C how to consume them
About MP3 and word - In these cases the information is stored inside the file, as part of its format.
How do I save my data...
(compressed/encrypted) not human readable in a portable file by
influencing on directory name and file name with using easily
addressable settings by "name" and "value" like in registry/ini
with the possibility to access the same one settings file (machine based)
with any executable
In VB I did that with INI Files, but now I have heard from MS that the Framework offers no function to access INI Files anymore. Since INI Files are also not any longer up-to-date for using them in a new application I wanted something similar. Just to write to a text file line by line is not what I want to do, I will explain why: If I will need a setting named "Label4" I have to read the entire file and search for the line containing it and then I have even to split the result in setting name and setting value (.NET has another syntax split function).
Let's say I have a thousand labels and a hundred textboxes in a form and wanna save its content easily to a file or whatever and read it the same easily back addressing the settings by a name already how would I do that? I am ready to use a DB but don't know how to setup and save information there. Easy would be to access the registry with "name" and "value", but I prefer a portable file.
The data shall be stored compressed or encrypted not in a plain text human readable way (I mean XML / html is not what I search for to store the data else I could research again how to use INI files with C# which might be possible I guess). I stumbled upon "application settings" but this doesn't save my data compressed nor can I simply influence on path and filename of the settings file. With each new EXE all the stored settings are lost again for those new EXE if I have used "application settings". I need not "user/application based" settings but "machine based", I hope you understand what I wanna say. Any App on my comp shall be able to get to my settings file
and I shall be able to choose the place/filename where to save the settings.
You should write to any xml file and get your desired results.
(compressed/encrypted) not human readable in a portable file
You can both encrypt/compress a normal file
influencing on directory name and file name with using easily
Can't understand what you mean
addressable settings by "name" and "value" like in registry/ini
xml gives you flexibility you need
with the possibility to access the same one settings file (machine based)
You can keep it machine based.
with any executable
What does it mean?
I need to develop an application to copy the contents of .jrn file as soon as it is generated in ATM.The generated file name(.jrn) will be saved on the basis of date,What i wanted is,my application should copy the content of .jrn file to one text file lets say "xy.txt" and all of these contents should be deleted in every one hour.I am planning to develop this in .NET platform.Can any one suggest me how can i do it or the steps involved in it?
In your case I would do some research as the following :
1) You say your application need to copy every 1 hour the content of the .jrn file to a .txt file. That imply you need to run your application as a 'background process'. You will need something like a Timer, see http://msdn.microsoft.com/en-us/library/system.timers.timer.aspx
2) In order to copy the content of .jrn to .txt, you will need something like stream and serialization. You will want to read the content of the .jrn file, do some parsing if you want only one type of information, and then serialize what you have extracted to the new .txt file.
Hope that can help.
As we all know that we can not get the full path of the file using File Upload control, we will follow the process for saving the file in to our application by creating a folder and by getting that folder path as follows
Server.MapPath
But i am having a scenario to select 1200 excel files, not at a time. I will select each and every excel file and read the requied content from that excel and saving the information to Database. While doing this i am saving the files to the Application Folder by creating a folder Excel. As i am having 1200 files all these files will be saved in to this folder after each and every run.
Is it the correct method to follow or not I don't know
I am looking for an alternative solution rather than saving the file to folder. I would like to save the full path of file temporarily until the process was executed.
So can any tell me the best way as per my requirement.
Grrbrr404 is correct. You can perfectly take the byte [] from the FileUpload.PostedFile and save it to the database directly without using the intermediate folder. You could store the file name with extension on a separate column so you know how to stream it later, in case you need to.
The debate of whether it's good or bad to store these things on the database itself or on the filesystem is very heated. I don't think either approach is best over the other; you'll have to look at your resources and your particular situation and make the appropriate decision. Search for "Store images on database or filesystem" in here or Google and you'll see what I mean.
See this one, for example.