I'm using MSVE, and I have my own tiles I'm displaying in layers on top. Problem is, there's a ton of them, and they're on a network server. In certain directories, there are something on the order of 30,000+ files. Initially I called Directory.GetFiles, but once I started testing in a pseudo-real environment, it timed out.
What's the best way to programatically list, and iterate through, this many files?
Edit: My coworker suggested using the MS indexing service. Has anyone tried this approach, and (how) has it worked?
I've worked on a SAN system in the past with telephony audio recordings which had issues with numbers of files in a single folder - that system became unusable somewhere near 5,000 (on Windows 2000 Advanced Server with an application in C#.Net 1.1)- the only sensible solution that we came up with was to change the folder structure so that there were a more reasonable number of files. Interestingly Explorer would also time out!
The convention we came up with was a structure that broke the structure up in years, months and days - but that will depend upon your system and whether you can control the directory structure...
Definitely split them up. That said, stay as far away from the Indexing Service as you can.
None. .NET relies on underlying Windows API calls that really, really hate that amount of files themselves.
As Ronnie says: split them up.
You could use DOS?
DIR /s/b > Files.txt
You could also look at either indexing the files yourself, or getting a third part app like google desktop or copernic to do it and then interface with their index. I know copernic has an API that you can use to search for any file in their index and it also supports mapping network drives.
Related
I am trying to recreate some features of Spotify in C# using the PostgreSQL database.
The reason is simple, I want to gain more knowledge, and I think this is a good challenge.
But I ran into an obstacle that I've been standing for days. Spotify he doesn't download the music, he does her streaming, plays the music while it's downloaded.
However, I can't do this in C#, I'm using the PostgreSQL database.
I'm well locked in this part, I tried several implementations, but I think I'm not on the right track, and on the internet I imagine I'm looking wrong, otherwise I would have found it.
Do you have any guidance for this streaming process in C#? I've tried to read the large_object bytes from PostgreSQL, but couldn't.
Any suggestions or guides about the process are welcome.
You start by getting the file into the database or its network location into the database, whichever gives you better performance; Then start with creating an implementation of a bytestream. You want to be transmitting raw data to c#.
you then build a real time interpreter that takes in using your file format, one byte at a time, and plays the value associated with that section. does that make sense? this is simple to do with many libraries and the brunt of it is just figuring those out.
You seem like you've PROBABLY got that first part down, and are instead having issues with the database. A lot of things we did at my last company involved saving file network locations and indexing files on disk. You might be able to instead point your streamer to a file locally using a server, and instead transmit data from one point to another in that manner instead.
You seem more than capable of doing this just judging by your speech. I hope this comment was helpful, and if it was not I apologize as well. I would be interested in seeing your finished result.
for clarification here would be that workflow:
request for a song listed in table dbo.Songs
matches that song onto dbo.songlocation
streams from dbo.songlocation.location from the filename dbo.songlocation.songname = dbo.song.name and verified directory returns true
enjoyment of that music
I would like to get a list of the files which have been opened by the user a) recently and/or
b) frequently.
Is this information available on Windows operating systems (Need support for everything down to XP) and if yes, is there a way to use this data with .NET Framework?
Searching the topic on Google is kind of hard, because the results are always referring to solutions where people want to achieve the same thing for their own application only (i.e. create a list of recently used files). I need the same thing on OS level, not just for files which have been opened with the appplication created by me.
An easy solution can be to use Environment.SpecialFolder.Recent:
string path = Environment.GetFolderPath(
Environment.SpecialFolder.Recent);
var files = Directory.EnumerateFiles(path);
Note that this solution only lists the recent opened documents. If you want a better solution, you can take a look to this article about how to use the Windows.Storage.AccessCache API:
http://msdn.microsoft.com/en-us/library/windows/apps/xaml/hh972344.aspx
If you are looking for Windows' MRU List you can find it in Registry.
This link might help you alot.
OK, I thought this was a fairly simple task, but apparently it isn't ...
I have a folder with +1000 photos in it. These are all photos taken with a camera, each about 3 MB. Users need to be able to view these pictures (as a list), rename or delete them. That's it.
A possible solution would be this control : ImageListView - CodeProject
but because it has an Apache license, we can't use it.
So how to do it? Any ideas or suggestions? I'm using .NET 2.0
.... EDIT : .....................................
OK, apparently we CAN use the Apache license. (Also see: https://stackoverflow.com/questions/1007338/can-i-use-a-library-under-the-apache-software-license-2-0-in-a-commercial-applic) However, using the license is very confusing for me. I read the following guide but still don't exactly know how to apply it to our project : http://blog.maestropublishing.com/how-to-apply-the-apache-20-license-to-your-pr
it says:
you need two files in the root or top directory of your distribution.
What's exactly meant by 'distribution'? Is that our installed application, and top directory meaning Program files/OurApp/ ?
It also says:
Replace all [bracketed] items in the above notice statement. There are only two of these items so should not be hard for you to do.
But that would give me a notice file, reading :
Copyright 2012 OUR_COMPANY
Licensed under the Apache License, etc...
But our app isn't licensed under the Apache license?
I'm sorry but I'm very confused and don't want to make any mistakes with this legal stuff...
What would I need to do exactly to be able to use this control?
Perhaps you need your own control for this task.
What i think is just a sketch of what i`d do in your place.
You need your own control with paging(to show only limited photos to user) or scroll-event-driven(to load photos on demand).
Perhaps you need some thumbnail generator.
Point is you probably face a huge pile of photos, so you cannot get them all in one time.
"Thats it" is not that simple.
For 1000+ that is over 3 GB.
Would need thumbnails for faster preview.
If users are going to access this files directly then they would need NTFS permission.
Maybe what you want.
What you are going to get into in locking problems.
If one user has a file open then you cannot rename or delete it.
I know you are not going to like this but to do it right you need a server app to manage that folder and users access via a WCF service so there is a single control point.
I recently got put on a project where they're having issues with too many files in a folder slowing down access. I believe it is 10,000+ files in a single folder where windows starts to slow down access, we have something on the order of 50,000. All the files are small and most of the time we only need to access the newest .1-2.% of them via windows file and print sharing. I'd look into dividing the files into subfolders, except that there is a bunch of legacy code that is only able to look at a single folder.
My idea - I don't know if it is possible or even plausible - is to create a small program that buffers the newest .1-.2% files in memory, and retrieves the rest from disk as needed.
I had thought that years ago I'd read of a protocol that could simulate a folder on a hard drive. Is it possible?
Is there something out there that already does this? Is there a better option without major changes to the system?
What to other systems use for serving up a large number of files? Is there some other product that serves files that we could map as a network drive? Or some way to blend 2 folders so they look like one?
Putting aside the "correct way to solve this problem" for the moment, what you're looking for is called "Shell namespace extensions". There are several .NET resources for writing these explorer extensions.
http://namespaceextension.codeplex.com/
http://www.codeproject.com/Articles/1649/The-Complete-Idiot-s-Guide-to-Writing-Namespace-Ex
http://www.codeproject.com/Articles/13515/A-Namespace-Extension-Toolkit
And perhaps many more.
Of course - we must remember why it isn't a good idea to write explorer extensions in .NET.
Hope this helps.
I have an application which store files (mostly Office documents) in various distant locations. I want my users to be able to search for these files based on some criteria on its own machine. I though I could use Windows Search to create an index. I've had that idea because a few years ago and to search for email in Outlook, I had to install Windows Search. So I suppose that Outlook is leveraging Windows Search to search in the PST file.
In brief, I am wondering if I can create my own index with Windows Search. Right now, I am unable to find any example online (ideally in C#). I was able to find IFilter example, but that's it.
Thanks for the help!
G'Day Martin,
I'll point out up front that the issue is that you'll have is an index being generated per machine. Which means that every machine on the network will at some point have to index every document. All that load over your network isn't a good thing and it won't scale well if you have to add more computers.
imho, the best thing to do would be install one of the free varients of SharePoint and leverage its search abilities. This will take care of any search UI that would need to be written and OOTB it's already very good at indexing files spread over drives across a network.
Hope that helps.
Cam