I'm planning to develop an application that will read a log file and display statistics.
The first question, I guess, is to know if I need a database or not?
Will it be quicker to run queries against the database ; or read the file each time a user wants to see the statistics?
If I choose the database method, I will have to read the log file and update the database on a regular basis (between 1 and 10 minutes).
Is this article still good do you think (as it's from 2005): http://www.codeproject.com/KB/aspnet/ASPNETService.aspx
Or is it better to develop a Windows service? In that case, can I add the Windows Serice in my ASP.NET project in Visual Studio, or does it need to be
You mentioned ASP.NET so I believe it is a web application. In such case I would suggest to use Data Base, this is more robust, flexible and distributed solution.
Any way consider using log4net and then you can easily switch on file/DB ouput in any time by simply adding an other one appender section into the configuration file.
If I choose the database method, I will have to read the log file and
update the database on a regular basis (between 1 and 10 minutes)
Exactly, you're going to have to do it anyway. The Database basically just becomes another bottleneck at that point. For this type of app, there's no need to do anything other than read the file when the user requests to see it, and display them the results on the fly.
No need to have a windows service either. I mean, I don't know all your details, but I'm assuming the log file is in a directory on your machine, so just access it, open it, parse it, and display it to the user when they choose to see it on the front end.
If the only data you going to work is LOG files, you don't need any database.
But I assume that your application would do parse logs files, create some statistics and STORE it somewhere, to make possible to users to get back and see statistics for some period of time. It is not cool if any time you will be "re-calculating" that statistics again (further more, you might loose original log files till that time).
Even if you could store it to some files also, I do not recommed that at all. Don't be afraid of using Database, don't be concered on application performace on such early stage. Do the most that helps you to solve the problem.. and as for me using Database will solve your problem;
Related
I have .txt files that are overwritten with data from software every 5-10 seconds, I then have a wpf application that reads and displays this data every second. Here are my issues:
Currently the text files are stored on a server and there are a bunch of users running this application to view this "live" data.
HOWEVER, due to:
An I/O bug in windows
The files "lock" up periodically and cause all of the applications to lock up (can't even close in task manager).
Therefore I decided to have the data copied from the text files to SQL, however from my understanding there's no way to overwrite the data in the SQL table. One must Drop the Table and Create a new one. This cause a 10+ second delay updating the data, which cannot happen.
My question is, there HAS to be a way to rapidly read and write data from somewhere, be it a database, etc. I am not sure where else to turn.
My constraints:
I'm stuck with Server 2008, have to use these text file, and I have to display it on my wpf application. Does anyone have any suggestions for a method that can handle this type of I/O?
All help is greatly appreciated, I'm at a complete loss..
It seems like you may not have extensive experience with database technology, so let me propose something different:
string text = System.IO.File.ReadAllText(path);
Then perhaps you can take the text and do what you want with it, dump it in a queue for action in another part of the application.
ReadAllText has some exceptions that are thrown:
https://msdn.microsoft.com/en-us/library/ms143368(v=vs.110).aspx
I'd be on the look out for UnauthorizedAccessException as you said, the file seems to lock up when multiple users are accessing it.
A question like this has been asked differently several times.
Yet here i am.
I am writing a standalone windows program which will get user input like three fields and have to store it in the disk.
Also i need to delete them, edit them and so on.
It should be UTF8.
Besides here is the actual need.
I have hosted this application in my server and users can download it.
I want the db automatically created when the program is executed for the first time.
What i mean is user can or should download only one file and that is the program.
The program will be one exe file and it will not have any dependencies.
meanwhile asking this question i tried sqllite for .net 2.0 and i got an installer from sourceforge and installed it.
I included in my application and it showed an error like there is a problem in that.
So, if people suggest sqlite then please give me reference of how to include in c#.net v2.0
I am new to .net so it is taking a very long time to fit things together so thought of posting this question.
any comments, suggestions, advices and references would do good.
EDIT
I have attached the error what i got
Edit after first reply
A user can save as many as set of three fields.
I mentioned three fields for an example.
They will save as many as records as they want.
It could 100 to infinity.
If that's only 3 fields to store - forget about databases and store the data in an XML file.
You can create a class that has these 3 properties, and then serialize/deserialize it on demand.
Here's a nice tutorial from Microsoft about XML serialization: http://support.microsoft.com/kb/815813
Deserialization is done in a very similar fashion.
I need to write an application that polls a directory which contains images on a file server and display 4 at a time.
This application will be run up to 50 times across the network at the same time.
I'm trying to think of the best architecture to complete this requirement.
I was working on the idea of opening a file with read/write access and no file share allowed so that if another PC came in to read it it would error and it would have to move on to the next one, the problem is, is that I need to access all 4 images in sequence on the same pc ensuring other pc's dont try to open them. So for example if PC1 tries to open 1.jpg it needs to be able to open 1,2,3,4.jpg. If another PC comes in at the same time to read them I need a way for it to then open 5,6,7,8.jpg and so on and so on.
It seems a simple requirement but a nightmare to try and build successfully.
You're basically dealing with a race condition here, and I don't see a way to handle it from separate instances of your application running on separate machines unless you can guarantee your file naming will always follow a standard naming convention that would allow you to work with the sequence of 4 files using only the name of the first.
The best way to handle this would be using a centralized resource to manage access to your files, either a database as was suggested in a comment or else a service (such as WCF) that would "hand out" each set of 4 files.
What about creating a 1.jpg.lock file? The presence of a the file indicates the images are locked and any other instance of the application should skip that set.
I need to create a patching routine for my application,
it's really small but I need to update it daily or weekly
how does the xdelta and the others work?
i've read around about those but I didn't understand much of it
the user shouldn't be prompted at all
Ok this post got flagged on meta for the answers given, so I'm going to weigh in on this.
xdelta is a binary difference program that, rather than providing you with a full image, only gives you what has changed and where. An example of a text diff will have + and - signs before lines of text showing you that these have been added or removed in the new version.
There are two ways to update a binary image: replace it using your own program or replace it using some form of package management. For example, Linux Systems use rpm etc to push out updates to packages. In a windows environment your options are limited by what is installed if you're not on a corporate network. If you are, try WSUS and MSI packaging. That'll give you an easier life, or ClickOnce as someone has mentioned.
If you're not however, you will need to bear in mind the following:
You need to be an administrator to update anything in certain folders as others have said. I would strongly encourage you to accept this behaviour.
If the user is an administrator, you can offer to check for updates. Then, you can do one of two things. You can download a whole new version of your application and write it over the image on the hard disk (i.e. the file - remember images are loaded into memory so you can re-write your own program file). You then need to tell the user the update has succeeded and reload the program as the new image will be different.
Or, you can apply a diff if bandwidth is a concern. Probably not in your case but you will need to know from the client program the two versions to diff between so that the update server gives you the correct patch. Otherwise, the diff might not succeed.
I don't think for your purposes xdelta is going to give you much gain anyway. Just replace the entire image.
Edit if the user must not be prompted at all, just reload the app. However, I would strongly encourage informing the user you are talking on their network and ask permission to do so / enable a manual update mode, otherwise people like me will block it.
What kind of application is this ? Perhaps you could use clickonce to deploy your application. Clickonce very easily allows you to push updates to your users.
The short story is, Clickonce creates an installation that allows your users to install the application from a web server or a file share, you enable automatic updates, and whenever you place a new version of the app on the server the app will automatically(or ask the user wether to) update the app. The clickonce framework takes care of the rest - fetching the update , figure out which files have changed and need to be downloaded again and performs the update. You can also check/perform the update programatically.
That said, clickonce leaves you with little control over the actual installation procedure, and you have nowhere close to the freedom of building your own .msi.
I wouldn't go with a patching solution, since it really complicates things when you have a lot of revisions. How will the patching solution handle different versions asking to be updated? What if user A is 10 revisions behind the current revision? Or 100 revisions, etc? It would probably be best to just download the latest exe(s) and dll(s) and replace them.
That said, I think this SO question on silent updates might help you.
There is a solution for efficient patching - it works on all platforms and can run in completely silent mode, without the user noticing anything. On .NET, it provides seamless integration of the update process using a custom UserControl declaratively bound to events from your own UI.
It's called wyUpdate.
While the updating client (wyUpdate) is open source, a paid for wybuild tool is used to build and publish the patches.
Depending on the size of your application, you'd probably have it split up into several dll's, an exe, and other files.
What you could do is have the main program check for updates. If updates are available, the main program would close and the update program would take over - updating old files, creating new ones, and deleting current files as specified by the instructions sent along with a patch file (probably a compressed format such as .zip) downloaded by the updater.
If your application is small (say, a single exe) it would suffice to simply have the updater replace that one exe.
Edit:
Another way to do this would be to (upon compilation of the new exe), compare the new one to the old one, and just send the differences over to the updater. It would then make the appropriate adjustments.
You can make your function reside in a separate DLL. So you can just replace the DLL instead of patching the whole program. (Assuming Windows as the target platform for a C# program.)
Recently I was working on displaying workflow diagram images in our web application. I managed to use the rehosted WF designer and create images on-the-fly on the server, but imagining how large the workflow diagrams can very quickly become, I wanted to give a better user experience by using some ajax control for displaying images that would support zoom & pan functionality.
I happened to come across the website of seadragon, which seems to be just an amazing piece of work that I could use. There is just one disadvantage - in order to use their library for generating deep zoom versions of images I have to use the file structure on a server. Because of the temporary nature of the images I am using (workflow diagrams with progress indicators), it is important to not only be able to create such images but also to get rid of them after some time.
Now the question is how can I best ensure that the temporary image files and the folder hierarchy can be created on a server (ASP.NET web app), and later cleaned up. I was thinking of using the cache functionality and by the expiration of the cache item delete the corresponding image folder hierarchy, or simply in the Application_Start and Application_End of Global.asax delete the content of the whole temporary folder, but I'm not really sure whether this is a good idea and whether there are some security restrictions or file-system-related troubles. What do you think ?
We do something similar for creating PDF reports and found the easiest way is to use a timestamp check to determine how "old" files are, and then delete them based on a period of time, in our case more then 2 hours old. This is done before the next PDF document is created, but as part of the creation process. We also created a specific folder and gave the ASP.Net user read/write access to the folder.
The only disadvantage is that if the process of creating PDF's is not used regularly there will be a build up of files, however they will be cleaned up eventually. In 2 years and close on 4000 PDF's we have yet to have an error doing it this way.
Use the App_Data folder. This folder is inside your application and writable by your app without having to go outside the context of the app, but it's also secured from casual browsing. It's meant to hold data files for your application.
Application_Start and Application_End will only fire once each, so if you need better cleanup than that, I would consider using a cache structure or a simple windows service to handle the cleanup.
First, you have to make sure your IIS worker process has rights to write/delete files from your cache directory (and NOT the rest of your site, just in case)
2nd, I would stay away from using App_Start and App_End, App end to clean up files is not 100% guaranteed to fire, and you could end up with a growing pile of orphaned images.
I would instead make a scheduled process, maybe runs once per hour, or once a day, depending on what you want. And have it check how old each image in your cache is, and if its older than your arbitrary "expiure time" then delete it.
Other than that there's not much to it.