What is the best way in logging actions, activities, etc done in an asp.net application. Also, which storage is best for logging these? XML? DB?
Thank you very much.
The answer I hate most, actually applies here: "it depends". Specifically, it depends on several things:
Who is the logging information for? Is it intended for business users (i.e., are there actual business requirements), is the information oriented at application management, do you need insight in frequently used features, etc.
What is the granularity of the logged info? For instance: do you only need to know if the search function was used, do you want to know the search query or do you also need info on the actual search results?
How accurate & complete does the info have to be? Audit trail requirements are usually very tight, technical ones often less so.
Do you want to be able to roll back the actions/activities? And if so, who is going to do that (business user, support personnel)
What does your deployment look like? If you have a single server, logging to text files or XML is more feasible than if you have a farm/load balanced environment.
For application logging, look at well-known providers such as log4net or the Enterprise Library logging application block; both allow you to configure where you want to log to (text file, database, etc).
For logging database actions, I suggest a solution in the database. Several versions of SQL Server 2008 have built-in support for auditing, Oracle has had this for years IIANM.
PostSharp probably. Log to a DB.
-- Edit:
This is for logging all code actions. To log all DB actions, I'd use triggers.
I would use :log4net
because you can configure and change the output(file, mail, DB, ...) in the config file so you do not have to rebuild your code.
If you don't need a complex auditing system, but just logging what your code is doing, I'll recommend you to use the tracing system integrated with .NET Framework and ASP.NET.
Using very simple classes in the framework, your code emit traces, and then, via configuration files you can send them to different storing systems (file, database, windows events, ...). You can even create your own store system trace provider
http://msdn.microsoft.com/en-us/library/system.diagnostics.trace.aspx
Related
I want to decide between semantic logging for azure vs traditional logging where all logs use to go to wadlogs table. Is there any specific reason we should go for one or other? what is the difference between two and which one is preferable?
Semantic logging is like strong typed logging messages. They give you consistency so an automated process (or a person) could filter through a log that could contain messages from multiple applications. It also centralizes the definition of what you can log. You can format a log message any way you want, but Azure and SQL sinks give the added benefit of persisting the entire payload. WAD logs are more geared for diagnostics and are not structured as they are stored as blobs.
I have a C# application that allows one user to enter information about customers and job sites. The information is very basic.
Customer: Name, number, address, email, associated job site.
Job Site: Name, location.
Here are my specs I need for this program.
No limit on amount of data entered.
Single user per application. No concurrent activity or multiple users.
Allow user entries/data to be exported to an external file that can be easily shared between applications/users.
Allows for user queries to display customers based on different combinations of customer information/job site information.
The data will never be viewed or manipulated outside of the application.
The program will be running almost always, minimized to the task bar.
Startup time is not very important, however I would like the queries to be considerably fast.
This all seems to point me towards a database, but a very lightweight one. However I also need it to have no limitations as far as data storage. If you agree I should use a database, please let me know what would be best suited for my needs. If you don't think I should use a database, please make some other suggestions on what you think would be best.
My suggestion would be to use SQLite. You can find it here: http://sqlite.org/. And you can find the C# wrapper version here: http://sqlite.phxsoftware.com/
SQLite is very lightweight and has some pretty powerful stuff for such a lightweight engine. Another option you can look into is Microsoft Access.
You're asking the wrong question again :)
The better question is "how do I build an application that lets me change the data storage implementation?"
If you apply the repository pattern and properly interface it you can build interchangable persistence layers. So you could start with one implementation and change it as-needed wihtout needing to re-engineer the business or application layers.
Once you have a repository interface you could try implementations in a lot of differnt approaches:
Flat File - You could persist the data as XML, and provided that it's not a lot of data you could store the full contents in-memory (just read the file at startup, write the file at shutdown). With in-memory XML you can get very high throughput without concern for database indexes, etc.
Distributable DB - SQLite or SQL Compact work great; they offer many DB benefits, and require no installation
Local DB - SQL Express is a good middle-ground between a lightweight and full-featured DB. Access, when used carefully, can suffice. The main benefit is that it's included with MS Office (although not installed by default), and some IT groups are more comfortable having Access installed on machines than SQL Express.
Full DB - MySql, SQL Server, PostGreSQL, et al.
Given your specific requirements I would advise you towards an XML-based flat file--with the only condition being that you are OK with the memory-usage of the application directly correlating to the size of the file (since your data is text, even with the weight of XML, this would take a lot of entries to become very large).
Here's the pros/cons--listed by your requirements:
Cons
No limit on amount of data entered.
using in-memory XML would mean your application would not scale. It could easily handle a 10MB data-file, 100MB shouldn't be an issue (unless your system is low on RAM), above that you have to seriously question "can I afford this much memory?".
Pros
Single user per application. No concurrent activity or multiple users.
XML can be read into memory and held by the process (AppDomain, really). It's perfectly suited for single-user scenarios where concurrency is a very narrow concern.
Allow user entries/data to be exported to an external file that can be easily shared between applications/users.
XML is perfect for exporting, and also easy to import to Excel, databases, etc...
Allows for user queries to display customers based on different combinations of customer information/job site information.
Linq-to-XML is your friend :D
The data will never be viewed or manipulated outside of the application.
....then holding it entirely in-memory doesn't cause any issues
The program will be running almost always, minimized to the task bar.
so loading the XML at startup, and writing at shutdown will be acceptible (if the file is very large it could take a while)
Startup time is not very important, however I would like the queries to be considerably fast
Reading the XML would be relatively slow at startup; but when it's loaded in-memory it will be hard to beat. Any given DB will require that the DB engine be started, that interop/cross-process/cross-network calls be made, that the results be loaded from disk (if not cached by the engine), etc...
It sounds to me like a database is 100% what you need. It offers both the data storage, data retrieval (including queries) and the ability to export data to a standard format (either direct from the database, or through your application.)
For a light database, I suggest SQLite (pronounced 'SQL Lite' ;) ). You can google for tutorials on how to set it up, and then how to interface with it via your C# code. I also found a reference to this C# wrapper for SQLite, which may be able to do much of the work for you!
How about SQLite? It sounds like it is a good fit for your application.
You can use System.Data.SQLite as the .NET wrapper.
You can get SQL Server Express for free. I would say the question is not so much why should you use a database, more why shouldn't you? This type of problem is exactly what databases are for, and SQL Server is a very powerful and widely used database, so if you are going to go for some other solution you need to provide a good reason why you wouldn't go with a database.
A database would be a good fit. SQLite is good as others have mentioned.
You could also use a local instance of SQL Server Express to take advantage of improved integration with other pieces of the Microsoft development stack (since you mention C#).
A third option is a document database like Raven which may fit from the sounds of your data.
edit
A fourth option would be to try Lightswitch when the beta comes out in a few days. (8-23-2010)
/edit
There is always going to be a limitation on data storage (the empty space of the hard disk). According to wikipedia, SQL Express is limited to 10 GB for SQL Server Express 2008 R2
I'm just starting to work on a logging library that everyone can use to keep track of any sort of system information while the user is running our application. The simplest example so far is to track Info, Warnings, and Errors.
I want all plugins to be able to use this feature, but since each developer might have a different idea of what's important to report, I want to keep this as generic as possible.
In the C++ world, I would normally use something like a stl::pair<string,string> to act as a key value pair structure, and have a stl::list of these to act as a "row" in the log. The log cache would then be a list<list<pair<string,string>>> (ugh!). This way, the developers can use a const string key like INFO, WARNING, ERROR to have a consistent naming for a column in the database (for SELECTing specific types of information).
I'd like the database to be able to deal with any number of distinct column names. For example, John might have an INFO row with a column called USER, and Bill might have an INFO row with a column called FILENAME. I want the log viewer to be able to display all information, and if one report doesn't have a value for INFO / FILENAME, those fields should just appear blank. So one option is to use List<List<KeyValuePair<String,String>>>, and the another is to have the log library consumer somehow "register" its schema, and then have the database do an ALTER TABLE to handle this situation. Yet another idea is to have a table that's just for key value pairs, with a foreign key that maps the key value pairs back to the original log entry.
I obviously don't want logging to bog down the system, so I only lock the log cache to make a copy of the data (and remove the already-copied data), then a background thread will dump the information to the database.
My specific questions regarding this are:
Do you see any performance issues? In other words, have you ever tried something like this and found that certain things just don't work well in practice?
Is there a more .NETish way to implement the key value pairs, other than List<List<KeyValuePair<String,String>>>?
Even if there is a way to do #2 better, is the ALTER TABLE idea I proposed above a Bad Thing?
Would you recommend multiple databases over a single one? I don't yet have an idea of how frequently the log would get written to, but we ideally would like to have lots of low level information. Perhaps there should be a DB with a fixed schema only for the low level stuff, and then another DB that's more flexible for reporting information back to users.
why don't you check log4net? It may be enough for your purposes and you would avoid re invent a wheel already invented many times :-)
Here you have some configuration examples about how to store the logging information on the database:
http://logging.apache.org/log4net/release/config-examples.html
Like others have already noted, there are several popular logging frameworks that have a lot of built-in functionality. While none of them have the flexibility you desire, my experience is that you never really need that kind of flexability. It's only logging :-).
Here is a list of some common logging libraries:
log4net
NLog
Enterprise Library
Termite
ELMAH
CuttingEdge.Logging
(did I miss one?)
And when you have trouble chosing one, use a logging facade to hide the implementation. For this you can pick:
Common.Logging
Simple Logging Facade.
Not until there are some (Knuth).
Create models that actually model your log entries, then cache them in a List
I'd provide standard fields and store all user configurables as XML. Allows for a flexible logging system, doesn't result in schema alterations, and is (at least in Sql Server) searchable.
More databases is more maintenance. Keep it simple. In fact, just use Log4net.
i have some WinForms app (Framework to develop some simple apps), written in C#. My framework later would be used to develop win forms applications. Other developers they are beginers often and sometimes do not use Parameters - they write direct SQL in code. So first i need somehow to do protection in my framework base classes in C#.
Do solve this, one developer suggested me to using an ORM such as NHibernate, which takes care of this issue for you (and you don't have to write SQL statements yourself most of the time).
So I want to ask, is there some general alternatives(other ways and techniques) when i want to get defense from SQL-injections.Some links or examples would be very nice.
I don't see how there is any means to protect any SQL-based library from developer misuse without crippling its functionality (i.e. never giving direct access to the database).
Even with NHibernate or Linq to SQL it's possible to bypass the mapping layers and directly write a SQL statement.
Personally I think your best option would be to write in BIG BOLD TEXT that people who use your library need to PARAMETERIZE THEIR QUERIES. Failing that, you could try to do some kind of clumsy input sanitization, but that is honestly a flimsy second-rate hack.
Parameterized queries have been around for so long now, there's no excuse for anyone writing code that touches any database to not be aware of it or understand how to use it. The only cure for ignorance is education.
Maybe if we knew more about what this library is supposed to do with respect to data access, we could offer more targeted suggestions...
Agree with Aaronaught, a framework will not completely prevent the possibility. I would never substitute stringent validation on the data layer. Also provide an abstraction layer around your data access that you open up as the API rather then allow developers to connect directly to database.
It sounds like you need to train your developers to use parameter binding instead of looking for a technical solution.
One other alternative would be to keep the database layer in a different project and only allow your SQL savy developers to code in it. The GUI can be in a different project. That way the GUI programmers won't mess up your DB.
Security is usually a process, not a product or api.
It is also an evolving process, we have to adapt or get hacked.
A heavy handed approach:
You can force everyone to write stored procedures,and not allow
direct table access from the accounts that are allowed to talk to
the database. (GRANT EXECUTE ON etc)
Then you would need to ensure that nobody writes any fancy stored procedures
that take a sql query as a parameter and evaluates it dynamically.
This tends to slow down development, and I personally would not use it,
but I have consulted at several shops that did.
I am working on a Silverlight client and associated ASP.NET web services (not WCF), and I need to implement some features containing user preferences such as a "favourite items" system and whether they'd like word-wrapping or not. In order to make a pleasant (rather than infuriating) user experience, I want to persist these settings across sessions. A brief investigation suggests that there are two main possibilities.
Silverlight isolated storage
ASP.NET-accessible database
I realise that option 2 is probably the best option as it ensures that even if a user disables isolated storage for Silverlight, their preferences still persist, but I would like to avoid the burden of maintaining a database at this time, and I like the idea that the preferences are available for loading and editing even when server connectivity is unavailable. However, I am open to reasoned arguments why it might be preferrable to take this hit now rather than later.
What I am looking for is suggestions on the best way to implement settings persistence, in either scenario. For example, if isolated storage is used, should I use an XML format, or some other file layout for persisting the settings; if the database approach is used, do I have to design a settings table or is there a built-in mechanism in ASP.NET to support this, and how do I serve the preferences to the client?
So:
Which solution is the better solution for user preference persistence? How might settings be persisted in that solution, and how might the client access and update them?
Prior Research
Note that I have conducted a little prior research on the matter and found the following links, which seem to advocate either solution depending on which article you read.
http://www.ddj.com/windows/208300036
http://tinesware.blogspot.com/2008/12/persisting-user-settings-in-silverlight.html
Update
It turns out that Microsoft have provided settings persistence in isolated storage as a built-in part of Silverlight (I somehow missed it until after implementing an alternative). My answer below has more details on this.
I'm keeping the question open as even though Microsoft provides client-side settings persistence, it doesn't necessarily mean this is the best approach for persisting user preferences and I'd like to canvas more opinions and suggestions on this.
After investigating some more and implementing my own XML file-based settings persistence using IsolatedStorage, I discovered the IsolatedStorageSettings class and the IsolatedStorageSettings.ApplicationSettings object that is a key/value collection specifically for storing user-specific, application settings.
It all seems obvious now. Of course, in the long term, a mechanism for backing up and restoring settings using a server database would be a good enhancement to this client-side settings persistence.
I think in general the default would be to store on the server; only when there are specific compelling reasons to attempt to store on the client should we do so. The more you rely on storing in a medium you can't control, the more risk you take on.
That having been said, and setting myself on the "database" side of the argument, I would ask what the downside of a database is? You mentioned using XML - is your data only semi-structured? If so, why not store XML in a SQL database? Setting up something this simple would not generally be considered a "burden" by most standards. A simple web service could act as a go-between between your Silverlight client and the settings database.
If it is an important feature for you that users have access to their preferences while offline, then it looks like isolated storage is the way to go for you. If it's more important that users be able to save preferences even if they have turned off isolated storage (is this really a problem? I'd be tempted to call YAGNI on this, but I'm not terribly experienced with the Silverlight platform...) then you need to host a database. If both are important, then you're probably looking at some kind of hybrid solution; using isolated storage if available, then falling back to a database.
In other words, I think the needs of your application are more important than some abstract best practice.