I have a web application, and I would like to audit most of the users actions on the application, for example login, insertion to db, update to db, fired exceptions, etc.
One of my senios suggested using a queue for faster performance, so you just enqeue an event and it is then processed automatically without having to wait for it to be processed.
What are your suggestions? How should I process them? Enqueueing them is no problem, but I'm not sure how they will be processed then without no body calling a method.
I am using C# with .NET 4.0
I've been working on a library that can probably help.
Audit.NET and its extensions to audit different systems (WCF, MVC, WebApi, EF) and store logs in different data storages (SQL, MongoDB, DocumentDB, File, EventLog) will give you the flexibility to configure what do you want to audit and where do you want to store the audit logs.
I would simply recommend an off the shelf logging framework that is stable and supported. Have you considered a logging framework, like log4net?
You could write a custom appender for logging into MSMQ if you'd like.
An alternative logger is called TracerX. It is written in C# and fast and flexible. Because the source code is available it means you can modify it as you wish to suit your needs. It comes with a viewer that allows for filtering the output.
https://github.com/MarkLTX/TracerX and an article on how to use it:
http://www.codeproject.com/KB/dotnet/TracerX.aspx
Two topics of interest actually:
Asynchronous logging
Aspect Oriented Features
Asynchronous logging may speed-up heavy processing 100-fold. Use a writer thread that dumps the queue into log sink every,say 100ms however that Logging engine must be deterministically started and stopped so it can flush the sinks on application stop.
Aspect Oriented Programming addressed your cross-cutting concern - audit/log calls shall be invoked in desired operation prologues/epilogues - look at PostSharp project.
(Little late on the answer, this post shows up high in google, so I thought it may be worth looking at some of the options)
If you are looking to actually audit. (By this I mean to record that an action took place, who did it and when, and for that auditable log to be able to be used as evidence to an external auditor)
(Debug Logging vs Auditing logging)
If so, you can consider some options, such as:
use an Audit logging library
adopt an EventStore database
use a logging library that fails loudly
1. using an audit library
Audit.NET has already been mentioned here and has an impressive number of downloads and is very feature-rich
auditable - an alternative to the above (disclaimer, its written by me)
both are pretty cool, as they allow you to bring your own datastore
2. Eventsourcing
EventStore
Postgres with Marten
The design here (which can impact your architecture to embrace Events) is that Events are immutable, and if you store them then you have an auditable store of things that happened in your system
note this does not look to solve the question above, but it does solve how to audit, so I have mentioned it
3. Logging library
Serilog - Issue
you have to confirm that the logging library if it fails to add an Audit Log, it will throw an exception.
if it does not do that then you will be missing auditable logs, which then you cannot build trust with your Auditors
Side note 1 - with options 1 and 3, you may need to ensure that the log is written in the same transaction as your primary data store. to ensure that all of the information is ACID. (this is similar to the issue people have with publishing an event which is outside of the database transaction)
Side note 2 - that audit logs should be able to identify who did what, so you may/should need to encrypt the datastore they eventually end up in.
Related
I want to implement logging in a console application. And since I'm using libraries that expect ILogger, I need to provide one.
For reasons I don't quite fully understand, every article seems to recommend I use SeriLog rather than writing my own logger. But no one has provided a convincing reason why.
In either case, I need the following:
Logged information should go to the console.
Logged information should go to a log file.
After my application runs, I want to put all the information logged into an email.
Can anyone tell me how I could accomplish all three, and if I really need something like SeriLog to do so?
NOTE: I do not think it makes sense to read from the log file and use that to email the log. The log file could include logging from many sessions and I only want to email data from the most recent session.
Based on the requirements you listed, it seems Serilog would be a perfect fit for you, as you can leverage a combination of the existing sinks such as Console, File, and possibly Memory - though you don't need the last one... You can easily create a unique log file per session, read that file and send over email if that's what you want to achieve...
If you're really particular about the last part, you can write your own Serilog sink that buffers the log events the way you want and then send by email at the end.
That said, sending logs to yourself via email is not really something people do anymore these days... There are better sinks that you can use to send logs directly to a store/service that can continuously ingest logs automatically and provide great UIs and APIs to query your logs... You might want to take a look at Seq, Sentry, DataDog, and others...
Serilog is a well designed library, very widely used, and thoroughly tested, with close to 27,000 open-source projects and hundreds of contributors, as of this writing.
If you think you can innovate on how logging is done and take it to the next level, then that would be a good reason to roll your own library, otherwise seems like a waste of time reinventing the wheel.
Innovation is actually how Serilog came about... Other widely used logging libraries such as Log4Net and NLog existed before, but Serilog made semantic logging mainstream in the .NET space coupled with a well designed way of creating extensions (sinks, enrichers, destructurers, ...). The rest is history... NLog soon after followed the footsteps of Serilog, and over time Log4Net became obsolete and is legacy at this point.
I need to implement Audit Trail in my application which is WinForm application.
I need to log all the activity done by user on application and in System to see if he had changed any security settings or anything.
Is there any way to do this by AOP or using PostSharp or any other such method which could be done with minimal changes in existing code as it is a very big application and implementing logging in every method is a time taking steps.
I am open to create a new application which could be for auditing purpose if it helps.
Please let me know any best practices I should follow to implement Auditing.
We are using .Net 4.5 and SQL Server 2005.
Sounds like you want an audit of business-level operations attempted via your WinForms application.
Since, you asked about aspect-oriented approach - yes you can certainly use PostSharp's OnMethodBoundaryAspect to plug in some logging/auditing behaviour with almost no change to existing code.
You will also get information about the caller and values of arguments passed which you can use to make your audits meaningful. Will update shortly with example. Further Reading
DISCLAIMER: I do not work for PostSharp. I just happened to try it out recently.
I'm wondering if there's any way of having some sort of Aspect-Oriented way of setting up logging of C# code. Or if the code could be instrumented for automatic logging.
At the moment the code is riddled with Log("Enter method XXX") and Log("Leaving method XXX") which make maintenance really tedious.
Ideally I'd like to have something that does the logging automatically the same way as the libraries are instrumented for profiling.
The next best thing would be to have some custom attributes maybe that I can tag my methods with. These would put some logging code on entrance and exit of the method.
And if the solution were compatible with the EntLib that would be perfect :)
Cheers.
If you're using the Enterprise Library, you have everything you need. Take a look at this article: http://www.codewrecks.com/blog/index.php/2009/01/31/unity-and-aop-in-enterprise-library/
You could use Log4PostSharp. I am not sure though what the future of this looks like as PostSharp went commercial.
What your referring too is a cross cutting concern, and not only affects your application but other applications that you might install at your establishment. The Enterprise Blocks are great and the inversion of control principal does help a lot with extracting repeating code from out of the system. However there is no way of logging without deciding some place in your code that you wish to record the event. for example exceptions, logging in, logging out, db actions, restricted actions etc. If you go the Enterrpise route its all done through configuration files and policies.
In the solutions I have provided, I have moved the logging functionality outside of the application space and it now sits aside every piece of code that I develop, ready and waiting to do the logging for me. On the last project I used a combination of Enterprise Blocks and Couchdb. Couchdb really helps with the aspect side as it works using REST and Json without involving itself too much in your application writing an interface to the log files is just a matter of a bit of HTML, it really is a fire and forget type affair, until that bad ol day when you need to scour the logs :)
The only problem that I have seen in applications where you automate the logging is that you use some sort of delegate process and pass things into them, which increases stack space. But this is so trivial that its beyond reason.
Program to interfaces and defined interfaces and you should be okay.
I remember something regarding Interceptors / Proxying to log entry/exit of methods.
Stack Overflow question - How do I intercept a method call in C#?
and check out this blog (ref'd in the same question) - http://madcoderspeak.blogspot.com/2005/09/essential-interception-using-contexts.html
I'm building a C#/Asp.Net (FW 2.0) web application and I was wondering where I should store my error logs entry.
I could write it on a file on the server, but that doesn't seems right to me. It would be insconsistent with the idea of a database.
The other obivous answer is on the database, but what happened if I do not have any connection with the database. If the client get a connection error to the database what do I do(because the server and the database aren't on the same machine obviously)?
Your suggestions are appreciated.
Than you
Edit : I'm talking about basic error logging here, please do not refer such things as big frameworks like log4net.
System Event log with appropriate event source (logging channel)
http://support.microsoft.com/kb/307024
Compared to logging to a DB log4net or nlog are not "big frameworks". Keep it simple and these two provide exactly what you need, with very little ramp up period.
Having a fallback mechanism is a very valid scenario. Logging to the windows event log is often not practical, because it isn't easy to analyze to logs, as is with relational databases. But allowing events to be logged to the event log when your database fails is not a luxury option IMO.
This scenario is one of the reasons I built a logging framework (I won't reference it, just as you requested). It supports a concept called 'fallbackProvider' that allow the event to be logged to an alternative logger in case the main logger fails. When you write your own logging mechanism, you could embrace such a concept.
Good luck.
Store it in the event log; it is designed for this purpose after all! Its the place most people would look for errors, messages and warning regardless of what they know about the application.
The enterprise library provides a framework for logging which you can use Enterprise Library. This allows you to change how/where events are logged based on a configuration file so you don't have to make a decision where events are logged.
In addition to this, if you use the exception handling block you can also log errors to the eventlog with minimal effort
I guess you really have three options:
Use a small database to store all the error logs on the local machine using something light-weight like SQLlite or SQLServer Compact.
Save it to a flat file (xml, or what have you) where you can view it.
Send it straight to the Event Log. (I'd probably do this).
log to the database and xml as a fallback, asp.net account will need perms to log errors to eventviewer which may not be such a good idea on the web server unless it is absolutely neccessasy.
If you're willing to consider a simple commercial product, take a look at Gibraltar. It stores logs locally then uploads them to a web service when a connection is available. The logs are then indexed in an embedded database and an analysis tool lets you review errors and other information at whatever level of detail you require.
I see that you're a student, Frank. Email me directly to discuss the possibility of us offering you a free license.
-System Event Viewer
-you may cache your error to local & lightweight db file (can be SqLite/Sql Compact), then when connection is available, you send it to server
First of all, apologies for the subjective sounding title. This is intended as a direct question.
At present I am working on a suite of tools:
A C# Windows Service, to primarily
maintain an Oracle database.
A C# Windows Service, (which will be
used on multiple node sites) to
process content of the database.
An ASP.NET web interface to
facilitate management of the overall
"system"
Currently the tho Windows Services have been developed as Console Applications (to ease debugging/development) and I am in the midst of converting these to Services. After testing for a couple days now with these services, I'm finding that I would like to increase the granularity of my logging. I'm finding that I miss Console.WriteLine() and I would like to provide an alternate log source like a flat-file for this type of output. This has lead me to think, "Should I be using a framework, or have I got enough?"
The reason I have mentioned the aspects I am developing is to provide insight to my situation. A "Core" DLL has been created, common across all components, abstracting the interaction layer between the applications and database. It is within this DLL that a class has been created which will attempt to "log to a table in the database" else on fail "log to local Event Log". This is it, that's the extent of logging.
Throughout the aforementioned tools, there are multiple instances of logging not dissimilar to:
Log.LogError("Code", e.Message + "\n" + e.StackTrace);
Although quite basic, this method does make use of reflection to Identify the source of the error.
My Question
Looking at my current logging solution it appears "sufficient" in terms of what it does and how it is integrated with all my solutions. However, I've been looking at logging frameworks (Notably log4net) and their features impress me. The ability to, if needed in the future, add another output format (such as an SMTP server) sounds kind of cool to me! :)
What I would like to know are the benefits of moving to a framework (like log4net)? The extent of how much I will have to adapt my code? Whether or not I am just looking at the greener grass on the other side? And finally, but probably most importantly, am I doing the right thing? Should I just add the ability to my Log class to "LogDebug" and be done with it? The last thing I would want to do is completely overhaul my suite, just for a "basic" feature, but if there are other benefits (to design, reliance, good practice? etc.) I'm interested.
Thanks,
Yes. Using an existing, proven logging framework (such as Log4net) is a good idea.
Log4Net is configurable at runtime (great for tracking down issues in production code).
As a commenter pointed out, it's also very simple to use.
Proper logging is especially beneficial when running code on multiple remote systems, as far as I recall, log4net will let you send your logs to a remote syslog server without much coding overhead (meaning you can view your logs from all machines in one centralized place) doing this will massively reduce the time it takes you to get information relating to a bug or problem with the system, and should also give you an indication of how prevalent the issue is.
As mentioned in other posts, log4net also allows for multiple appenders and multiple log levels, so determining where you want certain log information (i.e. in a database or in a local flat file, hey log4net even lets you spit logs out over telnet) to be stored is an absolute doddle.
As for implementing it, there are several good sites talking you through the setup. How you actually make use of the logging objects that log4net gives you is an architectural choice, but you could simply change the constructor of an object to take a log4net object and from within this object, just use the log4net object as you would Console.WriteLine.
I find the tutorial series here particularly useful, and it'll also go in to more depth than I can here about the benefits and the different ways of configuring log4net.
Yes, you definitely want to use a logging framework. A logging framework will allow you to:
Set the logging levels for the different logger instances.
Set the "appenders" or output for each of the different logger instances.
Perhaps, more importantly, if you use a logging framework, it is very easy to swap out one implementation of the logging framework for another (perhaps a null implementation that simply discards messages); whereas, if you write all your logging statements, directly, swapping out the implementation will be a nightmare.
I think you should use Log4net, simply because it's always better to reuse than to build your own thing. log4net has been used by a lot of developers and are pretty matured.
Think about your maintenance prospect; one or two months down the road, you might need to tweak your custom logging class a bit, to add some multithreading support etc. And when you are fixing the bugs arose from your logging class, you will miss Log4net.
Well one of the bigger benefits is not having to maintain the code yourself. Most of the time, logging frameworks have a lot more functionality than your own solution. Because they are so focused on logging, those frameworks usually are pretty complete in the both functionality and ways to implement it. And then there's reliability; there's nothing worse than a logging framework that's not logging anything because it's bugged. ;)
Take for example ELMAH for ASP.net applications. It also includes notifications, exports to various target formats, etc. Things that are pretty handy but you'll never build yourself unless you really need it.
How many changes to your code are needed obviously depends on both your code and the framework of choice. It's hard to say anything about that.
I am going to give a shout out to NLog (http://nlog-project.org/home) as it doesn't suffer from the 'Straight Java Port - then rewrite' syndrome of most oss .Net libs.
Some key benefits for us were the very fast Logger.IsFooEnabled (volatile read) and the overall performance of the system.
To each its own though, but I personally prefer NLog for my projects (and some of my clients too).
Cheers,
Florian
The advantage of using a good logging framework like Log4Net is that they have a small impact upon your code in terms of lines of code that you have to alter (in other words you only have to alter each existing logging line).
Also, if you are concerned about altering your code if you change frameworks, or if you feel you want to roll your own, then you could always create your own interface to a logging framework. Then you only ever have to change your code in one place after that.
I think sysadmins expect services to log to the application event log in windows.
Look up System.Diagnostics.EventLog, although log4net will write to that too..
The initial statement in the log4j website might help in some of your questions, the underlying principles are the same of log4net:
With log4j it is possible to enable
logging at runtime without modifying
the application binary. The log4j
package is designed so that these
statements can remain in shipped code
without incurring a heavy performance
cost. Logging behavior can be
controlled by editing a configuration
file, without touching the application
binary.
Using a logger hierarchy it is
possible to control which log
statements are output at arbitrarily
fine granularity but also great ease.
This helps reduce the volume of logged
output and minimize the cost of
logging.
In this case there's clearly no need to reinvent the wheel. Most Logging frameworks are somewhat straightforward, so the extend of changes will most likely depend on the size of your existing programs.
if you write your logger class properly it will be easily expendable to any of your needs. Any framework could impress you with many features but another framework is another variable in your debugging process as it can give you an error that does not exists or can make an error by itself in combination with your application. If you are ready to make beta testing for open source software project this is fine...
In your place i would write log class with ability to extend it features you find interesting to you project based on the list of features known frameworks have. I don't see any problem to log something to file and then send it over smpt, just one small function does the job.
Moreover, you can write your own class which will be pretty abstract and put your basic code in there, if you will ever need to use external framework for testing you class would be able to use it with minimal impact on code. Just take a look how there frameworks are implemented on the code level.
think of that you will need to learn how to properly use these frameworks when your only needs for now to log very small part of it...