I'm working with few applications, Two Dot Net (Standalone Windows Thick client) and SSIS (SQL Server ETL Engine) and right now all of them use their own logging implementations. Dot Net uses Log4Net and SSIS writes to a text file.
I want to redesign this whole thing where all the applications right to one Logging interface. There are few options which I'm considering: Windows Event Logging, ELK(Elastic/LogStash) or Custom Log provider
Details:
It's a file processing system where we do a bunch of operations using DotNet Apps and then load the files, using SSIS, into a SQL server DB.
I recommend going for log4net in the other application. It is very popular, has lots of features, is maintained, well documented and you can find tons of help on it on different forums (including stackoverflow). It can do much more than just logging into a file. It supports remote logging, SMTP, .NET Trace, DB and even Windows Event Log. Allows for extremely flexible configuration.
It is always good to have separate logging for separate apps but you can have both log into a single logging target as well.
Related
I have a web application, and I would like to audit most of the users actions on the application, for example login, insertion to db, update to db, fired exceptions, etc.
One of my senios suggested using a queue for faster performance, so you just enqeue an event and it is then processed automatically without having to wait for it to be processed.
What are your suggestions? How should I process them? Enqueueing them is no problem, but I'm not sure how they will be processed then without no body calling a method.
I am using C# with .NET 4.0
I've been working on a library that can probably help.
Audit.NET and its extensions to audit different systems (WCF, MVC, WebApi, EF) and store logs in different data storages (SQL, MongoDB, DocumentDB, File, EventLog) will give you the flexibility to configure what do you want to audit and where do you want to store the audit logs.
I would simply recommend an off the shelf logging framework that is stable and supported. Have you considered a logging framework, like log4net?
You could write a custom appender for logging into MSMQ if you'd like.
An alternative logger is called TracerX. It is written in C# and fast and flexible. Because the source code is available it means you can modify it as you wish to suit your needs. It comes with a viewer that allows for filtering the output.
https://github.com/MarkLTX/TracerX and an article on how to use it:
http://www.codeproject.com/KB/dotnet/TracerX.aspx
Two topics of interest actually:
Asynchronous logging
Aspect Oriented Features
Asynchronous logging may speed-up heavy processing 100-fold. Use a writer thread that dumps the queue into log sink every,say 100ms however that Logging engine must be deterministically started and stopped so it can flush the sinks on application stop.
Aspect Oriented Programming addressed your cross-cutting concern - audit/log calls shall be invoked in desired operation prologues/epilogues - look at PostSharp project.
(Little late on the answer, this post shows up high in google, so I thought it may be worth looking at some of the options)
If you are looking to actually audit. (By this I mean to record that an action took place, who did it and when, and for that auditable log to be able to be used as evidence to an external auditor)
(Debug Logging vs Auditing logging)
If so, you can consider some options, such as:
use an Audit logging library
adopt an EventStore database
use a logging library that fails loudly
1. using an audit library
Audit.NET has already been mentioned here and has an impressive number of downloads and is very feature-rich
auditable - an alternative to the above (disclaimer, its written by me)
both are pretty cool, as they allow you to bring your own datastore
2. Eventsourcing
EventStore
Postgres with Marten
The design here (which can impact your architecture to embrace Events) is that Events are immutable, and if you store them then you have an auditable store of things that happened in your system
note this does not look to solve the question above, but it does solve how to audit, so I have mentioned it
3. Logging library
Serilog - Issue
you have to confirm that the logging library if it fails to add an Audit Log, it will throw an exception.
if it does not do that then you will be missing auditable logs, which then you cannot build trust with your Auditors
Side note 1 - with options 1 and 3, you may need to ensure that the log is written in the same transaction as your primary data store. to ensure that all of the information is ACID. (this is similar to the issue people have with publishing an event which is outside of the database transaction)
Side note 2 - that audit logs should be able to identify who did what, so you may/should need to encrypt the datastore they eventually end up in.
In my team there is discussion about web application exception logging, regarding whether we should log to a text file or the event log.
Can anyone please tell me which is the better way of exception logging occuring in the application? Either one of the two I've already mentioned or is there a better option?
Don't reinvent the wheel - use an already existing logging library/tool such as log4net or ELMAH instead of writing your own logging library. Both give you multiple choices for where you log exceptions to, it's up to you which you prefer.
Two things to consider:
1 - Where do you log to if you're logging to the database, but the database is unavailable / under heavy load?
2 - Windows Event Logging is very light-weight and event log forwarding means you can automagically aggregate logs from multiple machines without writing any code.
Another great and easy to use Logging-Library is NLOG
From the site:
"Supported targets include:
Files – single file or multiple, with
automatic file naming and archival
Event Log – local or remote
Database
– store your logs in databases
supported by .NET
Network – using
TCP, UDP, SOAP, MSMQ protocols
Command-line console – including
color coding of messages
E-mail – you
can receive emails whenever
application errors occur
ASP.NET trace
… and many more
Other key features:
very easy to configure, both through configuration file and programmatically
easy-to-use logger pattern known from log4xxx
advanced routing using buffering, asynchronous logging, load balancing, failover, and more
cross-platform support: .NET Framework, .NET Compact Framework and Mono (on Windows and Unix)"
I prefer a text file because it allows for more flexibility and easier navigation. I suppose it's a matter of preference, but the navigation between errors in the Windows event log seems very cumbersome. In addition to the cumbersome navigation it includes errors that you don't care about. If you can define the format and the content then it's much more efficient.
At the app I work, we are used to log every exception at the database, so this way is easier to link the exceptions details to the "user error report" they are requested to fill (which is at our the default error page). It helps us on statistics and as already said, linking exception details to the user description of the error (like, which steps he took to get that error, etc...)
Disclaimer: i am the developer of KissLog.net logging application.
log4Net and Elmah are very good, but they have limited capabilities.
Elmah doesn't support trace / log messages, and logs only the exceptions, and log4Net doesn't provide a user interface to navigate through the logs.
You can try to use KissLog.net, it is an application where i tried to aggregate all the necessary logging features (capture errors, log / trace messages, easy to use user-interface)
I'm writing a small c# console application that needs to interogate Websphere Application Server ND (6.1) to retrieve a list of installed apps.
I can easily do this from the command line using the wsadmin command, but don't really want to launch wsadmin from my c# app.
Is there a way to natively get c# to talk to Websphere and get this sort of information ?
Wsadmin is just a scripting interface for JMX and everything you can do with it can be done with RMI and SOAP. For C# users that means querying for Management Beans via SOAP.
However there are no turn-key solutions available. What has been suggested before has been at least taking a look at the ws-jmx-connector and implementing your own library. You could probably easier just capture one of those queries with ie. SoapUI and replay the SOAP calls. This probably means too much work and that's probably also why there are no ready solutions.
Also, you could just read the XML files that describe the (properly) installed WebSpehre Application Server applications. That is probably much easier, and works just fine. Take a look at the server profile directory. You should see a directory called config, then under it cells, your management cell's name and under that you will find XML files that actually contain every setting you see in the management console. They are well parseable by the standard .NET libraries and a few of those will contain application lists. Take a look at serverindex.xml for instance.
The scenario that I bring forward is basically that of, interaction between two .NET executables.
I have made a .NET Windows Forms application in C# (Application-A) that runs on a user's machine and does some specific activity, due to which it collects some data.
Now I have another .NET Windows Forms executable (Application-B), also made in C#, which also does some specific activity based certain inputs or data provided.
Now what I want to do here is, call Application-B from the Application-A and pass the some data to it.
How do I accomplish this?
You can use several options. You have some resources for each option below.
.NET Remoting
WCF
Use a communication file
MSMQ
Two last options are valid only if the processes are in the same machine.
Since they are two separated processes I think that the easiest way to do this is using .NET Remoting. Here you can find documentation and examples about how to do it.
An alternative to Remoting is WCF (>= .NET 3.0). It performs better than remoting.
If the processes will be always in the same machine, if you don't want to use remoting on localhost you can communicate them through a file (simple solutions usually work fine!)
And other more complex solution is communicate them using a Message Queue (MSMQ). Here you cand find out an example about how to use it.
You can use MSMQ to communicate between the applications.
Any mechanism will do, however.
You could use file based communications (write to a known directory and read from it).
WCF is another solution.
WCF is going to allow you a lot of flexibility in the future : Should you ever decide to enhance this communication to support multiple modes of communication, app.config changes should be the majority of the work to support a different binding.
In some of the projects I've been involved in, there's a mix of communications technologies where one choice would've been far easier to maintain-- this leads me to embrace the WCF decision for its inherent flexibility (WCF also supports MSMQ should the need arise to have queued communication).
If you're concerned about the learning curve and you're sure that no future need will arise for you to embrace other communication topologies, remoting could be a useful solution. Remoting is probably the easiest, least-developer-work needed way of setting up IPC.
You should stay away from things like Web Services -- there's unnecessary overhead in the web service operations that WCF doesn't suffer from (WCF can still allow binary transport, for instance).
If both the applications are running on same user computer than
1- this can be achieved through inter-process communications channel (IPC channel)
2- If you are using .NET 4.0, you can use memory mapped files
If both the applications are running on different system
1- You can make use of .NET Remoting
2- You can have WCF service based communication
3- Web Service is also an option if using .NET 2.0 or lower versions
I'm building a C#/Asp.Net (FW 2.0) web application and I was wondering where I should store my error logs entry.
I could write it on a file on the server, but that doesn't seems right to me. It would be insconsistent with the idea of a database.
The other obivous answer is on the database, but what happened if I do not have any connection with the database. If the client get a connection error to the database what do I do(because the server and the database aren't on the same machine obviously)?
Your suggestions are appreciated.
Than you
Edit : I'm talking about basic error logging here, please do not refer such things as big frameworks like log4net.
System Event log with appropriate event source (logging channel)
http://support.microsoft.com/kb/307024
Compared to logging to a DB log4net or nlog are not "big frameworks". Keep it simple and these two provide exactly what you need, with very little ramp up period.
Having a fallback mechanism is a very valid scenario. Logging to the windows event log is often not practical, because it isn't easy to analyze to logs, as is with relational databases. But allowing events to be logged to the event log when your database fails is not a luxury option IMO.
This scenario is one of the reasons I built a logging framework (I won't reference it, just as you requested). It supports a concept called 'fallbackProvider' that allow the event to be logged to an alternative logger in case the main logger fails. When you write your own logging mechanism, you could embrace such a concept.
Good luck.
Store it in the event log; it is designed for this purpose after all! Its the place most people would look for errors, messages and warning regardless of what they know about the application.
The enterprise library provides a framework for logging which you can use Enterprise Library. This allows you to change how/where events are logged based on a configuration file so you don't have to make a decision where events are logged.
In addition to this, if you use the exception handling block you can also log errors to the eventlog with minimal effort
I guess you really have three options:
Use a small database to store all the error logs on the local machine using something light-weight like SQLlite or SQLServer Compact.
Save it to a flat file (xml, or what have you) where you can view it.
Send it straight to the Event Log. (I'd probably do this).
log to the database and xml as a fallback, asp.net account will need perms to log errors to eventviewer which may not be such a good idea on the web server unless it is absolutely neccessasy.
If you're willing to consider a simple commercial product, take a look at Gibraltar. It stores logs locally then uploads them to a web service when a connection is available. The logs are then indexed in an embedded database and an analysis tool lets you review errors and other information at whatever level of detail you require.
I see that you're a student, Frank. Email me directly to discuss the possibility of us offering you a free license.
-System Event Viewer
-you may cache your error to local & lightweight db file (can be SqLite/Sql Compact), then when connection is available, you send it to server