Considerations for useful trace messages in C# .NET - c#

I am trying to work up an error and logging framework, and while I think I have the errors well in hand at this point, I'm a little more sketchy some other things by virtue of having never tackled the problem before. The part I'm having most trouble with right now is trace messages, which are obviously not the same thing as outright errors and shouldn't be handled the same way. Of course, when I try to think about the way they should be handled, I draw a blank.
What are some things that you've found helpful in your trace messages, whether for debugging or filtering things later? Obviously, there are some things like the level (Info, Alert, Warn, Critical), the message, the AppDomain, but what else?
For a little background to the setup, the logging framework I'm working on will live in our application class libraries and put information into a database table, MSMQ line, or flat logging file (application configurable).
Please DO NOT suggest ready-made products like log4net, the Enterprise Library, ELMAH, or anything like that as answers to this question. We have already considered each popular addition and discarded them as not useful to our needs. If you'd like to suggest pieces of data from these products that's fine, but a lot of places I've asked for input on get hung up on recommending things that we have spent weeks vetting and discarding instead of answering the question. I am hoping Stack will show some more class.

For logging purposes I'd consider including the user logged in, timestamp and domain specific information (which you'll have to decide for yourself). For trace messages for debugging, I'd include the class/method and values of variables and parameters that could be troublesome.

I like to use the pattern of using different trace levels for things that "belong" in those levels. For example, in the case of a web service which handles requests and federates other requests out to other servers, i might log the fact that i received a request as 'Informational', whereas i might print out the contents of the full request as 'Verbose'.
I basically categorize different types of events with different levels, sprinkle various levels throughout the codebase, and then consumers of the framework / product can choose 'Verbose' mode to see every last parameter of each request and response object, vs 'Informational' to simply be able to see the flow of requests / responses.
Obvoiusly erroneous events ( non- OK status code received ), based on their severity, should either be a 'Warning' or an 'Error'. I like to think of things that cause the process to abort ( e.g. all external servers are down, i can't possibly fulfil your request ) as 'Error' worth, and things that are worth noting but don't interrupt workflow ( e.g. 1 replica of external server is unavailable or it's round trip response exceeds some latency threshold ) as 'Warning' worthy.
Some additional cool things i've done is put a method in your Logger implementation that automatically knows who the caller of the Logger.WriteLine(...) is and prints it accordingly:
/// <summary>
/// Any additional layers between this log class and calling classes will cause
/// this stack trace to return incorrect 'calling' method name
/// </summary>
/// <returns></returns>
private static string GetCallingMethod()
{
var stack = new StackTrace();
var className = string.Empty;
var methodName = string.Empty;
foreach ( var frame in stack.GetFrames() )
{
className = frame.GetMethod().DeclaringType.FullName;
methodName = frame.GetMethod().Name;
if ( className != typeof( Logger ).FullName )
{
break;
}
}
return string.Format( "{0}.{1}", className, methodName );
}

I think some of this depends on what your applications do. For example if your application is database intensive I would think you would definitely want to log your sql statements. Any dynamic input from the user can also be helpful as debug logging. Datestamps are always helpful when looking at logs. Stacktraces on errors like others have mentioned. There are other instances but these are a couple. I'm curious though about why you ruled out ready made products like log4net etc. If you asked this question before online I'd be curious to see links.

Related

Best practice for null testing [duplicate]

To avoid all standard-answers I could have Googled on, I will provide an example you all can attack at will.
C# and Java (and too many others) have with plenty of types some of ‘overflow’ behaviour I don’t like at all (e.g type.MaxValue + type.SmallestValue == type.MinValue for example : int.MaxValue + 1 == int.MinValue).
But, seen my vicious nature, I’ll add some insult to this injury by expanding this behaviour to, let’s say an Overridden DateTime type. (I know DateTime is sealed in .NET, but for the sake of this example, I’m using a pseudo language that is exactly like C#, except for the fact that DateTime isn’t sealed).
The overridden Add method:
/// <summary>
/// Increments this date with a timespan, but loops when
/// the maximum value for datetime is exceeded.
/// </summary>
/// <param name="ts">The timespan to (try to) add</param>
/// <returns>The Date, incremented with the given timespan.
/// If DateTime.MaxValue is exceeded, the sum wil 'overflow' and
/// continue from DateTime.MinValue.
/// </returns>
public DateTime override Add(TimeSpan ts)
{
try
{
return base.Add(ts);
}
catch (ArgumentOutOfRangeException nb)
{
// calculate how much the MaxValue is exceeded
// regular program flow
TimeSpan saldo = ts - (base.MaxValue - this);
return DateTime.MinValue.Add(saldo)
}
catch(Exception anyOther)
{
// 'real' exception handling.
}
}
Of course an if could solve this just as easy, but the fact remains that I just fail to see why you couldn’t use exceptions (logically that is, I can see that when performance is an issue that in certain cases exceptions should be avoided).
I think in many cases they are more clear than if-structures and don’t break any contract the method is making.
IMHO the “Never use them for regular program flow” reaction everybody seems to have is not that well underbuild as the strength of that reaction can justify.
Or am I mistaken?
I've read other posts, dealing with all kind of special cases, but my point is there's nothing wrong with it if you are both:
Clear
Honour the contract of your method
Shoot me.
Have you ever tried to debug a program raising five exceptions per second in the normal course of operation ?
I have.
The program was quite complex (it was a distributed calculation server), and a slight modification at one side of the program could easily break something in a totally different place.
I wish I could just have launched the program and wait for exceptions to occur, but there were around 200 exceptions during the start-up in the normal course of operations
My point : if you use exceptions for normal situations, how do you locate unusual (ie exceptional) situations ?
Of course, there are other strong reasons not to use exceptions too much, especially performance-wise
Exceptions are basically non-local goto statements with all the consequences of the latter. Using exceptions for flow control violates a principle of least astonishment, make programs hard to read (remember that programs are written for programmers first).
Moreover, this is not what compiler vendors expect. They expect exceptions to be thrown rarely, and they usually let the throw code be quite inefficient. Throwing exceptions is one of the most expensive operations in .NET.
However, some languages (notably Python) use exceptions as flow-control constructs. For example, iterators raise a StopIteration exception if there are no further items. Even standard language constructs (such as for) rely on this.
My rule of thumb is:
If you can do anything to recover from an error, catch exceptions
If the error is a very common one (eg. user tried to log in with the wrong password), use returnvalues
If you can't do anything to recover from an error, leave it uncaught (Or catch it in your main-catcher to do some semi-graceful shutdown of the application)
The problem I see with exceptions is from a purely syntax point of view (I'm pretty sure the perfomance overhead is minimal). I don't like try-blocks all over the place.
Take this example:
try
{
DoSomeMethod(); //Can throw Exception1
DoSomeOtherMethod(); //Can throw Exception1 and Exception2
}
catch(Exception1)
{
//Okay something messed up, but is it SomeMethod or SomeOtherMethod?
}
.. Another example could be when you need to assign something to a handle using a factory, and that factory could throw an exception:
Class1 myInstance;
try
{
myInstance = Class1Factory.Build();
}
catch(SomeException)
{
// Couldn't instantiate class, do something else..
}
myInstance.BestMethodEver(); // Will throw a compile-time error, saying that myInstance is uninitalized, which it potentially is.. :(
Soo, personally, I think you should keep exceptions for rare error-conditions (out of memory etc.) and use returnvalues (valueclasses, structs or enums) to do your error checking instead.
Hope I understood your question correct :)
A first reaction to a lot of answers :
you're writing for the programmers and the principle of least astonishment
Of course! But an if just isnot more clear all the time.
It shouldn't be astonishing eg : divide (1/x) catch (divisionByZero) is more clear than any if to me (at Conrad and others) . The fact this kind of programming isn't expected is purely conventional, and indeed, still relevant. Maybe in my example an if would be clearer.
But DivisionByZero and FileNotFound for that matter are clearer than ifs.
Of course if it's less performant and needed a zillion time per sec, you should of course avoid it, but still i haven't read any good reason to avoid the overal design.
As far as the principle of least astonishment goes : there's a danger of circular reasoning here : suppose a whole community uses a bad design, this design will become expected! Therefore the principle cannot be a grail and should be concidered carefully.
exceptions for normal situations, how do you locate unusual (ie exceptional) situations ?
In many reactions sth. like this shines trough. Just catch them, no? Your method should be clear, well documented, and hounouring it's contract. I don't get that question I must admit.
Debugging on all exceptions : the same, that's just done sometimes because the design not to use exceptions is common. My question was : why is it common in the first place?
Before exceptions, in C, there were setjmp and longjmp that could be used to accomplish a similar unrolling of the stack frame.
Then the same construct was given a name: "Exception". And most of the answers rely on the meaning of this name to argue about its usage, claiming that exceptions are intended to be used in exceptional conditions. That was never the intent in the original longjmp. There were just situations where you needed to break control flow across many stack frames.
Exceptions are slightly more general in that you can use them within the same stack frame too. This raises analogies with goto that I believe are wrong. Gotos are a tightly coupled pair (and so are setjmp and longjmp). Exceptions follow a loosely coupled publish/subscribe that is much cleaner! Therefore using them within the same stack frame is hardly the same thing as using gotos.
The third source of confusion relates to whether they are checked or unchecked exceptions. Of course, unchecked exceptions seem particularly awful to use for control flow and perhaps a lot of other things.
Checked exceptions however are great for control flow, once you get over all the Victorian hangups and live a little.
My favorite usage is a sequence of throw new Success() in a long fragment of code that tries one thing after the other until it finds what it is looking for. Each thing -- each piece of logic -- may have arbritrary nesting so break's are out as also any kind of condition tests. The if-else pattern is brittle. If I edit out an else or mess up the syntax in some other way, then there is a hairy bug.
Using throw new Success() linearizes the code flow. I use locally defined Success classes -- checked of course -- so that if I forget to catch it the code won't compile. And I don't catch another method's Successes.
Sometimes my code checks for one thing after the other and only succeeds if everything is OK. In this case I have a similar linearization using throw new Failure().
Using a separate function messes with the natural level of compartmentalization. So the return solution is not optimal. I prefer to have a page or two of code in one place for cognitive reasons. I don't believe in ultra-finely divided code.
What JVMs or compilers do is less relevant to me unless there is a hotspot. I cannot believe there is any fundamental reason for compilers to not detect locally thrown and caught Exceptions and simply treat them as very efficient gotos at the machine code level.
As far as using them across functions for control flow -- i. e. for common cases rather than exceptional ones -- I cannot see how they would be less efficient than multiple break, condition tests, returns to wade through three stack frames as opposed to just restore the stack pointer.
I personally do not use the pattern across stack frames and I can see how it would require design sophistication to do so elegantly. But used sparingly it should be fine.
Lastly, regarding surprising virgin programmers, it is not a compelling reason. If you gently introduce them to the practice, they will learn to love it. I remember C++ used to surprise and scare the heck out of C programmers.
The standard anwser is that exceptions are not regular and should be used in exceptional cases.
One reason, which is important to me, is that when I read a try-catch control structure in a software I maintain or debug, I try to find out why the original coder used an exception handling instead of an if-else structure. And I expect to find a good answer.
Remember that you write code not only for the computer but also for other coders. There is a semantic associated to an exception handler that you cannot throw away just because the machine doesn't mind.
Josh Bloch deals with this topic extensively in Effective Java. His suggestions are illuminating and should apply to .NET as well (except for the details).
In particular, exceptions should be used for exceptional circumstances. The reasons for this are usability-related, mainly. For a given method to be maximally usable, its input and output conditions should be maximally constrained.
For example, the second method is easier to use than the first:
/**
* Adds two positive numbers.
*
* #param addend1 greater than zero
* #param addend2 greater than zero
* #throws AdditionException if addend1 or addend2 is less than or equal to zero
*/
int addPositiveNumbers(int addend1, int addend2) throws AdditionException{
if( addend1 <= 0 ){
throw new AdditionException("addend1 is <= 0");
}
else if( addend2 <= 0 ){
throw new AdditionException("addend2 is <= 0");
}
return addend1 + addend2;
}
/**
* Adds two positive numbers.
*
* #param addend1 greater than zero
* #param addend2 greater than zero
*/
public int addPositiveNumbers(int addend1, int addend2) {
if( addend1 <= 0 ){
throw new IllegalArgumentException("addend1 is <= 0");
}
else if( addend2 <= 0 ){
throw new IllegalArgumentException("addend2 is <= 0");
}
return addend1 + addend2;
}
In either case, you need to check to make sure that the caller is using your API appropriately. But in the second case, you require it (implicitly). The soft Exceptions will still be thrown if the user didn't read the javadoc, but:
You don't need to document it.
You don't need to test for it (depending upon how aggresive your
unit testing strategy is).
You don't require the caller to handle three use cases.
The ground-level point is that Exceptions should not be used as return codes, largely because you've complicated not only YOUR API, but the caller's API as well.
Doing the right thing comes at a cost, of course. The cost is that everyone needs to understand that they need to read and follow the documentation. Hopefully that is the case anyway.
How about performance? While load testing a .NET web app we topped out at 100 simulated users per web server until we fixed a commonly-occuring exception and that number increased to 500 users.
I think that you can use Exceptions for flow control. There is, however, a flipside of this technique. Creating Exceptions is a costly thing, because they have to create a stack trace. So if you want to use Exceptions more often than for just signalling an exceptional situation you have to make sure that building the stack traces doesn't negatively influence your performance.
The best way to cut down the cost of creating exceptions is to override the fillInStackTrace() method like this:
public Throwable fillInStackTrace() { return this; }
Such an exception will have no stacktraces filled in.
Here are best practices I described in my blog post:
Throw an exception to state an unexpected situation in your software.
Use return values for input validation.
If you know how to deal with exceptions a library throws, catch them at the lowest level possible.
If you have an unexpected exception, discard current operation completely. Don’t pretend you know how to deal with them.
I don't really see how you're controlling program flow in the code you cited. You'll never see another exception besides the ArgumentOutOfRange exception. (So your second catch clause will never be hit). All you're doing is using an extremely costly throw to mimic an if statement.
Also you aren't performing the more sinister of operations where you just throw an exception purely for it to be caught somewhere else to perform flow control. You're actually handling an exceptional case.
Apart from the reasons stated, one reason not to use exceptions for flow control is that it can greatly complicate the debugging process.
For example, when I'm trying to track down a bug in VS I'll typically turn on "break on all exceptions". If you're using exceptions for flow control then I'm going to be breaking in the debugger on a regular basis and will have to keep ignoring these non-exceptional exceptions until I get to the real problem. This is likely to drive someone mad!!
Lets assume you have a method that does some calculations. There are many input parameters it has to validate, then to return a number greater then 0.
Using return values to signal validation error, it's simple: if method returned a number lesser then 0, an error occured. How to tell then which parameter didn't validate?
I remember from my C days a lot of functions returned error codes like this:
-1 - x lesser then MinX
-2 - x greater then MaxX
-3 - y lesser then MinY
etc.
Is it really less readable then throwing and catching an exception?
Because the code is hard to read, you may have troubles debugging it, you will introduce new bugs when fixing bugs after a long time, it is more expensive in terms of resources and time, and it annoys you if you are debugging your code and the debugger halts on the occurence of every exception ;)
If you are using exception handlers for control flow, you are being too general and lazy. As someone else mentioned, you know something happened if you are handling processing in the handler, but what exactly? Essentially you are using the exception for an else statement, if you are using it for control flow.
If you don't know what possible state could occur, then you can use an exception handler for unexpected states, for example when you have to use a third-party library, or you have to catch everything in the UI to show a nice error message and log the exception.
However, if you do know what might go wrong, and you don't put an if statement or something to check for it, then you are just being lazy. Allowing the exception handler to be the catch-all for stuff you know could happen is lazy, and it will come back to haunt you later, because you will be trying to fix a situation in your exception handler based on a possibly false assumption.
If you put logic in your exception handler to determine what exactly happened, then you would be quite stupid for not putting that logic inside the try block.
Exception handlers are the last resort, for when you run out of ideas/ways to stop something from going wrong, or things are beyond your ability to control. Like, the server is down and times out and you can't prevent that exception from being thrown.
Finally, having all the checks done up front shows what you know or expect will occur and makes it explicit. Code should be clear in intent. What would you rather read?
You can use a hammer's claw to turn a screw, just like you can use exceptions for control flow. That doesn't mean it is the intended usage of the feature. The if statement expresses conditions, whose intended usage is controlling flow.
If you are using a feature in an unintended way while choosing to not use the feature designed for that purpose, there will be an associated cost. In this case, clarity and performance suffer for no real added value. What does using exceptions buy you over the widely-accepted if statement?
Said another way: just because you can doesn't mean you should.
As others have mentioned numerously, the principle of least astonishment will forbid that you use exceptions excessively for control flow only purposes. On the other hand, no rule is 100% correct, and there are always those cases where an exception is "just the right tool" - much like goto itself, by the way, which ships in the form of break and continue in languages like Java, which are often the perfect way to jump out of heavily nested loops, which aren't always avoidable.
The following blog post explains a rather complex but also rather interesting use-case for a non-local ControlFlowException:
http://blog.jooq.org/2013/04/28/rare-uses-of-a-controlflowexception
It explains how inside of jOOQ (a SQL abstraction library for Java), such exceptions are occasionally used to abort the SQL rendering process early when some "rare" condition is met.
Examples of such conditions are:
Too many bind values are encountered. Some databases do not support arbitrary numbers of bind values in their SQL statements (SQLite: 999, Ingres 10.1.0: 1024, Sybase ASE 15.5: 2000, SQL Server 2008: 2100). In those cases, jOOQ aborts the SQL rendering phase and re-renders the SQL statement with inlined bind values. Example:
// Pseudo-code attaching a "handler" that will
// abort query rendering once the maximum number
// of bind values was exceeded:
context.attachBindValueCounter();
String sql;
try {
// In most cases, this will succeed:
sql = query.render();
}
catch (ReRenderWithInlinedVariables e) {
sql = query.renderWithInlinedBindValues();
}
If we explicitly extracted the bind values from the query AST to count them every time, we'd waste valuable CPU cycles for those 99.9% of the queries that don't suffer from this problem.
Some logic is available only indirectly via an API that we want to execute only "partially". The UpdatableRecord.store() method generates an INSERT or UPDATE statement, depending on the Record's internal flags. From the "outside", we don't know what kind of logic is contained in store() (e.g. optimistic locking, event listener handling, etc.) so we don't want to repeat that logic when we store several records in a batch statement, where we'd like to have store() only generate the SQL statement, not actually execute it. Example:
// Pseudo-code attaching a "handler" that will
// prevent query execution and throw exceptions
// instead:
context.attachQueryCollector();
// Collect the SQL for every store operation
for (int i = 0; i < records.length; i++) {
try {
records[i].store();
}
// The attached handler will result in this
// exception being thrown rather than actually
// storing records to the database
catch (QueryCollectorException e) {
// The exception is thrown after the rendered
// SQL statement is available
queries.add(e.query());
}
}
If we had externalised the store() logic into "re-usable" API that can be customised to optionally not execute the SQL, we'd be looking into creating a rather hard to maintain, hardly re-usable API.
Conclusion
In essence, our usage of these non-local gotos is just along the lines of what [Mason Wheeler][5] said in his answer:
"I just encountered a situation that I cannot deal with properly at this point, because I don't have enough context to handle it, but the routine that called me (or something further up the call stack) ought to know how to handle it."
Both usages of ControlFlowExceptions were rather easy to implement compared to their alternatives, allowing us to reuse a wide range of logic without refactoring it out of the relevant internals.
But the feeling of this being a bit of a surprise to future maintainers remains. The code feels rather delicate and while it was the right choice in this case, we'd always prefer not to use exceptions for local control flow, where it is easy to avoid using ordinary branching through if - else.
Typically there is nothing wrong, per se, with handling an exception at a low level. An exception IS a valid message that provides a lot of detail for why an operation cannot be performed. And if you can handle it, you ought to.
In general if you know there is a high probability of failure that you can check for... you should do the check... i.e. if(obj != null) obj.method()
In your case, i'm not familiar enough with the C# library to know if date time has an easy way to check whether a timestamp is out of bounds. If it does, just call if(.isvalid(ts))
otherwise your code is basically fine.
So, basically it comes down to whichever way creates cleaner code... if the operation to guard against an expected exception is more complex than just handling the exception; than you have my permission to handle the exception instead of creating complex guards everywhere.
You might be interested in having a look at Common Lisp's condition system which is a sort of generalization of exceptions done right. Because you can unwind the stack or not in a controlled way, you get "restarts" as well, which are extremely handy.
This doesn't have anything much to do with best practices in other languages, but it shows you what can be done with some design thought in (roughly) the direction you are thinking of.
Of course there are still performance considerations if you're bouncing up and down the stack like a yo-yo, but it's a much more general idea than "oh crap, lets bail" kind of approach that most catch/throw exception systems embody.
I don't think there is anything wrong with using Exceptions for flow-control. Exceptions are somewhat similar to continuations and in statically typed languages, Exceptions are more powerful than continuations, so, if you need continuations but your language doesn't have them, you can use Exceptions to implement them.
Well, actually, if you need continuations and your language doesn't have them, you chose the wrong language and you should rather be using a different one. But sometimes you don't have a choice: client-side web programming is the prime example – there's just no way to get around JavaScript.
An example: Microsoft Volta is a project to allow writing web applications in straight-forward .NET, and let the framework take care of figuring out which bits need to run where. One consequence of this is that Volta needs to be able to compile CIL to JavaScript, so that you can run code on the client. However, there is a problem: .NET has multithreading, JavaScript doesn't. So, Volta implements continuations in JavaScript using JavaScript Exceptions, then implements .NET Threads using those continuations. That way, Volta applications that use threads can be compiled to run in an unmodified browser – no Silverlight needed.
But you won't always know what happens in the Method/s that you call. You won't know exactly where the exception was thrown. Without examining the exception object in greater detail....
I feel that there is nothing wrong with your example. On the contrary, it would be a sin to ignore the exception thrown by the called function.
In the JVM, throwing an exception is not that expensive, only creating the exception with new xyzException(...), because the latter involves a stack walk. So if you have some exceptions created in advance, you may throw them many times without costs. Of course, this way you can't pass data along with the exception, but I think that is a bad thing to do anyway.
There are a few general mechanisms via which a language could allow for a method to exit without returning a value and unwind to the next "catch" block:
Have the method examine the stack frame to determine the call site, and use the metadata for the call site to find either information about a try block within the calling method, or the location where the calling method stored the address of its caller; in the latter situation, examine metadata for the caller's caller to determine in the same fashion as the immediate caller, repeating until one finds a try block or the stack is empty. This approach adds very little overhead to the no-exception case (it does preclude some optimizations) but is expensive when an exception occurs.
Have the method return a "hidden" flag which distinguishes a normal return from an exception, and have the caller check that flag and branch to an "exception" routine if it's set. This routine adds 1-2 instructions to the no-exception case, but relatively little overhead when an exception occurs.
Have the caller place exception-handling information or code at a fixed address relative to the stacked return address. For example, with the ARM, instead of using the instruction "BL subroutine", one could use the sequence:
adr lr,next_instr
b subroutine
b handle_exception
next_instr:
To exit normally, the subroutine would simply do bx lr or pop {pc}; in case of an abnormal exit, the subroutine would either subtract 4 from LR before performing the return or use sub lr,#4,pc (depending upon the ARM variation, execution mode, etc.) This approach will malfunction very badly if the caller is not designed to accommodate it.
A language or framework which uses checked exceptions might benefit from having those handled with a mechanism like #2 or #3 above, while unchecked exceptions are handled using #1. Although the implementation of checked exceptions in Java is rather nuisancesome, they would not be a bad concept if there were a means by which a call site could say, essentially, "This method is declared as throwing XX, but I don't expect it ever to do so; if it does, rethrow as an "unchecked" exception. In a framework where checked exceptions were handled in such fashion, they could be an effective means of flow control for things like parsing methods which in some contexts may have a high likelihood of failure, but where failure should return fundamentally different information than success. I'm unaware of any frameworks that use such a pattern, however. Instead, the more common pattern is to use the first approach above (minimal cost for the no-exception case, but high cost when exceptions are thrown) for all exceptions.
One aesthetic reason:
A try always comes with a catch, whereas an if doesn't have to come with an else.
if (PerformCheckSucceeded())
DoSomething();
With try/catch, it becomes much more verbose.
try
{
PerformCheckSucceeded();
DoSomething();
}
catch
{
}
That's 6 lines of code too many.

Querying the write model for duplicated aggregate root property

I'm implementing CQRS pattern with Event sourcing, I'm using NServiceBus, NEventStore and NES(Connects between NSB and NEventStore).
My application will check a web service regularly for any file to be downloaded and processed. when the a file is found, a command (DownloadFile) is sent to the bus, and received by FileCommandHandler which creates a new aggregate root (File) and handle the message.
Now inside the (File aggregate root) I have to check that the content of the file doesn't match with any other file content (Since the web service guarantee that only file name is unique, and the content may be duplicated with different name), by hashing it and comparing with the list of hashed contents.
The question is where I have to save the list of hash codes? is it allowed to query the read model?
public class File : AggregateBase
{
public File(DownloadFile cmd, IFileService fileDownloadService, IClaimSerializerService serializerService, IBus bus)
: this()
{
// code to download the file content, deserialize it, and publish an event.
}
}
public class FileCommandHandler : IHandleMessages<DownloadFile>, IHandleMessages<ExtractFile>
{
public void Handle(DownloadFile command)
{
//for example, is it possible to do this (honestly, I feel it is not, since read model should always considered stale !)
var file = readModelContext.GetFileByHashCode (Hash(command.FileContent));
if (file != null)
throw new Exception ("File content matched with another already downloaded file");
// Since there is no way to query the event source for file content like:
// eventSourceRepository.Find<File>(c=>c.HashCode == Hash(command.FileContent));
}
}
Seems like you're looking for deduplication.
Your command side is where you want things to be consistent. Queries will always leave you open to race conditions. So, instead of running a query, I'd reverse the logic and actually write the hash into a database table (any db with ACID guarantees). If this write is successful, process the file. If the write of the hash fails, skip processing.
There's no point putting this logic into a handler, because retrying the message in case of failure (ie storing the hash multiple times) will not make it succeed. You'd also end up with messages for duplicate files in the error q.
A good place for the deduplication logic is likely inside your web service client. Some pseudo logic
Get file
Open transaction
Insert hash into database & catch failure (not any failure, only failure to insert)
Bus.Send message to process file if # of records inserted in step 3 is not zero
commit transaction
Some example deduplication code in NServiceBus gateway here
Edit:
Looking at their code, I actually think the session.Get<DeduplicationMessage> is unnecessary. session.Save(gatewayMessage); should be enough and is the consistency boundary.
Doing a query would make sense only if the rate of failure is high, meaning you have a lot of duplicate content files. If 99%+ of inserts succeed, the duplicates can indeed be treated as exceptions.
This depends on a lot of things ... throughput being one of them. But since you're approaching this problem in a "pull based" fashion anyway (you're querying a webservice to poll for work (downloading and analysing a file)), you could make this whole process serial without having to worry about collisions. Now that might not give the desired rate at which you want to be handling "the work", but more importantly ... have you measured? Let's sidestep that for a minute and assume that serial isn't going to work. How many files are we talking about? A few 100, 1000, ... millions? Depending on that hashes might fit into memory and could be rebuilt if/when the process should come down. There might also be an opportunity to partition your problem along the axis of time or context. Every file since the beginning of dawn or just today, or maybe this month's worth of files? Really, I think you should dig deeper in your problem space. Apart from that, this feels like an awkward problem to solve using event sourcing, but YMMV.
When you have a true uniqueness-constraint in your domain, you can make the uniqueness-tester a domain service, whose implementation is part of the infrastructure -- similar to a repository, whose interface is part of the domain and whose implementation is part of the infrastructure. For the implementation, you can then use an in-memory hash or a database that is updated/queried as needed.

What should I identify with the id argument in TraceSource.TraceEvent method?

I use the TraceSource class for logging in my .NET projects.
However a point that has never been clear to me is, what the intent of the id parameter in the TraceEvent method. Currently, I always set it to 0.
But what is the expected or typical useful usage of it?
I can think of a few possibilities:
It is an ID for the occurrence of the event (i.e. the same line of code produces a different ID on each execution);
It is an ID for the method call (i.e. you can infer the line of code from the ID);
It is an ID for a family of similar events (e.g. all error messages that say that the database is absent share the same ID);
It is an ID for a set of events that are related to a logical operation, in combination with the TraceEventType.(Start|Stop|Suspend|Resume|Transfer) enumeration values;
I've asked myself the same question and I didn't found anything to clarify this in any Microsoft documentation.
What I've manage to find is an article written by a Microsoft MVP, Richard Grimes:
"The id parameter is whatever you choose it to be, there is no compulsion that a particular ID is associated with a particular format message."
He uses 0, for the id argument, in all examples.
In MSDN articles, I've seen it used random, not providing any additional info.
I believe that you can use in any way that helps you best when reading the logs, as long as you maintain the same code convention. It may prove useful afterwards in trace filtering, if you want to use the SourceFilter.ShouldTrace method, that accept an id argument too.
I use it to describe the error type, if I have an error, or use 0 for anything else.
As far as I've seen in the documentation, it's not specifically intended for one purpose. I think it's there for you to tie in with your own logic for tracing events. The ShouldTrace() method on SourceFilter takes a matching id parameter, so you can also use it to determine which events or event types go where.
Personally, when I use TraceSource (which admittedly isn't much, having only discovered it recently) I use it to track event types or categories. In one application I already had an enum for event types that I was using with another logging method, with values Debug, Info, Warn, Error, Fatal, so I cast that to int and used that as the id, which helped with filtering later so I could filter out anything below the level I was interested in to de-clutter the trace.
Another possibility is that you could use different values to relate to different parts of the application, so Data Access = 1, User Accounts = 2, Product Logic = 3, Notifications = 4, UI = 5 etc. Again, you could then use this to filter the trace down to only the type of thing you're looking at.
Alternatively, you could (as you suggested) use different id values to mean different event types, so you could use them like error codes so that (for example) any time you saw an id of 26 you'd know that the database connection could not be established, or whatever.
It doesn't particularly matter what you use the id parameter for, as long as:
It is useful to you in building and debugging the program
It is clear and understandable to programmers reading through your code
It is used consistently throughout the program
One possibility is that you could have a centralised class that manages the event ids and provides the values based on some sort of input to make sure that the whole application uses the same id for the same thing.

List-based publish subscribe pattern in c# (Wordpress hooks / filters)

I come from a PHP background and have used Wordpress quite a lot, I love how their plugin architecture works and the ability to hook events to event names. One of the best parts I like about it is being able to *add_filter()* to any database value just before it gets shown to the end user. My question is multi-part on how to replicate the whole plugin architecture in a C#.NET environment?
Part 1:
To create plug-ins I have researched the MEF framework would probably be the best (Managed Extensibility Framework -http://mef.codeplex.com/). This is designed specifically to take the grunt work out by giving you the ability to monitor directories for new plug-ins, tracking dependencies and other normal things. MEF ships with .NET 3.5+
Part 2
Hooking events? I can't seem to find much information about replicating a global channel based event system. From what I have upto yet I need a publish/subscribe pattern (which isn't that hard to make as you just create some concrete objects and give them events). The hard part is giving each event a 'channel' name and for all the events in the whole system to be part of a global collection (Mediator pattern).
To replicate: (http://codex.wordpress.org/Function_Reference/add_filter)
Example 1
// Add's my button to the end of the content
add_filter('the_content', 'my_plugin_button');
function my_plugin_button( $content ) {
// Add's my button to the end of the content
return $content . "<a href='#'>My button</a>";
}
OR
Example 2
// Add a new admin menu item by hooking in
add_action('admin_menu', 'my_plugin_menu');
function my_plugin_menu() {
add_options_page('My Plugin Options', 'My Plugin', 'manage_options', 'my-unique-identifier', 'my_plugin_options');
}
I hope your all with me upto yet? I have managed to replicate the functionality I need in Javascript and even jQuery has their .on() event function... same thing but channel or list based...
My 2 examples:
http://jsfiddle.net/AaronLayton/U3ucS/53/
http://jsfiddle.net/AaronLayton/eyNre/33/
Can anyone point me in the correct direction or is this the totaly wrong approach for c#?
I think NServiceBus can help you a lot with these issues. Udi Dahan which is the author of NServiceBus has also written a lot of articles about Domain Event pattern, which is a publish/subscribe mechanism.
Know it's been a long time since you posted this and you probably built something already. However I have been thinking about something like this myself. There are 2 options - really forget WordPress and try and build something much cleaner - it's a mess at the bottom of WordPress' code :D
Or this:
function the_content()
{
var result = get_the_content();
// do other stuff...if you want to.
execute_filters(ref result);
execute_actions(ref result);
return result;
}
function execute_filters(ref string result, string action_name)
{
var filters = get_pre_filters(action_name);
filters.ForEach(filter =>
{
/// somehow call the method name in filter. PHP is generally global. C# namespaced,
/// so you would need to think about that.
}
}
function execute_actions(ref string result, string action_name)
{
/// and so on....
}
When building something to mimic WordPress, you need to remember many of the issues of WordPress' plugin architecture (in my personal opinion)... It seems to want to run every plugin near enough on every page even if that page has nothing to do with that plugin. I onced installed a plugin that added 60 database queries to each page call, and it wasn't used.
Try and think smart about it when you are building it. Try and add a way to only have the plugins that are going to get used on the page/post of your new setup to be run e.g. in your database have a "Plugins" field on the post/page object with a list of plugins allowed to run on that page. That way you won't need to check all the plugins each time to see if it wants to run.
Anyways. Hope you got something working.

WCF Exception handling with batch operations

I need an hint on this architectural problem.
My application calls different wcf services that perform specific tasks returning a list of model objects.
Say, I have a service which polls stock exchange titles returning a List<SeTitle>. [Purely fictional example, I'm reading a book on this subject]
If an exception happens in one or few elements of this list I have two options in the wcf service:
catch the exception, log it and return the List<>, but in this way I'll lose evidence of all the faulty items in my application.
throw an exception on first error, but in this way I won't receive any info even if only one item was "faulty".
My will is to go for a third option, meaning that I'll always return my List but with the evidence of faulty items.
My idea is to let all my model object extend a simple interface like (pardon names, they are horrible)
public interface IWcfReturnObject
{
public List<WarningModel> Warnings;
}
and
public class SeTitle : IWcfReturnObject
in a such a way each element of the list could have an associated List of Warnings/Error to be returned to the application and then to the interface.
My questions are:
Is my question clear to understand?! ;)
Do you think my desire (receive all element from wcf with warnings attached) is architecturally good?
Do you think it exists a better approach to solve this problem?
Thanks
M.
EDIT: I was doing research and looking at the first answer I think it is useful to give one solution I found on the NET
Possible Solutions
My way
Log with some kind of correlation and refer it on the business layer as decyclone suggested
AggregateException
I think you should log your exceptions/errors/warnings instead of sending them along with result data. You can always refer to log and maybe create an interface to see error log. But, common result data should not be mixed with exception data.

Categories

Resources