Develop Reference

Selenium - disable test and do not show in test explorer [duplicate]

Is it possible to skip all tests from a specific class like in NUnit
[TestFixture]
[Ignore("Reason")]
public class TestClass {
}

No - there is no such facility at present, and the last time it was requested it was considered too low value to add,
One quick way of achieving the effect in xUnit is to comment out the public - private classes are not reflected over (obviously it won't appear on the skip list that way though).
UPDATE: Another way is to put a TraitAttribute on the class and then (assuming you're using the xunit.console runner) filter it out by running with /-trait traitName. (e.g. you can achieve ExplicitAttribute, some aspects of the BDD frameworky technique of Pending tests and similar semantics that way - of course the big problem is they don't show up in any reports when using any of these filtering techniques)
UPDATE 2: You can do
const string skip = "Class X disabled";
[Fact(Skip=skip)]
void Test() {}
Then you can change to to const string skip = null to undo the skip. The (dis)advantage of this is that the test is still shown as a Skipped test in the test list, generally with a reason included in the test run report (vs making it private which makes it likely to be forgotten)

Here is my hack to avoid error xUnit1000: Test classes must be public (checked on single Fact, I think Theories can be hacked this way, too).
// Uncomment to enable tests
//public class FactSwitch : FactAttribute { } // public! ahh, a bug!
// Uncomment to disable tests
internal class FactSwitch : Attribute { }
public class MyTests
{
[FactSwitch]
public void MyTest1()
{
"it".ShouldBe("it");
}
}
(3 years later)
While searching for the same solution I found there are better ways to do the same.
Let's rewrite the example above in a way Ruben Bartelink suggested (continuation of his idea).
public class MyTests
{
//const string SkipOrNot = null; // Run all tests
const string SkipOrNot = "reason"; // Skip all tests
[Fact(Skip = SkipOrNot)]
public void MyTest1()
{
"it".ShouldBe("it");
}
}
Nathan Cooper suggested a good improvement for my idea:
public class MyTests
{
// Uncomment to disable tests
//private class FactAttribute : Attribute { }
[Fact]
public void MyTest1()
{
"it".ShouldBe("it");
}
}
So I like both ideas from Ruben and Nathan. There is a subtle difference between using Skip="something" (Ruben) and not using Skip at all. Using "Skip" will put all your tests in a "Skipped tests" warning zone, while "FactAttribute : Attribute" will hide them.

I've found yet another way of temporary disabling entire class without compiler warning.
Disabled:
[System.Diagnostics.CodeAnalysis.SuppressMessage("Usage", "xUnit1000:Test classes must be public", Justification = "Disabled")]//*/
/*
public /**/class DatabaseTests
{
}
to enable move the /* one line up (i.e. using alt+up):
/*
[System.Diagnostics.CodeAnalysis.SuppressMessage("Usage", "xUnit1000:Test classes must be public", Justification = "Disabled")]//*/
public /**/class DatabaseTests
{
}
Note that using full namespace path for SupressMessage does not mess up with your usings.

You need to set the your class access level as as internal and surpress message as #Miq did:
[System.Diagnostics.CodeAnalysis.SuppressMessage("Usage", "xUnit1000:Test classes must be public", Justification = "Disabled")]
internal class MyClassThatIsNotATestClass
{ ... }

You can create LocalOnlyFactAttribute
public class LocalOnlyFactAttribute : FactAttribute
{
//uncomment to run on local
//const string skip = null;
//keep this to avoid slow running tests on other env
const string skip = "Disabled slow running tests.";
public override string Skip { get => skip; set => base.Skip = value; }
}

As far as I know, the simplest way to dynamically skip a whole xUnit test class at runtime is to use the TestFrameworkAttribute at the assembly level, to point to a class that implements the ITestFramework interface (or inherits from XunitTestFramework, which is simpler) and which overrides the CreateDiscoverer() method to return another class, that implements the ITestFrameworkDiscoverer interface (or inherits from XunitTestFrameworkDiscoverer, which is simpler), where you can finally override the IsValidTestClass() method, to decide whether a class should be skipped or not.
Here is some sample code:
[assembly: TestFramework("MyNamespace.Xunit.MyTestFramework", "MyAssembly")]
namespace MyNamespace.Xunit
{
public class MyTestFramework : XunitTestFramework
{
public MyTestFramework(IMessageSink messageSink)
: base(messageSink)
{
}
protected override ITestFrameworkDiscoverer CreateDiscoverer(
IAssemblyInfo assemblyInfo)
=> new MyTestFrameworkDiscoverer(
assemblyInfo,
SourceInformationProvider,
DiagnosticMessageSink);
}
public class MyTestFrameworkDiscoverer : XunitTestFrameworkDiscoverer
{
public MyTestFrameworkDiscoverer(
IAssemblyInfo assemblyInfo,
ISourceInformationProvider sourceProvider,
IMessageSink diagnosticMessageSink,
IXunitTestCollectionFactory collectionFactory = null)
: base(
assemblyInfo,
sourceProvider,
diagnosticMessageSink,
collectionFactory)
{
}
protected override bool IsValidTestClass(ITypeInfo type)
=> base.IsValidTestClass(type) &&
FilterType(type);
protected virtual bool FilterType(ITypeInfo type)
{
// Insert your custom filter conditions here.
return true;
}
}
}
Tested with xUnit 2.4.1.
We are using it in Pomelo.EntityFrameworkCore.MySql (see AssemblyInfo.cs and MySqlXunitTestFrameworkDiscoverer.cs) (a bit more complex than the sample code here).

You could achieve this through a custom ITestClassCommand.
See http://mariangemarcano.blogspot.be/2010/12/xunitnet-running-tests-testcategory.html

Here's another hack that requires minimal changes to code
using FactAttribute = System.Runtime.CompilerServices.CompilerGeneratedAttribute;
using TheoryAttribute = System.Runtime.CompilerServices.CompilerGeneratedAttribute;
Any compatible attribute can be used for the replacement.
If you also use the InlineDataAttribute then you'll need to define a replacement as I don't think there's an existing compatible attribute.
using InlineDataAttribute = DummyDataAttribute;
[AttributeUsage(AttributeTargets.Method, AllowMultiple = true)]
internal class DummyDataAttribute : Attribute
{
public DummyDataAttribute(params object[] data)
{
}
}

Adding a reason almost after one year after the initial question. I have a set of tests which are calling real server apis, and I would like to run then on demand. With nUnit, it has Ignore attribute : with that set, test runner will skip those tests, but I can still manually run it.
xUnit has no such feature. The nearest one is setting such a class level attribute, and comment it out when I want to run it.

Consider creating LocalOnlyFactAttribute, which can be reused across multiple test files.
public class LocalOnlyFactAttribute : FactAttribute
{
//uncomment to run on local
//const string skip = null;
//keep this to avoid slow running tests on other env
const string skip = "Disabled slow running tests.";
public override string Skip { get => skip; set => this.Skip = value; }
}

System.Diagnostics.Trace remove default Trace [duplicate]

I'm using a 3rd party library which makes several calls to the function:
Trace.WriteLine(string message);
This clutters up the visual studio output window and makes debugging my application difficult (for instance; XAML binding warnings).
I'm trying to find a way to stop all trace messages from a specific dll from dumping to the visual studio output window - is writing my own TraceListener the only path forward?
I can't make a TraceFilter / EventTypeFilter work for a string message without category -- although I can't find the documentation to back this up -- empirically:
TraceFilter.ShouldTrace(...)
is called by the following functions (not a complete set):
Trace.WriteLine(string message, string category);
Trace.TraceError(string message);
Trace.WriteLine(object o);
but isn't called by:
Trace.WriteLine(string message);
Does anyone know why this call avoids the ShouldTrace filter?

If you don't want to create your own TraceListener, the only way to suppress Trace messages from a problematic dll is to stop all Trace messages by using Trace.Listeners.Clear().
Note that this will stop your own Trace calls as well. I am mentioning this because I know of a few applications that never used Trace.WriteLine and were getting a severe performance hit from a very noisy library constantly writing to the output window.
I would recommend creating a TraceListener that uses reflection to look for the dll you want to ignore in the call stack.
It's not possible to override Trace.WriteLine, but it is possible to override some calls in the default TraceListener to achieve the same effect.
Using a TraceListener like the one below can help you clean up your output window in Visual Studio so you can focus on the events you are interested in, rather than getting bombarded by messages from a third party library.
See sample code below:
using System;
using System.Diagnostics;
using System.Reflection;
// The library that calls Trace, causing the messages you want to suppress.
using NoisyLibrary;
namespace TraceSuppress
{
/// <summary>
/// Trace listener that ignores trace messages from a specific assembly.
/// </summary>
public class AssemblyFilteredListener : DefaultTraceListener
{
private Assembly assemblyToIgnore;
public AssemblyFilteredListener(Assembly assemblyToIgnoreTracesFrom)
{
this.assemblyToIgnore = assemblyToIgnoreTracesFrom;
}
public bool TraceIsFromAssemblyToIgnore()
{
StackTrace traceCallStack = new StackTrace();
StackFrame[] traceStackFrames = traceCallStack.GetFrames();
// Look for the assembly to ignore in the call stack.
//
// This may be rather slow for very large call stacks. If you find that this is a bottleneck
// there are optimizations available.
foreach (StackFrame traceStackFrame in traceStackFrames)
{
MethodBase callStackMethod = traceStackFrame.GetMethod();
bool methodIsFromAssemblyToIgnore = (callStackMethod.Module.Assembly == this.assemblyToIgnore);
if (methodIsFromAssemblyToIgnore)
{
return true;
}
}
// The assembly to ignore was not found in the call stack.
return false;
}
public override void WriteLine(string message)
{
if (!this.TraceIsFromAssemblyToIgnore())
{
base.WriteLine(message);
}
}
public override void Write(string message)
{
if (!this.TraceIsFromAssemblyToIgnore())
{
base.Write(message);
}
}
}
class Program
{
static void SetupListeners()
{
// Clear out the default trace listener
Trace.Listeners.Clear();
// Grab the asssembly of the library, using any class from the library.
Assembly assemblyToIgnore = typeof(NoisyLibrary.LibraryClass).Assembly;
// Create a TraceListener that will ignore trace messages from that library
TraceListener thisApplicationOnlyListener = new AssemblyFilteredListener(assemblyToIgnore);
Trace.Listeners.Add(thisApplicationOnlyListener);
// Now the custom trace listener is the only listener in Trace.Listeners.
}
static void Main(string[] args)
{
SetupListeners();
// Testing
//-------------------------------------------------------------------------
// This still shows up in the output window in VS...
Trace.WriteLine("This is a trace from the application, we want to see this.");
// ...but the library function that calls trace no longer shows up.
LibraryClass.MethodThatCallsTrace();
// Now check the output window, the trace calls from that library will not be present.
}
}
}

According to ILSpy, the Trace.WriteLine(string message) is declared as abstract and needs to be overridden by derived classes:
public abstract void WriteLine(string message);
All other methods you mention check the ShouldTrace and ultimately call the Trace.WriteLine(string message) message.
E.g.:
public virtual void WriteLine(string message, string category)
{
if (Filter != null &&
!Filter.ShouldTrace(null, "", TraceEventType.Verbose, 0, message))
{
return;
}
if (category == null)
{
WriteLine(message);
return;
}
WriteLine(category + ": " + ((message == null) ? string.Empty : message));
}
So the real reason is in my opinion, a decision of the designer of the Trace class.
He could have made that Trace.WriteLine(string message) protected to incidate that it is not intended to be called directly, e.g.:
protected abstract void WriteLine(string message);

Where to define error messages

I'm making my first "real" C# program and I'm thinking about where I should define error messages? Should I do something like this:
static class Error
{
public static string Example { get { return "Example Error"; } }
}
I could also use values here instead of properties but that would still mean I can't do something like this:
public static string FailedToParse(string filepath, string exmessage)
{
return ("Failed to parse " + filepath + ".\n" + exmessage);
}
So, is that a good idea? Should I make a new class and write a method for each error? How do you guys implement this and why?
I already read
In C#, what's the best way to store a group of constants that my program uses?
The right way to use Globals Constants

I think this is something everything should figure out by themselves.
One like to display nice messages to users another just throw those default generated ones.
Personally I like to have codes for errors.
Something like this:
I create a static class called ExceptionFactory and just pass the code to the method called RaiseException.
public static class ExceptionRegions
{
public static int Internet = 0xA;
public static int FileSystem = 0xB;
}
public class InternetConnectionException : Exception
{
public InternetConnectionException () : base("No internet connection available") { }
}
public class FileSystemAccessException : Exception
{
public FileSystemAccessException () : base("Access to specified path caused an error") { }
}
public static class ExceptionFactory
{
public static void RaiseException(int code)
{
switch(code)
{
case ExceptionRegions.Internet : throw new InternetConnectionException();
...
...
}
}
}
Btw, this is a well known pattern called Factory Pattern. :)
Why I like this, because it allows me to set regions in my application.
Usually an application has many interfaces such as file system, or web services, or database and all I need to do is create a code for each area and the factory will throw a nice message to user without exposing to the user name of database and number of code line or whatever the default generated error message looks alike.

Pattern for raising errors/warnings during parsing in a library

I've got a library which has two input formats for the object model described by the library. I currently use an event subscription model for raising errors/warnings/verbose messages to the end user of the library. This hasn't proven to be the cleanest model, and I was wondering if there was a relevant design pattern or something similar found in the .Net Framework (when in Rome) to better handle this situation.
// Rough outline of the current code
public abstract class ModelReader : IDisposable
{
public abstract Model Read();
public event EventHandler<MessageAvailableEventArgs> MessageAvailable;
protected virtual void RaiseError(string message)
{
var handler = this.MessageAvailable;
if (handler != null)
{
handler(this, new MessageAvailaibleEventArgs(
TraceEventType.Error, message);
}
}
}
Edit: some clarification. The Read routine will already fail fast on all Fatal errors using an exception. The messages are being logged to potentially multiple sources on the user's end, thus any pattern should avoid limiting the number of potential sources.

I can give you a real world example. The Html Agility Pack library is a parsing library. It simply defines a list of parsing errors on the the reader class. Extending your example, it would be something like:
public abstract class ModelReader
{
private List<ParseError> _errors = new List<ParseError>();
private bool _throwOnError;
public ModelReader()
:this(true)
{
}
public ModelReader(bool throwOnError)
{
_throwOnError = throwOnError;
}
// use AddError in implementation when an error is detected
public abstract Model Read();
public virtual IEnumerable<ParseError> Errors
{
get {return _errors;}
}
protected virtual void AddError(ParseError error)
{
if (_throwOnError) // fail fast?
throw new ParseException(error);
_errors.Add(error);
}
}
public class ParseError
{
public ParseError(...)
{
}
public ParseErrorCode Code { get; private set; }
public int Line { get; private set; }
public int LinePosition { get; private set; }
public string Reason { get; private set; }
public string SourceText { get; private set; }
public int StreamPosition { get; private set; }
}
public enum ParseErrorCode
{
InvalidSyntax,
ClosingQuoteNotFound,
... whatever...
}
public class ParseException: Exception
{
...
}
And you still can add an event if the library caller wants on-the-fly events.

You seem to want a composable validator where your users can plug in their own logic to flag specific non fatal errors as either fatal, warning or informational messages. Throwing an exception does not fit the bill since you have left the method already if another part wanted to transform it into a warning but wanted to continue parsing. This sounds much like the two pass exception handling model in Windows for Structured Exception Handling.
It basically goes like
Register as many exception handlers as you wish
When a problem is detected in pass all handlers are asked (no exeption throwing yet!) which one wants to handle the error. The first one which says yes will become the actual handler which decides what to do.
When a handler was found that can handle we come to pass 2 and call it. This time it is exception throwing time or not. It is entirely up to the handler.
The beauty of this two pass model is that during the first pass all handlers are asked but you still are able to continue parsing where you left. No state has been destroyed yet.
In C# you have of course much more freedom to turn this into a more flexible error transformation and reporting pipeline where you can transform for a strict reader e.g. all warnings to an error.
Which route you need to go depends on how your library is beeing used and how skilled the users of your library are and how many service requests you want to handle because some users make dumb errors. There is always a tradeoff between beeing as strict as possible (possibly too strict and you get a hard to use library) or too relaxed (nice parser but skips silently half of my input due to internal errors).
The Windows SDK Libraries are sometimes pretty hard to use because the engineers there optimize more towards less serivce calls. They throw a Win32 error code or HResult at you and you have to figure out which principle (memory alignment, buffer size, cross threading, ...) you have violated.

I think event subscription mode is OK. But you can consider interface instead. This might give you more flexibility. Something like this:
public interface IMessageHandler
{
void HandleMessage(object sender, MessageAvailaibleEventArgs eventArgs);
}
public abstract class ModelReader : IDisposable
{
private readonly IMessageHandler handler; // Should be initialized somewhere, e.g. in constructor
public abstract Model Read();
public event EventHandler<MessageAvailableEventArgs> MessageAvailable;
protected virtual void RaiseError(string message)
{
MessageAvailaibleEventArgs eventArgs =
new MessageAvailaibleEventArgs(TraceEventType.Error, message);
this.handler.HandleMessage(this, eventArgs);
}
}
So now you can use any implementation of message handler, for exmaple your event subscription one:
public class EventMessageHandler : IMessageHandler
{
public event EventHandler<MessageAvailaibleEventArgs> MessageAvailable;
public void HandleMessage(object sender, MessageAvailaibleEventArgs eventArgs)
{
var handler = this.MessageAvailable;
if (handler != null)
{
handler(this, new MessageAvailaibleEventArgs(
TraceEventType.Error, message);
}
}
}

Your current approach isn't the cleanest model because it has two contradictory execution styles, pull (reading the input) and push (handling error callbacks), and that means both your reader and its clients require more state management to provide a coherent whole.
I recommend you avoid the XmlReader-like interface, and use the visitor pattern to push the input data and errors to the client application.

If there are error that stop the library from doing it's work I would use exception.
If this is strictly for tracing and instrumentation I would either go with your pattern or a TextWriter pattern, where the consumer would pass in a text writer and you would write your trace info to that, the only problem with that is you can only have one external subscriber. but it results in a slightly cleaner code.
public TextWriter Log { get; set; }
private void WriteToLog(string Message)
{
if (Log != null) Log.WriteLine(message);
}

I know you mentioned that you may have several subscribers, so Event handlers and an Injected interface call are both good solutions as already mentioned above. For completeness I will also offer providing an optional Read() parameter as a Func. for example:
void Read(Func<string, bool> WarningHandler)
{
bool cancel = false;
if (HasAWarning)
cancel = WarningHandler("Warning!");
}
Then of course you could do whatever you wanted in the Func delegate, like divy the warning out to several sources. But when coupled with the Event model allows you to handle all Warnings in a general way (like a logger perhaps), and use the Func for individual specialized actions and control flow (if necessary).

Publishing to multiple subscribes in RX

I am investigating how to develop a plugin framework for a project and Rx seems like a good fit for what i am trying to achieve. Ultimately, the project will be a set of plugins (modular functionality) that can be configured via xml to do different things. The requirements are as follows
Enforce a modular architecture even within a plugin. This encourages loose coupling and potentially minimizes complexity. This hopefully should make individual plugin functionality easier to model and test
Enforce immutability with respect to data to reduce complexity and ensure that state management within modules is kept to a minimum
Discourage manual thread creation by providing thread pool threads to do work within modules wherever possible
In my mind, a plugin is essentially a data transformation entity. This means a plugin either
Takes in some data and transforms it in some way to produce new data (Not shown here)
Generates data in itself and pushes it out to observers
Takes in some data and does some work on the data without notifying outsiders
If you take the concept further, a plugin can consist of a number of all three types above.For example within a plugin you can have an IntGenerator module that generates some data to a ConsoleWorkUnit module etc. So what I am trying to model in the main function is the wiring that a plugin would have to do its work.
To that end, I have the following base classes using the Immutable nuget from Microsoft. What I am trying to achieve is to abstract away the Rx calls so they can be used in modules so the ultimate aim would be to wrap up calls to buffer etc in abstract classes that can be used to compose complex queries and modules. This way the code is a bit more self documenting than having to actually read all the code within a module to find out it subscribes to a buffer or window of type x etc.
public abstract class OutputBase<TOutput> : SendOutputBase<TOutput>
{
public abstract void Work();
}
public interface IBufferedBase<TOutput>
{
void Work(IList<ImmutableList<Data<TOutput>>> list);
}
public abstract class BufferedWorkBase<TInput> : IBufferedBase<TInput>
{
public abstract void Work(IList<ImmutableList<Data<TInput>>> input);
}
public abstract class SendOutputBase<TOutput>
{
private readonly ReplaySubject<ImmutableList<Data<TOutput>>> _outputNotifier;
private readonly IObservable<ImmutableList<Data<TOutput>>> _observable;
protected SendOutputBase()
{
_outputNotifier = new ReplaySubject<ImmutableList<Data<TOutput>>>(10);
_observable = _outputNotifier.SubscribeOn(ThreadPoolScheduler.Instance);
_observable = _outputNotifier.ObserveOn(ThreadPoolScheduler.Instance);
}
protected void SetOutputTo(ImmutableList<Data<TOutput>> output)
{
_outputNotifier.OnNext(output);
}
public void ConnectOutputTo(IWorkBase<TOutput> unit)
{
_observable.Subscribe(unit.Work);
}
public void BufferOutputTo(int count, IBufferedBase<TOutput> unit)
{
_observable.Buffer(count).Subscribe(unit.Work);
}
}
public abstract class WorkBase<TInput> : IWorkBase<TInput>
{
public abstract void Work(ImmutableList<Data<TInput>> input);
}
public interface IWorkBase<TInput>
{
void Work(ImmutableList<Data<TInput>> input);
}
public class Data<T>
{
private readonly T _value;
private Data(T value)
{
_value = value;
}
public static Data<TData> Create<TData>(TData value)
{
return new Data<TData>(value);
}
public T Value { get { return _value; } }
}
These base classes are used to create three classes; one for generating some int data, one to print out the data when they occur and the last to buffer the data as it comes in and sum the values in threes.
public class IntGenerator : OutputBase<int>
{
public override void Work()
{
var list = ImmutableList<Data<int>>.Empty;
var builder = list.ToBuilder();
for (var i = 0; i < 1000; i++)
{
builder.Add(Data<int>.Create(i));
}
SetOutputTo(builder.ToImmutable());
}
}
public class ConsoleWorkUnit : WorkBase<int>
{
public override void Work(ImmutableList<Data<int>> input)
{
foreach (var data in input)
{
Console.WriteLine("ConsoleWorkUnit printing {0}", data.Value);
}
}
}
public class SumPrinter : WorkBase<int>
{
public override void Work(ImmutableList<Data<int>> input)
{
input.ToObservable().Buffer(2).Subscribe(PrintSum);
}
private void PrintSum(IList<Data<int>> obj)
{
Console.WriteLine("Sum of {0}, {1} is {2} ", obj.First().Value,obj.Last().Value ,obj.Sum(x=>x.Value) );
}
}
These are run in a main like this
var intgen = new IntGenerator();
var cons = new ConsoleWorkUnit();
var sumPrinter = new SumPrinter();
intgen.ConnectOutputTo(cons);
intgen.BufferOutputTo(3,sumPrinter);
Task.Factory.StartNew(intgen.Work);
Console.ReadLine();
Is this architecture sound?

You are buffering your observable (.Buffer(count)) so that it only signals after count notifications arrive.
However, your IntGenerator.DoWork only ever produces a single value. Thus you never "fill" the buffer and trigger downstream notifications.
Either change DoWork so that it eventually produces more values, or have it complete the observable stream when it finishes its work. Buffer will release the remaining buffered values when the stream completes. To do this, it means somewhere IntGenerator.DoWork needs to cause a call to _outputNotifier.OnCompleted()