Is there any way to access the test results (success/fail, maybe even asserts, etc) from a Specflow AfterScenario hook? I don't see anything, but it seems like something that would be included.
You can get hold of the test result by peeking into the ScenarioContext.Current. There's a TestError property that may help you.
See this wiki (https://github.com/techtalk/SpecFlow/wiki/ScenarioContext) for more information.
Yes, there is, but you need to use reflection. In your [AfterScenario] block use the following:
PropertyInfo pInfo = typeof(ScenarioContext).GetProperty("TestStatus", BindingFlags.Instance | BindingFlags.NonPublic);
MethodInfo getter = pInfo.GetGetMethod(nonPublic: true);
object TestResult = getter.Invoke(ScenarioContext.Current, null);
TestResult will be OK, MissingStepDefinition etc.
I use ScenarioContext to do this. Here is some example code which hopefully makes sense (doesn't get the actual assert value - not possible as far as I know), but does mean I can leave the browser open if the test fails:
[AfterScenario]
public void AfterScenario() {
if (ScenarioContext.Current.TestError == null) {
// Test failed (use ScenarioContext.Current.TestError to print out error to logs if required)
_driver.Quit
}
}
Related
Take this example:
var client = new HttpClient();
await client.GetAsync("http://www.google.com?q=%2D");
this actually sends a request to 'http://www.google.com?q=-'. I don't want .NET to alter my url.
This behavior is from System.Uri, which seems to unescape those character.
How can I prevent Uri/HttpClient from changing my url?
.NET Framework 4.7.2
Update: this behavior seems by design. I still can't believe there is not a way around this. What if I actually want to send '?q=what does %2D mean' to google.com? Now this gets send as 'http://www.google.com/?q=what%20does%20-%20mean'. Which is NOT what I meant to do.
Possible partial solution based on reflection.
I think the problem is that - is listed as a special character, here: https://referencesource.microsoft.com/#System/net/System/UriHelper.cs,657 . I don't think there's a way to modify the http scheme to change that behavior.
There was a previous bug, which has since been fixed, relating to how file paths are parsed by Uri. At the time, the workaround was to change the private flags of the related UriParser using reflection : https://stackoverflow.com/a/2285321/1462295
Here is a quick demo which you'll have to evaluate if it helps or not. It depends on whether uri.ToString() is called (then this might help), or uri.GetComponents (then you'll have to figure something else out). This code reaches into the Uri object and replaces the parsed string with something else. Here's the code and console output:
static void Main(string[] args)
{
var surl = "http://www.google.com?q=%2D";
var url = new Uri(surl);
Console.WriteLine("Broken: " + url.ToString());
// Declared in Uri class as
// private UriInfo m_Info;
// https://referencesource.microsoft.com/#System/net/System/URI.cs,129
FieldInfo infoField = url.GetType().GetField("m_Info", System.Reflection.BindingFlags.Instance | System.Reflection.BindingFlags.NonPublic);
// Immediately after m_Info is declared, the private class definition is given:
// private class UriInfo {
// public string String;
// ...
// };
object info = infoField.GetValue(url);
FieldInfo infoStringField = info.GetType().GetField("String");
// If you check the value of m_Info.String, you'll see it has the
// modified string with '?q=-'.
// The idea with this block of code is to replace the parsed
// string with the one that you want.
// This just replaces the string with the original value.
infoStringField.SetValue(info, surl);
// ToString() # https://referencesource.microsoft.com/#System/net/System/URI.cs,1661
// There are a couple of 'if' branches, but the last line is
// return m_Info.String;
// This is the idea behind the above code.
Console.WriteLine("Fixed: " + url.ToString());
// However, GetComponents uses entirely different logic:
Console.WriteLine($"Still broken: {url.GetComponents(UriComponents.AbsoluteUri, UriFormat.Unescaped)}");
Console.WriteLine($"Still broken: {url.GetComponents(UriComponents.AbsoluteUri, UriFormat.SafeUnescaped)}");
Console.WriteLine($"Still broken: {url.GetComponents(UriComponents.AbsoluteUri, UriFormat.UriEscaped)}");
Console.WriteLine("Press ENTER to exit ...");
Console.ReadLine();
}
Console output:
Broken: http://www.google.com/?q=-
Fixed: http://www.google.com?q=%2D
Still broken: http://www.google.com/?q=-
Still broken: http://www.google.com/?q=-
Still broken: http://www.google.com/?q=-
Press ENTER to exit ...
You might find some other inspiration from the code here which does use reflection, but also defines its own scheme to work with. Note the trust issues mentioned.
You mention .Net Framework 4.7.2, which should work with the above code. dotnet core will not.
The Allure framework is a really beautiful framework for test reporting.
Yet it has rather bad documentation for C#.
I want to add some things to my allure report:
Debug log (like all things I write to debug)
Screenshot
A file
How to do it? I have no idea, please help me if you know how to do it. It seems like AllureLifecycle class can help me but I'm not sure how to use it.
In case it matters I use Allure together with SpecFlow and MS test.
I searched more and seems I found the Truth.
And the Truth is it's possible to add all attachments I wanted but they can be added only as a file:
byte[] log = Encoding.ASCII.GetBytes(Log.GetAllLog());
AllureLifecycle.Instance.AddAttachment("DebugLog", "application/json", log, "json");
If you want to add a file from actually a path (location) you can do it with the same method but a different overload.
So just place this code in a "teardown\afterscenario" method or at any other place (for example at "afterstep" method) where you want to make this attachment. I use SpecFlow so if I add this to "AfterStep" hook then Allure displays those files attached to a specific step! That's amazing!)
it seems that allure has some events that can be used.
See : https://github.com/allure-framework/allure-csharp-commons/blob/master/AllureCSharpCommons.Tests/IntegrationTests.cs for more information.
haven't tried it myself, but something like this should work according to the documentation.
_lifecycle = Allure.DefaultLifecycle;
_lifecycle.Fire(new
MakeAttachmentEvent(AllureResultsUtils.TakeScreenShot(),
"Screenshot",
"image/png"));
_lifecycle.Fire(new MakeAttachmentEvent(File.ReadAllBytes("TestData/attachment.json"),
"JsonAttachment",
"application/json"));
Hope this helps.
Using this kind of code in AfterScenario method:
if (_scenarioContext.TestError != null)
{
var path = WebElementsUtils.MakeScreenshot(_driver);
_allureLifecycle.AddAttachment(path);
}
First it verifies, if Scenario passed, if not then
WebElementsUtils.MakeScreenshot(_driver)
method makes screenshot and returns it's path. Then this path I giving to Allure. As a second parameter in the same method I can give a name of the screenshot. As a result I am getting a screenshot in AfterScenario block in Allure report.
P.S. This is only for screenshots, about logs can't tell nothing.
With this example you can add an attachment exactly to the failed step
[AfterStep(Order = 0)]
public void RecordScreenFailure(ScenarioContext scenarioContext)
{
if (scenarioContext.TestError != null)
{
Allure.Commons.AllureLifecycle allureInstance = Allure.Commons.AllureLifecycle.Instance;
string screenshotPath = MagicMethodMakingScreenshotAndReturningPathToIt();
allureInstance.UpdateTestCase(testResult => {
Allure.Commons.StepResult failedStepRsult =
testResult.steps.First(step => step.status == Allure.Commons.Status.failed);
failedStepRsult.attachments.Add(new Allure.Commons.Attachment() {
name = "failure screen",
source = screenshotPath,
type = "image/png"
});
});
}
}
Background:
On an application I am working on, I am writing tests using a mixture of Visual Studio 2015, SpecFlow, and ReSharper 2016.3 (I'll abbreviate this as R#, because I'm lazy.)
The application I am working on sends HTML-formatted emails, based on a template, which are stored in HTML files that are set as Copy Always in Visual Studio 2015.
Problem:
When I attempt to run my tests, I get the following exception:
System.IO.DirectoryNotFoundException: Could not find a part of the path 'C:\Users\[Me]\AppData\Local\JetBrains\Installations\ReSharperPlatformVs14_001\Resources\SomeEmailTemplate.html`
The directory was not the output directory of the project I am working on, so I double-checked my R# settings, and confirmed that Shadow Copy was turned off. To be perfectly clear, my R# Shadow Copy checkbox is indeed unchecked.
The offending code is really pretty simple. Normal remedies like TestContext.CurrentContext.TestDirectory is not something I can, should, or even want to do, due to the fact that this code is needed by the application itself. It would be in appropriate to put test framework code in the application under test.
public class HtmlTemplateLog : ISectionedLog, IRenderableLog
{
#region Variables / Properties
private readonly string _rawHtml;
private readonly Dictionary<string, StringBuilder> _sectionDictionary = new Dictionary<string, StringBuilder>();
private StringBuilder _workingSection;
#endregion Variables / Properties
#region Constructor
public HtmlTemplateLog(string templateFile)
{
// This is specifically what breaks the tests.
_rawHtml = File.ReadAllText(templateFile)
.Replace("\r\n", string.Empty); // Replace all newlines with empty strings.
}
#endregion Constructor
#region Methods
// Methods work with the section dictionary.
// The RenderLog method does a string.Replace on all section names in the HTML.
// These methods aren't important for the question.
#endregion Methods
This is invoked as in the example below:
_someLog = new HtmlTemplateLog("Resources/SomeEmailTemplate.html");
// ...code...
_someLog.WriteLineInSection("{someSection}", "This is a message!");
string finalHtml = _someLog.RenderLog();
Questions:
1. I've turned off Shadow Copy on my R# tests. Why is this still doing Shadow Copies?
2. In what ways can I work around the fact that R# is not respecting the Shadow Copy checkbox, given that this is not test code, and thus that remedies that would normally be appropriate for test code aren't for this case?
I've discovered an answer for #2...though, it's rather clunky. I was inspired by the answer from #mcdon for a less-detailed version of the question.
Pretty much, if I don't want to resort to TestContext.CurrentContext.TestDirectory, then I need to make my local filenames into absolute paths. Unfortunately, R#'s broken Shadow Copy setting creates more work, since I can't just interrogate the currently-executing assembly - it will tell me the wrong thing. I need to get at the codebase instead and interrogate that.
I'm still a bit worried about what this code when we try to run it on the build server, however - I'm expecting 'unexpected' results. In that light, I'm wondering if the unexpected results can truly be called unexpected, given that I'm expecting that this won't work...
Anyways, the fix I came up with was this field-property system:
private string _presentWorkingDirectory;
private string PresentWorkingDirectory
{
get
{
if (!string.IsNullOrEmpty(_presentWorkingDirectory))
return _presentWorkingDirectory;
var assembly = Assembly.GetExecutingAssembly();
var codebase = new Uri(assembly.CodeBase);
var filePath = codebase.LocalPath;
var path = Directory.GetParent(filePath);
_presentWorkingDirectory = path.ToString();
return _presentWorkingDirectory;
}
}
Could anyone perhaps tell me how should I use the nsIWebBrowserFind interface in GeckoFX to find strings on a webpage?
I tried the following code, but this throws me an ArgumentNullException - parameter cannot be null (pUnk).
I have no idea what this means, I have never used interfaces before.
GeckoWebBrowser browser = getCurrentBrowser();
nsIWebBrowserFind finder = browser.GetInterface<nsIWebBrowserFind>();
finder.SetSearchStringAttribute(searchBox1.Text);
finder.FindNext();
I have also tried
nsIWebBrowserFind finder = Gecko.Xpcom.GetInterface<nsIWebBrowserFind>(browser);
With the same results:(
Please help:)
Thanks!
This works in GeckoFX 29.0:
var field = typeof(GeckoWebBrowser).GetField("WebBrowser", BindingFlags.Instance | BindingFlags.NonPublic);
nsIWebBrowser browser = (nsIWebBrowser)field.GetValue(webBrowser1);
var browserFind = Xpcom.QueryInterface<nsIWebBrowserFind>(browser);
browserFind.SetSearchStringAttribute(search);
try
{
browserFind.SetWrapFindAttribute(true);
browserFind.FindNext();
}
catch { }
Some websites with frames throws an exception after the last result however, this seems to be a bug in GeckoFX. We did a workaround by falling back to a javascript find (javascript:window.find) when the exception was thrown.
I am attempting to implement a Language Service in a VSPackage using the MPF, and it's not working quite as I understand it should.
I have several implementations already, such as ParseSource parsing the input file with a ParseRequest. However, when it finds an error, it adds it with AuthoringSink.AddError. The documentation for this implies it adds it to the Error List for me; it doesn't.
I also have a simple MySource class, a subclass of Source. I return this new class with an overridden LanguageService.CreateSource method. The documentation for OnCommand says it's fired 'when a command is entered'. However, it's not.
There's obviously some intermediate step which I haven't done correctly. I've already rambled enough, so I'll be glad to give any additional details by request.
Any clarification is much appreciated.
For the AuthoringSink error list question, I use this behavior in my Language Service. In ParseSource, the ParseRequest class has an AuthoringSink. You can also create a new ErrorListProvider if you want to work outside of the parser's behavior. Here is some example code:
error_list = new ErrorListProvider(this.Site);
error_list.ProviderName = "MyLanguageService Errors";
error_list.ProviderGuid = new Guid(this.errorlistGUIDstring.);
}
ErrorTask task = new ErrorTask();
task.Document = filename;
task.CanDelete = true;
task.Category = TaskCategory.CodeSense;
task.Column = column;
task.Line = line;
task.Text = message;
task.ErrorCategory = TaskErrorCategory.Error;
task.Navigate += NavigateToParseError;
error_list.Tasks.Add(task);
I hope this was helpful.
OnCommand should be firing every time there is a command, in your MySource class you can do something like this (pulled from working code):
public override void OnCommand(IVsTextView textView, VsCommands2K command, char ch)
{
if (textView == null || this.LanguageService == null
|| !this.LanguageService.Preferences.EnableCodeSense)
return;
if (command == Microsoft.VisualStudio.VSConstants.VSStd2KCmdID.TYPECHAR)
{
if (char.IsLetterOrDigit(ch))
{
//do something cool
}
}
base.OnCommand(textView, command, ch);
}
If that doesn't work double check that CodeSense = true in your ProvideLanguageService attribute when you setup your LanguageService package. A whole lot of what is cool to do in the LanguageService requires these attributes to be correctly turned on. Some even give cool behaviors for free!
Another thing to be careful of is that some behaviors like colorizer don't function correctly in the hive in my experience. I don't think these were ones that gave me trouble, but I implemented these a couple of years ago so I'm mostly just looking back at old code.
AuthoringSink.AddError only adds errors to the error list if ParseRequest.Reason is ParseReason.Check. When your ParseSource function attempts to add errors while parsing for any other ParseReason, nothing will happen.
It's possible that your language service is never calling ParseSource with this ParseReason. As far as I know, the only way to get a ParseReason of Check (outside of manually calling BeginParse or ParseSource yourself) is to proffer your service with an idle timer.