C# TestProject - How do I get CodeFilePath & LineNumber for each test? - c#

Given:
I have a project/solution containing tests. It is written in C# and is using nUnit.
What I want:
Somehow to retrieve all the tests in the Project/Solution without executing them and also giving me the CodeFilePath and LineNumber for each test.
What I tried / investigated:
dotnet test --list-tests : is giving me nothing (just the displayname)
NUnit3TestAdapter : is at somepoint exposing those values (see NUnitTestAdapter).
But how can I retrieve them?

Thanks #Charlie for pointing out a possible solution. I did basically exactly as you stated.
IMPORTANT NOTICE:
This seems to not work with dot.net CORE tests currently because of this issue: https://github.com/nunit/nunit-console/issues/710
Also DiaSession runs into an exception when used like this in dot.net CORE :-(
using System;
using System.Xml;
using Microsoft.VisualStudio.TestPlatform.ObjectModel;
using NUnit.Engine;
namespace ConsoleApp1
{
class Program
{
static void Main(string[] args)
{
var testAssemblyPath = #"C:\src\qata\src\tests\external\SomethingProject.Tests\bin\Debug\net461\SomethingProject.Tests.dll";
var package = new TestPackage(testAssemblyPath);
var testEngine = new TestEngine();
var runner = testEngine.GetRunner(package);
var nUnitXml = runner.Explore(TestFilter.Empty);
var session = new DiaSession(testAssemblyPath);
foreach (XmlNode testNode in nUnitXml.SelectNodes("//test-case"))
{
var testName = testNode.Attributes["fullname"]?.Value;
var className = testNode.Attributes["classname"]?.Value;
var methodName = testNode.Attributes["methodname"]?.Value;
var navigationData = session.GetNavigationData(className, methodName);
Console.WriteLine($"{testName} - {navigationData.FileName} - {navigationData.MinLineNumber}.");
}
}
}
}

Related

Retrieve TFS Test Steps always returns count of 0

I am trying to retrieve test steps (aka "actions") that have been added to a test case in TFS (2017.2) using the API (Microsoft.TeamFoundationServer.ExtendedClient v15.112.1). My current implementation always returns 0 test steps, although the actual test case has steps. I tried this as well in a clean new Team Project without any Work Item customization and even there it returns 0 steps. My implementation uses the older API (based on SOAP webservices), because it seems the newer http based API does not yet implement test steps. This is the code I have used:
private void GetTestStepsForTestCase(int testCaseId, int testSuiteId,
string teamProjectName, Uri tfsUrl)
{
TfsTeamProjectCollection tpc = new TfsTeamProjectCollection(tfsUrl);
ITestManagementService itms = tpc.GetService<ITestManagementService>();
ITestManagementTeamProject ittp = itms.GetTeamProject(teamProjectName);
ITestSuiteBase suite = ittp.TestSuites.Find(testSuiteId);
ITestCaseCollection testCaseCollection = suite.AllTestCases;
ITestCase itestCase = testCaseCollection.FirstOrDefault(t => t.Id == testCaseId);
foreach (Microsoft.TeamFoundation.TestManagement.Client.ITestAction itestAction in itestCase.Actions)
{
// Do something
}
}
Anyone?
You can use below sample to retrieve the test case steps from a specific test suite, it works on my side:
Install the nuget package : Microsoft.TeamFoundationServer.ExtendedClient - 15.112.1
using Microsoft.TeamFoundation.Client;
using Microsoft.TeamFoundation.TestManagement.Client;
using Microsoft.VisualStudio.Services.Client;
using System;
namespace RetrieveTestSteps
{
class Program
{
static void Main(string[] args)
{
var u = new Uri("http://server:8080/tfs/DefaultCollection");
var c = new VssClientCredentials();
TfsTeamProjectCollection tpc = new TfsTeamProjectCollection(u, c);
tpc.EnsureAuthenticated();
ITestManagementService itms = tpc.GetService<ITestManagementService>();
ITestManagementTeamProject ittp = itms.GetTeamProject("LCScrum");
ITestSuiteBase suite = ittp.TestSuites.Find(352);
ITestCaseCollection testCaseCollection = suite.AllTestCases;
foreach (var tc in testCaseCollection)
{
ITestCase testcase = ittp.TestCases.Find(tc.Id);
foreach (ITestAction action in testcase.Actions)
{
Console.WriteLine(String.Format("{0} - {1}", testcase.Id, action));
}
}
Console.Read();
}
}
}
OK, I finally figured out this one myself. The answer and comments of Andy helped me validate that my code is correct. I just discovered that my code worked fine when NOT debugging! When debugging, at some point I noticed this:
So probably because of lazy loading somewhere, it is not possible to verify the count of the attachments debug-time (see post here: Lazy<T>: "The function evaluation requires all threads to run").

Visual Studio 2015 build drop location

Our company recently updated TFS to 2015 update 1. After that context menu item named Drop folder disappeared from completed builds. I found nothing about it and how to bring it back. When I click Open on completed build, VS opens web version of TFS where I forced to click through the menus and copy drop folder path manually. So I decided to write a simple extension that will add this item to the menu.
Some googling brought me to this page. But it seems that the example code is quite old and not working in VS2015:
IVsTeamFoundationBuild vsTfBuild = (IVsTeamFoundationBuild)GetService(typeof(IVsTeamFoundationBuild));
IBuildDetail[] builds = vsTfBuild.BuildExplorer.CompletedView.SelectedBuilds;
Property SelectedBuilds is always empty. I suppose that it relates to old window from VS2010. It returns items that are instance of IBuildDetail interface.
So I found this piece of code here:
var teamExplorer = (ITeamExplorer)ServiceProvider.GetService(typeof(ITeamExplorer));
var page = teamExplorer.CurrentPage;
var buildsPageExt = (IBuildsPageExt)page.GetExtensibilityService(typeof(IBuildsPageExt));
var build = buildsPageExt.SelectedBuilds[0];
Here build is the instance of IBuildModel interface. It lacks DropLocation property.
Is there any way to found drop location of selected build? Or maybe latest build?
You can use IBuildDedetail.DropLocation in .NET client libraries for Visual Studio Team Services (and TFS). Basic code for your reference:
using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.Threading.Tasks;
using Microsoft.TeamFoundation.Build.Client;
using Microsoft.TeamFoundation.Client;
namespace BuildAPI
{
class Program
{
static void Main(string[] args)
{
string project = "http://xxx.xxx.xxx.xxx";
TfsTeamProjectCollection tpc = new TfsTeamProjectCollection(new Uri(project));
IBuildServer ibs = tpc.GetService<IBuildServer>();
var builds = ibs.QueryBuilds("TeamProjectName");
foreach (IBuildDetail ibd in builds)
{
Console.WriteLine(ibd.DropLocation);
Console.ReadLine();
}
}
}
}
So, after digging through TFS API, I ended up with this workaround.
private void MenuItemCallback(object sender, EventArgs e)
{
var context = (ITeamFoundationContextManager)ServiceProvider.GetService(typeof(ITeamFoundationContextManager));
IBuildServer buildServer = context.CurrentContext.TeamProjectCollection.GetService<IBuildServer>();
var teamExplorer = (ITeamExplorer)ServiceProvider.GetService(typeof(ITeamExplorer));
var buildsPageExt = (IBuildsPageExt)teamExplorer.CurrentPage.GetExtensibilityService(typeof(IBuildsPageExt));
var menuCommand = (MenuCommand)sender;
if (menuCommand.CommandID.Guid == CommandSetCompleted)
{
foreach (var buildDetail in buildsPageExt.SelectedBuilds)
Process.Start("explorer.exe", GetBuild(buildServer, buildDetail).DropLocation);
}
if (menuCommand.CommandID.Guid == CommandSetFavorite)
{
var definitions = buildsPageExt.SelectedFavoriteDefinitions.Concat(buildsPageExt.SelectedXamlDefinitions).ToArray();
foreach (var build in GetLatestSuccessfulBuild(buildServer, definitions))
Process.Start("explorer.exe", build.DropLocation);
}
}
private IBuildDetail GetBuild(IBuildServer buildServer, IBuildModel buildModel)
{
Uri buildUri = new Uri(buildModel.GetType().GetProperty("UriToOpen").GetValue(buildModel).ToString());
return buildServer.GetBuild(buildUri);
}
private IBuildDetail[] GetLatestSuccessfulBuild(IBuildServer buildServer, IDefinitionModel[] buildDefinitions)
{
var spec = buildServer.CreateBuildDetailSpec(buildDefinitions.Select(bd => bd.Uri));
spec.MaxBuildsPerDefinition = 1;
spec.QueryOrder = BuildQueryOrder.FinishTimeDescending;
spec.Status = BuildStatus.Succeeded;
var builds = buildServer.QueryBuilds(spec);
return builds.Builds;
}

test build agents of a build controller using tfs api

I want to automate testing of newly created build agents in TFS.
Now, I have written code to queue builds of particular build definition on TFS.
Problem I am facing is to queue build definition on agents of given controller.
Previously there was one method which I've commented in my code
for doing this, i.e. IbuildRequest.buildagent, but now it has been deprecated in newer tfs api.
Edit:
I have already tested IbuildRequest.BuildController property which tends to randomly pick agent, which is currently free.
What I want to know is can I force build definition to use one build Agent using tfs api.
using System;
using System.Linq;
using System.Security.Principal;
using System.Collections.Generic;
using Microsoft.TeamFoundation.Client;
using Microsoft.TeamFoundation.Build.Client;
using Microsoft.TeamFoundation.WorkItemTracking.Client;
namespace BuildAgentsTestApp
{
class Program
{
static void Main(string[] args)
{
Uri collectionURI = new Uri("https://tfs-uri");
var tfsCreds = new TfsClientCredentials(new WindowsCredential(), true);
TfsTeamProjectCollection tfs = new TfsTeamProjectCollection(collectionURI,tfsCreds);
WindowsIdentity user = WindowsIdentity.GetCurrent();
tfs.EnsureAuthenticated();
if (tfs.HasAuthenticated)
{
WorkItemStore workItemStore = tfs.GetService<WorkItemStore>();
Project teamProject = workItemStore.Projects["myProjectName"];
IBuildServer buildServer = (IBuildServer)tfs.GetService(typeof(IBuildServer));
IBuildDefinition buildDef = buildServer.GetBuildDefinition(teamProject.Name, "myBuildDefinitionName");
var controller = GetRequestedController(buildServer, "myControllerName");
var AgentsList = GetAgentCollection(controller);
//Queue each build definition
IBuildRequest buildRequest = buildDef.CreateBuildRequest();
buildRequest.GetOption = GetOption.Custom;
//buildRequest.BuildAgent = AgentsList.First();
buildServer.QueueBuild(buildRequest);
}
Console.WriteLine("Build Queued!");
Console.ReadKey();
}
static IBuildController GetRequestedController(IBuildServer reqBuildServer, string reqControllerName)
{
var requiredController = reqBuildServer.QueryBuildControllers()
.Where(ctrl =>
ctrl.Name == reqControllerName
).FirstOrDefault();
return requiredController;
}
static List<IBuildAgent> GetAgentCollection(IBuildController controller)
{
var ListOfAgents = from agent in controller.Agents
select agent;
return ListOfAgents.ToList();
}
}
}

Run Xunit and UI Tests test from code with reflection

I am trying to run CodedUI Tests from MTM as well as running Xunit tests on our code base. I am supposed to be able per my requirements to run these tests from ONE project so as to be able to gather data on the tests later on. I have been able to get the tests to run in both instances from the same code using reflection to call test methods based on their respective attributes.
The problem is, then for both the UI and the Xunit tests, the Test Explorer is not registering that the tests are passing or failing. I assume that there is some initialization that must occur for the testing environment, but I can't find any resource on it.
using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.Threading.Tasks;
using CodedUITestProject1;
using Microsoft.VisualStudio.TestTools.UITesting;
using CodedUITestProject2;
using System.Reflection;
using ConsoleApplication1;
namespace Run_UI_From_Outside
{
class Program
{
static void Main(string[] args)
{
// This section uses reflection to find Test Methods in the CodedUITest class bases on their attributes
var assembly = typeof(CodedUITestProject2.CodedUITest1).Assembly;
var attributes = assembly.GetCustomAttributes(typeof(CodedUITestProject2.CodedUITest1), true);
var types = assembly.GetTypes();
foreach (var t in types)
{
var ca = t.CustomAttributes;
foreach (var item in ca)
{
if (item.AttributeType == typeof(Microsoft.VisualStudio.TestTools.UITesting.CodedUITestAttribute))
{
object myInstance = Activator.CreateInstance(t);
MethodInfo myInstanceMethod = t.GetMethod("CodedUITestMethod1");
Playback.Initialize();
var test = myInstanceMethod.Invoke(myInstance, null);
Playback.Cleanup();
}
}
}
// This section uses reflection to find Xunit Methods in the CodedUITest class bases on their attributes
var assemblyXunit = typeof(ConsoleApplication1.Tests.SampleTest).Assembly;
var xunitMethods = assemblyXunit.GetTypes().SelectMany(t => t.GetMethods())
.Where(a => a.GetCustomAttributes(typeof(Xunit.FactAttribute), false).Length > 0)
.ToArray();
foreach (var t in xunitMethods)
{
object myMethodInstance = Activator.CreateInstance(t.DeclaringType);
t.Invoke(myMethodInstance, null);
}
}
}
}

Roslyn / Find References - Can't properly load Workspace

I'm trying to write some code to find all method invocations of any given method as I am looking to create an open source UML Sequence Diagramming tool. I'm having trouble, however, getting past the first few lines of code :/
The API appears to have changed drastically and I can't seem to infer proper usage by looking at the code.
When I do:
var workspace = new CustomWorkspace();
string solutionPath = #"C:\Workspace\RoslynTest\RoslynTest.sln";
var solution = workspace.CurrentSolution;
I find that workspace.CurrentSolution has 0 Projects. I figured this would be the equivalent to what was previously Workspace.LoadSolution( string solutionFile ) which would then supposedly contain any Projects in the Solution, but I am not finding any success with this path.
I am terribly confused 0.o
If someone could offer some additional guidance as to how I can use the FindReferences API to identify all invocations of a particular method, it would be very much appreciated!
Alternatively, would I be better off taking a static-analysis approach? I would like to support things like lambdas, iterator methods and async.
====================================================================
Edit -
Here is a full example based on the accepted answer:
using System.Linq;
using Microsoft.CodeAnalysis.CSharp;
using Microsoft.CodeAnalysis.CSharp.Syntax;
using Microsoft.CodeAnalysis.MSBuild;
using Microsoft.CodeAnalysis.FindSymbols;
using System.Diagnostics;
namespace RoslynTest
{
class Program
{
static void Main(string[] args)
{
string solutionPath = #"C:\Workspace\RoslynTest\RoslynTest.sln";
var workspace = MSBuildWorkspace.Create();
var solution = workspace.OpenSolutionAsync(solutionPath).Result;
var project = solution.Projects.Where(p => p.Name == "RoslynTest").First();
var compilation = project.GetCompilationAsync().Result;
var programClass = compilation.GetTypeByMetadataName("RoslynTest.Program");
var barMethod = programClass.GetMembers("Bar").First();
var fooMethod = programClass.GetMembers("Foo").First();
var barResult = SymbolFinder.FindReferencesAsync(barMethod, solution).Result.ToList();
var fooResult = SymbolFinder.FindReferencesAsync(fooMethod, solution).Result.ToList();
Debug.Assert(barResult.First().Locations.Count() == 1);
Debug.Assert(fooResult.First().Locations.Count() == 0);
}
public bool Foo()
{
return "Bar" == Bar();
}
public string Bar()
{
return "Bar";
}
}
}
CustomWorkspace is
A workspace that allows manual addition of projects and documents.
Since you're trying to load a solution, you should use the MSBuildWorkspace, which is
A workspace that can be populated by opening MSBuild solution and project files.
You can create a new MSBuildWorkspace and call OpenSolutionAsync with your solutionPath. For the reference finding part, take a look at the SymbolFinder.
Solutions are an MSBuild concept.
You need to create an MSBuildWorkspace and call OpenSolutionAsync().
string solutionPath = #"C:\Workspace\RoslynTest\RoslynTest.sln";
creates a local variable. It has no influence on your CustomWorkspace object.

Categories

Resources