We have around ~ 80 C# projects in a single Git repository.
Our CI server is TeamCity, and we use it to build upon every commit to our git master repository.
I am trying to come up with the best way to build all of the projects on the server.
These projects are currently grouped together in a few .sln files (around 20 projects each), is it a common practice to create a "master" solution file with ALL projects such that the build server will only need to build this solution?
Or is it preferable to keep some MSBuild or other script file that will itself execute the building of each solution/project whatever it sees fit?
We are also using TeamCity with about 130 projects all in the one solution. There is not fits all answer. For CI I believe the critical issue is how long it takes to build and run tests. Both on your build farm and on developer boxes. Do the simplest possible thing for efficient software development.
Mind you the tricky thing is what is fast? Speed is relative and different developers can work very different. Some like to only commit once a day for example. So it needs to be context.
Have your TeamCity build build the same as you do on your developer boxes. Less layers the better. Keep it simple.
See similar answer: Visual Studio Large Solution.
Related
After reading about using GIT with Visual Studio solution, I understand that it is preferable to use GIT per project, at least for project which represent a (re-usable) common library. If you have a solution with many projects specific to that solution, using only one repository could be acceptable. For common library, it appears logical to me to have one repository for it in order to be able to fix bugs in the common library from where you detect it and have only one history for all changes.
Using one GIT per common project means that you have more than one GIT repository for any solution that produce an executable that use one or more common library.
According to this: Visual Studio suggestion - Allow multiple Git repositories to be active at once, Visual Studio seems to not support many GIT repository seamlessly. (1995 request)
Does Visual Studio 2017 implement the previous suggestion and manage many GIT repository per solution? (ex: one per project for some projects and one per solution for the solution itself and all other specific projects to this solution)? In other words, does Microsoft will see and manage changes per project/GIT or do I have to works with only one GIT at the solution Level?
Just as a side one (this is not the primary question - the real question is in the previous paragraph): If Visual Studio does not allow multiple GIT repo to be active at once, wouldn't be better to stick with TFS for the moment for any development with common library?
This is a controversial topic. You have a few options:
Mono-repo. All your code lives in a single repository, whether they're split up into separate solutions or not is up to you. If your dependencies are "possibly reusable" then this is the simplest starting point. If you ever really start to reuse your components, you can always break them out.
Separate repositories + Package Manager (npm, Nuget). Put each reusable project in a separate solution, optionally in a separate repository. Have a CI build automatically create a package and publish it to the VSTS package management feed.
SubModules: Create a master repository with your solution, create a separate repository for each reusable component with just the csproj and the sources for that component. Visual Studio 2017 has rudimentary support for submodules. But with a separate git client on the side (Tower, SourceTree) or simply with some commandline mastery, you can make this work just fine.
Subtree: Create a master repository with your solution, create a separate repository for each reusable component with just the csproj and the sources for that component. Visual Studio 2017 has no support for subtrees at the moment. But with a separate git client on the side (Tower, SourceTree) or simply with some commandline mastery, you can make this work just fine.
Multi-repo: Create separate repositories for each project, put a solution alongside it. Manage each sub-component separately and there is no concept of a submodule tracking. Visual Studio will actively fight you in this setup, as it will only support a single master repository at a time
In the end it is your choice. Each solution has its own benefits and there are examples of each out there. The Windows sources are all stored in a single monstrous mono-repo with all its reusable components in the same repository. It has great advantages with regards to knowing which files and which versions work together and which won't. But the repository is so big that Microsoft built GVFS in order to allow their developers to work on a 300GB working directory.
If your components are currently not being re-used and if your tests are more integration tests than real unit tests, then it makes sense to join them up into the same repository.
You can always decide to fix this when the need arises. You can always try to keep these projects as separate as possible. The .NET route will most logically lead you to Nuget though... Also because it handles the versioning aspects and the ease of sharing between projects without having to build the sources everywhere.
While there has not been great support for multi repo solutions in the past, this is about to change. In Visual Studio 16.11 Preview 1, there is now a much more fully featured Git client, multiple repositories are detected and users can easily switch between them using a repository selector in the status bar.
See this blog for further information: Enhanced Productivity with Git in Visual Studio
I have a solution that has 4 projects in it. 3 are dependencies for my tests and the other is just my tests.
DL
BI
MySite (web site)
MyTests
Some unit tests in the MyTests project reference namespaces in the web site MySite for some MVC contollers.
Question is how do I get just the MyTests project to build and deploy with a TFS build. NO matter what I try the _publishedWebSites folder on the TFS build machine always has the web site and not the MyTests folder. For some reason it thinks it is building the web site and not the tests. Any help would be appreciated from the build definition or solution perspective.
The purpose is to build the tests and distribute them to a server where they can be run (selectively) using the command line tool in the task manager. I cannot distribute them if I cannot get the solution to build properly.
Alright so there are a few things. Firstly, you need to make sure that the outer solution recognizes MyTests as the start up project and has the other projects as build dependencies. However, this likely won't solve your problems. In order to do this you'll probably have to create a custom build script or edit your solution/project files by hand. The problem with the latter approach is that if other people are building MySite from this solution editing the project file to exclude it's output from the drop is going to cause problems for them.
My personal approach would be to make an MSBuild script which specifies the order in which to build the projects and which files you want in the drop. It's fairly straight forward and it will probably be easy to specify the output you want (this is sometimes very tedious if the projects build is messy to begin with or it has excessive and convoluted dependencies).
Here's the outer most resource for MSBuild. I'd look it over and think about what the simplest solution is but I wouldn't be surprised if you can just make every project build using their project files then add a single build step to cleanse your output.
I have a C# project built using NANT. Is there a convenient way to generate VS solution and project files to open and compile this project using Visual Studio?
Yes there is. Shameless plug: generate-msbuild task on GitHub. It doesn't do a full faithful conversion, but it does create a Visual Studio solution that you can open in IDE and compile from there. Also, might give you a nice starting point if you think about migrating your build from NAnt to MSBuild. If you have any problems (or bugs), please ask on the GitHub page. That was tested on a couple large projects, including CruiseControl.NET, NAnt and Boo, so should handle most scenarios.
Also, new MSBuild (starting with .NET 4.0) can do everything NAnt can - most users just don't realise it.
No, nant (while-complete) allows to much flexibility in building that you could represent in VS solution and projects (Fixed List of Files). Maybe you could create a way of generating solution and project files from nant build files for a specific subset of build scripts or record how the C# is assembled but I would not consider that convenient, at least for small to medium sized solutions.
Fortunately if sufficient recent versions of VS I find it quite convenient and easy to import files into new solutions and projects.
I have the application (solution in VS2010) with +-10 projects. Some of these projects are quite large (dozens of folders and hundreds of files).
When I make a change in project on lower level and I want to run my tests (+-10 seconds) then I have to wait 2 minutes for projects build.
This is very inefficient.
Is there any way to speed-up build? For example, split current projects into multiple projects or something else?
Or are there some general advice and recommendations to speed-up building projects in Visual Studio?
Try this:
Go to "Tools" menu -> "Options" -> "Projects And Solutions" -> "Build And Run"
And increase the value of "Maximum number of parallel project builds" to 32 or greater.
In my experience solutions with large number of projects build slowly on vs2010 and there is not much you can do about it:
Reduce number of projects - larger single project will build faster then many smaller.
Prepare set of build configurations Build->ConfigruationManager to build only parts of your project since you don't have to build projects that depend on changed project if public interface is not changed. This can be tricky to use and some unexpected errors might occur in runtime. Also make sure that all your projects point to single Bin\Debug Bin\Release folder so that new dlls are loaded on application start. This is requited because if you don't build project its dependencies wont be copied to its output directory.
Upgrade to vs 2012. This is the best option you can choose.
Try to use tools that allow concurrent test execution coupled with shadow building - NCrunch or MightyMoose.
Also try to get rid of unused references - for example move all tests that test only your core project to separate project so that they get executed right after core is built. Try not to use MS accessors, because they force to fully recompile all projects that they are referencing. And of course use SSD disks.
Ok, so here's my current setup and my problem at the moment. I have a growing set of projects in a Visual Studio solution. The solution contains about 15 projects (give or take a few) and a quickly growing code-base. I realize that I should have had a continuous build system set-up before I got to this, but its never too late I suppose. So after doing a little research, I believe that my perfect setup would be:
NUnit 2.5.x (we are already tied to this... so a necessity)
Integration with CruiseControl.Net (open to other options, but only free ones with Git support)
Integration with a code-coverage tool (NCover, DotCover) would be nice
Integration to run shell commands (for JSLint and compression tools, etc.)
What I am missing is a tool to run the automated build. I looked at NAnt, but it's support for running MSBuild (to build the project) seemed fairly outdated (we're using VS2010) and utilizing the solution files in our build process would be a HUGE time saver. I also looked at MSBuild (for the obvious reasons) but the process that I found for running NUnit tests only supports 2.4.x (MSBuild extensions project).
I am curious how everyone else has organized their continuous build systems. NUnit if fairly popular, so I must not be the only one who is wondering about this.
my first question is how may build projects will you have?
Teamcity Professional is free for 20 build configurations per server and will make your like sooooo much easier, has dotcover build in, and is really easy to setup, run your tests, etc. and it's by far the most fully baked CI server out there.
Jenkins is the next runner up, it's a fork of Hudson and is very flexible with plugins
to do just about anything making it a little more flexible then Teamcity but it's not as easy to set up, code coverage is a pain to set up and has some annoying quirks, but is completely free.
unless you have some really strong reason to use CruiseControl.Net, don't bother, for it's time it was very powerful but it's now sadly out dated and painful to use.
As far as setting up builds, both Teamcity and Jenkins support MSBuild, NAnt, Rake, etc. they also support multiple build steps like would do in an msbuild or Nant file. What I have done in the past is just use the .sln file to do the build with one build step, used the build in task for unit tests, then used the built in task for code coverage then used a another build task for pushing the files.
I have used TeamCity, Jenkins, TFS, and I tried to used CruiseControl.Net but found it painfully clunky. By far Teamcity is the best, with Jenkins a close second, I would not willing use TFS even if I had it.
If you have any questions please feel free to contact me.
You can use NAnt to build your Visual Studio 2010 solutions. I do it all the time. I provided sample NAnt script in my answer here: <msbuild> task or msbuild.exe with NAnt?
If all you need is compile and run tests you can't go wrong with TeamCity it has great support for NUnit/VS and a bunch of reports built in.
If you need to run a more complex build script I suggest you use FinalBuilder for creating the build script and TeamCity command runner to execute that script.
By importing the test result into TeamCity you can still get the reports and there is a simple way to output build status from FinalBuilder to TeamCity:
How can I output messages from FinalBuilder that will be captured by TeamCity?
Outputting build status from FinalBuilder to TeamCity