I have about 10-15 projects with separate solutions that reference 3rd party DLL's in a microsoft .NET shop. One of the problems that we want to address is consistency of the versions of the DLL's used across projects (E.G. Netwonsoft 8.0.3 in all projects as opposed to separate versions depending when the project was created).
I have seen this done in two separate ways in my previous positions and was wondering if there are any other options to solve this problem.
I have used a corporate NuGet for all third party DLL's referenced within a solution for any project within the company. The DLL's would be updated and then made available to the developers in the projects to pull down and upgrade (if needed) within the solutions on their own.
Another company had an assemblies folder in source that housed all "approved" third party DLL's and all references lived within this directory.
I did see this question but it only offered one of the two solutions above: Where should you store 3rd party assemblies?
Are there other options aside from the ones listed above?
Whenever possible use NuGet. Primary reason being that Git doesn't very much handle large binaries well and using LFS for this doesn't make much sense, since there is a valid alternative. TFVC has fewer issues with large binaries, but I'd keep future migration to Git in mind if you're on TFVC.
Keep in mind that not just NuGet, but likely also npm and other package sources are of interest in this case.
If you want to enforce a certain version being used, create a custom task that you hook into the CI pipeline. That way you can easily give off warnings or setup some kind of policy. The custom task could take the packages.config file, scan the referenced packages and then query the TFS/VSTS package management feed to see if it's using the latest version (or the is using the latest minor version)... (or is using at least x versions back)... or fetches the approved versions from a json file or xml file from somewhere and validates against that...
In your source control, Commit and Push to Master with the desired dependency DLLs when the repository is first populated. Since all users, even on other branches, will then be pulling from the repository, you're ensuring they receive all the DLLs they need. This only becomes a problem if you're referring to DLLs in the GAC, which is resolved either by GACUtil or just making sure everyone is using the same Windows version.
Related
After searching quite a bit, I seem to be unable to find an answer to my problem. Usually, I find that means it is a non existent or incorrect approach, but I think it worth it to have an answer floating around on the internet nonetheless.
Essentially, we have 4 applications referencing 5 different "source" projects. So the scenario is, when we add a 5th application (for example), we will need to create project references to the other 5 different projects, as the application requires their output.
Its not a difficult task because the amount of projects is small, but it got us thinking. What if you could create a single project, maybe called Libs or something, reference all 5 projects in that project, and then the applications must only reference Libs. The idea seems cool, but I don't know if it will work because when you create a project reference, it points to Libs single output libs.dll.
So to actually ask a question, is this possible, and if so, how can it be done? Currently, having Libs reference the other "source" projects, and then having the applications reference the Lib project does not work, as it says there are missing assemblies.
And just to go over how this was created. The 5 source projects reside in a couple different solutions, so the only tedious part of this approach is "add existing project" at the initial start of the application's solution.
The way we manage this kind of thing in my organisation is to make a NuGet package for each of these shared "source" projects (e.g. in our case we have an error logging library, an XML utils library, a bespoke HTTP client, and others). These are published to our private NuGet feed URL (hosted on Azure DevOps, but you can just use a standard Windows fileshare if necessary) for our developers to use in their applications.
This has some clear advantages over your approach:
1) Dependencies - this seems most relevant to your question. If the project you built the NuGet package from itself depends on any other NuGet packages (either publicly available ones, or others from our private feed) then when someone installs that package in their project it will automatically install all the other packages it depends on.
So in your case you could create a shell "libs" package which doesn't deliver any content itself, but has dependencies on all your other packages, causing them to be installed automatically. In our case we have several cases of dependency (e.g. a "base" error logging package which is relied on by error handling modules which are tailored to different app types, e.g. MVC, Web API, Windows Services), and it works very well.
2) Updates and maintenance. In your scenario if you make a breaking change to one of your "source" projects, then, because you have a direct project reference declared in Visual Studio, any project which references the source one will have to make related changes to cope with the updates to the source project, before you can re-compile it and do whatever feature changes you're trying to achieve. This could be a pain, and be an untimely problem, especially in the case of major updates. However if instead you install a NuGet package containing that functionality, the developer of the application can choose if and when to install an updated version of the package.
There are other minor benefits as well which I won't go into, but there are some very good reasons why almost all major programming languages now provide similar "package" and "feed" functionality as a way of managing dependency on external projects and libraries. Your approach is, I feel, outdated and naive, resulting in the issue you've described and the potential for other irritations as well.
This is much of an architectural level question where I am stuck at. I was planning to implement nuget onto our enterprise project to get rid of multiple levels of deployment/re-building of projects. Here is the Project structure:
Framework (class lib solution): It is referenced in every other solution
ParentClient(WPF App Solution): It is the host client, it can contain many other module inside it which are different solutions as (3). This also uses Framework dlls
MyClient(WPF solution): This client is a module which also uses Framework dlls
Now presently whenever there is a change in Framework's dlls, the corresponding other solutions/projects must be updated, since in the actual deployment environment(a windows machine/ Client Machine) all the dlls resides in the same bin folder. If I do not update the framework dlls, then the present module won't function, and if I do, it might impact the existing application which do not require the latest framework update although due to the latest changes, it somehow breaks the current functioning. (Remember: This issue is due to the fact that in the ParentClient, we have multiple modules say MyClient, YourClient etc)
I would like to know whether or not we can use Nuget or does Nuget will might help me in resolving the issue...???
Yes, NuGet is the way to go. You can set up a very simple NuGet Server by simply creating an empty MVC application and then getting the NuGet Server project template from... nuget :)
An even easier solution, is to simply host your NuGet packages from a network share. You can then keep all versions of your dll's there, and address them from your CI/Build server. Nuget supports adding a network share as a source, and this is common practice
I'm looking for some thoughts on which approach would be considered best practice.
Scenario:
.NET SVN repository consisting out of multiple projects, some sharing the same NuGet package. Multiple solution files also exist with a subset, some overlapping, of projects in each.
Example:
a.sln
--> proj1 - log4net, nunit(v1.0.0)
--> proj2 - log4net, json
--> proj3 - log4net, nunit(v1.0.0)
b.sln
--> proj2 - log4net, json
--> proj4 - nunit(v1.0.0)
The problem arrises when a developer opens a.sln and updates the nunit(v1.0.0) package for all projects within that solution to say, v2.0.0. This leaves proj4's nunit still on v1.0.0, and assuming all binaries are copied to one output folder, the solution that is built first will determine which version of nunit becomes available.
Approach #1:
Compose one project, used by both a.sln and b.sln, that's sole purpose is contain all the NuGet references for all the projects, and output all files to a folder, say Externals. Modify all projects to manually reference the dll's inside this folder.
Approach #2:
Be deligent when updating packages and repeat the process for each solution.
Approach #3:
Create one solution that contains all the projects and avoid multiple solutions.
I have a strong preference but would like to get community feedback and be open minded.
Approach #1 - the problem with that I see is that it forces updates onto other projects, leading to potential compilation issues. Also, you may have a project that would prefer the older NuGet package
Approach #2 - as per #1 and in addition, unless developers are across the other solutions, they might not think to update them. We developers can be finicky when working on a bug with blinders on
Approach #3 - though somewhat easier from a maintenance perspective, it does not guarantee that some projects are all using the same NuGet packet. e.g. two different versions of nunit. In Visual Studio when installing and/or updating a NuGet package, the developer can override which projects will receive the package. Depending on the size of your projects, some developers might not like the idea of creating a monolithic solution (even if projects can be right-click Unload Project) due to general performance impact
In my opinion Approach #3 may be easier due to the convenience of mass installing the same package to all your projects in one step. Approach #1 comes close too but I fear you will find that you're going to have to deploy the NuGet packages into the other projects regardless due to dependencies - "Type xxx is defined in an assembly that is not referenced. Please add it to references".
Incompatible Versions
This leaves proj4's nunit still on v1.0.0, and assuming all binaries are copied to one output folder, the solution that is built first will determine which version of nunit becomes available.
Agreed, this strikes me as the ultimate problem and something we have run into. Even if your team is disciplined with using say log4net version X everywhere, good chances you will run into a situation where using some third party library also requires log4net but version Y. So the issue of incompatible versions rises again.
AppDomains
If you find that your solution must be deployed with multiple versions of 3rd party assemblies, you may want to look into child .NET AppDomains. The idea is that you deploy your assemblies not into one big folder (where older files can be clobbered by newer ones or vice versa), but a root folder and child folder for each assembly bound to a particular third-party .dll.
The reason you need child AppDomains is that an assembly may not be loaded more than once per AppDomain even if the other assemblies are located in different folders.
e.g.
<my app folder>
|--folder a
|--log4net v1.dll
|--nunit 3.dll
|--folder b
|--log4net v2.dll
|--nunit 4.dll
After all, nUnit uses child AppDomains when running your tests.
I need to get my external dll dependencies (automap, others ...) into a situation where I can build them on my build server. I would also like to get then them into subversion, so the build server can pick them up.
So I'm new to the whole 'lib' folder thing. I've searched Google, but it seems it's kind of assumed, there are no basics of what to do here. The books I own don't go into it. It's been a long time since I had a mentor at work, or even someone I could ask questions of ... and I'd really love to understand the fundamentals of what I should be doing here.
I write in .Net, use Jenkins as my CI server (new to that) and msbuild (new to that too). I'm hearing svn:externals (don't compute), NuGet ....
Please help!
Suppose my solution is called MySolution and is stored in C:\MySolution, then I have have three directories for binaries, all managed by source control.
vendor the source code of third party frameworks. If needed they are built and signed (with my key) and treated as if the code was my own. This is sometimes necessary to "fix" defects in the framework or debug their source to understand why it fails.
src\packages modules managed by nuget (I wished to combine this with my "lib" folder, but that isn't yet supported)
lib compiled libraries for which I don't have the source and that are not managed by nuget.
(I have omitted folders like "src", "sample", "setup", "documentation" and "scripts" to keep the answer specific to the OP).
The recent months I started to create my own nuget packages for "packages" in the lib folder so I can migrate all of them to "packages". Its published to a private nuget server. It also simplify managing the binaries across solutions.
I use to use externs, but they pose a branching nightmare after a while because you have to branch and pin the external dependencies to. With nuget this is no longer needed.
I would definitely avoid putting binaries in source control. You'd be far better off creating your own nuget repository containing your preferred versions of packages and either using nuget restore or some other way of "rehydrating" your dependencies for building. I use a simple batch file called nuget-update.bat which just looks at all packages.config files and gets any dependencies it finds.
It seems that you posted a sequence of questions on the same topic. I recommend NuGet, as it becomes critical and promoted hard by Microsoft. However, many old libraries are not available there, and you may still need to keep a lib folder. My open source project #SNMP is a good example,
I tried to use as many NuGet packages as possible, and even stepped up to maintain some of the dependencies, such as DockPanel Suite.
I have an issue with creating an easy solution for my build system, based on mono.
Current situation is that I keep my referenced libraries inside the git repository, which is not good, for many obvious reasons.
What I want to achieve is something like what NuGet provides - automatically download dlls from the Web, put them in some directory and forget about them.
I want to do this at build time, so it would not require any additional actions with downloading libraries etc. The best option would be an msbuild (xbuild on mono) task, but I want it to be system independent, so the popular one, executing NuGet.exe, is out of question (consider parallel mono installations, etc.).
I've tried Pepita project, but it's... wrong. No, really, it is, it has too many design mistakes to be easy to use or repair. To make a proper configuration would require a serious rewrite of the whole project.
What I would love is a library, that would employ NuGet.Core library and be available as a task. If such a lib is not there, I could use any solution, that would download a nuget package and unpack it to a directory specified in .csproj.
Even better, it would be nice if such a library could resolve dependencies without specifying them explicitly in packages.config (or similar) file, e.g. if I want to include Castle.Windsor I don't want to include Castle.Core in my config file.
I know about the OpenWrap project (with NuGet Gallery), it looks promising, but I can't find the solution where I would just put a constant set of libraries in my repo once, modify csproj files, some configs and have it done.
I can tell you that OpenWrap at the core has everything built-in to do what you want. Everything you can do with the openwrap-shell is also available to be called from msbuild. So, it seems to me that you would just need to add a before build hook to call out to openwrap to perform an "update-wrap". Several months back I actually looked into doing something similar. AFAIR I actually wrote an msbuild script to call openwrap tasks, but didn't really hook them into the normal build process.
I don't know exactly what you mean with "put a constant set of libraries in your repo once"? For OpenWrap, all you need to do is maintain the "openwrap descriptor" for your project. That file contains all direct dependencies of your project (with or without restrictions on version numbers). (Indirect dependencies are pulled in automatically) Are you wondering about how you get started when you have a bunch of binary dlls to start with? I can tell you what I did. Basically, I do not use any NuGet packages, I created OpenWrap packages for everything. I also created OpenWrap packages for all our binary dependencies (some of which are open-source). This is really super simple: you fill in correct dependencies in the OpenWrap descriptor and specify that the package must only contain the given dlls. We had a bunch of binary dependencies, but once you start packaging them, it's definitely not that much work.
If you want to see an example, you can check this one:
http://code.google.com/p/ppwcode/source/browse/dotnet/External/Apache.Log4Net/trunk/Apache.Log4Net.wrapdesc
That is all you have to do to package your binary dependencies. This is a package I created and we currently use it in the company where I work. I know Log4Net is probably available as a NuGet package, and I could probably use that. The advantage of creating those binary packages myself, is that I have full control over the packages, over the version numbering of the packages, over how a big project is split over several smaller packages and so on.
As an OpenWrap repo, you can use a folder on your local filesystem, or a folder on a network share. What we use, is actually a webdav repository that we mount locally on a drive (using Windows 7). This works fine for us and also allows us to specify who has read and write access to the repository.
You mention mono.... well, that might be a problem: the currently released version of OpenWrap (2.0.2) does not run on mono AFAIK. But the good news is that Sebastien Lambla has been working hard to get OpenWrap to run on mono+xbuild for the new version that is going to be released very soon: 2.0.3. No alpha/beta builds available yet, but you can build from git. (In that case you would need to build both openwrap-shell and openwrap). Sebastien Lambla, who created OpenWrap, normally keeps an eye on questions on StackOverflow and will probably be able to give you a more complete answer on the mono status.
Btw, where I work, we are using OpenWrap already for over a year. Back then we compared both NuGet and OpenWrap, and at that moment OpenWrap was way way way ahead of NuGet. Basically, to me, NuGet was not a tool for dependency management, but a tool to assist you in Visual Studio to pull in binary dependencies from a remote server (meaning: copy dll from remote server to local folder and add reference to local dll in project file). In the mean time, NuGet has been playing catch-up with OpenWrap and has added functionality that already existed in OpenWrap. There are in my opinion only 2 things that NuGet has over OpenWrap and that is integration in Visual Studio (aka overview of remotely available packages and click-click-click adding of packages) and the fact that it is maintained by Microsoft people (AFAIK). Both things are just political: it's easier to convince people with a pretty interface and microsoft support. Personally, however, I think that OpenWrap is technically superior and I think it's really a pity that it doesn't get the attention that it deserves.