I have a solution containing a lot of projects and installer projects. One project uses a third party package. The package comes with a native DLL and a .net wrapper DLL. In order for the code to work, the .net wrapper DLL needs to find the native DLL in runtime. But the code never directly refers to the native DLL in compile time (the code talks to .net wrapper DLL in compile time).
Now I have to choose proper way to deploy the native DLL, during compile time on a programmer's machine and during installing time on a user machine.
Basically I have two options, either to put the native DLL to Windows System folder or to put the native DLL in the local folder containing the exe file.
To put the native DLL to system folder, I need a post-build script to xcopy the file to the directory after building the solution. I also need to create a System Folder output in installer project for installer to work on a client machine. I don't know whether copying files to system folder is a good idea or not. For this solution, every time someone (else in the big team) creates a new installer, he or she has to remember to create System Folder output and add the native DLL under the configuration otherwise his or her installer will not install a workable piece of software on a user machine.
To put the native DLL to local folder, I have the following two ways:
1. Use a post-build script. This solution, I have to find out every executable project in my big solution that has a reference to the project using the native DLL and link the post-build script to every such executable project. In the future, when someone (else in the big team) creates a new executable project with the same kind, he or she has to remember to link the same post-build script otherwise the executable wouldn't be able to find the native DLL. This is what I really don't like, people tend to forget.
I can add the native DLL to the project that uses it and in the DLL's property configuration, set it to "Copy If Newer". This way, the native DLL will be copied to the project's output folder and to every project that refers to the project. This way, I don't have to remember anything. The native DLL will be copied to the local folder of every dependent project.
This seems a good solution. But msbuild command seems not being able to handle this situation cleverly. For example, suppose project A directly uses native DLL and I add the native DLL to project A. Project B refers to project A and project C refers to project B and project A. if using msbuild to build the solution, the native DLL will be copied repeatedly to the output folder of project C 3 times, one for reference to project A, one for reference to project B, one for reference of project B to project A. In my big solution, towards the end of the dependency link, the native DLL will be copied exponentially many times to the same output folder, regardless of the "Copy If Newer" setting. This takes up tremendous amount of time to build the whole solution.
Now I totally have no idea what is the best solution for my situation. For anyone who uses native DLLs, how do you deploy the DLL so 1. it is convenient for both deploying on developer machine (compile and run) and on user machine (install and run), 2. developers in big team don't have to remember anything when he or she adds new project/installer to the solution, 3. smart enough build manner that avoids unnecessary redundant actions. Thank you for any hint, tutorial on Web, suggestion or clever teach in advance.
What we've done in general is to make all \bin directories into symlinks to a common directory. This avoids all the transitive copies that VS does for all references, and can speed up a clean build of a large solution up to a factor of 10-20.
We use the following powershell script to set up the symlinks:
param($linkTarget={throw "Link target must be specified"}, $rootDir=".")
ls $rootDir -Recurse -Include *.csproj | % { $_.DirectoryName } | % { Join-Path $_ -ChildPath "bin" } | % {
Write-Host "Creating link from $_ to $linkTarget"
if (Test-Path $_)
{
Remove-Item -Force -Recurse $_
cmd /c rd $_
}
cmd /c mklink /D $_ $linkTarget
}
Symbolic links require Vista or Win7; if on XP, junctions can be used instead by replacing the call to mklink with a call to junction.exe.
By default, this must be run as administrator.
For starters, avoid using post build events unless there's no other choice. Post build events unmanaged nature tend to fail builds and have low maintainability.
"Copy If Newer" flag may have lame effectiveness, but it will assure you that: 1) the required dependencies will be copied to output 2) it won't fail the build 3) has virtually no maintainability.
Secondly, you can reduce redundant copying by setting only the last project in the build chain with the "Copy If Newer" flag, e.g.: project A directly refers the DLL but does not copy it to output, project C refers indirectly to project A through project B, and has dummy reference to the DLL with "Copy If Newer" enabled.
Steve,
I add native dlls to my projects all the time and I haven't had a problem with adding the dll as content to the project. You can actually add the necessary dll as a project link rather than actual content. This way, the dll will not be copied into the project folder as well as into the target folder.
Here's an explanation from devx:
To add a shared file, open the dialog
to select an existing file with the
Project | Add Existing Item menu item
and select the file you want to
include. Then, instead of clicking the
Open button, click the arrow on the
left of that button, and click Link
File from the list that drops down.
This way you link to the original
file, not to a local copy of it.
Do you build all the projects into a common solution target directory, i.e. ..\bin\debug? This may curtail unnecessary copies.
Adding the project link to the dll is definitely easier to maintain than copying in a build event.
There must be something "dodgy" in your project setup (like a circular reference) to cause an infinite copy loop - but finding it could be tricky and/or time consuming.
Two things I would try:
Turn off the "newer" flag and see if it has any effect on your problem. Copying a single dll is unlikely to impact build times very much, and problems can come about with this option when file datestamps get confused. (I wouldn't expect this to cause repeated copy attempts though)
Try using a post-build event with xcopy to copy the file instead of relying on the automatic system. Then you'll be in full control of a simple system instead of wondering why a clever system doesn't work.
Personally, I'd treat the native DLL and assembly DLL as one entity. Adding a Link to the file is one way. Alternatively, you could have a separate main build script that massages your project and installer.
For instance, I have a Library.msbuild file that runs a MsBuild task of a source .csproj as my Build target. My Deploy target uses a copy task to copy the dlls from source\library\LibraryName\bin\Configuration*.dll to library\LibraryName. I have one library location for all the source files I need to build and the other is all the bins that my main project references.
Here's the entire Library.msbuild script:`
<?xml version="1.0" encoding="utf-8"?>
<Project DefaultTargets="Build" xmlns="http://schemas.microsoft.com/developer/msbuild/2003">
<PropertyGroup>
<Configuration Condition=" '$(Configuration)' == '' ">Release</Configuration>
<ProjectDirectory>library\AspNetMvc\DataAnnotationsModelBinder\src</ProjectDirectory>
<LibraryDirectory>..\library\AspNetMvc</LibraryDirectory>
</PropertyGroup>
<ItemGroup>
<CompiledBinaries Include="$(ProjectDirectory)\bin\$(Configuration)\*.dll" />
</ItemGroup>
<Target Name="Clean">
<MSBuild Projects="$(ProjectDirectory)\Microsoft.Web.Mvc.DataAnnotations.csproj"
Targets="Clean" Properties="Configuration=$(Configuration)" />
</Target>
<Target Name="PreBuild" DependsOnTargets="Clean">
</Target>
<Target Name="Build" DependsOnTargets="PreBuild">
<MSBuild Projects="$(ProjectDirectory)\Microsoft.Web.Mvc.DataAnnotations.csproj"
Targets="Build" Properties="Configuration=$(Configuration)" />
</Target>
<Target Name="Deploy" DependsOnTargets="Build">
<Copy SourceFiles="#(CompiledBinaries)" DestinationFolder="$(LibraryDirectory)"/>
</Target>
</Project>
`
In your instance, I would move the Deploy copy task to PreBuild or remove it entirely to depend on the Add Existing method copying the dll. The goal is that when you run\debug your project that everything will be bin relative. You could even have bin\win32 if you're separating native from .NET DLLs.
Every build is now done from a .build.cmd batch file (1-click build) that is simply this: C:\Windows\Microsoft.NET\Framework\v3.5\MSBuild Library.msbuild /t:Deploy. You're free to add more targets in the chain like test after build but before deploy.
If you are using WiX or any other type of installer, you should be able to use relative paths for the files themselves. I have a top-level staging\ directory that I have no problem getting into from source\install\ (..\..\staging\ icky, yes but works). Worst case you can place your staging directory somewhere inside the install path.
In most instances you can get away with not having to abstract MsBuild further than your .csproj or .wixproj files. There's a very low ceiling that Before/AfterBuild runs into but I find myself opting for this approach in almost every situation now.
See this post: http://blog.alexyakunin.com/2009/09/making-msbuild-visual-studio-to.html
It explains how to automatize copying of N-th level dependencies into Bin folder.
If you have some assembly, that isn't referenced even indirectly, you can create another one (e.g. MyProject.ThirdPartyMagnet), reference this assembly from this project and reference MyProject.ThirdPartyMagnet from the project you need to copy all the assemblies to.
Related
I have a solution with round about 50 Projects.
The default behavior of Visual Studio is to build each project into the project subfolders bin/debug and bin/release.
In my company there are guidelines that all projects should end up in a common output directory. The advantage should be that the files are not copied so often.
This is done defining the <OutputDirectory> in the .csproj file.
Checking the buildoutput showed me many deletes and copies of the same file. So I would expect that there are no advantages in copy count.
So are there any other benefits by sharing the output directory? Are there any problems that I not noticed yet?
What is the best practice handling this?
Thanks for your help :D
You can do this simply by using the MSBuild switch --output and specifying a folder for all the outputs when building a project/solution. All the dependent projects etc will be built and copied into one output folder.
I don't think it's particularly relevant "to reduce the number of files that are copied" - hard disks copy files all day, every day. Outputting into one folder can have a benefit when packaging for release; our Jenkins build server bundles all files into one folder before it puts them in a package (zip) file for Octopus deploy
You need to be careful htough, because it's possible for different projects to be dependent on different versions of the same DLL, and by mashing all the files together in one folder, DLLs of the same name (but different version) overwrite each other. You can then end up with a situation where your app doesn't load because it's trying for find XYZ DLL version 1.0 and some other project has a reference to version 1.1, and because that project built later, it's the 1.1 version of the DLL that ends up in the folder. You can then experience runtime errors indicating "The located assembly's manifest definition does not match the assembly reference" - it's saying "I was looking for 1.0, I found a DLL with the right name, but the version of the one I found was 1.1"
This can usually be solved with binding redirects, but it isn't 100% guaranteed that future versions of DLLs are backwards compatible with earlier versions. If you're arranging all your DLLs to copy into one folder youre creating a bit of a lottery for yourself as to whether things will break in production, when DLL versions go out of sync with what you expect
If your company operates a build and deploy process that uses locally copied versions of DLLs then it might be that they insist you do copy all DLLs to one folder to prove that the app will work in production - this is a much better reason (i.e. arrange a way for you to break the dev server so you can see and fix problems caused by DLL versions before putting to production and hitting the same) than "because we don't want the hard disk to get tired copying extra files"
If you insist to do that. You can
Right Click Your Project
Choose "Properties" => "Build Events" => "Post-build event command line"
Put xcopy "$(ProjectDir)\bing\debug{yourprojectdll}" "{path to shared folder}" /Y /I /R
I have a solution that contains 2 projects: a class library called plugin and a console app called myconsole. They both use a NuGet package i.e. log4net. They have to meet these requirements:
myconsole is not allowed to reference plugin.dll, but it needs to load it via reflection at run time
plugin.dll has to be available in a directory called plugins that is relative to myconsole.exe
log4net.dll has to be available in a directory called dlls that is also relative to myconsole.exe
I need this deployment to happen when I build the solution in the IDE. Ideally I need this file structure to be produced by the IDE build:
[Z:\Staging]
- [Dlls]
- log4net.dll
- [Plugins]
- plugin.dll
- myconsole.exe
I was thinking to update each project and add a MSBuild target that runs after the build to copy the content of the the output directory (bin\debug) to the destination directory that I need. This approach has some issues:
It does not scale well if I need to add more projects with different deployment requirements.
If the projects are built in parallel then they both need to copy log4net.dll to the same location. If this happens at the same time then I get warnings because the file is in use and retries are attempted. It works eventually, but I take it as a hint that I am doing something wrong. To workaround this I could filter the project output to deploy only what is produced by the project - which means I am able to deploy only myconsole.exe and plugin.dll, but not log4net.dll. I would have to deploy this one manually, probably after being done with both projects (solution-level event?). This seems too complicated...
What is the correct way to implement this in MSBuild?
In my opinion, you ask a few questions:
How to output the references to a separate folder.
How to export a dll without its references.
How to output the references to a separate folder
This question has already been asked, and its answer is somewhat complex, so it is not worth duplicating it.
How to export a dll without its references
You don't want plugin.dll to use its own log4net.dll, because if there is a static field in log4net.dll, it will be reinitialized when you load the plugin.dll at runtime.
Therefore, although you are exporting the plugin.dll to a separate folder, you need to leave the dlls in your executable folder:
Add this to your plugin.csproj file:
<Target Name="DontCopyReferencesToLocal" AfterTargets="ResolveAssemblyReferences">
<ItemGroup>
<ReferenceCopyLocalPaths Remove="#(ReferenceCopyLocalPaths)" />
</ItemGroup>
</Target>
Disclaimer:
I have not checked how the combination of the 2 requirements works together.
But I think that as long as each of them works on its own, there is no reason why they should not work together.
I used the method suggested in the following post to include an App.config file to Class Library project.
Equivalent to 'app.config' for a library (DLL)
It works as mentioned and creates the .dll.config file in respective Class Library projects output directory (e.g. ApiCommunicator\bin\Debug).
I have referenced this Class Library project as a "Project Reference" from another Console Application project within the same Visual Studio solution.
Now the problem is, the .dll is getting copied to the Console Projects output directory (i.e. Engine\bin\Debug) but the .dll.config doesn't. If I manually copy it, everything works fine but is there a way to configure Visual Studio to auto copy .dll.config to any other project's output directory which references the Class Library project?
Thanks,
Bathiya
Although I am late, but my answer can help others. I used the following command as pre-build event:
copy /Y $(SolutionDir)\[YOUR_LIBRARY_PROJECT]\app.config $(ProjectDir)$(OutputPath)[YOUR_LIBRARY_NAME].dll.config
I tried to be dynamic as much as possible, and the above command worked.
I posted the same answer at the question Can we autocopy the *.dll.config? as well.
It would have to be the other way around: Projects referencing your dll could copy it's dll.config from a known location as a post-build event.
However I think it would be much better to provide the configuration from within the consumer application's configuration. That way each referencing project would have an option to adjust the configuration to its needs, which I would think is a critical feature.
If that is not the case, then I don't see the reason for having the .dll.config, and you can just hardcode the values.
You can use the 'Build Events' tab of your project properties to run command line instructions post-build or even pre-build. This way, you can use a simple
copy /Y "<yourconfigfilepath>" "<yourprojectfilepath>\Engine\bin\Debug\"
This will copy the dll.config file you are needing over to the correct directory.
In brief, I have a separate WCF VisualStudio Project, to compile the CLient and Contract dlls. Once I build the project using MS-BUILD(command line), i have a post-build nant task to copy over the Client and Contract dlls to another location.
Now it is easier for me to pick these dlls from obj/Debug folder(as they have only the CLient and Contract dlls and not their dependencies).
However I've heard that the obj/Debug folder is temporary.. and we need to rely on bin/Debug to get the dlls.
Is it necessary to pick it up from bin/debug and not obj/Debug and why ?
The role of obj\Debug is an undocumented implementation detail of MSBuild. It exists because MSBuild likes to conditionally copy files based on project settings. The exact rules are convoluted, to put it mildly, you could only gain insight by studying the .targets files in the framework directory. Which in themselves are highly dependent on the .NET version you target.
If you really want to know what makes it tick then do copy files from obj\Debug and see what hits the fan. Do beware that this may happen long after you created the project so there will forever be a cast of FUD when builds fail or produce the wrong file. If that doesn't sound very productive, it is not, then avoid breaking the warranty and copy from the project's output directory. bin\Debug and bin\Release by default. If that produces a problem then at least you can ask a question about it at SO.
Yes. You are supposed to pick up binaries from bin/Debug, or the variable OutputPath in MSBuild context more specifically,
http://msdn.microsoft.com/en-us/library/bb629394.aspx
We have a solution comprising of a windows application and various library files. Not all of the library files are referenced by the main windows application however we would like to have all the library files included in the output build folder "bin".
Obviously one solution is to simply reference every single library from the Windows application however we would like to avoid any unnecessary referencing.
How can we include additional files into our build folder?
This is a C# project.
You can always use the pre-build or post-build events in the project settings to copy the additional files.
You can do this simply by doing a bunch of copy source target, or you could even be fancy and write an nmake file. You do have to maintain the list of source files however...
Edit:
One other thought. Your assumption is that this is "unnecessary referencing". However, if your application depends on these assemblies to run, whether or not they are compile time references, then don't these dependencies become "necessary" references? In that case, isn't adding them as references and letting Studio's build system work for you the best (and simplest) approach?
The solution was to change the build location for all "libraries" within the solution to the main output "bin" location. The main Windows application only references the libraries that it depends upon however all the libraries are built to the one "common" location.
Thanks to Nader Shirazie for help with this question.