I never understand , What actually happens to the the external DLLs when code is compiled by the compiler and converted to Intermediate code to run on CLR.
Does DLL code added to the Intermediate code and the references are not longer needed on the new machine or just the path of added DLLs are stored and we need those DLLs on our drive to run the program.
Generally (ie. there are some exceptions) the referenced assembly needs to be on the deployment system. All that is included in your assembly is the assembly name (this includes version and possibly signature).
Assembly binding (involving the GAC, config overrides etc.) is applied at runtime to get the right assembly. The rules depend on how the app domain was set up (eg. extra folders can be added, which is why ASP.NET web apps apply different rules).
There are various cases where the reference can be embedded (including referencing ActiveX when the right options are set and use of the assembly binding tool).
I'm writing a plug-in for a 3rd party application (.NET). This application lets me choose the plug-in (as a .dll library file) to load. However, if I have two versions of the same library---they have the same name but are in different directories---and try to load one after the other, it only loads the first plug-in and treats the second as if it was the first. In other words, if the first plug-in is supposed to show a message box saying "First plug-in" and the second plug-in is supposed to show a message box saying "Second plug-in," then loading the second plug-in after the first will actually show a message saying "First plug-in" (i.e., the second plug-in was actually never loaded).
After searching and reading online, I believe that the problem is that the 3rd party application loads its plug-ins into its primary AppDomain. Therefore, plug-in libraries are never unloaded (become locked?) and subsequent attempts to load a plug-in with the same name simply uses the library that's already been loaded. I thought perhaps signing my plug-in libraries would fix the problem, but unfortunately, I'm unable to sign them because I depend on a .dll provided by the 3rd party application and it is not signed. Also, I cannot change the 3rd party application's config file, so I cannot play around with probing.
Our current solution is to re-name the assembly for every version of the plug-in library we have (for example, "PlugIn-1.0.dll" and "PlugIn-2.0.dll"), including re-naming all their dependent assemblies. I don't mean just changing their file name, but changing the AssemblyName property and re-compiling. This works, but I'd like to see if there's a cleaner solution. It wouldn't be so bad if it was just the plug-in assembly name we had to change, but we are also forced to change all their dependent .dll's (because different plug-ins may use different versions of these .dll's as well). I tried creating a config file for the plug-in library to change the probing directory, but this doesn't work. It looks like it is the application itself that does the probing, not the library that depends on the .dll's (am I correct in inferring this?)
Finally, I tried having my plug-in create an AppDomain and load its dependent .dll's into it, but unfortunately my plug-in directory location (and dependent .dll's) must be on a remote location relative to the 3rd party app. There are security/permission issues with loading assemblies over network locations in .NET 4.0 (which I'm using) that I haven't been able to solve.
What are my options? Thanks in advance.
I think you're knocking on the right door with spinning up your own AppDomain. That is the only way I know of to have different versions of the same assembly loaded into the same process.
And yes - as soon as a .NET appdomain has loaded an assembly, it will not load another version of that assembly. Whoever is first wins. And of course, there is no way to unload an assembly once it has been loaded.
As for your permission issues.. Can you copy the assemblies from that network location to a user folder (where the user has write permissions) and load them from there? I've been able to do this successfully for an auto-updating application.
If I host CLR in my C++ executable, is there any way to set base directory of default AppDomain to something other than location of the executable?
Here's why I need it. I have a rather complex application that loads .NET plugins using a plugin loader executable, PluginBox.exe. The plugins are located outside of the main application folder. We run one instance of PluginBox.exe per plugin.
Currently PluginBox.exe is written in C++. I want to convert it to a managed app. PluginBox uses unmanaged CLR hosting (ICLRMetaHost, ICLRRuntimeInfo, etc.), and locates plugin assemblies by implementing IHostAssemblyStore interface. There is only one AppDomain, and its base directory is the application directory. When searching for assemblies, CLR invokes the assembly store before looking at the application base directory. So, if the plugin and the main app contain an assembly with the same name, the assembly store can load the plugin-specific version.
As a first step towards making PluginBox.exe managed, I would like to get rid of the assembly store and replace it with an assembly resolver. The trouble is, unlike assembly store, assembly resolver is called after the application base directory has been considered. So, if the plugin and the main app contain an assembly with the same name, main app's assembly wins. This disrupts plugin execution.
I would like to switch base directory of the hosted CLR to where the plugin is located. So far, I found only two ways to do that, both of them unacceptabe: move PluginBox.exe to the plugin directory, or create a second AppDomain, which is problematic for a variety of internal reasons. This is a huge application with a lot of history, and any drastic moves are bound to cause problems.
Any thoughts and ideas are appreciated.
PS. Current CLR version is 4.0.
I know it's a Windows PE32, but I also know that the unit of deployment in .NET is an assembly which in turn has a manifest and can be made up of multiple managed modules.
My questions are :
1) How would you create multiple managed modules when building a project such as a class lib or a console app etc.
2) Is there a way to specify this to the compiler(via the project properties for example) to partition your source code files into multiple managed modules.
If so what is the benefit of doing so?
3)Can managed modules span assemblies?
4)Are separate file created on disk when the source code is compiled or are these created in memory and directly embedded in an assembly?
EDIT:
#Jon:
For 2):So, does compiling/building source in visual studio always create a single managed module? If so then I fail to understand as to why VS doesn't provide a mechanism to do so in spite of the fact that .NET supports doing so.
I agree that it would be unmanageable to create an assembly with modules from different languages. Is that the only reason why .NET allows creating multi module assemblies?
I read in Richter's CLR via C# that modules can also span assemblies, and this can help keep assembly sizes down, and reduce memory footprint by downloading assembles on demand when certain functionality is invoked for the first time, but I'm not quite sure as to why would one want to span a module across assemblies, why not just create a new assembly which implicity creates a new module in the process. You would still gain the same benefits.
Item 4) was in regards to ".netmodule" files.
As part of the VS build process I haven't seen any ".netmodule" files created in the obj directory. I've typically noticed .pdb, .dll/.exe and a *FileListAbsolute file and hence the question on whether any separate files are created for managed modules.
EDIT:
#Jon: Here is the excerpt from CLR via C#(3rd edition) Pg 43:
Maybe I'm misreading this but it sounds to me that a module (which is a file belonging to an assembly) can be downloaded on demand.
"For example, an assembly can consist of several types.
You could put the frequently used types in one file and the less frequently used types in
another file. If your assembly is deployed by downloading it via the Internet, the file with
the infrequently used types might not ever have to be downloaded to the client if the client
never accesses the types. For example, an independent software vendor (ISV) specializing in
UI controls might choose to implement Active Accessibility types in a separate module (to
satisfy Microsoft’s Logo requirements). Only users who require the additional accessibility
features would require this module to be downloaded.
You configure an application to download assembly files by specifying a codeBase element
(discussed in Chapter 3) in the application’s configuration file. The codeBase element identifies
a URL pointing to where all of an assembly’s files can be found."
1) You can't do this in Visual Studio. You can do it from the command line using:
csc /target:module Foo.cs Bar.cs
In this case you'd end up with a fle called Foo.netmodule
2) See question 1 - you can't do this from Visual Studio, but you can do it from the command line. I don't know of any benefits. EDIT: I agree with Andrew's statement that you could create an assembly from multiple languages this way - but I believe it would be impractical. You'd have to work out an appropriate dependency chain so that you could build one complete module first, then the next etc... at that point, why not just build separate assemblies in the first place? It would effectively be an extra accessibility domain, admittedly... but that's about all. I believe the disadvantages of this are likely to outweigh the advantages in almost all scenarios. If you really want to build a single assembly, you can always use ilmerge after building separate assemblies.
3) Well, in theory a single module could be included in multiple assemblies, but there'd be no point in doing so - it would create a very confusing system.
4) I'm not really sure what you mean. Visual Studio creates some intermediate files in the obj directory, if that's what you mean. The command line compiler doesn't leave any extra files lying around, but it may create intermediate files which it deletes on completion - I don't really know.
EDIT: I don't believe VS builds modules as an intermediate step. Compiling in Visual Studio always creates a single assembly per project, and that assembly has a single module. When you say that CLR via C# says that "modules can span assemblies" are you sure you don't mean that assemblies can span multiple modules? You can download modules of an assembly on demand, but not the other way round. If you have a specific reference, I could look it up...
You cannot create modules using VS, but you can do it using compiler. Modules are separate files on the file system, it is possible to have several modules in one assembly written in different languages.
EDIT: Also you can put rarely used classes in the separate modules. Such modules will be loaded only when classes are needed.
To preface, I've been working with C# for a few months, but I'm completely unfamiliar with concepts like deployment and assemblies, etc. My questions are many and varied, although I'm furiously Googling and reading about them to no avail (I currently have Pro C# 2008 and the .NET 3.5 Platform in front of me).
We have this process and it's composed of three components: an engine, a filter, and logic for the process. We love this process so much we want it reused in other projects. So now I'm starting to explore the space beyond one solution, one project.
Does this sound correct? One huge Solution:
Process A, exe
Process B, exe
Process C, exe
Filter, dll
Engine, dll
The engine is shared code for all of the processes, so I'm assuming that can be a shared assembly? If a shared assembly is in the same solution as a project that consumes it, how does it get consumed if it's supposed to be in the GAC? I've read something about a post build event. Does that mean the engine.dll has to be reployed on every build?
Also, the principle reason we separated the filter from the process (only one process uses it) is so that we can deploy the filter independently from the process so that the process executable doesn't need to be updated. Regardless of if that's best practice, let's just roll with it. Is this possible? I've read that assemblies link to specific versions of other assemblies, so if I update the DLL only, it's actually considered tampering. How can I update the DLL without changing the EXE? Is that what a publisher policy is for?
By the way, is any of this stuff Google-able or Amazon-able? What should I look for? I see lots of books about C# and .NET, but none about deployment or building or testing or things not related to the language itself.
I agree with Aequitarum's analysis. Just a couple additional points:
The engine is shared code for all of the processes, so I'm assuming that can be a shared assembly?
That seems reasonable.
If a shared assembly is in the same solution as a project that consumes it, how does it get consumed if it's supposed to be in the GAC?
Magic.
OK, its not magic. Let's suppose that in your solution your process project has a reference to the engine project. When you build the solution, you'll produce a project assembly that has a reference to the engine assembly. Visual Studio then copies the various files to the right directories. When you execute the process assembly, the runtime loader knows to look in the current directory for the engine assembly. If it cannot find it there, it looks in the global assembly cache. (This is a highly simplified view of loading policy; the real policy is considerably more complex than that.)
Stuff in the GAC should be truly global code; code that you reasonably expect large numbers of disparate projects to use.
Does that mean the engine.dll has to be reployed on every build?
I'm not sure what you mean by "redeployed". Like I said, if you have a project-to-project reference, the build system will automatically copy the files around to the right places.
the principle reason we separated the filter from the process (only one process uses it) is so that we can deploy the filter independently from the process so that the process executable doesn't need to be updated
I question whether that's actually valuable. Scenario one: no filter assembly, all filter code is in project.exe. You wish to update the filter code; you update project.exe. Scenario two: filter.dll, project.exe. You wish to update the filter code; you update filter.dll. How is scenario two cheaper or easier than scenario one? In both scenarios you're updating a file; why does it matter what the name of the file is?
However, perhaps it really is cheaper and easier for your particular scenario. The key thing to understand about assemblies is assemblies are the smallest unit of independently versionable and redistributable code. If you have two things and it makes sense to version and ship them independently of each other, then they should be in different assemblies; if it does not make sense to do that, then they should be in the same assembly.
I've read that assemblies link to specific versions of other assemblies, so if I update the DLL only, it's actually considered tampering. How can I update the DLL without changing the EXE? Is that what a publisher policy is for?
An assembly may be given a "strong name". When you name your assembly Foo.DLL, and you write Bar.EXE to say "Bar.EXE depends on Foo.DLL", then the runtime will load anything that happens to be named Foo.DLL; file names are not strong. If an evil hacker gets their own version of Foo.DLL onto the client machine, the loader will load it. A strong name lets Bar.EXE say "Bar.exe version 1.2 written by Bar Corporation depends on Foo.DLL version 1.4 written by Foo Corporation", and all the verifications are done against the cryptographically strong keys associated with Foo Corp and Bar Corp.
So yes, an assembly may be configured to bind only against a specific version from a specific company, to prevent tampering. What you can do to update an assembly to use a newer version is create a little XML file that tells the loader "you know how I said I wanted Foo.DLL v1.4? Well, actually if 1.5 is available, its OK to use that too."
What should I look for? I see lots of books about C# and .NET, but none about deployment or building or testing or things not related to the language itself.
Deployment is frequently neglected in books, I agree.
I would start by searching for "ClickOnce" if you're interested in deployment of managed Windows applications.
Projects can reference assemblies or projects.
When you reference another assembly/project, you are allowed to use all the public classes/enums/structs etc in the referenced assembly.
You do not need to have all of them in one solution. You can have three solutions, one for each Process, and all three solutions can load Engine and Filter.
Also, you could have Process B and Process C reference the compiled assemblies (the .dll's) of the Engine and Filter and have similar effect.
As long as you don't set the property in the reference to an assembly to require a specific version, you can freely update DLLs without much concern, providing the only code changes were to the DLL.
Also, the principle reason we
separated the filter from the process
(only one process uses it) is so that
we can deploy the filter independently
from the process so that the process
executable doesn't need to be updated.
Regardless of if that's best practice,
let's just roll with it. Is this
possible?
I actually prefer this method of updating. Less overhead to update only files that changed rather than everything everytime.
As for using the GAC, whole other level of complexity I won't get into.
Tamper proofing your assemblies can be done by signing them, which is required to use the GAC in the first place, but you should still be fine so long as a specific version is not required.
My recommendation is to read a book about the .NET framework. This will really help you understand the CLR and what you're doing.
Applied Microsoft .NET Framework Programming was a book I really enjoyed reading.
You mention the engine is shared code, which is why you put it in a separate project under your solution. There's nothing wrong with doing it this way, and it's not necessary to add this DLL to the GAC. During your development phase, you can just add a reference to your engine project, and you'll be able to call the code from that assembly. When you want to deploy this application, you can either deploy the engine DLL with it, or you can add the engine DLL to the GAC (which is another ball of wax in and of itself). I tend to lean against GAC deployments unless it's truly necessary. One of the best features of .NET is the ability to deploy everything you need to run your application in one folder without having to copy stuff to system folders (i.e. the GAC).
If you want to achieve something like dynamically loading DLL's and calling member methods from your processor without caring about specific version, you can go a couple of routes. The easiest route is to just set the Specific Version property to False when you add the reference. This will give you the liberty of changing the DLL later, and as long as you don't mess with method signatures, it shouldn't be a problem. The second option is the MEF (which uses Reflection and will be part of the framework in .NET 4.0). The idea with the MEF is that you can scan a "plugins" style folder for DLL's that implement specific functionality and then call them dynamically. This gives you some additional flexibility in that you can add new assemblies later without the need to modify your references.
Another thing to note is that there are Setup and Deployment project templates built into Visual Studio that you can use to generate MSI packages for deploying your projects. MSDN has lots of documentation related to this subject that you can check out, here:
http://msdn.microsoft.com/en-us/library/ybshs20f%28VS.80%29.aspx
Do not use the GAC on your build machine, it is a deployment detail. Visual Studio automatically copies the DLL into build directory of your application when you reference the DLL. That ensures that you'll run and debug with the expected version of the DLL.
When you deploy, you've got a choice. You can ship the DLL along with the application that uses it, stored in the EXE installation folder. Nothing special is needed, the CLR can always find the DLL and you don't have to worry about strong names or versions. A bug fix update is deployed simply by copying the new DLL into the EXE folder.
When you have several installed apps with a dependency on the DLL then deploying bug fix updates can start to get awkward. Since you have to copy to the DLL repeatedly, once for each app. And you can get into trouble when you update some apps but not others. Especially so when there's a breaking change in the DLL interface that requires the app to be recompiled. That's DLL Hell knocking, the GAC can solve that.
We found some guidance on this issue at MSDN. We started with two separate solution with no shared code, and then abstracted the commonalities to a shared assemblies. We struggled with ways to isolate changes in the shared code to impact only the projects that were ready for it. We were terrible at Open/Close.
We tried
branching the shared code for each project that used it and including it in the solution
copying the shared assembly from the shared solution when we made changes
coding pre-build events to build the shared code solution and copy the assembly
Everything was a real pain. We ended up using one large solution with all the projects in it. We branch each project as we want to stage features closer to production. This branches the shared code as well. It's simplified things a lot and we get a better idea of what tests fail across all projects, as the common code changes.
As far as deployment, our build scripts are setup to build the code and copy only the files that have changed, including the assemblies, to our environments.
By default, you have a hardcoded version number in your project (1.0.0.0). As long as you don't change it, you can use all Filter builds with the Process assembly (it only knows it should use the 1.0.0.0 version). This is not the best solution, however, because how do you distinguish between various builds yourself?
Another option is use different versions of the Filter by the same Process. You should add an app.config file to the Process project, and include a bindingRedirect element (see the docs). Whenever the Runtime looks for a particular version of the Filter, it's "redirected" to a version indicated in the config. Unfortunately, this means that although you don't have to update the Process assembly, you'll have to update the config file with the new version.
Whenever you encounter versioning problems, you can use Fuslogvw.exe (fusion log viewer) to troubleshoot these.
Have fun!
ulu