Develop Reference

How to trace resolving process of nuget package references

I have a solution with multiple projects in it (~400). A lot of them are using newtonsoft.json library. Some of them are using different version of the library. In a host project for web app, there is no direct reference to newtonsoft.json. Also, I believe, there are dependencies to packages that depend on newtonsoft.json. However, in bin folder of that project I began getting newtonsoft.json 9.0 instead of newtonsoft.json 11.0 as it used to be. And it causes load exception in runtime due to invalid version. If I include direct reference to newtonsoft.json 11.0, it still puts version 9.0 in the bin folder even if I clear all bin,obj folders. I use PackageReference for managing dependencies, and everything is in .net 4.6.1; I use binding redirects to resolve issues with different versions of the same library.
My question is if there is a way to diagnose how particular dll of package ref appears in bin folder? I would like to see some sort of comprehensive trace of dependency resolution so that I can fix it without using "trial and error" approach.
Upd.
Actually, thanks guys for pointing to structured logging. You are the best! So the issue was that one of the project had <OutputPath> pointing to the bin folder of the host project. So when the project was built, it was overriding binaries in the host project. Apparently, build order has changed due to continuous reference shuffling and the project with wrong <OutputPath> started building last. To find this out 1) I Newtonsoft.Json was recorded in DoubleWrites section 2) analyzing location from double writes I found out in _CopyFilesMarkedLocal section
Copying file from "\VenomousProject\bin\Debug\Newtonsoft.Json.dll" to "HostProject\bin\Debug\Newtonsoft.Json.dll".
That was it.

You can try MSBuild Binary and Structured Log Viewer (https://msbuildlog.com/)
build the project from command line with msbuild -bl - you will get msbuild.binlog
open the binlog with the log viewer and use the dll name as a search term
inspect all records related to the dll and backtrack a place (project and msbuild' target) where the wrong file version is taken.

With PackageReference, NuGet will write a file named project.assets.json in the project's obj folder. This file is used by the rest of the build to determine what files from packages should be included in the build, but it also contains a list of all the packages that were selected, what version was selected, and what dependencies each package has (package id and version). This is the closest thing to dependency resolving debugging that NuGet has.
In the file, search for "Newtonsoft.Json/, and you should find which version of Newtonsoft.Json that NuGet selected. Remove the / and replace it with ", and you can find all the packages that have dependencies on Newtonsoft.Josn. Search for newtonsoft.json.dll to find all the packages that ship that dll in its package (sometimes package authors perfer to ship multiple dlls in their package, rather than adding dependencies, which prevents NuGet from being able to version selection.
If there are multiple packages with which contain a dll with the same filename, NuGet will tell MSBuild about all of them, and it's up to MSBuild to select which one to use (pass to the compiler and copy to the bin/publish directory). As #Serg wrote in their answer, you can use binlogs (with the -bl argument on any MSBuild command, including dotnet restore or dotnet build). NuGet's inner workings are not output to MSBuild, so when your package graph has multiple packages that list Newtonsoft.Json as a dependency, it won't tell you why NuGet chose a specific version, but binlogs are very useful at debugging other build related issues.

How can my NuGet package add another package reference at runtime based on the current runtime identifier?

I'm creating multiple OS-specific NuGet packages. Each one of them contains a fairly large native process. I can't combine them in a single NuGet package, it'll be too big. So it looks like this:
- MyProject.win-x86-x64
- MyProject.macos-x86-x64
- MyProject.ubuntu1804-x86-x64
- MyProject.ubuntu1804-arm64
- MyProject.debian10-x86-x64
- ...
I'd like to create another netstandard2.0 package that references the right OS-specific package based on the current OS runtime identifier, using a .targets file included in the build and buildTransitive package directory.
So far the best I can do is use $([MSBuild]::IsOsPlatform()) which only tells if the consumer is targeting Windows, OSX or Linux. It doesn't provide the current Linux distribution.
I know it's also possible to retrieve the runtime identifier using the source code of the soon deprecated Microsoft.DotNet.PlatformAbstractions package, as suggested by Dan Moseley from MS. However, I have no idea how I cound integrate this code into my custom .targets file.
Does anybody knows how I could reference the right OS-specific package in a custom .targets file?

could not load file or assembly exception NET 5/6 of NuGet package for NET Standard

I'm at the end of my rope trying to figure out why my working DLL cannot load a NuGet package that it could always load before (like for a whole year). My system has 70 NET 6 projects and half a dozen of them use the same NuGet package to read XML files. Recently (a few months ago) I upgraded everything to NET 6, and even after that upgrade, everything worked fine.
But now, one of my C# class libraries generates an exception in a constructor because it can't find the NuGet package that sits right beside it in the filesystem app folder where everything is loaded from. The error message was:
.ctor Could not load file or assembly xxx blah blah. The system could not find the file specified.
For what it is worth, here is a trace of the build options in the batch file that I have been using for months.
Deleting hscore\hscore\bin Deleting hscore\hscore\obj Deleting
hscore\hscore\packages "C:\Program Files\Microsoft Visual
Studio\2022\Community\MSBuild\Current\Bin\msbuild.exe" -nologo -m
-V:minimal -t:clean;restore;publish /p:Platform=AnyCPU /p:PlatformTarget=AnyCPU /p:Configuration=Debug
/p:TargetFramework=net6.0-windows7.0 /p:RuntimeIdentifier=win-x64
/p:SelfContained=false /p:PublishProtcol=FileSystem
/p:DeleteExistingFiles=true /p:PublishDir=c:\dev\holding\core.plt
-nowarn:MSB3305 c:\dev\products\hscore\hscore\hscore.csproj
Determining projects to restore... Restored
c:\dev\products\hscore\hscore\hscore.csproj (in 874 ms).
hscore -> c:\dev\products\hscore\hscore\bin\Debug\net6.0-windows7.0\win-x64\hscore.dll
hscore -> c:\dev\holding\core.plt\
I have done the following to debug it, without success:
deleted the bin/obj folders of the failing class library
checked the target framework of all projects (= net6.0-windows7.0)
checked the runtime identifier of all projects (= win-x64)
checked the build configuration (Debug, AnyCPU)
removed and re-added the NuGet package (with a specific version)
the Nuget package depends on NET Standard, compatible with NET 6
and I use the same version in other NET 6 programs that are working fine
restored and rebuilt the DLL
published it to the destination folder where it is run from
checked that the expected Nuget package is there (it was)
and yet it still fails to find the Nuget package
I tried to trace DLL loads of the process with the SysInternals Process Monitor, but I am not good enough to do that (if it is even possible). I can filter events to see the parent app process load, but the failing DLL is called by the parent process and I don't know how to see the search paths it is using to find the Nuget package.
I want to believe that the problem is a version mismatch between NET 5 and NET 6 and the version of the Nuget package sitting in the folder, because I once had a problem like that. For example, if I had a NET Framework version of the Nuget package, it would not be "found" by a NET 5 DLL that wanted to load it. (I am NOT saying that is my problem, because I have no NET Framework projects anymore.)
The most recent thing I have been working on is the build system and options. I can build the whole system with batch files or with parallel builds (when the system is working). I use the same msbuild options in both cases, and the batch method has been reliable and shows no build errors, no publish errors, all the correct build options, and no warnings or errors whatsoever.
The Nuget package pulls in 10 or 20 dependencies, but that is all automatic, and the package works with other console programs and apps that are working fine. So, I don't think there is anything wrong with Nuget package. All the working apps use the same Nuget package from the same runtime folder. I don't know why this one DLL is having a problem. AND I have not changed anything in the failing DLL class library for months (and it has been working fine).
Does anyone have suggestions on what else I might try to debug the problem and get the system working again? Thank you.
UPDATE - New version and build, but still a runtime failure.
Just to be sure, I upgraded the Nuget package that could not be found to the latest version and replaced all references in my 70 projects to the new version. Then I rebuilt the system (no warnings, no errors). But that did not make a difference. Now the new version cannot be found.
All unit tests of the code in question work flawlessly in VStudio, presumably because VS loads the Nuget package properly. I am mystified and would appreciate thoughts on how to proceed. Thank you.
UPDATE 2 - installed 'dotnet-trace' and traced DLL loads
I found a page that described how to dotnet tool install --global dotnet-trace and then captured a trace of my app trying to find the elusive Nuget package that is sitting right beside the requesting DLL. Although I could my app loading various app DLLs (and tons of system DLLs) with found pathnames, the trace showed nothing useful about the Nuget package.
The trace entries show the system searching in stages FindInLoadContext, then ApplicationAssemblies, then AppDomainAssemblyResolveEvent. After that, the system throws an exception because it can't find the Nuget package.
I would have thought the system FindInLoadContext would find it in the same folder as the executing and requesting assemblies, but it seems not so.
Ideas? I'm lost. And everything used to work fine.
UPDATE 3 - A small console app calls the DLL and it finds the Nuget package
The problem scenario in this question is: WindowsFormsApp -> loads MyCore.dll and calls new MyCore.MyObject(), which in the constructor tries to read some XML files using MyUtils.dll, which tries to reference the mystery Nuget package for reading XML files. MyUtils.dll cannot find the package, no matter what I do.
I wrote a small console program to load MyUtils.dll and called the exact same MyUtils.ReadMyXmlFiles API to read the files. The console app (actually, the system assembly loader) correctly found the Nuget package and read the XML files properly. This all occurred in the same folder that contains the Forms app and the Nuget package.
The only difference now is that the WindowsForms app calls an intermediate DLL that calls a constructor that references MyUtils.dll.
Could the intermediate DLL + constructor call be changing the Assembly Load Context (FindInLoadContext search rules) for the Nuget package load operation? Very strange.

The overall problem was that my Windows Forms app failed during boot (in the Form_Load event) because a Nuget package could not be found to read some XML configuration files.
Four assemblies were involved in the problem. Assembly1 (the app) called Assembly2 (.. new Assembly2Object()), whose constructor called an Assembly3 method (utilities.ReadXMLFile) which called Assembly4 (NugetPackage.XMLReader).
You can see that only Assembly4 references the Nugetpackage which could not be found. Normally, there should be no reason for Assemblies 1 or 2 to know about the existence of the NugetXmlReader used by Assembly3. But, the exceptions and assembly load traces of dotnet-trace and procmon clearly showed that Assembly3 (utilities.ReadXmlFile) could not find the NugetPackage at runtime, even though the right package was in the folder beside the executing assemblies (1->3).
The "solution" for my case was to add a Nuget package dependency to
Assembly1 (the WinFormsApp). As soon as I did that, Assembly3 (utilities.ReadXmlFile) could find Assembly4 (Nugetpackage).
My working theory is that during a WinForms boot sequence (including the Form_Shown event), the AssemblyLoadContext from Assembly1 is used to look up all assemblies in any call chain (like Assembly2 calling Assembly3 calling NugetPackage4).
Because the load context of Assembly1 has no reference to NugetPackage, and because the .deps.json file for Assembly1 is present (the doc says it is used to form the load context), Assembly3 - using the assembly load context for Assembly1 - could not find the NugetPackage.
When I added a dependency to Assembly4 (Nugetpackage) to Assembly1, then the assembly load context from Assembly1 was used by the System AssemblyLoader FindInLoadContext phase to search for (and find) the NugetPackage4 wanted by (the referencing assembly) Assembly3.
I used to think that each assembly in a calling chain would have its own assembly load context used by AssemblyLoad/FindInLoadContext. But I think that no longer. At least for my case of a Windows Forms app booting up, the assembly load context from the top-level app is being used to look up Nuget packages far down the calling chain.
It's worth repeating that my little test console apps always found the Nuget package without adding the package as a dependency to the top-level console program. I wrote console programs to test the calling chain from the bottom up: Test1) console calls Nuget directly - found; Test2) console (w/o Nuget dependency) calls Assembly3 - Nuget found; (Test3) console (w/o Nuget dependency) calls Assembly2 - Nuget found.
Then I wrote a skeleton WindowsFormsTestApp to call Assembly2 (just like the console program did) - Nuget NOT found. When I added a Nuget dependency to the WindowsFormsTestApp - Nuget was found by Assembly3.
The final step was to add a Nuget package dependency to the WinFormsApp that started this whole mess. Presto! Assembly3 found the Nuget package immediately.
My (unproven) belief is that somewhere along the line of upgrades from NET 5 - NET 6 (several SDK versions of each one), the assembly load context rules changed somehow. I could be wrong, but I lean toward this belief because my code from App->Assembly2->Assembly3->Nuget did not change during several months of successful operation. And just a couple of weeks ago after another NET 6 SDK upgrade, things broke.
Hopefully, this record might help someone someday. It seems completely unintuitive to me to add a Nuget package dependency to the top-level Forms app to help Assembly3 find a Nuget package sitting in the same folder as all the other assemblies.
After all this, I am coming around to the idea that the top-level app must/should include dependencies on anything the app ever calls, including packages used by dependent assemblies. (Although I am still puzzled as to why the code worked for months before without the Nuget package dependency.)

MSBuild -t:pack ProjectReference

I have a C# solution that contains two net472 projects: Foo.csproj, and Bar.csproj.
The projects are in classic format (no SDK version).
The projects use PackageRefernce as package management.
Foo.csproj depends on Bar.csproj.
Bar.csproj isn't a nuget package.
I want to create a nuget package of Foo.
I followed this instruction, and I execute the command msbuild -t:pack Foo.csproj, that produces the Foo.nupkg file.
The problem is that Foo.nupkg contains Foo.dll, but it doesn't contain Bar.dll.
So, when I try to install via NuGet the package Foo, I receive the following error:
Unable to resolve dependency 'Bar'.
What I'm missing?
I hope I have provided all the necessary info.

I had the same problem as explained above.
My goal was to automate Azure Devops nuget package generation of two dependant assemblies and publishing one NuGet package.
One of the steps in the Azure pipeline, actually includes execution of msbuild with -t:pack arguments.
After reading this thread: about advanced technics of creating NuGet packages, I've concluded that it's better to have 2 different packages.
So I have ended up with modifiying inital pipeline job with additional 2 steps ("set assembly manifest data" and "Copy files") as on the picture.
One set (of these steps) for A assembly and the other set for B assembly.
At the end result was two published Nuget packeges to the feed, where installation of A package in another project resulted auto installation of dependent B referenced package.
Hope this will help further others.

loading multiple.net assemblies with same AssemblyVersion from a plugin

I'm in dll hell.
I'm building a plugin for a huge, ancient and very powerful software suite called ANSYS. They have a plugin framework. I had hoped that they would magically handle everything for me via AssemblyContexts or AppDomains or some other clever dotnet device that I don't understand. They do not.
The result is that I've created an application that depends on GRPC.core 1.16.0 via nuget. I wrote a little application that drives my plugin with a winform host. It loads and works perfectly, finding my library in ~/myproject/bin/debug/grpc.core.1.1.16.dll that exists right beside the class-library that is my plugin, no problem.
When I run my plugin in the ANSYS process space, which happens to also depend on grpc 1.0.0.0, the linker finds C:\Program FIles\ANSYS\...\WIN64\grpc.core.dll. No Good.
One odd thing about the Nuget GRPC package is that it adds a reference with a "reference version" of 1.0.0.0, where most other nuget packages have their reference version match the nuget package version. If i manually change the reference version the compiler wont find the library.
<Reference Include="Grpc.Core, Version=1.0.0.0, Culture=neutral, PublicKeyToken=d754f35622e28bad">
<HintPath>..\packages\Grpc.Core.1.16.1\lib\net45\Grpc.Core.dll</HintPath>
</Reference>
edit: the key is in the above line. The Nuget published Grpc.core artifact is at AssemblyInformationVersion=1.16.1.0, AssemblyFileVersion=1.16.1.0, AssemblyVersion=1.0.0.0. I logged this as a request against GRPC. More Below.
Thus I need to tell the runtime linking facilities not to use grpc.core...dll found in ANSYS's own binary directoryWhats more, there is exactly one dll (and its dependents) that I wish to load from my parent processes context: and that's ANSYS API dlls themselves, which are probably already in the GAC. In my project I've included this as a non-nuget reference with "build action: do not copy" selected.
So my questions:
is there something simple and easy I can do at runtime to tell the runtime-linker "when somebody loads a type from an assembly you think should be grpc.core, do not load 1.0.0.0, find 1.16.0.0 exactly"?
the runtime was already matching the needed library by "strong
name". The problem is that the 1.16.0 is a misnomer. That version
string was informational, but the assembly itself was version
1.0.0.0. Fusion was loading the library I wanted by exact match already.
is there something smarter I can do with appdomains or contexts or another C# device to explictly enter some kind of nested scope? Could I go so far as to log this as a bug in ANSYS's API?
I've tried digging into this myself, but I'm not a dotnet expert and finding out whether I'm looking at a nuget package configuration option --which isn't relevant to me, or an old-fashioned dotnet runtime option, has been very tricky.
update 1:
I've tried using AppDomain.CreateDomain, and it does indeed solve my problem, but it also requires me to provide a marshalling strategy for the already-loaded API objects. In other words, if you're programming against a plugin framework that has an api similar to:
public void DoMyPluginsFunctionality(ApiProvidedInputContext context){
var myPlugin = AppDomain.Create(
strongName: "MyCompany.MyPlugin.; Version=1.2.3.4 ...",
baseDirectory: "C:\\Program Files\\MyPlugin\\bin"
)
//success! MyCompany.MyPlugin loads the version of GRPC I want!
myPlugin.unWrapAsDynamicProxy().doFunctionality(context)
//error: No marshalling strategy and/or not serializable and/or swizzling errors
}
Then the runtime will require you to marshall (serialize) the context variable, because .net will not let you share memory across AppDomain boundaries.
So my new question:
- given I cant use AppDomains myself
- given that Grpc.core is always published as AssemblyVersion=1.0.0.0
What are my options?
Stop using newer features of GRPC.core and live in fear of my parent processes dependencies
use a strategy similar to shading. Is there something like shading in the .net world?
Edit the published binary's version metadata. Can I dynamically edit a published binaries version?
rebuild GRPC myself with the version string updated --effectively a private fork of GRPC.
update 2:
The GRPC build system seems like its quite large and well maintained, so I'm hoping I can simply build it and change a vcproj file to include an updated version string.
Unfortunately it also seems quite complex, and I haven't quite got the targeting/cross-compiling (x64 targeting x86) worked out.