I am trying to extract a dll from a nuget package programatically and load the dll at runtime.
I want to avoid using any command line tools - I want my program to be completely self contained, and not rely on external executables.
I am trying to use the various nuget.client nuget packages listed at https://www.nuget.org/profiles/nuget, but there is no documentation for them whatsoever and I can't work out how.
I have the nupkg, and I am able to work out the location of the dll in the nupkg via a PackageReader, but I don't know how to extract the nupkg so that I can get the file out.
Edit
Thanks to the people who have pointed out that a nupkg is just a zip. I've now done the following:
var archive = new ZipArchive(downloadResourceResult.PackageStream);
var entry = archive.GetEntry(dllPath);
var assemblyLoadContext = new System.Runtime.Loader.AssemblyLoadContext(null, isCollectible: true);
var assembly = assemblyLoadContext.LoadFromStream(entry.Open());
However this throws a NotSupportedException with the following stack trace
System.IO.Compression.DeflateStream.get_Length() at System.Runtime.Loader.AssemblyLoadContext.LoadFromStream(Stream assembly, Stream assemblySymbols) at System.Runtime.Loader.AssemblyLoadContext.LoadFromStream(Stream assembly)
Here is a full method to download a nuget package and load it. It's just a POC - you'll want to configure it for your use case.
public async Task<Assembly> LoadFromNuget(string id, string version, string? nugetFeedUrl = null, CancellationToken cancellationToken = default)
{
var repository = Repository.Factory.GetCoreV3(nugetFeedUrl ?? "https://api.nuget.org/v3/index.json");
var downloadResource = await repository.GetResourceAsync<DownloadResource>();
if (!NuGetVersion.TryParse(version, out var nuGetVersion))
{
throw new Exception($"invalid version {version} for nuget package {id}");
}
using var downloadResourceResult = await downloadResource.GetDownloadResourceResultAsync(
new PackageIdentity(id, nuGetVersion),
new PackageDownloadContext(new SourceCacheContext()),
globalPackagesFolder: Path.GetTempDirectory(),
logger: _nugetLogger,
token: cancellationToken);
if (downloadResourceResult.Status != DownloadResourceResultStatus.Available)
{
throw new Exception($"Download of NuGet package failed. DownloadResult Status: {downloadResourceResult.Status}");
}
var reader = downloadResourceResult.PackageReader;
var archive = new ZipArchive(downloadResourceResult.PackageStream);
var lib = reader.GetLibItems().First()?.Items.First();
var entry = archive.GetEntry(lib);
using var decompressed = new MemoryStream();
entry.Open().CopyTo(decompressed);
var assemblyLoadContext = new System.Runtime.Loader.AssemblyLoadContext(null, isCollectible: true);
decompressed.Position = 0;
return assemblyLoadContext.LoadFromStream(decompressed);
}
You'll have to implement or use a version of the Nuget ILogger to download the nupkg.
You can rename the nuget extension to zip extension and should be able to extract to folder.
Now, you can get the dlls from the extracted folder.
Related
I have to extract PDB signature from both .pdb and .dll file.
That's the code I use to extract it from .pdb file. Unfortunately I haven't found similiar way of extracting it from a DLL.
public static string GetPdbSignature(string pdbFilePath)
{
using (var pdbFileStream = File.OpenRead(pdbFilePath))
{
var metadataProvider = MetadataReaderProvider.FromPortablePdbStream(pdbFileStream);
var metadataReader = metadataProvider.GetMetadataReader();
var id = new BlobContentId(metadataReader.DebugMetadataHeader.Id);
return $"{id.Guid.ToString("N")}ffffff";
}
}
I found out that a PeNet nuget package can be used to perform the extraction, yet I'd prefer to achieve that without installing external dependancies.
Also, I managed to find the desired data using a dotPeek (screen), but as I need to resolve the problem programatically it doesn't solve my issue either.
I'd apreciate any hint how to aproach that problem. Either by using some built in dotnet mechanism or by some smart low level byte extraction.
I managed to find an official Microsoft's package - Microsoft.Diagnostics.Tracing.TraceEvent that contains PEFile class allowing to extract the exact data I need.
public static string GetDllSignature(string dllFilePath)
{
var peFile = new PEFile.PEFile(dllFilePath);
peFile.GetPdbSignature(out string pdbName, out Guid pdbGuid, out int pdbAge);
return $"{pdbGuid.ToString("N")}ffffff";
}
UPDATE:
Actually there also is a PEReader class in System.Reflection.PortableExecutable namespace that makes the reading possible using only the system libraries. However it requires some knowledge of the portable executable format, as the PEReader does not provide an explicit, user-friendly method for extracting the signature, instead it just allows getting all kind of data that the PE file contains.
public static string GetDllSignatureV2(string dllFilePath)
{
using (var pdbStream = File.OpenRead(pdbPath))
using (var peReader = new PEReader(pdbStream))
{
var debugDirectory = peReader.ReadDebugDirectory().First(entry => entry.Type == DebugDirectoryEntryType.CodeView);
var codeViewData = peReader.ReadCodeViewDebugDirectoryData(debugDirectory);
return $"{codeViewData.Guid.ToString("N").Replace("-", string.Empty)}FFFFFFFF".ToUpper();
}
}
My goal is to test this code to make sure that Stanford Core NLP installed properly.
First I installed StanfordCOreNLP package using NuGet package manager and then I downloaded a zip file that contained a jar file that needed to be installed using jar -xf command , and then I ran the code.
At (var pipeline = new StanfordCoreNLP(props);)
I'm getting an error that says:
edu.stanford.nlp.io.RuntimeIOException: Error while loading a tagger model(probably missing model file)"
Inner Exception IOException:Unable to open"edu/stanford/nlp/models/pos-tagger/english-left3words/english-left3words-distsim.tagger" as class path, filename or URL
var jarRoot = #"D:/VisualStudioProjects/C#MachineLearningProjects/Chapter3TwiterSentiment/CoreNLPTest2/CoreNLPTest2/edu/stanford/nlp/models/pos-tagger";
var text = "We're going to test our CoreNLP instalation!!";
Properties props = new Properties();
props.setProperty("annotators", "tokenize, ssplit, pos, lemma, ner, parse, dcoref");
props.setProperty("ner.useSUTime", "0");
var curDir = Environment.CurrentDirectory;
Directory.SetCurrentDirectory(jarRoot);
var pipeline = new StanfordCoreNLP(props);
Directory.SetCurrentDirectory(curDir);
var annotation = new Annotation(text);
pipeline.annotate(annotation);
using (var stream = new ByteArrayOutputStream())
{
pipeline.prettyPrint(annotation, new PrintWriter(stream));
Console.WriteLine(stream.toString());
stream.close();
}
Console.ReadKey();
Please follow the below steps:
Step 1: Download Core NLP
Step 2: Unzip d:\stanford-corenlp-full-2018-10-05
Step 3: Unzip d:\stanford-corenlp-full-2018-10-05\stanford-corenlp-3.9.2-models.jar
Step 4: Change var jarRoot = #"d:/stanford-corenlp-full-2018-10-05/stanford-corenlp-3.9.2-models";
Step 5: Change props.setProperty("ner.useSUTime", "0"); to props.setProperty("sutime.binders", "0")
I want to programmatically install a NuGet package to a project, and update the .csproj file, and the packages.config file.
I am using the official Nuget.core framework which source code is available here: https://github.com/NuGet/NuGet2
I am not using the NuGet package: https://www.nuget.org/packages/NuGet.Core/
But the source code found on GitHub to be able to do some debugging.
Note: I am using the version 2.11 and not the 2.13
I am able to download a package at a desired directory and update the packages.config file:
// ---- Download and install a package at a desired path ----
string packageID = "Newtonsoft.json";
var sourceUri = new Uri("https://packages.nuget.org/api/v2");
// Return an IPackage
var package = GetNugetPackage(packageID, sourceUri);
IPackageRepository sourceRepository = PackageRepositoryFactory.Default.CreateRepository(sourceUri.ToString());
string packagesPath = "../../TestFiles/packages";
PackageManager packageManager = new PackageManager(sourceRepository, packagesPath);
packageManager.InstallPackage(packageID, SemanticVersion.Parse(package.Version.ToFullString()));
// ---- Update the ‘packages.config’ file ----
var packageReferenceFile = new PackageReferenceFile("../../TestFiles/packages.config");
// Get the target framework of the current project to add --> targetframework="net452" attribute in the package.config file
var currentTargetFw = Assembly.GetExecutingAssembly().GetCustomAttributes(typeof(TargetFrameworkAttribute), false);
var targetFrameworkAttribute = ((TargetFrameworkAttribute[])currentTargetFw).FirstOrDefault();
// Update the packages.config file
packageReferenceFile.AddEntry(package.GetFullName(), SemanticVersion.Parse(package.Version.ToFullString()), false, new FrameworkName(targetFrameworkAttribute.FrameworkName));
Now I need to update the .csproj and here is the tricky part...
Here's what I tried so far:
string csprojFilePath = "../../TestFiles/test.csproj";
var project = new MSBuildProjectSystem(csprojFilePath);
string pathToAnExistingNugetPackageDll = "../../TestFiles/packages/Newtonsoft.json/lib/net45/Newtonsoft.json.dll"
project.AddReference(pathToAnExistingNugetPackageDll, Stream.Null);
project.Save();
This piece of code update the .csproj file, it add a new reference node like this:
<Reference Include="Newtonsoft.json">
<HintPath>..\packages\Newtonsoft.json\lib\net45\Newtonsoft.json.dll</HintPath>
</Reference>
But I need a complete reference node like this:
<Reference Include="Newtonsoft.json, Version=9.0.8, Culture=neutral, PublicKeyToken=b03f4f7d11d50a3a, processorArchitecture=MSIL">
<HintPath>..\packages\Newtonsoft.json\lib\net45\Newtonsoft.json.dll</HintPath>
<Private>True</Private>
</Reference>
How can I do it ?
Thanks to #MattWard and #bashis, i've written my own code generator:
So, to add properly a reference into a given .csproj here's what I do:
var CsprojDoc = new XmlDocument();
CsprojDoc.LoadXml("*your_csproj_content*");
var Nsmgr = new XmlNamespaceManager(CsprojDoc.NameTable);
Nsmgr.AddNamespace("x", "http://schemas.microsoft.com/developer/msbuild/2003");
IPackage packageInfos = GetNugetPackage(packageId, packageRepositoryUri);
XmlNode referenceNode = CsprojDoc.CreateNode(XmlNodeType.Element, "Reference", XmlNamespaceValue);
XmlAttribute includeAttribute = CsprojDoc.CreateAttribute("Include");
var targetFwProfile = CsprojDoc.SelectSingleNode("//x:TargetFrameworkProfile", Nsmgr);
string targetFrameworkProfile = string.Empty;
if (!string.IsNullOrEmpty(targetFwProfile?.InnerXml))
{
targetFrameworkProfile = targetFwProfile.InnerXml;
}
var targetFwAttribute = GetTargetFrameworkFromCsproj();
Regex p = new Regex(#"\d+(\.\d+)+");
Match m = p.Match(targetFwAttribute.FrameworkName);
Version targetFwVersion = Version.Parse(m.Value);
// Get the package's assembly reference matching the target framework from the given '.csproj'.
var assemblyReference =
packageInfos.AssemblyReferences
.Where(a => a.TargetFramework.Identifier.Equals(targetFwAttribute.FrameworkName.Split(',').First()))
.Where(a => a.TargetFramework.Profile.Equals(targetFrameworkProfile))
.Last(a => (a.TargetFramework.Version.Major.Equals(targetFwVersion.Major) && a.TargetFramework.Version.Minor.Equals(targetFwVersion.Minor)) ||
a.TargetFramework.Version.Major.Equals(targetFwVersion.Major));
DownloadNugetPackage(packageInfos.Id, packageRepositoryUri, packagesFolderPath, packageInfos.Version.ToFullString());
string dllAbsolutePath = Path.GetFullPath($"{packagesFolderPath}\\{packageInfos.GetFullName().Replace(' ', '.')}\\{assemblyReference.Path}");
var assemblyInfos = Assembly.LoadFile(dllAbsolutePath);
includeAttribute.Value = $"{assemblyInfos.FullName}, processorArchitecture=MSIL";
referenceNode.Attributes.Append(includeAttribute);
XmlNode hintPathNode = CsprojDoc.CreateNode(XmlNodeType.Element, "HintPath", XmlNamespaceValue);
XmlNode privateNode = CsprojDoc.CreateNode(XmlNodeType.Element, "Private", XmlNamespaceValue);
hintPathNode.InnerXml = $"$(SolutionDir)\\packages\\{assemblyReference.Path}";
privateNode.InnerXml = "True";
referenceNode.AppendChild(hintPathNode);
referenceNode.AppendChild(privateNode);
var itemGroupNode = CsprojDoc.SelectSingleNode("//x:Project/x:ItemGroup/x:Reference", Nsmgr).ParentNode;
itemGroupNode.AppendChild(referenceNode);
Here's my DownloadNugetPackage method:
private static void DownloadNugetPackage(string packageId, Uri repoUri, string packagesFolderPath, string version)
{
IPackageRepository packageRepository = PackageRepositoryFactory.Default.CreateRepository(repoUri.ToString());
PackageManager packageManager = new PackageManager(packageRepository, packagesFolderPath);
packageManager.InstallPackage(packageId, SemanticVersion.Parse(version));
}
My GetTargetFrameworkFromCsproj
public static TargetFrameworkAttribute GetTargetFrameworkFromCsproj()
{
XmlNode targetFrameworkNode = CsprojDoc.SelectSingleNode("//x:TargetFrameworkVersion", Nsmgr);
return new TargetFrameworkAttribute($".NETFramework, Version={targetFrameworkNode.InnerXml}");
}
And my GetNugetPackage method:
public static IPackage GetNugetPackage(string packageId, Uri repoUri, string version = null)
{
IPackageRepository packageRepository = PackageRepositoryFactory.Default.CreateRepository(repoUri.ToString());
IPackage package;
if (!string.IsNullOrEmpty(version))
{
package = packageRepository.FindPackagesById(packageId).SingleOrDefault(p => p.Version.ToFullString().Equals(version));
}
else
{
package = packageRepository.FindPackagesById(packageId).SingleOrDefault(p => p.IsLatestVersion);
}
return package;
}
Note: This time, i'm using the official NuGet package: Nuget.core 2.14: https://www.nuget.org/packages/NuGet.Core/
Note 2: When I add a new Reference node, in the processorArchitecture attribute, I've hard coded the value: MSIL
If you want to test it, you might tweak it a bit.
This work fine for me.
(Assuming you are writing the code generator on your own)
I am not sure this is the best solution but you could try something like this:
var assembly = Assembly.LoadFile("path_to_dll");
string info = assembly.FullName; // contains something like "AssemblyName, Version=0.0.0.0, Culture=neutral, PublicKeyToken=foobar"
which will add some overhead by loading the assembly, but will presumably do the job.
I'm looking for a way to run code by executing the following steps:
Receiving a list of NuGet packages (a list of tuples ("package name", "package version", "path to main class").
Retrieving them in a local directory (cf code sample #1)
Loading them in my program at run-time
Running the main classes by introspection (cf code sample #2)
By now I am struggling with the third step. I can't find out how to load my package at run-time.
My main question are:
How can I find out in which folders were stored the retrieved packages?
How can I load the content of those directories into my program?
Code Sample #1:
private static void getPackageByNameAndVersion(string packageID, string version)
{
IPackageRepository repo =
PackageRepositoryFactory.Default
.CreateRepository("https://packages.nuget.org/api/v2");
string path = "C:/tmp_repo";
PackageManager packageManager = new PackageManager(repo, path);
Console.WriteLine("before dl pkg");
packageManager.InstallPackage(packageID, SemanticVersion.Parse(version));
}
Code sample #2:
private static void loadByAssemblyNameAndTypeName(string assemblyName, string typeName)
{
AppDomain isolationAppDomain = AppDomain.CreateDomain("tmp");
object a = isolationAppDomain.CreateInstanceAndUnwrap(assemblyName, typeName);
Type x = a.GetType();
MethodInfo m = x.GetMethod("Main");
m.Invoke(a, new object[] { });
}
Grab a cup of coffee :)
Downloading the nuget package?
Nuget.Core (nuget package) is a good choice, and here is a snippet of code that I have that should be able to download a nuget package by id and version
var repo = PackageRepositoryFactory.Default
.CreateRepository("https://packages.nuget.org/api/v2");
string path = "c:\\temp";
var packageManager = new PackageManager(repo, path);
packageManager.PackageInstalled += PackageManager_PackageInstalled;
var package = repo.FindPackage("packageName", SemanticVersion.Parse("1.0.0"));
if (package != null)
{
packageManager.InstallPackage(package, false, true);
}
Notice that I plugged an event handler to the PackageInstalled event of the PackageManager class.
How do we load an assembly in an isolated app domain?
Since reflection API does not provide a way to load an assembly in a specific domain, We will create a proxy class that act as a loader in our isolated domain:
public class TypeProxy : MarshalByRefObject
{
public Type LoadFromAssembly(string assemblyPath, string typeName)
{
try
{
var asm = Assembly.LoadFile(assemblyPath);
return asm.GetType(typeName);
}
catch (Exception) { return null; }
}
}
And now, is how to put it all together?
Here comes the complex part:
private static void PackageManager_PackageInstalled(object sender,
PackageOperationEventArgs e)
{
var files = e.FileSystem.GetFiles(e.InstallPath, "*.dll", true);
foreach (var file in files)
{
try
{
AppDomain domain = AppDomain.CreateDomain("tmp");
Type typeProxyType = typeof(TypeProxy);
var typeProxyInstance = (TypeProxy)domain.CreateInstanceAndUnwrap(
typeProxyType.Assembly.FullName,
typeProxyType.FullName);
var type = typeProxyInstance.LoadFromAssembly(file, "<KnownTypeName>");
object instance =
domain.CreateInstanceAndUnwrap(type.Assembly.FullName, type.FullName);
}
catch (Exception ex)
{
Console.WriteLine("failed to load {0}", file);
Console.WriteLine(ex.ToString());
}
}
}
Notice that this method is the event handler that gets executed after downloading the nuget package
Also
Note that you will need to replace <KnownTypeName> with the expected type name coming from the assembly (or maybe run a discovery of all public types in the assembly)
Worth noting that I haven't executed this code myself and cannot guarantee that it will work out of the box, and still might need some tweaking. but Hopefully it is the concept that allows you to solve the problem.
Don't do that! You are probably trying to load NuGet content at a customers computer to save some space on distribution of your software. Isn't it that?
The common recommended approach is to download the NuGet content as the second step of an automated build (after downloading the source code), build the software and run the automated tests with the NuGet content you have downloaded. And then distribute the build with the NuGet content you have tested as the complex whole unit.
I am attempting to create a PCL assembly using Roslyn (Microsoft.CodeAnalysis).
I'm referencing PCL assemblies located at "C:\Program Files (x86)\Reference Assemblies\Microsoft\Framework.NETPortable\v4.5\Profile\Profile259". Here is my code that compiles the actual assembly.
var assemblyName = string.Concat("ODataHQ.SDK.", accountKey, ".dll");
var source = GenerateAccountSource(accountKey, workspace);
var assemblyInfoSource = GetAssemblyInfo(assemblyName);
var assemblyTree = CSharpSyntaxTree.ParseText(assemblyInfoSource);
var tree = CSharpSyntaxTree.ParseText(source);
// Lets add PCL framework assemblies.
var frameworkFiles = new[] {"mscorlib.dll", "Microsoft.CSharp.dll", "System.dll", "System.Core.dll", "System.Runtime.dll"};
var references = frameworkFiles
.Select(file => Path.Combine(Settings.PCLProfilePath, file))
.Select(fullPath => MetadataReference.CreateFromFile(fullPath))
.Cast<MetadataReference>()
.ToList();
// Lets add third-party dependent assemblies.
references.AddRange(Directory.GetFiles(Settings.SDKDependencyPath, "*.dll")
.Select(file => MetadataReference.CreateFromFile(file)));
var options = new CSharpCompilationOptions(OutputKind.DynamicallyLinkedLibrary);
var compilation = CSharpCompilation.Create(assemblyName, new [] { assemblyTree, tree }, references, options);
var ms = new MemoryStream();
var result = compilation.Emit(ms);
if (result.Success)
{
// Reset stream position to read from beginning.
ms.Position = 0;
return ms;
}
// Destroy memory stream since it didn't compile successfully.
ms.Dispose();
var firstError = result.Diagnostics
.Where(d => d.Severity == DiagnosticSeverity.Error)
.Select(d => d.GetMessage())
.FirstOrDefault();
throw new CompilationException(firstError);
Here is my GetAssemblyInfo method:
private string GetAssemblyInfo(string assemblyName)
{
return #"using System.Reflection;
using System.Runtime.Versioning;
[assembly: AssemblyTitle(""" + assemblyName + #""")]
[assembly: AssemblyVersion(""1.0.*"")]
[assembly: AssemblyFileVersion(""1.0.*"")]
[assembly: TargetFramework("".NETPortable,Version=v4.5,Profile=Profile259"", FrameworkDisplayName="".NET Portable Subset"")]";
}
I take the assembly that is generated and save it to disk. Then reference it in another console app project. However, when I run the console app and try to use a type from the dynamically generated PCL assembly, I get the following error.
An unhandled exception of type 'System.IO.FileNotFoundException' occurred in mscorlib.dll
Additional information: Could not load file or assembly 'ODataHQ.SDK.dvester.dll, Version=1.0.5635.36199, Culture=neutral, PublicKeyToken=null' or one of its dependencies. The system cannot find the file specified.
Can someone help me figure out what's going wrong?
Check the FusionLog property of the exception (you may need to enable that in the registry).
This will tell you why it failed.