I don't understand the first part of the last sentence on packages from google's Python protobuf docs:
The .proto file starts with a package declaration, which helps to prevent naming conflicts between different projects. In Python, packages are normally determined by directory structure, so the package you define in your .proto file will have no effect on the generated code. However, you should still declare one to avoid name collisions in the Protocol Buffers name space as well as in non-Python languages.
It's actually false that "the package you define in your .proto file will have no effect on the generated code" in Python (and presumably therefore in languages which do use namespaces) because the generated ..._pb2.py files contain descriptors which have a package field.
DESCRIPTOR = _descriptor.FileDescriptor(
name='example.proto',
package='example',
syntax='proto3',
...
)
I have checked and you cannot import example into a regular python file after having generated the code. i.e. declaring package example; in your .proto file and then protoc compiling it doesn't automatically make your package available for import in python files in your environment (you must still use the import ..._pb2 to access the classes).
Is anyone able to explain how packages declared in .proto files are used both in Python and say C# as an example of a language which makes explicit use of namespaces?
Quoting from the documentation itself:
In Python, the package directive is ignored, since Python modules are
organized according to their location in the file system.
So what exactly is the use of packages declaration then?
Two things I believe:
To avoid the possibility of naming conflicts in the message objects. If supposedly you have a message which has been declared differently under the same application, then you can consider providing the package specifier at the beginning of the proto file, and then use the package specified when defining fields of your other message types. For example, if Foo is defined under package bar and package baz both; then use bar.Foo and baz.Foo wherever required.
For portability. Package directive is ignored both in Python and Golang. But languages like C++ and C# which create namespaces explicitly, do consider the package specified. So for Foo in the above example, a message type named Foo would be declared both in namespaces of bar and baz. The proto file you created with your Python application can be compiled for your C# application in much the same way. It is to be noted that Protobuf is language agnostic mechanism for serializing and deserializing data.
Having said that, I haven't personally much found the use of the package specifier in my protobuf message with my Python applications.
Ref: https://developers.google.com/protocol-buffers/docs/proto
Related
My C# project (a Unity Package library) is not getting warnings on missing xmldoc of public classes, properties or methods.
I want to ensure all of my public API has XML documentation in code, but I've noticed there's undocumented public methods with no warnings. The csproj for my lib was basically copied from the csproj auto-generated by Unity, with a few changes. I'm not suppressing any warnings related to documentation as far as I can tell.
What I wanted was basically CS1591:
Missing XML comment for publicly visible type or member 'Type_or_Member'
The DocumentationFile compiler option was specified, but one or more constructs did not have comments.
It seems I have to make sure the <DocumentationFile> tag is specified to a non-empty file path (and WarningLevel 4 is not disabled) in the csproj.
<DocumentationFile>some/path/AssemblyName.xml</DocumentationFile>
I hadn't specified it before because I don't actually need documentation in a separate xml if my lib will be distributed as source code (Unity already picks the docs from code comments). I would need it if I had to compile the dll.
After specifying the DocumentationFile to some temp path, VSCode / Omnisharp will emit warnings like CS1591, CS0419.
I wanted to download the lame library for use in my C# project to convert audio. I found the files to use here (libmp3lame)
When I downloaded the archive, I found the .dll I was looking for but along with them, there were 2 others files:
libmp3lame.dll
libmp3lame.exp
libmp3lame.lib
My question:
1 What are these files used for? And how can I make use of them in my project apart from the .dll file
What benefit do these files give me that the .dll library cant?
Edit: I have a feeling that these files are not for use in C#. More for C++. But either way, what are those files? And what are they used for?
Both lib files and exp files are used with native code development (e.g. C++/VB6/Win32) and are not used at all in the .NET world.
lib files contain a collection of reusable code or data or both that can be statically linked and reused with your existing code. They are similar to DLL files, except the big difference being DLL files are linked dynamically and LIB files are linked statically, when you compile your program.
exp files are export files. There are kind of like metadata files that tell your compiler which functions in a DLL have been exported, and therefore can be reused in your statically linked program. .NET program do not require this because each .NET DLL has a special section called the CLR Metadata section. .NET programs can use reflection to pull this metadata information and see which methods are available.
If you think about the method for consuming a native DLL in .NET (p/invoke), your p/invoke declarations have several parts:
The function signature (return time, argument types)
DllImportAttribute specifying the DLL where the function is found and its key in the export table (mangled name or ordinal)
In C and C++, the header file provides the first, and the import library, ending in .lib, provides the second.
Unlike p/invoke's DllImportAttribute, in C and C++ the calling convention is stored with the function signature in the header file.
Basically, I developped a small library with some common fonctionnalities that I use in all my projects. For some political reasons, I cannot choose a generic name for that library (including namespace and assembly name). Usually, it must include the name of the enterprise, something like this for the namespace: Enterprise.ProjectName.XXX.YYY.
For the moment, I'm doing a copy of my library, then I'm renaming the namespaces manually with Visual Studio, and finally I'm recompiling the whole thing.
So my question is the following: Is it possible to create a small program that takes an assembly as input, rename all namespaces from MyLibrary.XXX.YYY to Enterprise.ProjectName.XXX.YYY as well as the assembly name?
What are the steps to follow?
[Edit]
Generating the assembly automatically seems to much work. I will use resharper and/or CTRL+ALT+F like I did so far. Thanks for the answers...
You could use Mono's Cecil project to disassemble the assembly, inspect each type, rename or recreate the type with a new namespace, and generate the resulting assembly.
That being said, it might be simpler to use a tool like Resharper which allows you to rename namespaces correctly within the code base.
Some options:
If you are copying the entire source code for your library into your new project, you can use a refactoring tool like Resharper to "Adjust Namespaces". This is a pretty quick and safe refactoring.
If you just need to avoid shipping the internally named assembly, you may be able to use ILMerge to 'hide' the internal assembly during a post-build step. This is viable if it's just a perception issue for the final assembly names in the binary output directory.
Deal with the issue at the political level by describing your internal library as being no different from any other third-party dependency. Then the naming is no longer a problem. This may solve other problems if you're shipping the source code of this library to multiple clients, as it clarifies that you are not giving full ownership of your 'shared' code to each client. Otherwise they could potentially argue that you are not allowed to use that 'shared' code in projects for other clients, since it is clearly owned by them, having their enterprise name in the namespace.
I have a solution with many F# and C# projects. My goal is to merge them all to one library using ILMerge. Resulting merged dll will be put in a NuGet package and referenced in other projects. However, I'm running into few issues when merged dll is referenced in F# projects.
The problem I have is that if primary assembly given to ILMerge is F# then referencing resulting dll in F# project allows to only access F# types. If C# dll is chosen as primary assembly for merge then extension methods from merged F# assemblies were not available when referencing in F# project. Also modules with AutoOpen attribute were no longer implicitly opened when opening enclosing namespace.
Is there a way to merge F# and C# assemblies so that all types (including extension methods) would be available?
In part of our code base, a big chunk of the library is done in F# and the rest in C#. Both F# and C# code are front facing.
We have a hellish batch file to take care of mergeing and what I see is that we are merging with this code:
echo merging %mergeapp% /keyfile:"%keyfile%" /target:library /attr:"%dstpath%%csharpdll%" /targetplatform:%targetplatform%,%targetlib% /lib:%sllib% /lib:%targetlib% /lib:"%libpath%lib" /out:"%mergedpath%..\%csharpdll%" "%dstpath%%csharpdll%" "%dstpath%%fsharpdll%"
%mergeapp% /keyfile:"%keyfile%" /target:library /attr:"%dstpath%%csharpdll%" /targetplatform:%targetplatform%,%targetlib% /lib:%sllib% /lib:%targetlib% /lib:"%libpath%lib" /out:"%mergedpath%..\%csharpdll%" "%dstpath%%csharpdll%" "%dstpath%%fsharpdll%"
and that does what we intend. However, we do not publish any extension methods nor do we do any AutoOpen. What we discovered was a bug in the F# compiler that, up until we started putting obfuscation in the mix, required us to run ildasm on the F# assembly and rip out the offending code. The other issue is that F# doesn't properly support the protected modifier on members (F# makes them public) so we created an attribute that we could hang on class members that were meant to be protected. Then we wrote a tool that uses Cecil to blow the assembly apart, rip out our attribute and change the access to those members to protected (code is in the accepted answer here).
I didn't know about AutoOpen, but I had to do a similar task, so I created class called a registrant that did that kind of work like this:
type FSharpRegistrant() =
do
// do whatever I need to get going
Then in a static constructor within the C# module, I wrote some code that instantiates the F# registrant using reflection to find the class (since in my code base the C# code builds first and doesn't know there's F# code at all). This is ugly code with a lot of error checking, but it works.
I have a large application that I can build through the command line. I want to specify a flag that enables me to compile it into either one of two modes, Actual or Simulated.
So the main issue is, how can I use the preprocessor to programmatically add a reference?
For example:
#if SIMULATED
include SimulatedFiles;
myFactory = new SimulatedFiles.simFactory();
#else
myFactory = new realFactory();
#endif
I don't want any simulated files to compiled into my "actual" application. Since there is no "include" directive in C#, I am stuck on how to accomplish this.
You cannot do this via a C# preprocessor statement because the language doesn't support the notion of references via preprocessor macros.
What you can do is use a msbuild file and alter the set of references added based on msbuild parameters.
nant/msbuild and dependency injection tool with xml configuration?
In C#, there is no real preprocessor, as you can read on the C# Preprocessor's documentation.
From the documentation (emphasis mine):
While the compiler does not have a separate preprocessor, the directives described in this section are processed as if there was one; these directives are used to aid in conditional compilation. Unlike C and C++ directives, you cannot use these directives to create macros.
Are the include files your own source code, or third-party assembly dlls?
If they are your own sources, then you can easily use conditional compilation to remove the "simulated" code from your release build, exactly as you have done in your example (just replace 'include' with 'using'). This is common practice with debugging classes for example.
If you don't "control" the source code for the includes, then you can still add a project reference, but if you conditionally compile all the code that uses the assembly, your applicaton won't ever attempt to access the assembly, so it doesn't need to be be present when the code is running.
(Another possiblity that seems less useful for you is to write a "dummy" version of the referenced assembly that you ship in place of the "real" one, or a proxy that calls the real third-party dll in simulated builds only. If it supplies the public classes and methods that you call, you can ship the dummy instead of the simulated assembly to your customers)