Preserving proto comments when generating C# with protobuf-net - c#

We are using protobuf-net to handle our Protocol Buffer needs in a C# application. Since we share our .proto files with other, non-managed applications, we are generating our code from the .proto files (not using the code-first protobuf-net approach). In order to stay as DRY as possible, we keep a lot of interface documentation inside the .proto files themselves. We generate the C# code by means of protogen.exe, called by a project build target.
Now, is there any way to (automatically) transfer these comments into the compiled C# code?
Basically, given a .proto like this:
// This message is used to request a resource from the server
message GetResource
{
// The identifier of the requested resource
required string resourceId = 1;
}
...I would like something like this (IExtensible methods omitted for readability):
/// <summary>
/// This message is used to request a resource from the server
/// </summary>
[global::System.Serializable,global::ProtoBuf.ProtoContract(Name=#"GetResource")]
public partial class GetResource : global::ProtoBuf.IExtensible
{
public GetResource() {}
private string _resourceId;
/// <summary>
/// The identifier of the requested resource
/// [Required] <-- Would be nice...
/// </summary>
[global::ProtoBuf.ProtoMember(1, IsRequired = true, Name=#"resourceId",
DataFormat = global::ProtoBuf.DataFormat.Default)]
public string ResourceId
{
get { return _resourceId; }
set { _resourceId = value; }
}
}

Actually current version does support comments. It may be enabled with --include_source_info.
Comments are available in descriptor.Location[n].leading_comments and trailing_comments :
https://code.google.com/p/protobuf/source/browse/trunk/src/google/protobuf/descriptor.proto
I've added the corresponding properties to protobuf-net Location class:
private string _leading_comments = "";
[global::ProtoBuf.ProtoMember(3, IsRequired = false, Name = #"leading_comments", DataFormat = global::ProtoBuf.DataFormat.Default)]
[global::System.ComponentModel.DefaultValue("")]
public string leading_comments
{
get { return _leading_comments; }
set { _leading_comments = value; }
}
private string _trailing_comments = "";
[global::ProtoBuf.ProtoMember(4, IsRequired = false, Name = #"trailing_comments", DataFormat = global::ProtoBuf.DataFormat.Default)]
[global::System.ComponentModel.DefaultValue("")]
public string trailing_comments
{
get { return _trailing_comments; }
set { _trailing_comments = value; }
}
And added --include_source_info to protoc call (ProtoBuf.CodeGenerator.InputFileLoader)
And locations with comments were added to xml generated:
<?xml version="1.0" encoding="utf-16"?>
<FileDescriptorSet xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:xsd="http://www.w3.org/2001/XMLSchema">
<file>
<FileDescriptorProto>
<name>Test.proto</name>
<dependency />
<message_type>
<DescriptorProto>
<name>Test2</name>
<field>
<FieldDescriptorProto>
<name>IntValue</name>
<number>1</number>
<type>TYPE_INT32</type>
</FieldDescriptorProto>
</field>
<extension />
<nested_type />
<enum_type />
<extension_range />
</DescriptorProto>
</message_type>
<enum_type />
<service />
<extension />
<source_code_info>
<location>
...
<Location>
<path>
<int>4</int>
<int>0</int>
<int>2</int>
<int>0</int>
</path>
<span>
<int>1</int>
<int>0</int>
<int>28</int>
</span>
<trailing_comments> some comment
</trailing_comments>
</Location>
...
</location>
</source_code_info>
</FileDescriptorProto>
</file>
</FileDescriptorSet>
source .proto:
message Test2{
optional int32 IntValue = 1;// some comment
}
But I'm not strong in xslt to update ProtoGen/csharp.xslt to include comments into CS file generated

At the current time, I believe the answer is "no". To the best of my knowledge, "protoc" (Google's tool for parsing .proto files, which is used under the hood) silently discards the comments - so there is nothing available to read from. If a custom parser was written, then yes it would be possible, but there is also a language ambiguity about which comments apply to which lines, for example:
// this probably relates to resourceId
required string resourceId = 1;
required int foo = 2; // but... is this foo? or bar?
// and what about this?
// what does this relate to? and why?
// and this? what are the rules?
required int bar = 3;
So for 2 different reasons: at the moment, no. All suggestions considered, though... especially if they come with a custom parser included :)
Note that AFAIK this information is missing from most (all?) implementations for this reason. I'm happy to be corrected, though.

Related

Creating a dictionary with Unity Application Block IoC

I would like to create a dictionary and populate it in the configuration file for Unity like shown here. However, that example seems to be from an older version of Unity IoC but I would like to know how to do this with Unity 4.
<type ... to="MyTypeServer">
<typeConfig>
<property Name="Converters" KeyType"string" ValueType="IConverter">
<entry>
<key>csv</key>
<value><dependency name="csvConverter"/></value>
</entry>
<entry>
<key>xml</key>
<value><dependency name="xmlConverter"/></value>
</entry>
</property>
</typeConfig>
</type>
<type name="csvConverter" from="IConverter" to="MyCsvConverter">
</type>
<type name="xmlConverter" from="IConverter" to="MyXmlConverter">
</type>
And here is the class:
public class MyTypeServer
{
public IDictionary<string, IConverter> Converters
{
set;
private get;
}
public void DoConversion(string fileName)
{
string fileType = Path.GetFileExtension(fileName);
IConverter converter = Converters[fileType];
if (converter != null)
converter.DoConversion(fileName);
..
...
}
}
I have been trying for hours and researching but no luck.
From the codeplex link you posted:
This is a quick pseudo code of the type of stuff we can have with dictionary.
To me, this reads "we could do something like this if we implemented the feature". Aligns with my experience with unity, I've never come across something like this.
What you can do however: register all the converters, have them all injected as array and then build the dictionary yourself.
// register in code or in xml...
container.RegisterType<IConverter, XmlConverter>( "xml-Converter" );
container.RegisterType<IConverter, JsonConverter>( "json-Converter" );
internal class ConverterConsumer
{
public ConverterConsumer( IConverter[] converters )
{
_converters = converters.ToDictionary( x => x.FileType, x => x );
}
#region private
private Dictionary<string, IConverter> _converters;
#endregion
}
public interface IConverter
{
string FileType { get; }
void DoConversion( string fileName );
}
Posting as an answer because I do not have sufficient points to comment. This is what I did to solve my problem. Not exactly a Dictionary approach but it might help.
My requirements - Store app settings in the Unity XML file as opposed to app.config
I had various objects which were registered in the Unity XML and they had properties like connection strings, Azure queue names , Azure blob container names, etc. I found myself duplicating these values in the XML very often. I could make my objects read values from *appSettings** element of app.config or some other configuration section. However, I chose not to use the app.config for the sake of keeping my objects more testable.
My solution - Use the <instance> element to register reusable string values
I registered all reusable connection strings in a single location as shown below and without any duplications:
<instance name="cnstring1" value="blah connection string 1"></instance>
<instance name="cnstring2" value="blah connection string 2"></instance>
<instance name="azurequeue1" value="name of receiver queue "></instance>
<instance name="azurequeue2" value="name of sender queue "></instance>
<instance name="azurestoragecnstring" value="your azure storage account connection string 0001"></instance>
Reference the name-value pairs using the dependency element whereever required.
Example XML:
<register name="i2" mapTo="someimplementation" type="someinterface">
<property name="Database">
<dependency name="cnstring1" />
</property>
</register>
Example C# code snippet:
public string Database { get; set; }
At run time the property Database of the object with the registration i2 will be set to the value of blah connection string 1
Caveats
Storing connection strings in plain text XML might not be safe especially if the connection string has username and password values.

xsd2code creates extra nested collection when serializing lists

I've been using xsd2code v3.4.
So far I'm pretty close to getting it to work, however I'm facing one glaring issue and I can't seem to find any solutions. When my XML gets generated after I serialize my object, it's adding an additional complex type that is named exactly like the class. This is what I currently get. Notice how it's adding an unnecessary collection right after an order line:
<?xml version="1.0" encoding="utf-8"?>
<CORE_PO_INBOUND_V2 xmlns:xsd="http://www.w3.org/2001/XMLSchema" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
<INTEGRATION_MESSAGE_CONTROL>
<ACTION>FULL_UPDATE</ACTION>
<COMPANY_CODE>COMPANY</COMPANY_CODE>
<ORG_CODE>COMPANY</ORG_CODE>
<MESSAGE_TYPE>INBOUND_ENTITY_INTEGRATION</MESSAGE_TYPE>
<USERID>COMPANY</USERID>
<RECEIVER>TA15</RECEIVER>
<SENDER>COMPANY</SENDER>
<BATCH_ID>1234</BATCH_ID>
<BUS_KEY>
<ORG_CODE>COMPANY</ORG_CODE>
<PO_NUMBER>1234</PO_NUMBER>
</BUS_KEY>
</INTEGRATION_MESSAGE_CONTROL>
<PURCHASE_ORDER_HEADER>
<CTRY_OF_EXPORT>TR</CTRY_OF_EXPORT>
<CTRY_OF_IMPORT>US</CTRY_OF_IMPORT>
<CURRENCY_CODE>USD</CURRENCY_CODE>
<INCOTERM_CODE>011</INCOTERM_CODE>
<ORG_CODE>COMPANY</ORG_CODE>
<SOURCE_TX_ID>THING</SOURCE_TX_ID>
<PO_NUMBER>1234</PO_NUMBER>
<PURCHASE_ORDER_LINE>
<CORE_PO_INBOUND_V2PURCHASE_ORDER_HEADERPURCHASE_ORDER_LINE>
<BUSINESS_UNIT>BCA</BUSINESS_UNIT>
<COMMERCIAL_UOM>EA</COMMERCIAL_UOM>
<CTRY_OF_IMPORT>US</CTRY_OF_IMPORT>
<CURRENCY_CODE>USD</CURRENCY_CODE>
<DEPARTMENT>602</DEPARTMENT>
<LINE_ID>1</LINE_ID>
</CORE_PO_INBOUND_V2PURCHASE_ORDER_HEADERPURCHASE_ORDER_LINE>
</PURCHASE_ORDER_LINE>
<PURCHASE_ORDER_HEADER_PARTNER>
<CORE_PO_INBOUND_V2PURCHASE_ORDER_HEADERPURCHASE_ORDER_HEADER_PARTNER>
<REF_RESOLUTION_PARTNER>Stuff</REF_RESOLUTION_PARTNER>
</CORE_PO_INBOUND_V2PURCHASE_ORDER_HEADERPURCHASE_ORDER_HEADER_PARTNER>
</PURCHASE_ORDER_HEADER_PARTNER>
</PURCHASE_ORDER_HEADER>
</CORE_PO_INBOUND_V2>
This is what I actually want:
<?xml version="1.0" encoding="utf-8"?>
<CORE_PO_INBOUND_V2 xmlns:xsd="http://www.w3.org/2001/XMLSchema" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
<INTEGRATION_MESSAGE_CONTROL>
<ACTION>FULL_UPDATE</ACTION>
<COMPANY_CODE>COMPANY</COMPANY_CODE>
<ORG_CODE>COMPANY</ORG_CODE>
<MESSAGE_TYPE>INBOUND_ENTITY_INTEGRATION</MESSAGE_TYPE>
<USERID>COMPANY</USERID>
<RECEIVER>TA15</RECEIVER>
<SENDER>COMPANY</SENDER>
<BATCH_ID>1234</BATCH_ID>
<BUS_KEY>
<ORG_CODE>COMPANY</ORG_CODE>
<PO_NUMBER>1234</PO_NUMBER>
</BUS_KEY>
</INTEGRATION_MESSAGE_CONTROL>
<PURCHASE_ORDER_HEADER>
<CTRY_OF_EXPORT>TR</CTRY_OF_EXPORT>
<CTRY_OF_IMPORT>US</CTRY_OF_IMPORT>
<CURRENCY_CODE>USD</CURRENCY_CODE>
<INCOTERM_CODE>011</INCOTERM_CODE>
<ORG_CODE>COMPANY</ORG_CODE>
<SOURCE_TX_ID>THING</SOURCE_TX_ID>
<PO_NUMBER>1234</PO_NUMBER>
<PURCHASE_ORDER_LINE>
<BUSINESS_UNIT>BCA</BUSINESS_UNIT>
<COMMERCIAL_UOM>EA</COMMERCIAL_UOM>
<CTRY_OF_IMPORT>US</CTRY_OF_IMPORT>
<CURRENCY_CODE>USD</CURRENCY_CODE>
<DEPARTMENT>602</DEPARTMENT>
<LINE_ID>1</LINE_ID>
</PURCHASE_ORDER_LINE>
<PURCHASE_ORDER_HEADER_PARTNER>
<REF_RESOLUTION_PARTNER>Stuff</REF_RESOLUTION_PARTNER>
</PURCHASE_ORDER_HEADER_PARTNER>
</PURCHASE_ORDER_HEADER>
</CORE_PO_INBOUND_V2>
Is there some setting I'm using incorrectly? I have it set to work with List collections. Seems as if this problem only exists for collections of a class that's generated from this tool.
Edit: Adding some snippets of the designer class that gets generated by xsd2code. Note that this file is pretty large (almost 10k lines...), so I'm not going to post the whole thing here, but rather the chunks that pertain to the purchase order line collections of elements:
public partial class CORE_PO_INBOUND_V2PURCHASE_ORDER_HEADER : EntityBase<CORE_PO_INBOUND_V2PURCHASE_ORDER_HEADER>
{
private List<CORE_PO_INBOUND_V2PURCHASE_ORDER_HEADERPURCHASE_ORDER_LINE> pURCHASE_ORDER_LINEField;
public List<CORE_PO_INBOUND_V2PURCHASE_ORDER_HEADERPURCHASE_ORDER_LINE> PURCHASE_ORDER_LINE
{
get
{
if ((this.pURCHASE_ORDER_LINEField == null))
{
this.pURCHASE_ORDER_LINEField = new List<CORE_PO_INBOUND_V2PURCHASE_ORDER_HEADERPURCHASE_ORDER_LINE>();
}
return this.pURCHASE_ORDER_LINEField;
}
set
{
this.pURCHASE_ORDER_LINEField = value;
}
}
}
public partial class CORE_PO_INBOUND_V2PURCHASE_ORDER_HEADERPURCHASE_ORDER_LINE : EntityBase<CORE_PO_INBOUND_V2PURCHASE_ORDER_HEADERPURCHASE_ORDER_LINE>
{
private System.Nullable<decimal> aREAField;
private bool aREAFieldSpecified;
private string aREA_UOMField;
...
}
I think I found a solution. I spoke with a colleague whom had done something similar. He said he used the native "xsd" and not "xsd2code". We did a compare on what got generated and noticed that on the arrays (in my case, I use lists...), he had the following annotation:
[System.Xml.Serialization.XmlElementAttribute(Form=System.Xml.Schema.XmlSchemaForm.Unqualified, IsNullable=true)]
Is there a way to trigger this same annotation via xsd2code? Looks like without it extra elements get generated upon executing the serializer.

Classes generated from XSD does not work with XmlSerializer

I need to read some XML files that follow the ONIX standard
See: http://www.editeur.org/93/Release-3.0-Downloads/
To do this i downloaded the ONIX 3.0 XSD:
http://www.editeur.org/files/ONIX%203/ONIX_BookProduct_XSD_schema+codes_Issue_25.zip
Using the downloaded XSD and this command "xsd your.xsd /classes" i created classes that i want to use.
When trying to create a new Xml Serializer like so:
var xmls = new XmlSerializer(typeof(Model.ONIX.editeur.ONIXMessage));
I get and exception
"There was an error reflecting type 'Model.ONIX.editeur.ONIXMessage'."
When i drill down through the inner exceptions i end up with this message:
"{"Member 'Text' cannot be encoded using the XmlText attribute. You
may use the XmlText attribute to encode primitives, enumerations,
arrays of strings, or arrays of XmlNode."}"
I am not sure what to do, is something wrong with the XSD? Any suggestions?!
Edit
public static List<Model.ONIX.editeur.Product> GetProductsDataFromOnixFile(string onixFileLocation)
{
var xmls = new XmlSerializer(typeof(Model.ONIX.editeur.ONIXMessageRefname));
using (var reader = XmlReader.Create(onixFileLocation))
{
if (xmls.CanDeserialize(reader))
{
var onixMessage = (Model.ONIX.editeur.ONIXMessage)xmls.Deserialize(reader);
return onixMessage.Items.OfType<Model.ONIX.editeur.Product>().ToList();
}
throw new Exception(string.Format("Cant read the file {0} as Onix", onixFileLocation));
}
}
I know this question is old but I assume others with specific Onix issues will run into this.
Here is how I got it to work.
In the reference xsd are two includes in the top. Here I copy/pasted the other two files in.
<xs:include schemaLocation="ONIX_BookProduct_CodeLists.xsd" />
<xs:include schemaLocation="ONIX_XHTML_Subset.xsd" />
I.e. these lines are replaced in the file with the corresponding file.
Then I did the
xsd ONIX_BookProduct_3.0_reference.xsd /classes
And then it generates the .cs file. And the only issue I had here was I had to remove a text attribute from all fields that was e.g. List147, but not from the fields that was string. E.g. I had to remove the attribute from generated code like this:
/// <remarks/>
[System.Xml.Serialization.XmlTextAttribute()]
public List121 textscript {
get {
return this.textscriptField;
but not from attributes like this
/// <remarks/>
[System.Xml.Serialization.XmlTextAttribute()]
public string Value {
get {
return this.valueField;

How to read nested configuration element from the web.config?

I need to read configuration elements from the web.config.
Let this be my web.config.
<family>
<parents>
<child name="Hello"/>
<child name="World"/>
</parents>
<parents>
<child name="Hello1"/>
<child name="World2"/>
</parents>
</family>
So I have something like this, I need to read this into a collection.
How can i do this????
In general, you can store simple application settings and connection string in web.config (or app.config), but anything more complex, like an object graph or XML (as in your case) and you should consider a different method.
These may be helpful:
How do I store an XML value in my .NET App.Config file
(it suggests encoding the XML in an app setting)
However it would be better to have a separate XML data file and convert it to an object graph using Linq-To-XML (see reference) or XPath and the XmlDocument and related classes.
Edit: see the other answer, which does allow XML in the config file. That's a more direct answer to your exact questions but I will leave this here for reference. On the whole it looks like your data is not configuration data (more like runtime / user data) and does not belong in a .config file: so I would recommend storing it in a separate XML file, and having a config file entry pointing to the filename of the separate XML file.
Hope that helps!
You need to define your own custom configuration section, which will allow you to read the nested configuration element properly. BTW, this is the same method that all the others use, for instance the Enterprise Library components, NHibernate, etc.
The steps you need to take are very straightforward, and a tutorial is provided here:
http://msdn.microsoft.com/en-us/library/2tw134k3.aspx
You need to use the ConfigurationElementCollection Class.
See this sample on the MSDN
public struct Child
{
public string name;
public Child(string name)
{
this.name = name;
}
}
public class Parent
{
public List<Child> childs = new List<Child>();
public static List<Parent> ReadParentsFromXml(string fileName)
{
List<Parent> parents = new List<Parent>();
System.Xml.XmlTextReader doc = new System.Xml.XmlTextReader(fileName);
Parent element = new Parent();
while (doc.Read())
{
switch (doc.Name)
{
case "parents":
if (doc.NodeType == System.Xml.XmlNodeType.EndElement)
{
parents.Add(element);
element = new Parent();
}
break;
case "child":
if(doc.NodeType != System.Xml.XmlNodeType.EndElement)
element.childs.Add(new Child(doc.GetAttribute(0)));
break;
}
}
return parents;
}
}

XML Serialization of HTML

Okay this one DID it! Thanks to all of you!
public class Result
{
public String htmlEscaped
{
set;
get;
}
[XmlIgnore]
public String htmlValue
{ set; get; }
[XmlElement("htmlValue")]
public XmlCDataSection htmlValueCData
{
get
{
XmlDocument _dummyDoc = new XmlDocument();
return _dummyDoc.CreateCDataSection(htmlValue);
}
set { htmlValue = (value != null) ? value.Data : null; }
}
}
Result r = new Result();
r.htmlValue = ("<b>Hello</b>");
r.htmlEscaped = ("<b>Hello</b>");
XmlSerializer xml = new XmlSerializer(r.GetType());
TextWriter file = new StreamWriter(Environment.CurrentDirectory + "\\results\\result.xml", false, System.Text.Encoding.Default);
xml.Serialize(file, r);
file.Close();
RESULT:
<?xml version="1.0" encoding="Windows-1252"?>
<Result xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:xsd="http://www.w3.org/2001/XMLSchema">
<htmlEscaped><b>Hello</b></htmlEscaped>
<htmlValue><![CDATA[<b>Hello</b>]]></htmlValue>
</Result>
As you can see, after CDATA is return type, no more escaped html in XML file on filesystem.
The JSON Serialization isn't working anymore, but this can be fixed with a little type extention.
QUESTION WAS:
Maybe someone knows how to make do it...
I have this Class:
public class Result
{
public String htmlValue
{
get;
set;
}
}
I use this to serialize it to XML
Result res = new Result();
res.htmlValue = "<p>Hello World</p>";
XmlSerializer s = new XmlSerializer(res.GetType());
TextWriter w = new StreamWriter(Environment.CurrentDirectory + "\\result.xml", false, System.Text.Encoding.Default);
s.Serialize(w, res);
w.Close();
Works fine i get this:
<?xml version="1.0" encoding="Windows-1252"?>
<Result xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:xsd="http://www.w3.org/2001/XMLSchema">
<htmlValue><b>Hello World</b></htmlValue>
</Result>
What can do i have to change to get this:
<?xml version="1.0" encoding="Windows-1252"?>
<Result xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:xsd="http://www.w3.org/2001/XMLSchema">
<htmlValue><![CDATA[<b>Hello World</b>]]></htmlValue>
</Result>
I've already searched but I can't find anything. The type of htmlValue
have to stay String, because of other Serialisations JSON, etc.
Tricky one... Thanks in advance for suggestions
HTML is correct in String within C#. Why decode or encode?
XmlSerializer saved the HTML escaped to XML file.
Don't use C# for consuming.
Is external tool which accept this:
<htmlValue><![CDATA[<b>Hello World</b>]]></htmlValue>
but not
<htmlValue><b>Hello World</b></htmlValue>
I do the same with JSON Serializer, in file on hard drive the HTML is saved correct.
Why and where to use HTTP Utility to prevent that? And how to get <![CDATA[ ]]> around it.
Can you give a code sample?
Are there any other Serializer than the C# own one?
I've found this Link .NET XML Serialization of CDATA ATTRIBUTE from Marco André Silva, which does I need to do, but it's different, how to include this without changing Types?
Here's a simple trick to do achieve what you want. You just need to serialize a XmlCDataSection property instead of the string property :
(it's almost the same as John's suggestion, but a bit simpler...)
public class Result
{
[XmlIgnore]
public String htmlValue
{
get;
set;
}
private static XmlDocument _dummyDoc;
[XmlElement("htmlValue")]
public XmlCDataSection htmlValueCData
{
get { return _dummyDoc.CreateCDataSection(htmlValue); }
set { htmlValue = (value != null) ? value.Data : null; }
}
}
See "CDATA serialization with XMLSerializer" for the same problem, and for the solution.
BTW, it seems to me that if the vendor no longer exists, it's time to use a different product. Possibly one that understands the XML specifications which have only existed for over a decade.
It is my understanding that you need the XML to feed it to some utility. Do you also plan to use it to de-serialize the object?
If not then why do not do it yourself - serialize your object that is? Roundtrip object -> XML -> object is somewhat tricky, but the first part is not.

Categories

Resources