Mainframe Flat file to C# classes - c#

I have to communicate with a IBM main frame using IBM WebSphere.
The service on the main frame side can only use flat files.
On my side I want to use CQRS (Command / Query)
In other words I want to serialize command / queries and deserialize query results
I could do it with standard reflection offcourse, but my question is if there is a nicer way of doing it?
Can I make use of dynamics?
Flatfile > ParsedObjectStructured > Dynamic type > static type

This would depend an awful lot on what the format of the flat file is, and how the schema works - is it self-describing, for example? However, it sounds to me like most of the work here would be in understanding the flat-file format (and the schema-binding). From there, the choice of "deserialize into the static type" vs "deserialize into a dynamic type" is kinda moot, and I would say that there is very little point deserializing into a dynamic type just to have to map it all to the static type. Additionally, the static type can (again, depending on the file-format specifics) be a handy place to decorate the types to say "here's how to interpret this", if the file-format needs specification. For example (and I'm totally making this up as I go along - don't expect this to relate to your format):
[Frobber(Offset = 4, Format = DataFormat.LittleEndianInt32)]
public int Id {get;set;}
[Frobber(Offset = 0, Format = DataFormat.LittleEndianInt32)]
public int Index {get;set;}
[Frobber(Offset = 8, Format = DataFormat.FixedAscii, Size = 20)]
public string Name {get;set;}
[Frobber(Offset = 28, Format = DataFormat.Blob)] // implicit Size=16 as Guid
public Guid UniqueKey {get;set;}
where FrobberAttribute is just something you might invent to specify the file format. Of course, if the schema is defined internally to the file, this may not be necessary.
Re reflection: basic reflection will work fine if the data is fairly light usage; but overall, reflection can be quite expensive. If you need it to be optimal, you would probably want the implementation to consider strategy-caching (i.e. only doing the discovery work once) and meta-programming (turning the strategy into ready-baked IL, rather than incurring the overhead of reflection at runtime).
If the file format is a common / popular one, you might find that there are existing tools for reading that format. If not, you can either roll your own, or find some crazy person who enjoys writing serialization and meta-programming tools. Such people do exist...

Related

C# attributes used for information or to import extern functions?

I'm a bit confused about C#'s use of attributes. At first I thought it was simply used to give program code additional information through the use of the [Obsolete] attribute. Now I find that [Dllimport] can be used to import a dynamic linked library and its functions. Can attributes import .exe files and other kind of files?
A last question, for programmers working in C# every day, how much do you use attributes, and do you use it for anything else than extending information and importing dll's?
Simply said, attributes are just metadata attached to classes or methods, at the very base.
The compiler, however, reads through your code, and runs specific actions for specific attributes it encounters while doing so, hardcoded into it. E.g., when it finds a DllImportAttribute on a method, it will resolve it to an external symbol (again, this is a very simplified explanation).
When it finds an ObsoleteAttribute, it emits a warning of deprecation.
Your own attributes (which you can create with a class inheriting from the Attribute base class) will not have an effect on the default compiler. But you (or other libraries) can also scan for them at runtime, opening up many possibilities and leading to your second question:
I typically use them to do meta programming. For example, imagine a custom network server handling packets of a specific format, implemented in different classes. Each packet format is recognized by reading an integer value. Now I need to find the correct class to instantiate for that integer.
I could do that with a switch..case or dictionary mapping integer -> packet which I extend every time I add a packet, but that is ugly since I have to touch code possibly far away from the actual Packet class whenever I add or delete a packet. I may not even know about the switch or dictionary in case the server is implemented in another assembly than my packets (modularity / extensibility)!
Instead, I create a custom PacketAttribute, storing an integer property set via the attribute, and decorate all my Packet classes with it. The server only has to scan through my assembly types at startup (via reflection) and build a dictionary of integer -> packet pairs automatically. Of course I could scan my assembly every time I need a packet, but that's probably a bit slow performance-wise.
There are APIs which are much more attribute heavy, like controllers in ASP.NET Core: You map full request URLs to methods in handler classes with them, which then execute the server code. Even URL parameters are mapped to parameters in that way.
Debuggers can also make use of attributes. For example, decorating a class with the DebuggerDisplayAttribute lets you provide a custom string displayed for the instances of the class when inspecting them in Visual Studio, which has a specific format and can directly show the values of important members.
You can see, attributes can be very powerful if utilized nicely. The comments give some more references! :)
To answer the second part of your questions, they are also used, for example, in setting validation and display attributes for both client and server side use in a web application. For example:
[Display(Name = "Person's age")]
[Required(ErrorMessage = "Persons's age is required")]
[RangeCheck(13, 59, ErrorMessage = "The age must be between 13 and 59")]
public int? PersonsAgeAtBooking { get; set; }
Or to decorate enums for use in display
public enum YesNoOnlyEnum
{
[Description("Yes")]
Yes = 1,
[Description("No")]
No = 2
}
There are many other uses.

Serialize a decimal to a human readable currency format

I am using json.net library to serialize an object which has a decimal value that represents a cost. When serializing, I want the json to look something like '$400,000' instead of '400000.0'.
Is there a way that I can accomplish this in an easy and efficient way? This object contains many other secondary objects and subsequent properties.
The application will be used locally and the specs say that the output should be in human readable format. Culture variance isn't of any importance in this context.
There's no easy way to perform this since you can't work around a fact that serializer is directly accessing your properties.
If you need a formatted JSON output I would recommend writing a helper class that wraps the class you want to expose. Unfortunately I've done this once for some XML report and really the easiest way went something like this:
class PersonXml
{
Person _person;
void PersonXml(Person person) { _person = person; }
string Age { get { return _person.Age + " years"; } }
}
This is just a simple example I wrote on the fly but the principle is always the same. Even if some advanced JSON serializer offered me ways to format my output, I'd really keep this concept out of my main objects in a separate file with separate helper classes.
Again this isn't quite the solution, in my opinion it should never come to this but if it does, this is the lesser of the evils in my experience.
Also, just note that if you reference the class in your property getters there's a danger of null reference.
As D Stanley suggested, I changed the field type to string from decimal as data in it was only being read and not used for data manipulation.

Generate a C# object based on an xml file?

This may be way out in left field, crazy, but I just need to ask before I go on implementing this massive set of classes.
Basically, I'm writing a binary message parser that decodes a certain military message format into an object. The problem is that there are literally hundreds of different message types and they share almost nothing in common with each other. So the way I'm planning to implement this is to create hundreds of different objects.
However, even though the message attributes share nothing in common, the method for decoding them is fairly straightforward and follows a pattern. So I'm planning to write a code generator to generate all the objects and the decode logic for each message type.
What would be really sweet is if there was some way to dynamically create an object based on some schema. It doesn't necessarily have to be XML, but XML is pretty easy to work with.
Is this possible in C#?
I would like the interface to look something like this:
var decodedMessage = MessageDecoder.Decode(byteArray);
Where the MessageDecoder figures out what type of message it is and then returns the appropriate object. It will probably return an interface which implements a MessageType Property or something like that.
Basically what I'm wondering is if there is a way to have one object called Message, which implements a MessageType Property. And then Depending on the MessageType, the Message object transforms into whatever type of message it is, so I don't have to spend the time creating all of these message types.
ExpandOobject Where you can dynamically add fields to an object.
A good starting point is here.
Is xsd.exe what you are looking for? It can take an XML file or a schema and generate the c# classes. One problem that you might encounter though is that some of the military message formats are VERY obtuse. You could end up with some very large code files.
Look at T4 templates. They let you write code to generate code, they are integrated into the IDE, and they are quite easy really.
EDIT: There is no way to do what you are after with var, because var requires the right-hand side of the assignment to be statically typed (at compile time). I suppose that you could dynamically generate that statement, then compile and run it, but that's a very painful approach.
If you have XSD's for all of the message types, then you can use xsd.exe as #jle suggests. If not, then I am curious about the following:
// Let's assume this works
var decodedMessage = MessageDecoder.Decode(byteArray);
// Now what? I don't know what properties there are on decodedMessage, so I cant do anything with it.

Transferring data from C# to Java via serialization

I have a Java application for Android that will have some static data, which will be loaded at start up. This data is generated by a C# app as a bunch of classes with lists and Dictionaries, something like this:
public class DataContainer
{
public List<A> As = new List<A>();
public Dictionary<int, B> Bs = new Dictionary<int, B>();
// ...
}
public class A
{
public int IdA;
public int IdSomething;
public DateTime dt;
//...
}
public class B
{
public int IdB;
public string Name;
List<C> Cs = new List<C>();
//...
}
public class C
{
//...
}
(The DataContainer class is the root class that contains all the data to be transferred. I then have a similar class structure in the Java project into which I want the data to be imported.)
My issue is that I can't seem to find a combination of a .NET and Java tools/libraries that would be able to write and read the same format.
I've tried JSON with DataContractJsonSerializer and then loading with GSON but that doesn't work with Dictionaries/Maps (works fine with Lists but I need the dictionaries). The .NET exporter uses format like
"Bs":[{"Key":1,"Value":{...}}, ...]
while it looks like GSON uses
{key:value,key:value,...}
I've also looked at some XML but since there isn't a standard formatting all the libraries use something different by default and I really don't want to write all the writing/parsing rules by hand. Also, JSON has much less overhead compared to XML.
So, is there some pair of libraries for .NET export and Java import that can handle nested classes with Lists and Dictionaries? The things I've managed to find were either in the other direction or did just simple one class serialization.
BTW: I'm also using Tuples for Dictionary keys, but if nothing can handle that I'll just use nested Dictionaries, that is not a big deal.
EDIT: Since I can't answer my own question due to low rep here is the solution that worked for me:
So, I've tried a few more combinations and serializing using JSon.NET on the C#/.NET side and then loading using GSON on the Java side works great, except Tuples. For example, using Tuple as a key for dictionary will get serialized as:
{ "(False, 3)" : { ... }, "(True, 1)" : { ... }, ... }
So I have to use String keys in Java for now but that is not an issue. Possible solution is to write a custom class for the Tuple in both .NET and Java.
Example code:
string serializeObject = JsonConvert.SerializeObject(dataContainer, new IsoDateTimeConverter());
File.WriteAllText("test.json", serializeObject);
and Java:
InputStream is = getResources().openRawResource(R.raw.test); // open from /res/raw/test.json
Gson gson = new Gson();
InputStreamReader r = new InputStreamReader(is);
DataContainer dc = gson.fromJson(r, DataContainer.class);
Log.i("jsondebug", dc.Packers.get(0).Something);
You can have a look at Google Protocol Buffers. Protocol buffers are Google's lingua franca for data. Below is an excerpt from Protocol Buffers Developer Guide:
Protocol buffers are a flexible, efficient, automated mechanism for
serializing structured data – think XML, but smaller, faster, and
simpler. You define how you want your data to be structured once, then
you can use special generated source code to easily write and read
your structured data to and from a variety of data streams and using a
variety of languages. You can even update your data structure without
breaking deployed programs that are compiled against the "old" format.
Protocol buffers have many advantages over XML for serializing
structured data. Protocol buffers:
are simpler
are 3 to 10 times smaller
are 20 to 100 times faster
are less ambiguous
generate data access classes that are easier to use programmatically
An XML mapping would work just fine. Serialize as XML and send it over. You'll need marshallers and unmarshallers on either end.
Java has its JAXB standard for binding XML to objects. Perhaps you can manage it if you follow that standard.
I think you need to approach this from a different perspective. Think first about a JSON or XML format that will work well with both the C# and Java side. Think of that as an API or contract between the C# and Android apps. Instead of using libraries that marshall/unmarshall according to their own formats, make the code on each side read/write to the contract you defined.
Alternatively, you might want to look into using a binary format like Apache Thrift or Protocol Buffers.

Limiting a String length on a property

This question arose when I was trying to figure out a larger problem that, for simplicity sake, I'm omitting.
I have to represent a certain data structure in C#. Its a protocol that will be used to communicate with an external system. As such, it has a sequence of strings with predefined lengths and integer (or other, more complicated data). Let's assume:
SYSTEM : four chars
APPLICATION : eight chars
ID : four-byte integer
Now, my preferred way to represent this would be using strings, so
class Message
{
string System {get; set; }; // four characters only!
string Application {get; set; }; // eight chars
int Id {get; set; };
}
Problem is: I have to ensure that string doesn't have more than the predefined length. Furthermore, this header will actually have tenths of fields, are those will change every now and then (we are still deciding the message layout).
How is the best way to describe such structure? I thought, for example, to use a XML with the data description and use reflection in order to create a class that adheres to the implementation (since I need to access it programatically).
And, like I said, there is more trouble. I have other types of data types that limits the number of characters/digits...
For starters: the whole length issue. That's easily solved by not using auto-properties, but instead declaring your own field and writing the property the "old-fashioned" way. You can then validate your requirement in the setter, and throw an exception or discard the new value if it's invalid.
For the changing structure: If it's not possible to just go in and alter the class, you could write a solution which uses a Dictionary (well, perhaps one per data type you want to store) to associate a name with a value. Add a file of some sort (perhaps XML) which describes the fields allowed, their type, and validation requirements.
However, if it's just changing because you haven't decided on a final structure yet, I would probably prefer just changing the class - if you don't need that sort of dynamic structure when you deploy your application, it seems like a waste of time, since you'll probably end up spending more time writing the dynamic stuff than you would altering the class.

Categories

Resources