How would you design an application (classes, interfaces in class library) in .NET when we have a fixed database design on our side and we need to support imports of data from third party data sources, which will most likely be in XML?
For instance, let us say we have a Products table in our DB which has columns
Id
Title
Description
TaxLevel
Price
and on the other side we have for instance Products:
ProductId
ProdTitle
Text
BasicPrice
Quantity.
Currently I do it like this:
Have the third party XML convert to classes and XSD's and then deserialize its contents into strong typed objects (what we get as a result of this process is classes like ThirdPartyProduct, ThirdPartyClassification, etc.).
Then I have methods like this:
InsertProduct(ThirdPartyProduct newproduct)
I do not use interfaces at the moment but I would like to. What I would like is implement something like
public class Contoso_ProductSynchronization : ProductSynchronization
{
public void InsertProduct(ContosoProduct p)
{
Product product = new Product(); // this is our Entity class
// do the assignments from p to product here
using(SyncEntities db = new SyncEntities())
{
// ....
db.AddToProducts(product);
}
}
// the problem is Product and ContosoProduct have no arhitectural connection right now
// so I cannot do this
public void InsertProduct(ContosoProduct p)
{
Product product = (Product)p;
using(SyncEntities db = new SyncEntities())
{
// ....
db.AddToProducts(product);
}
}
}
where ProductSynchronization will be an interface or abstract class. There will most likely be many implementations of ProductSynchronization. I cannot hardcode the types - classes like ContosoProduct, NorthwindProduct might be created from the third party XML's (so preferably I would continue to use deserialization).
Hopefully someone will understand what I'm trying to explain here. Just imagine you are the seller and you have numerous providers and each one uses their own proprietary XML format. I don't mind the development, which will of course be needed everytime new format appears, because it will only require 10-20 methods to be implemented, I just want the architecture to be open and support that.
In your replies, please focus on design and not so much on data access technologies because most are pretty straightforward to use (if you need to know, EF will be used for interacting with our database).
[EDIT: Design note]
Ok, from a design perspective I would do xslt on the incoming xml to transform it to a unified format. Also very easy to verify the result xml towards a schema.
Using xslt I would stay away from any interface or abstract class, and just have one class implementation in my code, the internal class. It would keep the code base clean, and the xslt's themselves should be pretty short if the data is as simple as you state.
Documenting the transformations can easily be done wherever you have your project documentation.
If you decide you absolutely want to have one class per xml (or if you perhaps got a .net dll instead of xml from one customer), then I would make the proxy class inherit an interface or abstract class (based off your internal class, and implement the mappings per property as needed in the proxy classes. This way you can cast any class to your base/internal class.
But seems to me doing the conversion/mapping in code will make the code design a bit more messy.
[Original Answer]
If I understand you correctly you want to map a ThirdPartyProduct class over to your own internal class.
Initially I am thinking class mapping. Use something like Automapper and configure up the mappings as you create your xml deserializing proxy's. If you make your deserialization end up with the same property names as your internal class, then there's less config to do for the mapper. Convention over Configuration.
I'd like to hear anyones thoughts on going this route.
Another approach would be to add a .ToInternalProduct( ThirdPartyClass ) in a Converter class. And keep adding more as you add more external classes.
The third approach is for XSLT guys. If you love XSLT you could transform the xml into something which can be deserialized into your internal product class.
Which one of these three I'd choose would depend on the skills of the programmer, and who will maintain adding new external classes. The XSLT approach would require no recompiling or compiling of code as new formats arrived. That might be an advantage.
Related
Background
I like Jeffrey Palermo's Onion Architecture model (similar to Hexagonal Architecture) which prescribes that the Domain Model be at the 'center' and the concrete implementations of infrastructure, specifically Concrete Repositories be on the periphery.
So say I have a Domain Model:
//https://libphonenumber.codeplex.com/
using libphonenumber;
namespace MyApplication.Domain
{
public class Speaker
{
public virtual string Name {get;set;}
public virtual PhoneNumber PhoneNumber {get;set;}
}
}
Now I need to expose this Domain Model to other teams:
The UI Team hypothetically wants to add several Data Validation attributes and custom JSON serialization attributes.
The Infrastructure Team hypothetically wants to add XML Serialization attributes and some custom attributes from a 3rd party Database implementation.
The Public API Team hypothetically wants to add WCF attributes.
I don't want to give every team carte blanche to add their Attributes to my Domain Model and I especially don't want them adding all of their "layer specific" dependencies to my model assembly.
And this case is made more complicated because I'm using 3rd party 'domain models' in my own (in this case using Google's LibPhoneNumber to handle the Phone Number).
Ideally, they'd each need to create their own wrapper class like:
using MyApplication.Domain;
namespace MyApplication.UI.DomainWrappers
{
public class UISpeaker
{
private Speaker _speaker;
public class UISpeaker(Speaker speaker = null)
{
_speaker = speaker ?? new Speaker();
}
[Required]
public virtual string Name {
get{ return _speaker.Name; }
set{ _speaker.Name = value; }
}
[Required]
public virtual PhoneNumber PhoneNumber {
get{ return _speaker.PhoneNumber ; }
set{ _speaker.PhoneNumber = value; }
}
//Conversion operators
public static implicit operator UISpeaker(Speaker s)
{
return new UISpeaker(s);
}
public static implicit operator Speaker(UISpeaker s)
{
return s._speaker;
}
}
}
Question
Writing and maintaining the UISpeaker class is a pain and is boring boilerplate code.
Is there either a better way to add the Attributes each team wants to add without letting them directly edit the Domain Model? Or is there some tooling that can help generate these wrapper classes (I was thinking possibly a weaving tool like Fody or T4 Templates, but I'm not familiar enough with either to know if they could help in this use case).
Research
I looked around Stackoverflow and found some similar questions, but none that hit the full scope I'm looking for:
Avoid using the JsonIgnore attribute in a domain model - Concluded to just use .NET native attributes on the Domain Model so you didn't have to take a dependency on Json.Net
Add an attribute to another assembly's class - Discussed using CustomReflectionContext to add Attributes to an existing type. This looks really cool, but unfortunatly, the model would be handed off to 3rd party code (ORMs, EF, Json.Net, etc) for reflection so I don't think this will work here.
Having Separate Domain Model and Persistence Model in DDD - Confirmed that each layer should have it's own version of the Domain Model, but didn't discuss if there's any tooling / strategies to make writing / maintaining that code any easier.
You can use these options to simplify the job:
Metadata Classes
Object to Object Mappers
Code Generation
Metadata Classes
You can create metadata classes and add attributes like data annotations and validation attributes to those metadata classes and then relate these metadata classes to your main domain classes using AssociatedMetadataTypeTypeDescriptionProvider. Such metadata classes are only attribute containers and using type descriptor mechanisms add attributes to your main classes.
For example, you can register a metadata class for your model this way, and let all infrastructures that benefit TypeDescriptor see your metadata attributes for your model:
var provider = new AssociatedMetadataTypeTypeDescriptionProvider(typeof(Model),
typeof(ModelMetadata));
TypeDescriptor.AddProvider(provider, typeof(Model));
Object to Object Mappers
You can have view models, business models and domain models in different layers and using decorate them with attributes that you need for each layer, then using an object to object mapper like AutoMapper simplify the task of mapping those classes to each other.
AutoMapper is an object-object mapper. Object-object mapping works by
transforming an input object of one type into an output object of a
different type. What makes AutoMapper interesting is that it provides
some interesting conventions to take the dirty work out of figuring
out how to map type A to type B. As long as type B follows
AutoMapper's established convention, almost zero configuration is
needed to map two types.
Code Generation
You can make creating metadata classes or view model classes more easy using some code generation tools. For example you can create the wrapper classes simply using a code generation mechanism like T4 Templates.
In Visual Studio, a T4 text template is a mixture of text blocks and
control logic that can generate a text file. The control logic is
written as fragments of program code in Visual C# or Visual Basic. The
generated file can be text of any kind, such as a Web page, or a
resource file, or program source code in any language.
New to MongoDB. Set up a C# web project in VS 2013.
Need to insert data as document into MongoDB. The number of Key-Value pair every time could be different.
For example,
document 1: Id is "1", data is one pair key-value: "order":"shoes"
document 2: Id is "2", data is a 3-pair key-value: "order":"shoes", "package":"big", "country":"Norway"
In this "Getting Started" says because it is so much easier to work with your own domain classes this quick-start will assume that you are going to do that. suggests make our own class like:
public class Entity
{
public ObjectId Id { get; set; }
public string Name { get; set; }
}
then use it like:
var entity = new Entity { Name = "Tom" };
...
entity.Name = "Dick";
collection.Save(entity);
Well, it defeats the idea of no-fixed columns, right?
So, I guess BsonDocument is the the model to use and is there any good samples for beginners?
I'm amazed how often this topic comes up... Essentially, this is more of a 'statically typed language limitation' than a MongoDB issue:
Schemaless doesn't mean you don't have any schema per se, it basically means you don't have to tell the database up front what you're going to store. It's basically "code first" - the code just writes to the database like it would to RAM, with all the flexibility involved.
Of course, the typical application will have some sort of reoccurring data structure, some classes, some object-oriented paradigm in one way or another. That is also true for the indexes: indexes are (usually) 'static' in the sense that you do have to tell mongodb about which field to index up front.
However, there is also the use case where you don't know what to store. If your data is really that unforeseeable, it makes sense to think "code first": what would you do in C#? Would you use the BsonDocument? Probably not. Maybe an embedded Dictionary does the trick, e.g.
public class Product {
public ObjectId Id {get;set;}
public decimal Price {get;set;}
public Dictionary<string, string> Attributes {get;set;}
// ...
}
This solution can also work with multikeys to simulate a large number of indexes to make queries on the attributes reasonably fast (though the lack of static typing makes range queries tricky). See
It really depends on your needs. If you want to have nested objects and static typing, things get a lot more complicated than this. Then again, the consumer of such a data structure (i.e. the frontend or client application) often needs to make assumptions that make it easy to digest this information, so it's often not possible to make this type safe anyway.
Other options include indeed using the BsonDocument, which I find too invasive in the sense that you make your business models depend on the database driver implementation; or using a common base class like ProductAttributes that can be extended by classes such as ProductAttributesShoes, etc. This question really revolves around the whole system design - do you know the properties at compile time? Do you have dropdowns for the property values in your frontend? Where do they come from?
If you want something reusable and flexible, you could simply use a JSON library, serialize the object to string and store that to the database. In any case, the interaction with such objects will be ugly from the C# side because they're not statically typed.
I have a database that contains "widgets", let's say. Widgets have properties like Length and Width, for example. The original lower-level API for creating wdigets is a mess, so I'm writing a higher-level set of functions to make things easier for callers. The database is strange, and I don't have good control over the timing of the creation of a widget object. Specifically, it can't be created until the later stages of processing, after certain other things have happened first. But I'd like my callers to think that a widget object has been created at an earlier stage, so that they can get/set its properties from the outset.
So, I implemented a "ProxyWidget" object that my callers can play with. It has private fields like private_Length and private_Width that can store the desired values. Then, it also has public properties Length and Width, that my callers can access. If the caller tells me to set the value of the Width property, the logic is:
If the corresponding widget object already exists in the database, then set
its Width property
If not, store the given width value in the private_Width field for later use.
At some later stage, when I'm sure that the widget object has been created in the database, I copy all the values: copy from private_Width to the database Width field, and so on (one field/property at a time, unfortunately).
This works OK for one type of widget. But I have about 50 types, each with about 20 different fields/properties, and this leads to an unmaintainable mess. I'm wondering if there is a smarter approach. Perhaps I could use reflection to create the "proxy" objects and copy field/property data in a generic way, rather than writing reams of repetitive code? Factor out common code somehow? Can I learn anything from "data binding" patterns? I'm a mathematician, not a programmer, and I have an uneasy feeling that my current approach is just plain dumb. My code is in C#.
First, in my experience, manually coding a data access layer can feel like a lot of repetitive work (putting an ORM in place, such as NHibernate or Entity Framework, might somewhat alleviate this issue), and updating a legacy data access layer is awful work, especially when it consists of many parts.
Some things are unclear in your question, but I suppose it is still possible to give a high-level answer. These are meant to give you some ideas:
You can build ProxyWidget either as an alternative implementation for Widget (or whatever the widget class from the existing low-level API is called), or you can implement it "on top of", or as a "wrapper around", Widget. This is the Adapter design pattern.
public sealed class ExistingTerribleWidget { … }
public sealed class ShinyWidget // this is the wrapper that sits on top of the above
{
public ShinyWidget(ExistingTerribleWidget underlying) { … }
private ExistingTerribleWidget underlying;
… // perform all real work by delegating to `underlying` as appropriate
}
I would recommend that (at least while there is still code using the existing low-level API) you use this pattern instead of creating a completely separate Widget implementation, because if ever there is a database schema change, you will have to update two different APIs. If you build your new EasyWidget class as a wrapper on top of the existing API, it could remain unchanged and only the underlying implementation would have to be updated.
You describe ProxyWidget having two functions (1) Allow modifications to an already persisted widget; and (2) Buffer for a new widget, which will be added to the database later.
You could perhaps simplify your design if you have one common base type and two sub-classes: One for new widgets that haven't been persisted yet, and one for already persisted widgets. The latter subtype possibly has an additional database ID property so that the existing widget can be identified, loaded, modified, and updated in the database:
interface IWidget { /* define all the properties required for a widget */ }
interface IWidgetTemplate : IWidget
{
IPersistedWidget Create();
bool TryLoadFrom(IWidgetRepository repository, out IPersistedWidget matching);
}
interface IPersistedWidget : IWidget
{
Guid Id { get; }
void SaveChanges();
}
This is one example for the Builder design pattern.
If you need to write similar code for many classes (for example, your 50+ database object types) you could consider using T4 text templates. This just makes writing code less repetitive; but you will still have to define your 50+ objects somewhere.
I'm trying to design a interface for iTextsharp (PDF creation library) that I'm using for my project. I don't want any reference to iTextsharp in my project, just the interface.
Lets say, I have
interface IPdfTable { /* */ }
public class PdfTable : IPdfTable { /* */ }
interface IPdfCell { /* */ }
public class PdfCell : PdfCell { /* */ }
While I can easily build interfaces for each class individualy, I'm having difficulty on the implementation when these classes interact with each other. Somewhere in the code, I need tables to be able to accept an collection of cells.
The problem arrives when I have a have an collection of cells, and I need to add it to the table. Somehow I need to transform the IPdfCell into the original element that is accepted by the library (iTextSharp). I believe the quick and easy implementation was downcasting, but not a good design.
The only other solution I can think is using the interface to collection varies settings and create the orginal element (accepted by iTextsharp) on the fly when it is being passed around into other elements.
Is there a better implementation?
I don't understand the goal of such an interface. No matter what, your pdf generation code is going to be tied to the library you're using, building proxy classes/interfaces that exactly mirror the library doesn't prevent that, it just adds another layer. If you were to switch to a different PDF generation library, it is extremely unlikely that the new library would match up 1:1 with equivalents in iText, and you'd end up changing the interface and calling code anyway.
My recommendation would be to create an IPDFGenerator, and then create an iTextPDFGenerator:IPDFGenerator which IS coupled to iText, and leave it at that. You'll have the separation you want, the ability to keep iText out of your core services, but it won't require a bunch of pointless mapping between identical classes.
I think you should reevaluate the S, I, and D in SOLID and make sure you're not over-doing it.
Typically, you would have a translation layer that translates your interface facade implementation into the types supported natively by iTextSharp.
AutoMapper can help for this if you require property to property mappings and can remove a lot of the translation legwork for you.
If my domain object should contain string properties in 2 languages, should I create 2 separate properties or create a new type BiLingualString?
For example in plant classification application, the plant domain object can contain Plant.LatName and Plant.EngName.
The number of bi-lingual properties for the whole domain is not big, about 6-8, I need only to support two languages, information should be presented to UI in both languages at the same time. (so this is not locallization). The requirements will not change during development.
It may look like an easy question, but this decision will have impact on validation, persistance, object cloning and many other things.
Negative sides I can think of using new dualString type:
Validation: If i'm going to use DataAnattations, Enterprise Library validation block, Flued validation this will require more work, object graph validation is harder than simple property validation.
Persistance: iether NH or EF will require more work with complex properties.
OOP: more complex object initialization, I will have to initialize this new Type in constructor before I can use it.
Architecture: converting objects for passing them between layers is harder, auto mapping tools will require more hand work.
While reading your question I was thinking about why not localization all the time but when I read information should be presented to UI in both languages at the same time. I think it makes sense to use properties.
In this case I would go for a class with one string for each languages as you have mentioned BiLingualString
public class Names
{
public string EngName {get;set;}
public string LatName {get;set;}
}
Then I would use this class in my main Plant Class like this
public class Plant: Names
{
}
If you 100% sure that it will always be only Latin and English I would just stick with simplest solution - 2 string properties. It also more flexible in UI then having BiLingualString. And you won't have to deal with Complex types when persisting.
To help decide, I suggest considering how consistent this behavior will be at all layers. If you expose these as two separate properties on the business object, I would also expect to see it stored as two separate columns in a database record, for example, rather than two translations for the same property stored in a separate table. It does seem odd to store translations this way, but your justifications sound reasonable, and 6 properties is not un-managable. But be sure that you don't intend to add more languages in the future.
If you expect this system to by somewhat dynamic in that you may need to add another language at some point, it would seem to make more sense to me to implement this differently so that you don't have to alter the schema when a new language needs to be supported.
I guess the thing to balance is this: consider the likelihood of having to adjust the languages or properties to accommodate a new language against the advantage (simplicity) you gain by exposing these directly as separate properties rather than having to load translations as a separate level.