Dynamically mapping nested objects with CSVHelper - c#

I'm using CSVHelper (thanks Josh Close) to read a CSV file which works great. I'm now trying to use it to map that file to some internal classes; however, the CSV's I'm mapping vary by customer but all need to map to my internal classes. I need to allow the customer to define how the CSV maps to my POCO objects.
I'm storing the customer defined mappings as Dictionary<string,int> -- ["Firstname", 20],["Lastname",21],["Address.Line1",30], ["Address.Line2",31], etc.
I have a dynamic map class that works for dynamically mapping based on a given mapping at runtime. My problem lies in dealing with reference type properties. Here is what I have so far.
POCO Classes
public class Client
{
public int Id {get; set;}
public string Firstname {get; set;}
public string Lastname {get; set;}
public Address Address {get; set;}
...
}
public class Address
{
public string Line1 {get; set;}
public string Line2 {get; set;}
public string City {get; set;}
...
}
Based on a few posts that I've run across here and here, I came up with the following that uses a defined mapping to map a CSV dynamically.
Dynamic Map
public class BaseCSVMap<T> : ClassMap<T> where T : class
{
public void CreateMap(Dictionary<string,int> mappings)
{
foreach(var mapping in mappings)
{
var propname = mapping.Key;
var csvIndex = mapping.Value;
var member = typeof(T).GetProperty(propname);
Map(typeof(T), member).Index(csvIndex);
}
}
}
Using Dynamic Map
var id = 2; //Customer 2
var mappings = dataContext.Mappings.Where(m => m.id = id); //Get customer 2's map
using(var reader = File.OpenText(#"c:\temp\testfile.csv"))
{
var csv = new CsvReader(reader);
csv.Configuration.HasHeaderRecord = true; //hardcoded for now
var map = new BaseCSVMap<Client>();
map.CreateMap(mappings);
csv.Configuration.RegisterClassMap(map);
var records = csv.GetRecords<Client>();
}
I added the following in my BaseCSVMap<T> class, which works great if all my reference properties are strings, but doesn't work so well when a property is something else.
var member = typeof(T).GetProperty(propname);
//New code
//Mapping would look like ["Address.Line1",78]
if(member.GetType().IsClass)
{
string exp = $"c.{propname}";
var p = Expression.Parameter(typeof(T), "c");
var e = System.Linq.Dynamic.Core.DynamicExpressionParser.ParseLambda(new [] {p}, null, exp);
Map((Expression<Func<T,string>>)e).Index(csvIndex);
}
I also looked for a way to take advantage of the Reference mapping that CSVHelper makes available, but was unable to figure out how to do that in a dynamic fashion.
Looking for some guidance in how to accomplish defining a dynamic map for a reference type with CSVHelper.

Related

Circular reference issue in webapi2

Scenario:
An Intern can learn multiple technologies
db design
ef view
Result
controller code:
private InternEntities db = new InternEntities();
// GET: api/Interns
public IQueryable<Intern> GetInterns()
{
return db.Interns;
}
What am i doing wrong here?
This is an expected error, and the reason is because your types reference each other like an Infinity Mirror. In order to solve the problem, you have several options.
1- You can develop a ViewModel and then serialize that one:
public class InternViewModel{
public int Id {get; set;}
public String Name {get; set;}
public List<String> Tehcnologies {get; set;}
}
2- You can select the properties that you need when returning the entity in your actions:
public async Task<List<Technology>> Get() {
var data = dbContext.Set<Technology>().Select(x=> new Technology{
Id = x.Id,
Name = x.Name,
Intern= new Intern {
Id = x.Technology.Id,
Name = x.Technology.Name,
Technologies = null
}
});
return await data.ToListAsync();
}
3- Load only the what you need which is known as Explicit Loading.

automapper with runtime mapping configuration

In my ASP.NET MVC application I need to implemenet mapping from one object to another with some kind of UI for mapping configuration in runtime, so the user can define mapping "on the go". Is there any libraries that supports such functionality?
Description
This is objects in my application. I need to somehow allow user to configure mapping of this objects via UI during application runs. For exmaple some kind of page in my application where user will be able to define mapping in simple way like so map Amout of OrderDTO to Order Qty and later without application recompile change this mapping for exmaple for ExactAmmount
//Object in DAL
public class Order
{
public int Id {get; set;}
public string Name {get; set;}
public decimal Qty {get; set;}
//Lots of other fields
}
//Object from XSD generation (for example)
public class OrderDTO
{
public int Id {get; set;}
public string Description {get; set;}
public decimal Ammout {get; set;}
public decimal VAT {get; set;}
public decimal ExactAmmount {get; set;}
//Lots of other fields
}
Note: for legacy reasons I based this answer on AutoMapper 4.2.1 instead of the current 5.x version. The overall approach should be similar with the new version.
It is possible to create different mapping configurations and different mappers within a program. Also, it is possible to create member mappings by member names (string) instead of lambda expressions. However, some static type information is still necessary (as far as my example goes).
See the following example of a profile, that prepares a custom mapping based on property names:
class MemberProfile : Profile
{
private string from;
private string to;
public MemberProfile(string from, string to)
{
this.from = from;
this.to = to;
}
protected override void Configure()
{
this.CreateMap<Order, OrderDTO>()
.ForMember(to, c => c.MapFrom<decimal>(from));
}
}
This could be extended to support different source property types and a collection of custom mappings instead of a single one.
Usage example:
var order = new Order() { Id = 1, Name = "Test", Qty = 0.5m };
var conf1 = new MapperConfiguration(c => c.AddProfile(new MemberProfile("Qty", "Ammout")));
var conf2 = new MapperConfiguration(c => c.AddProfile(new MemberProfile("Qty", "ExactAmmount")));
var res1 = conf1.CreateMapper().Map<OrderDTO>(order);
var res2 = conf2.CreateMapper().Map<OrderDTO>(order);
For res1, Qty is mapped to Ammout and for res2, Qty is mapped to ExactAmmount. Since the difference is described as string property names, it should be possible to let the user influence this configuration.

RavenDB SelectMany alternative

Am building a Photo gallery using Nancy and RavenDB
I have my model classes as below:
[JsonObject(IsReference=true)]
public class Album
{
public Album()
{
this.Photos = new List<Photo>();
this.Tags = new List<String>();
}
public String Id {get; set;}
public String Name {get; set;}
public String Path {get; set;}
public DateTime ModifiedDate{get; set;}
public IList<String> Tags {get; set;}
public IList<Photo> Photos {get; set;}
}
public class Photo
{
public String Id {get; set;}
public String Name {get; set;}
public String Path {get; set;}
public IList<String> Tags {get; set;}
public Album Album {get; set;}
public DateTime ModifiedDate{get; set;}
public bool IsPrivate{get; set;}
}
And my attempt at map reduce index on Photo->Tags:
public class TaggedPhotosIndex:AbstractIndexCreationTask<Album, TaggedPhotos>
{
public TaggedPhotosIndex()
{
Map = albums =>
from a in albums
from p in a.Photos
from t in p.Tags
select new TaggedPhotos
{
Tag = t,
PhotoIds = new List<String> {p.Id}
};
Reduce = results =>
from result in results
group result by result.Tag into agg
select new TaggedPhotos
{
Tag = agg.Key,
PhotoIds = agg.SelectMany(a => a.PhotoIds).ToList()
};
}
}
public class TaggedPhotos
{
public String Tag {get; set;}
public IList<String> PhotoIds {get; set;}
}
Here is what I want to achieve:
Given an array of tags, I want to get all the Photo objects that have atleast one matching tag.
RavenDB doesn't allow SelectMany in the query and am out of ideas.
Found a solution (using LuceneQuery # Workaround for selectmany in ravendb using client api) , but looking forward for any other alternatives.
There are some issues with your document structure that are going to make your query difficult.
Queries are designed to return whole documents. You are asking for a partial result from the document. That can be done, but only if you store all fields in the index and project the results from there. You give up the ACID guarantees of the document store, which is one of RavenDB's strongest features.
You have a reference from Photo back to Album which would normally be embedded, causing a circular reference, except you are setting [JsonObject(IsReference=true)] to avoid that. This may work for basic serialization within a single document, but it is meaningless when it comes to referencing a whole document back from a projected index value. You've set up your own "chicken & egg" problem.
You still are missing basic functionality one would expect from this problem domain. Specifically, you should be able to load a single photo without having to load the whole album.
I strongly suggest that you put photos in their own documents. Then you can have much simpler indexes and avoid the circular references. In DDD terms, both Photo and Album are aggregates. And a DDD aggregate == a RavenDB document. You are currently modeling the Photo as an entity that is not an aggregate, but then you are asking for operations that one would only do against an aggregate - like the search.
If you were asking only "give me all albums that contain a photo tagged with one of these tags", then you would be ok. But you're not asking for the albums, you're asking for the photos.
Assuming you did model it that way, Brett's answer is close - but has some extraneous stuff. Here is a unit test showing a full implementation. Posting on gist since it does not directly address the original question. https://gist.github.com/4499724
This one took me a while but I verified the solution below is working.
First, modify your Index and index result as shown below.
public class TaggedPhotosIndex:AbstractIndexCreationTask<Photo, TaggedPhotos>
{
public TaggedPhotosIndex()
{
Map = photos =>
from p in Photos
from t in p.Tags
select new TaggedPhotos
{
Tag = t,
PhotoId = p.Id
};
}
}
public class TaggedPhotos
{
public string Tag {get; set;}
public string PhotoId {get; set;}
}
Now that the index is created, here is how you query against it to get your photos.
var tagsToSearch = new List<string>(){"test1", "test2", "test3"};
var photoIds = documentSession.Query<TaggedPhotos, TaggedPhotoIndex>()
.Customize(x => x.Include<Photo>(p => p.Id))
.Where(x => x.Tag.In(tagsToSearch))
.Select(x => x.PhotoId)
.Distinct()
.ToArray();
// This doesn't actually make a second call as the photos are already loaded
// in the document session
var photos = documentSession.Load<Photo>(photoIds);
Based on the requirement
Here is what I want to achieve:
Given an array of tags, I want to get all the Photo objects that have
at least one matching tag.
I didn't see how album really tied into it all

C# dynamic casts when using Entity Framework for insertion

I have a database which contains tables of mapping values for a variety of objects.
For example the Colour table contains BLK > Black, BLU > BLUE, ORA > ORANGE etc.. and the CarType table contains 4DH > Four-door hatchback, 4WD > Four wheel drive etc...
I'm using Entity Framework code-first so I have a context set up something like this.
public class MappingContext : DbContext
{
public MappingContext ()
: base("Mappings")
{
}
public DbSet<Colour> ColourMappings { get; set; }
public DbSet<CarType> CarTypeMappings { get; set; }
}
Every object that relates to each table in my Mapping database inherits from a base class like so:
public class Mapping
{
public int Id { get; set; }
public string OrignalValue { get; set; }
public string MappedValue { get; set; }
}
public class CarType : Mapping{}
public class Colour : Mapping{}
Now what I want to do is read these mappings in from an XML file filled with "Templates" which contain the mappings and insert them in the DB.
I have the following method to do this:
public void InsertMappings(XDocument xml, string templateName, Type typeToInsert)
{
// Here I find the relevent mappings
using (var repo = new Repository())
{
var mapppings = mappings.Select(mapping => new Mapping
{
MappedValue = mapping.Value,
OrignalValue = GetCode(mapping)
});
foreach (var mapping in mapppings.ToList())
{
var map = (typeToInsert)mapping // << This line will not compile
repo.Insert(map);
}
repo.Save();
}
}
This will not complie as it doesnt recognise the attempted cast "(typeToInsert)mapping".
So basically what I need to know is how to I cast this Mapping object to a Specific Mapping object when it comes to inserting it into the db? Or any suggestions for a better way of doing this!
From the looks of it you are trying to cast an instance of Mapping as a CarType or Colour which won't work because Mapping doesn't know anything about those types as it's the base class.
Your code would need to create an instance of the concrete type i.e. typeToInsert and cast it as Mapping. You could do something like:
public void InsertMappings(XDocument xml, string templateName, Type typeToInsert)
{
// Here I find the relevent mappings
using (var repo = new Repository())
{
foreach (var m in mappings)
{
// XML -> Entity
var mapping = (typeToInsert)Activator.CreateInstance(typeToInsert);
(mapping as Mapping).MappedValue = m.Value;
(mapping as Mapping).OriginalValue = GetCode(m);
// Update database
repo.Insert(mapping);
}
repo.Save();
}
}
You should probably be making use of generics here as well, you could refactor your method to look like:
public void InsertMappings<T>(XDocument xml, string templateName)
{
// Here I find the relevent mappings
using (var repo = new Repository())
{
foreach (var m in mappings)
{
// XML -> Entity
var mapping = (T)Activator.CreateInstance(typeof(T));
(mapping as Mapping).MappedValue = m.Value;
(mapping as Mapping).OriginalValue = GetCode(m);
// Update database
repo.Insert(mapping);
}
repo.Save();
}
}
Usage
InsertMappings<CarType>(xmlDoc, "CarTemplate");
InsertMappings<Colour>(xmlDoc, "ColourTemplate");

How to set this from Entity Framework using Linq

A while back, I fell into the fat controller trap when I was first working with MVC. My first app used EF4 to make all the models I needed. I just put all my logic into my controller actions. While it worked, it's definitely not the best practice way. To do it the right way I started trying to build my models based on my EF objects in an effort to follow the skinny controller concept.
I've run into a roadblock in trying to find the best way to populate my models. Is there a way to run a LINQ query and have it populate your model without having to iterate through the properties to set to another class?
Something like this:
// from EF model built from database
public class MyEFObject
{
public int ID {get; set;}
public string Name {get; set;}
public string Title {get; set;}
}
public class MyObjectModel : MyEFObject
{
private Entities _data = new Entities();
public MyObjectModel(int? id)
{
if(id.HasValue) // get an existing record
{
this = _data.MyEFObjects.Where(m => m.ID.Equals(id)).Single();
// or populate right out of the query
_data.MyEFObjects.Where(m => m.ID.Equals(id))
.Select(o => new {
this.ID = o.ID,
this.Name = o.Name,
this.Title = o.Title
});
}
else
{
// set defaults for a new MyObjectModel
}
}
public void Save()
{
// takes the current object and saves changes
}
}
I know you can add a function to the EF Entity object, but I like having the option to Create or Update all tied up in one call (Save method). I don't see the point messing with a model if I have to essentially recreate what I already have from my EF Object. If I simply have a method on a class that accepts a populated object, the concept of a usable model for my views is negated.
Slauma is right. LINQ to Entities won't accept it. I tried a couple of versions of what was posted and I only found my self with a kludgy mess. I got it to the point where I could set instance values, but by then EF wouldn't register a change had been made and defeating the whole purpose. There may be a way to do this, but as of now, the steps to make it work seem to be overkill.
I ended up with something like this:
public class MyObjectModel : MyEFObject
{
public void Save(int? id, MyObjectModel model)
{
var data = new Entities();
MyEFObject foo;
if(id.HasValue)
{
foo = data.MyEFObjects.Where(e => e.ID.Equals(id.Value)).Single();
}
else
{
foo = new MyEFObject();
}
foo.Name = model.Name;
foo.Title = model.Title;
if(!id.HasValue)
{
data.MyEFObjects.AddObject(foo);
}
data.SaveChanges();
}
}
I didn't want to have to work with two instances of my model, but it works and I have my lean controller action.
What you could do is have a domain model, ef model and and adapter. I think this keeps the code pretty clean and nicely separates the mapping logic.
//Domain model to decouple from EF
public class MyObjectModel
{
public int ID {get; set;}
public string Name {get; set;}
public string Title {get; set;}
}
//Auto generated Entity Framework class
public class MyEFObject
{
public int ID {get; set;}
public string Name {get; set;}
public string Title {get; set;}
}
//Adapter responsible for mapping your data to your domain model
public class MyObjectModelAdapter : MyEFObject
{
public MyObjectModelAdapter(MyEFObject entity)
{
if(entity != null)
{
this.ID = entity.ID;
this.Name = entity.Name;
this.Title = entity.Title;
}
else
{
// set defaults for a new MyObjectModel
}
}
}
Then the basic usage would be:
new Entities().MyEFObjects.ToList().Select(x => new MyObjectModelAdapter(x));
OR
new MyObjectModelAdapter(new Entities().MyEFObjects.FirstOrDefault(x => x.ID.Equals(objectId)));
If you specifically require a list of MyObjectModel then you could do the following:
new Entities().MyEFObjects.ToList().Select(x => new MyObjectModelAdapter(x) as MyObjectModel);
OR
new MyObjectModelAdapter(new Entities().MyEFObjects.FirstOrDefault(x => x.ID.Equals(objectId)) as MyObjectModel;
Of course you don't want to chain your entity context together like that, it is just to show usage.

Categories

Resources