Develop Reference

IEnumerable<T> GroupBy with Fluent Validation

So the problem I am trying to solve is that I am using a lot of generic classes with <T> that need to execute a .NET Async REST call to retrieve an IEnumerable<T> list of objects from an API. At runtime things are resolved fine with the T stuff because I have some concrete instances higher up the chain.
I have a worker class:
public class Worker<T> where T : class, new()
That has a REST client factory:
IBatchClientFactory batchClientFactory
where in that factory basically creates an instance of this:
public class BatchClient<T> where T : class, new()
That BatchClient has an important method:
public BaseBatchResponse<T> RetrieveManyAsync(int top = 100, int skip = 0)
so that the worker class's method does something like:
var batchClient = this.batchClientFactory.Create<T>(siteId);
var batchResponse = await batchClient.RetrieveManyAsync(top, skip);
Batch Response looks like:
public class BaseBatchResponse<T>
{
public List<T> Value { get; set; }
public BaseBatchResponse<T> Combine(BaseBatchResponse<T> baseBatchResponse)
{
return new BaseBatchResponse<T>
{
Value = this.Value.Concat(baseBatchResponse.Value).ToList()
};
}
}
Now at runtime things are ok because higher up the chain i will instantiate Worker into something like.. new Worker<Appointment>(); And the T's will all just work perfectly since everything down the chain is just doing generics.
My problem now is that I would like to evaluate my batchResponse and go through the List and run some validation against each element in the list. I saw this article on stack overflow that seems to let you split a list into 2 lists using GroupBy via a Dictionary where some SomeProp is the thing you're splitting around.. but can you do that GroupBy logic using a method call? And more importantly can I use FluentValidation as that method call? Ideally my code would look like:
var groups = allValues.GroupBy(val => validationService.Validate(val)).ToDictionary(g => g.Key, g => g.ToList());
List<T> valids = groups[true];
List<T> invalids= groups[false];
Where the result would be a List of my objects that are valid, and a second List of my objects that are invalid.
Ideally I would then just make a FluentValidation class that binds to my concreate Appointment class and has a rule inside it:
this.When(x => !string.IsNullOrWhiteSpace(x.Description), () =>
this.RuleFor(x => x.Description).Length(1, 4000));
Which will hook everything together and be used to determine if my object at runtime belongs in the valids or invalids list

I'm not sure fluent means, there is a approch to achieve that using LINQ:
using System.Collections.Generic;
using System.Linq;
namespace Investigate.Samples.Linq
{
class Program
{
public class SomeEntity
{
public string Description { get; set; }
}
static void Main(string[] args)
{
//Mock some entities
List<SomeEntity> someEntities = new List<SomeEntity>()
{
new SomeEntity() { Description = "" },
new SomeEntity() { Description = "1" },
new SomeEntity() { Description = "I am good" },
};
//Linq: Where to filter out invalids, then category to result with ToDictionary
Dictionary<bool, SomeEntity> filteredAndVlidated = someEntities.Where(p => !string.IsNullOrWhiteSpace(p.Description)).ToDictionary(p => (p.Description.Length > 1));
/* Output:
* False: new SomeEntity() { Description = "1" }
* True: new SomeEntity() { Description = "I am good" }
* */
}
}
}
Code segment:
Dictionary<bool, SomeEntity> filteredAndVlidated = someEntities.Where(p => !string.IsNullOrWhiteSpace(p.Description)).ToDictionary(p => (p.Description.Length > 1));

Automapper does not map properly null List member, when the condition != null is specified

There is a problem when I try to map a null list (member) of an object, considering that I specified:
.ForAllMembers(opts => opts.Condition((src, dest, srcMember) =>
srcMember != null
));
cfg.AllowNullCollections = true; // didn't help also
short example from code:
gi.PersonList = new List<Person>();
gi.PersonList.Add(new Person { Num = 1, Name = "John", Surname = "Scott" });
GeneralInfo gi2 = new GeneralInfo();
gi2.Qty = 3;
Mapper.Map<GeneralInfo, GeneralInfo>(gi2, gi);
gi.PersonList.Count = 0, how to fix that?
using System;
using System.Collections.Generic;
using AutoMapper;
public class Program
{
public static void Main(string[] args)
{
Mapper.Initialize(cfg =>
{
cfg.AllowNullCollections = true;
cfg.CreateMap<GeneralInfo, GeneralInfo>()
.ForAllMembers(opts => opts.Condition((src, dest, srcMember) =>
srcMember != null
));
});
GeneralInfo gi = new GeneralInfo();
gi.Descr = "Test";
gi.Dt = DateTime.Now;
gi.Qty = 1;
gi.PersonList = new List<Person>();
gi.PersonList.Add(new Person { Num = 1, Name = "John", Surname = "Scott" });
GeneralInfo gi2 = new GeneralInfo();
gi2.Qty = 3;
Console.WriteLine("Count antes de mapeo = " + gi.PersonList.Count);
Mapper.Map<GeneralInfo, GeneralInfo>(gi2, gi);
Console.WriteLine("Count despues de mapeo = " + gi.PersonList.Count);
// Error : gi.PersonList.Count == 0 !!!!
//por que? si arriba esta: Condition((src, dest, srcMember) => srcMember != null ...
}
}
class Person
{
public int Num { get; set; }
public string Name { get; set; }
public string Surname { get; set; }
}
class GeneralInfo
{
public int? Qty { get; set; }
public DateTime? Dt { get; set; }
public string Descr { get; set; }
public List<Person> PersonList { get; set; }
}
https://dotnetfiddle.net/N8fyJh

This should work but I'm not sure if you want to micro manage it like that:
cfg.AllowNullCollections = true;
cfg.CreateMap<GeneralInfo, GeneralInfo>()
.ForMember(x => x.PersonList, opts => opts.PreCondition((src) => src.PersonList != null));
Problem is the collections that are handled specifically (that's true for most mappers though AutoMapper is a bit weird in this case, it's not my favorite) and seem to require the destination collection to be initialized. As I can see, collections are not copied in entirety which makes sense, but you need to initialize and copy individual items (this is my deduction but does sound right).
I.e. even if you skip the source, destination would still end up reinitialized (empty).
Problem as it seems is the Condition which is, given their documentation, applied at some later point, at which time the destination has already been initialized.
PreCondition on the other hand has a different signature to be used like you intended, as it doesn't take actual values, just source is available.
The only solution that seems to work is to use "per member" PreCondition (like the above).
EDIT:
...or this (using the ForAllMembers), but a bit ugly, reflection etc.
cfg.CreateMap<GeneralInfo, GeneralInfo>()
.ForAllMembers(opts =>
{
opts.PreCondition((src, context) =>
{
// we can do this as you have a mapping in between the same types and no special handling
// (i.e. destination member is the same as the source property)
var property = opts.DestinationMember as System.Reflection.PropertyInfo;
if (property == null) throw new InvalidOperationException();
var value = property.GetValue(src);
return value != null;
});
}
);
...but there doesn't seem to be any cleaner support for this.
EDIT (BUG & FINAL THOUGHTS):
Conditional mapping to existing collection doesn't work from version 5.2.0 #1918
As pointed out in the comment (by #LucianBargaoanu), this seems to be a bug really, as it's inconsistent in this 'corner' case (though I wouldn't agree on that, it's a pretty typical scenario) when mapping collections and passing the destination. And it pretty much renders the Condition useless in this case as the destination is already initialized/cleared.
The only solution indeed seems to be the PreCondition (but it has issues given the different signature, I'm personally not sure why they don't pass the same plethora of parameters into the PreCondition as well?).
And some more info here:
the relevant code (I think)
nest collection is clear when using Condition but not Ignore #1940
Collection property on destination object is overwritten despite Condition() returning false #2111
Null source collection emptying destination collection #2031

Try like this;
gi = Mapper.Map<GeneralInfo, GeneralInfo>(gi2);
I encounterted with this problem recently and somehow if the destination and sources are same type, the destination parameter doesn't work as expected.
Also, I want to notify that it isn't relevant with just Collection objects. If you debug the gi object you will see that the other properties are remaining with old values too. Somehow Automapper doesn't change the property values which assigned before if you pass the destination instance as destionation parameter. I think, it is mostly relevant that Automapper was not designed for creating copy/clone objects.

MongoDB Linq OfType() on fields

This is my MongoDB document structure:
{
string _id;
ObservableCollection<DataElement> PartData;
ObservableCollection<DataElement> SensorData;
...
other ObservableCollection<DataElement> fields
...
other types and fields
...
}
Is there any possibility to retrieve a concatenation of fields with the type ObservableCollection<DataElement>? Using LINQ I would do something like
var query = dbCollection
.AsQueryable()
.Select(x => new {
data = x
.OfType(typeof(ObservableCollection<DataElement>))
.SelectMany(x => x)
.ToList()
});
or alternatively
data = x.Where(y => typeof(y) == typeof(ObservableCollection<DataElement>)
.SelectMany(x => x).ToList()
Unfortunately .Where() and .OfType() do not work on documents, only on queryables/lists, so is there another possibility to achieve this? The document structure must stay the same.
Edit:
After dnickless answer I tried it with method 1b), which works pretty well for getting the fields thy way they are in the collection. Thank you!
Unfortunately it wasn't precisely what I was looking for, as I wanted to be all those fields with that specific type put together in one List, at it would be returned by the OfType or Where(typeof) statement.
e.g. data = [x.PartData , x.SensorData, ...] with data being an ObsverableCollection<DataElement>[], so that I can use SelectMany() on that to finally get the concatenation of all sequences.
Sorry for asking the question unprecisely and not including the last step of doing a SelectMany()/Concat()
Finally I found a solution doing this, but it doesn't seem very elegant to me, as it needs one concat() for every element (and I have more of them) and it needs to make a new collection when finding a non-existing field:
query.Select(x => new
{
part = x.PartData ?? new ObservableCollection<DataElement>(),
sensor = x.SensorData ?? new ObservableCollection<DataElement>(),
}
)
.Select(x => new
{
dataElements = x.part.Concat(x.sensor)
}
).ToList()

In order to limit the fields returned you would need to use the MongoDB Projection feature in one way or the other.
There's a few alternatives depending on your specific requirements that I can think of:
Option 1a (fairly static approach): Create a custom type with only the fields that you are interested in if you know them upfront. Something like this:
public class OnlyWhatWeAreInterestedIn
{
public ObservableCollection<DataElement> PartData { get; set; }
public ObservableCollection<DataElement> SensorData { get; set; }
// ...
}
Then you can query your Collection like that:
var collection = new MongoClient().GetDatabase("test").GetCollection<OnlyWhatWeAreInterestedIn>("test");
var result = collection.Find(FilterDefinition<OnlyWhatWeAreInterestedIn>.Empty);
Using this approach you get a nicely typed result back without the need for custom projections.
Option 1b (still pretty static): A minor variation of Option 1a, just without a new explicit type but a projection stage instead to limit the returned fields. Kind of like that:
var collection = new MongoClient().GetDatabase("test").GetCollection<Test>("test");
var result = collection.Find(FilterDefinition<Test>.Empty).Project(t => new { t.PartData, t.SensorData }).ToList();
Again, you get a nicely typed C# entity back that you can continue to operate on.
Option 2: Use some dark reflection magic in order to dynamically create a projection stage. Downside: You won't get a typed instance reflecting your properties but instead a BsonDocument so you will have to deal with that afterwards. Also, if you have any custom MongoDB mappings in place, you would need to add some code to deal with them.
Here's the full example code:
First, your entities:
public class Test
{
string _id;
public ObservableCollection<DataElement> PartData { get; set; }
public ObservableCollection<DataElement> SensorData { get; set; }
// just to have one additional property that will not be part of the returned document
public string TestString { get; set; }
}
public class DataElement
{
}
And then the test program:
public class Program
{
static void Main(string[] args)
{
var collection = new MongoClient().GetDatabase("test").GetCollection<Test>("test");
// insert test record
collection.InsertOne(
new Test
{
PartData = new ObservableCollection<DataElement>(
new ObservableCollection<DataElement>
{
new DataElement(),
new DataElement()
}),
SensorData = new ObservableCollection<DataElement>(
new ObservableCollection<DataElement>
{
new DataElement(),
new DataElement()
}),
TestString = "SomeString"
});
// here, we use reflection to find the relevant properties
var allPropertiesThatWeAreLookingFor = typeof(Test).GetProperties().Where(p => typeof(ObservableCollection<DataElement>).IsAssignableFrom(p.PropertyType));
// create a string of all properties that we are interested in with a ":1" appended so MongoDB will return these fields only
// in our example, this will look like
// "PartData:1,SensorData:1"
var mongoDbProjection = string.Join(",", allPropertiesThatWeAreLookingFor.Select(p => $"{p.Name}:1"));
// we do not want MongoDB to return the _id field because it's not of the selected type but would be returned by default otherwise
mongoDbProjection += ",_id:0";
var result = collection.Find(FilterDefinition<Test>.Empty).Project($"{{{mongoDbProjection}}}").ToList();
Console.ReadLine();
}
}

List<T> LINQ Projection to Anonymous or Dynamic type

I'm trying to convert a List<Topic> to an anonymous or dynamic type via linq projection... I'm am using the following code, but it doesn't seem to work properly. It returns the dynamic type without error, however, if I try to enumerate the children field (list<object/topic>) then it says
Results View = The type '<>f__AnonymousType6<id,title,children>' exists in both 'MyWebCore.dll' and 'MvcExtensions.dll'
Strange.
Here is the code I am using:
protected dynamic FlattenTopics()
{
Func<List<Topic>, object> _Flatten = null; // satisfy recursion re-use
_Flatten = (topList) =>
{
if (topList == null) return null;
var projection = from tops in topList
select new
{
id = tops.Id,
title = tops.Name,
children = _Flatten(childs.Children.ToList<Topic>())
};
dynamic transformed = projection;
return transformed;
};
var topics = from tops in Repository.Query<Topic>().ToList()
select new
{
id = tops.Id,
title = tops.Name,
children = _Flatten(tops.Children.ToList<Topic>())
};
return topics;
}
All i'm doing is flattening a list of self containing objects - basically it's a list of POCOs that will be stuffed into a tree view (jstree).
The Topic class is defined as:
public class Topic
{
public Guid Id {get;set;}
public string Name {get;set;}
public List<Topic> Children {get;set;}
}
And here is an example of what the first member of the returned dynamic object looks like:
[0] = {
id = {566697be-b336-42bc-9549-9feb0022f348},
title = "AUTO",
children = {System.Linq.Enumerable.SelectManyIterator
<MyWeb.Models.Topic,
MyWeb.Models.Topic,
<>f__AnonymousType6<System.Guid,string,object>
>}
}

Why do you have the same LINQ code twice? After you define your _Flatten func, you can just call it immediately - var topics = _Flatten(Repository.Query<Topic>().ToList().
It looks like you're creating two identical anonymous types, one inside the _Flatten func and one outside it. I would think the compiler is smart enough to handle that, but try changing your call to explicitly use _Flatten, see if it solves the problem.

Here is the proper way - have to load into the a DTO / POCO and return that:
_Flatten = (topList) =>
{
if (topList == null) return null;
var projection = from tops in topList
//from childs in tops.Children
select new JsTreeJsonNode
{
//id = tops.Id.ToString(),
data = tops.Name,
attr = setAttributes(tops.Id.ToString(), tops.URI),
state = "closed",
children = _Flatten(tops.Children)
};
return projection.ToList();
};

A more concise syntax in C# constructor?

I have a constructor something like the following:
using Microsoft.Data.Extensions;
public class Complaint
{
public int Id {get; set;}
public int Transcript {get; set;}
//... etc. ... Lots more properties
public Complaint(int id)
{
var command = dataContext.CreateStoreCommand(
"dbo.stp_Complaint_Get",
CommandType.StoredProcedure,
new SqlParameter("Id", id));
var complaint = command.Materialize(x =>
new Complaint
{
Id = x.Field<int>("Id"),
Transcript = x.Field<string>("Transcript");
//... etc. ... Lots more fields from db
}
this.Id = complaint.Id;
this.Transcript = complaint.Transcript;
//... etc. ... Lots more properties to set
}
}
Is there a syntax in C# that would allow me to carry out the last part in one step instead of two? i.e. conceptually something like this:
this = command.Materialize(x =>
new Complaint
{
Id = x.Field<int>("Id"),
Transcript = x.Field<string>("Transcript");
}

Well, you could use a static method instead of a constructor:
public static Complaint FromId(int id)
{
var command = dataContext.CreateStoreCommand(
"dbo.stp_Complaint_Get",
CommandType.StoredProcedure,
new SqlParameter("Id", id));
return command.Materialize(x =>
new Complaint
{
Id = x.Field<int>("Id"),
Transcript = x.Field<string>("Transcript");
//... etc. ... Lots more fields from db
});
}

Not inherently. You could store the complaint object inside the class, and have all the properties point off that rather than setting them all from the constructor.
eg
public class Complaint
{
private readonly {type of complaint} m_Complaint;
public int Id
{
get { return m_Complaint.Id; }
}
// ...etc
}
You could get more complicated if you don't have a setter on m_Complaint - keeping nullable backing fields, and check that before you access the m_Complaint properties

I believe you may try something like this:
var currentInstance = this;
var complaint = command.Materialize(x =>
new Complaint
{
Id = currentInstance.Id = x.Field("Id"),
Transcript = currentInstance.Transcript = x.Field("Transcript");
//... etc. ... Lots more fields from db
});
I don't think you can capture this in the closure, so using that currentInstance trick should help, however it may as well turn out to be redundant.
Besides, code such as that one is sort of obfuscated compared to your current solution. I believe that your code is pretty fine as it is.

I gather that you're trying to avoid setting all of those properties twice, because any change to the structure requires you to update the properties in two places, is that correct?. You could try to use some reflection to copy the properties from one instance to another.
var complaint = command.Materialize(x =>
new Complaint
{
Id = x.Field<int>("Id"),
Transcript = x.Field<string>("Transcript");
//... etc. ... Lots more fields from db
}
foreach (var prop in typeof(Complaint).GetProperties())
...

Have the complaint object as a member variable and the get/set properties accesses the underlying complaint's properties?