Develop Reference

Map [name,value] string values to class without reflection

I'm having a huge performance issue about mapping string property names and string property values to classes using reflection.
My issue now:
public class Person
{
public string Property1 { get; set; }
public string Property2 { get; set; }
public string Property3 { get; set; }
public string Property4 { get; set; }
// My class has around 100 properties
public string Property100 { get; set; }
}
I am mapping a key value pair collection to the class using reflection
[{"Property1": "some value"}, {"Property2": "something else"},{"Property3","Property4","value" }.....{"Property100","val"}]
It got to the point that I am now mapping around 10 000 class instances using reflection and the performance is to say it lightly bad.
Any ideas for eliminating the reflection would be greatly appreciated.

I see two options, if you need to avoid reflection for tasks like this(when code could be programatically generated).
First is Expressions I use it often, e.g. I saw some people write something like this
public class A
{
public Prop1 ...
....
public Prop100
public override ToString() => $"{nameof(Prop1)}={Prop1};...";
and so for all 100 properties, and always doing this manually.
And with Expression it can be easily automated, you just need to generate Expression for String.Concat and pass list of properties and names there.
For your example, it is not clear what are your data. How do you do lookup in the list?
Let's assume there is a dictionary<string,string>(you can transform your list of tuples to a dictionary), and all properties are strings as well.
Then we would need to generate a list assignment expressions like this
if(data.ContainsKey("Prop1")) result.Prop1 = data["Prop1"];
And the code would be complicated, anyway it would look like this
private static class CompiledDelegate<T>
{
public static Action<T, Dictionary<string, string>> initObject;
static CompiledDelegate()
{
var i = Expression.Parameter(typeof(Dictionary<string, string>), "i");
var v = Expression.Parameter(typeof(T), "v");
var propertyInfos = typeof(T).GetProperties().ToArray();
var t = new Dictionary<string, string>();
var contains = typeof(Dictionary<string, string>).GetMethod(nameof(Dictionary<string, string>.ContainsKey));
var getter = typeof(Dictionary<string, string>).GetProperties().First(x => x.GetIndexParameters().Length > 0);
var result = new List<Expression>();
foreach (var propertyInfo in propertyInfos)
{
var cst = Expression.Constant(propertyInfo.Name);
var assignExpression =
Expression.IfThen(Expression.Call(i, contains, cst),
Expression.Assign(Expression.PropertyOrField(v, propertyInfo.Name), Expression.MakeIndex(i, getter, new[] { cst })));
result.Add(assignExpression);
}
var block = Expression.Block(result);
initObject = Expression.Lambda<Action<T, Dictionary<string, string>>>(block, new ParameterExpression[] { v, i }).Compile();
}
}
It is an example, it would fail if there were non-string properties.
And it could be used like this
static void Main(string[] args)
{
var tst = new Test();
CompiledDelegate<Test>.initObject(tst, new Dictionary<string, string>
{
{ "S3", "Value3" },
{ "S2", "Value2" },
});
CompiledDelegate<Test>.initObject(tst, new Dictionary<string, string>
{
{ "S3", "Value3" },
{ "S1", "Value1" },
});
Console.ReadKey();
}
The second option is, actually, what it should be ideally imlemented like Using source generators I think such things do have to be done just in build time.
There is a lot of articles on msdn, for instance with samples. But it turned out to be not very easy to implement, even just a sample.
I can say, it didn't work for me, while I tried to do it according to samples.
In order to get it work I had to change TargetFramework to netstandard2.0, do something else...
But after all, when build was green, Visual Studio still showed an error.
Ok, it disappeared after VS restart, but still, that doesn't look very usable.
So, this is a generator, that creates a converter for every class with attribute.
It is again a sample, it doesn't check many things.
[Generator]
public class ConverterGenerator : ISourceGenerator
{
private static string mytemplate = #"using System.Collections.Generic;
using {2};
namespace GeneratedConverters
{{
public static class {0}Converter
{{
public static {0} Convert(Dictionary<string, string> data)
{{
var result = new {0}();
{1}
return result;
}}
}}
}}";
public static string GetNamespaceFrom(SyntaxNode s)
{
if (s.Parent is NamespaceDeclarationSyntax namespaceDeclarationSyntax)
{
return namespaceDeclarationSyntax.Name.ToString();
}
if (s.Parent == null)
return "";
return GetNamespaceFrom(s.Parent);
}
public void Execute(GeneratorExecutionContext context)
{
GetMenuComponents(context, context.Compilation);
}
private static void GetMenuComponents(GeneratorExecutionContext context, Compilation compilation)
{
var allNodes = compilation.SyntaxTrees.SelectMany(s => s.GetRoot().DescendantNodes());
var allClasses = allNodes.Where(d => d.IsKind(SyntaxKind.ClassDeclaration)).OfType<ClassDeclarationSyntax>();
var classes = allClasses
.Where(c => c.AttributeLists.SelectMany(a => a.Attributes).Select(a => a.Name).Any(s => s.ToString().Contains("DictionaryConverter")))
.ToImmutableArray();
foreach (var item in classes.Distinct().Take(1))
{
context.AddSource(item.Identifier.Text + "Converter", String.Format(mytemplate, item.Identifier.Text, SourceText.From(GenerateProperties(item)), GetNamespaceFrom(item)));
}
}
private static string GenerateProperties(ClassDeclarationSyntax s)
{
var properties = s.Members.OfType<PropertyDeclarationSyntax>();
return String.Join(Environment.NewLine,
properties.Select(p =>
{
var name = p.Identifier.Text;
return $"if(data.ContainsKey(\"{name}\")) result.{name} = data[\"{name}\"];";
}));
}
public void Initialize(GeneratorInitializationContext context)
{
}
}
and
static void Main(string[] args)
{
var t1 = GeneratedConverters.TestConverter.Convert(new Dictionary<string, string>
{
{ "S3", "Value3" },
{ "S2", "Value2" },
});
}

Best performance without reflection would be manual mapping.
It seems your key/value pair collection is regular JSON. So you could use the JSONTextReader from JSON.NET and read the string. Then manually map the JSON properties to the class properties.
Like so:
JsonTextReader reader = new JsonTextReader(new StringReader(jsonString));
while (reader.Read())
{
if (reader.Value != null)
{
// check reader.Value.ToString() and assign to correct class property
}
}
More info can be found on the JSON.NET website : https://www.newtonsoft.com/json/help/html/ReadingWritingJSON.htm

Is there a more efficient way to loop through multiple lists

I have 4 lists of strings - These are all identical in length
var nameList;
var dateList;
var versionList;
var downloadList;
I'm currently looping through each list individually and then adding the contents of each list into a concurrent queue
var concurrentNameQueue= new ConcurrentQueue<string>();
var concurrentDateQueue= new ConcurrentQueue<string>();
var concurrentVersionQueue= new ConcurrentQueue<string>();
var concurrentDownloadQueue= new ConcurrentQueue<string>();
foreach (var name in nameList)
{
concurrentNameQueue.Enqeue(name);
}
foreach (var date in dateList)
{
concurrentDateQueue.Enqeue(date);
}
foreach (var version in versionList)
{
concurrentVersionQueue.Enqeue(version);
}
foreach (var download in downloadList)
{
concurrentDownloadQueue.Enqeue(download);
}
This process seems awfully repetitive and got me wondering if there is a more efficient way to loop through all these lists
Is there a more efficent way to do this?

You can write an extension method and then call it:
public static void Enqueue<T>(this ConcurrentQueue<T> queue, IEnumerable<T> items)
{
foreach (var item in items)
queue.Enqueue(item);
}
concurrentNameQueue.Enqueue(nameList);
concurrentDateQueue.Enqueue(dateList);
concurrentVersionQueue.Enqueue(versionList);
concurrentDownloadQueue.Enqueue(downloadList);
Or as you are initializing the list just above then use the second
constructor that requires an IEnumerable<T> as input:
var concurrentNameQueue = new ConcurrentQueue<string>(nameList);
var concurrentDateQueue = new ConcurrentQueue<string>(dateList);
var concurrentVersionQueue = new ConcurrentQueue<string>(versionList);
var concurrentDownloadQueue = new ConcurrentQueue<string>(downloadList);
But as I mentioned in my comment it seems like you should create a single class with 4 properties, one for each of name, date, version and download. Then you have one list of that class instead of keeping 4 lists synced.

Instead of making multiple list, use data structure that has multiple properties
Thats the advantage of OOP. Create single list of this instead
public class DownloadItem
{
public string Name { get; set; }
public string Date { get; set; }
public string Version { get; set; }
public string Download { get; set; }
}

Get coursework from google classroom course

I'm trying to get a list of CourseWork objects from a list of courses and add each coursework object to a list. However I have no idea how to get the Coursework object, I'm thinking that it's to do with either eTags or Ids, but have no way of knowing. The code I'm using is below:
public static IList<Course> GetAllCourses()
{
CoursesResource.ListRequest request = ClassService.Courses.List();
request.PageSize = 100;
ListCoursesResponse response = request.Execute();
GoogleClassroomCourses = response.Courses;
return GoogleClassroomCourses;
}
public static IList<CourseWork> GetCourseWork()
{
IList<string> courseIds = new List<string>();
IList<string> eTags = new List<string>();
foreach (Course course in GetAllCourses())
{
string id = course.Id;
courseIds.Add(id);
foreach (string cId in courseIds)
{
Course c = ClassService.Courses.Get(cId).Execute();
eTags.Add(c.ETag);
}
}
}

You may want to use Method: courses.courseWork.list using this HTTP request format:
GET https://classroom.googleapis.com/v1/courses/{courseId}/courseWork
which, if successful, returns course work items that match the request in the response body.
Sample JSON representation:
{
"courseWork": [
{
object(CourseWork)
}
],
"nextPageToken": string,
}

set type dynamically using Createmany

I have different types of documents that are derived from a base type called Topic. I'd like to use:
Client.Bulk(b => b.CreateMany(documents)
to be able to process all the documents with a single call to Bulk, how can I set the type for each document?
Here is a snippet of code:
public IEnumerable<IBulkResponse> CreateBulkTopics(IEnumerable<Topic> topics)
{
var results = new List<IBulkResponse>();
results.Add(IndexDocuments(TopicFactory.ConvertDrugsToDocuments(topics)));
results.Add(IndexDocuments(TopicFactory.ConvertTreatmentSummariesToDocuments(topics)));
return results;
}
public IBulkResponse IndexDocuments(IEnumerable<Common.Elastic.Models.Topic> documents)
{
return ElasticConnector.Client.Bulk(b => b.CreateMany(documents));
}
The problem at this minute is all the documents are being stored as "topic" as opposed to the derived types such as drugs and treatmentsummaries.

How many types inherit from Topic? Are they constant and small? Then something like this can help. Lets say TopicA and TopicB inherit from Topic:
public IEnumerable<IBulkResponse> IndexDocuments(IEnumerable<Common.Elastic.Models.Topic> documents)
{
yield return ElasticConnector.Client.Bulk(b => b.CreateMany(documents.OfType<TopicA>()));
yield return ElasticConnector.Client.Bulk(b => b.CreateMany(documents.OfType<TopicB>()));
}
and then in CreateBulkTopics:
results.AddRange(IndexDocuments(....
Of course this is only effective if the number of subclasses is small and available to this code. Otherwise, you can use reflection to achieve the same result. The sample code is a bit more complex, but tell me if you need it. Also, this will degrade performance in case the number of subclasses is very high, as it will send each type in a separate request to Bulk api. I can think of no better apprach in the client.
EDIT: This is how you do it using reflection:
class MyClass
{
public IBulkResponse IndexDocuments<T>(IEnumerable<Topic> documents)
where T : Topic
{
var derived = documents.OfType<T>();
return ElasticConnector.Client.Bulk(b => b.CreateMany(derived));
}
public IEnumerable<IBulkResponse> IndexDocumentsByType(IEnumerable<Topic> documents)
{
var groups = documents.GroupBy(x => x.GetType());
var method = typeof(MyClass).GetMethod(nameof(IndexDocuments)); //prior to c#6, typeof(MyClass).GetMethod("IndexDocuments")
foreach (var group in groups)
{
var generic = method.MakeGenericMethod(group.Key);
var result = generic.Invoke(this, new object[] { group });
yield return result as IBulkResponse;
}
}
}
class Program
{
static void Main(string[] args)
{
var documents = new Topic[] { new TopicA(), new TopicA(), new TopicB(), new Topic() };
var result = new MyClass().IndexDocumentsByType(documents);
Console.WriteLine(result.Count()); //writes 3
}
}

I've managed to do it using a generic class:
public class IndexOperations<T> where T:Topic
{
public ElasticConnector ElasticConnector { get; set; }
public IndexOperations(ElasticConnector elasticConnector)
{
ElasticConnector = elasticConnector;
}
public IBulkResponse CreateMany(IEnumerable<T> t)
{
return ElasticConnector.Client.Bulk(b => b.CreateMany(t));
}
}
Client code:
var documents = TopicFactory.ConvertToDocuments(topics);
SaveDrugs(documents);
public IBulkResponse SaveDrugs(IEnumerable<Common.Elastic.Models.Topic> documents)
{
var indexOperations = new IndexOperations<Drug>(ElasticConnector);
return indexOperations.CreateMany(documents.OfType<Drug>());
}

Retrieve sections from config.json in ASP.NET 5

Let's say I have a config.json like this:
{
"CustomSection": {
"A": 1,
"B": 2
}
}
I know I can use an IConfiguration object to get specific settings, i.e., configuration.Get("CustomSection:A"), but can I grab the whole hierarchy (in any type - even a raw string would be fine)? When I try configuration.Get("CustomSection"), I get a null result, so I think this isn't supported by default.
My use case is grabbing entire configuration dictionaries at once without having to grab each individual setting - some properties may not be known at compile time.

I have solved a similar problem where I wanted to bind the entire IConfigurationRoot or IConfigurationSection to a Dictionary. Here is an extension class:
public class ConfigurationExtensions
{
public static Dictionary<string, string> ToDictionary(this IConfiguration config, bool stripSectionPath = true)
{
var data = new Dictionary<string, string>();
var section = stripSectionPath ? config as IConfigurationSection : null;
ConvertToDictionary(config, data, section);
return data;
}
static void ConvertToDictionary(IConfiguration config, Dictionary<string, string> data = null, IConfigurationSection top = null)
{
if (data == null) data = new Dictionary<string, string>();
var children = config.GetChildren();
foreach (var child in children)
{
if (child.Value == null)
{
ConvertToDictionary(config.GetSection(child.Key), data);
continue;
}
var key = top != null ? child.Path.Substring(top.Path.Length + 1) : child.Path;
data[key] = child.Value;
}
}
}
And using the extension:
IConfigurationRoot config;
var data = config.ToDictionary();
var data = config.GetSection("CustomSection").ToDictionary();
There is an optional parameter (stripSectionPath) to either retain the full section key path or to strip the section path out, leaving a relative path.
var data = config.GetSection("CustomSection").ToDictionary(false);

configuration.Get is for getting a value to get a section you need
IConfiguration mysection = configuration.GetConfigurationSection("SectionKey");

I was able to load and bind multiple sub sections with unknown keys like this (the syntax has changed slightly since your post; I recommend keeping an eye on the github project and their unit tests to see how it is currently working):
var objectSections = Configuration.GetSection("CustomObjects").GetChildren();
var objects = objectSections.ToDictionary(x => x.Key, x =>
{
var obj = new CustomObject();
ConfigurationBinder.Bind(x, obj);
return obj ;
});

Edit: updating this answer for the 1.0 release of Core.
This is possible now if you use a strongly typed object, for example:
public class CustomSection
{
public int A {get;set;}
public int B {get;set;}
}
//In Startup.cs
services.Configure<CustomSection>(Configuration.GetSection("CustomSection"));
//You can then inject an IOptions instance
public HomeController(IOptions<CustomSection> options)
{
var settings = options.Value;
}

For a detailed explanation, see
https://dotnetcodr.com/2017/01/20/introduction-to-asp-net-core-part-3-the-configuration-file/
Below is an example from the site:
Config file has this:
"Languages": {
".NET": [ "C#", "VB.NET", "F#" ],
"JVM": ["Java", "Scala", "Clojure"]
}
Load this configuration as follows:
IConfigurationSection languagesSection = configRoot.GetSection("Languages");
IEnumerable<IConfigurationSection> languagesSectionMembers = languagesSection.GetChildren();
Dictionary<string, List<string>> platformLanguages = new Dictionary<string, List<string>>();
foreach (var platform in languagesSectionMembers)
{
List<string> langs = (from p in platform.GetChildren() select p.Value).ToList();
platformLanguages[platform.Key] = langs;
}

You can use the ConfigurationBinder and read everything as Dictionary<string, string>
Here are some test cases that you can use as example: https://github.com/aspnet/Configuration/blob/dev/test/Microsoft.Framework.Configuration.Binder.Test/ConfigurationCollectionBindingTests.cs