I am trying to have a column not provided in the CSV file populated using CSVHelper.
This is an example of the CSV I need to import
Id
10
123
45
213
The class I am trying to deserialize to is this one:
public class Foo {
public int Id { get; set }
public string Name { get; set }
}
With the default configuration I get this error:
CsvHelper.HeaderValidationException: 'Header with name 'Name' was not found.
I would like to have the possibility to define a mapper so that the column Name could be populated by the parser (e.g. by providing a dictionary of values). Any way to do this?
Thanks
---------- EDIT
Just to clarify, the idea is to have something like a converter that, associated to a field, would be used to decode the Id to the Name
public class NameConverter
{
public NameConverter()
{
employeesList = new Dictionary<int, string>()
{
{ 10, "Mary" },
{ 45, "Mike" },
{ 123, "Jack" },
{ 213, "Suzanne" },
};
}
IDictionary<int, string> employeesList;
public string GetValue(int id) => employeesList[id];
}
The alternative, I would imagine, is to ignore the Name field as suggested and inject the NameConverter in the Foo class and make the Name a get only property.
void Main()
{
var s = new StringBuilder();
s.AppendLine("Id");
s.AppendLine("45");
s.AppendLine("123");
using (var reader = new StringReader(s.ToString()))
using (var csv = new CsvReader(reader, CultureInfo.InvariantCulture))
{
csv.Configuration.RegisterClassMap<FooMap>();
csv.GetRecords<Foo>().ToList().Dump();
}
}
public class Foo
{
public int Id { get; set; }
public string Name { get; set; }
}
public class FooMap : ClassMap<Foo>
{
public FooMap()
{
var nameConverter = new NameConverter();
Map(m => m.Id);
Map(m => m.Name).ConvertUsing(row => nameConverter.GetValue(row.GetField<int>(nameof(Foo.Id))));
}
}
public class NameConverter
{
public NameConverter()
{
employeesList = new Dictionary<int, string>()
{
{ 10, "Mary" },
{ 45, "Mike" },
{ 123, "Jack" },
{ 213, "Suzanne" },
};
}
IDictionary<int, string> employeesList;
public string GetValue(int id) => employeesList[id];
}
Output:
Related
Hi and thanks in advance everyone!
I have a collection of the following objects:
public class ItemsModel
{
public List<int> IdCollection { get; set; }
public string Name { get; set; }
public int Weight { get; set; }
}
List<ItemsModel> col = ...;
I want to optimally store this with LiteDb and be able to modify the records.
Each ItemsModel has a unique Name+Weight set.
In the entire col, the elements of the IdCollection are also unique.
Body example:
List<ItemsModel>:
[{
IdCollection: [1,3,5,6,...],
Name: "first name",
Weight: 10
},
{
IdCollection: [2,4,...],
Name: "second name",
Weight: 5
}]
I want to index by Id
I want to expand into two tables for easy storage in LiteDb:
[{
_id: 1,
NameAndWeight: {&ref: "names"}
},
{
_id: 2,
NameAndWeight: {&ref: "names"}
},
{
_id: 3,
NameAndWeight: {&ref: "names"}
},
...
]
[{
Name: "first name",
Weight: 10
},
{
Name: "second name",
Weight: 5
}]
For this I have to make new storage classes:
public class ItemsModel
{
[BsonId]
public int Id { get; set; }
[BsonRef("names")]
public NamesModel NameAndWeight { get; set; }
}
public class NamesModel
{
[BsonId(true)]
public ObjectId Id { get; set; }
public string Name { get; set; }
public int Weight { get; set; }
}
But next step I'm having trouble...
Tell me, can I somehow save data using Insert array and Include in one operation?
Or should I use foreach to first write the NamesModel in "names" DB, get the generated _id, then write the ItemsModel with a link to the NamesModel already written to the database?
using (var db = new LiteDatabase(_strConnection))
{
var itemsDb = db.GetCollection<ItemsModel>("items");
var namesDb = db.GetCollection<NamesModel>("names");
itemsDb.EnsureIndex(x => x.Id, true);
foreach (var group in col)
{
var name = new NamesModel(group.Name, group.Weight);
namesDb.Insert(name);
var itemDb = group.IdCollection.Select(el => new ItemsModel(el, name));
var h = itemsDb.Insert(itemDb);
}
}
it is too long(
Now I did like this:
using (var db = new LiteDatabase(_strConnection))
{
var itemsDb = db.GetCollection<ItemsModel>("items");
var namesDb = db.GetCollection<NamesModel>("names");
itemsDb.EnsureIndex(x => x.Id, true);
namesDb.EnsureIndex(x => x.Name);
var temp = col.Select(el => (el.IdCollection, new NamesModel(el.Name, el.Weight))).ToList();
namesDb.Insert(temp.Select(el => el.Item2));
var temp2 = temp.SelectMany(gr => gr.IdCollection.Select(el => new ItemsModel(el, gr.Item2)));
eventsIdDB.Insert(temp2);
}
Performed basic operations in linq to reduce the number of hits in liteDb
I loaded a csv file in my database with a DbId column not in the file.
I want to export it back to the original format.
My csvhelper mapping is in MyCsvClass with Map(m => m.DbId).Ignore();
Header is fine, but output data is still showing values of DbId column:
https://dotnetfiddle.net/XP2Vvq
using CsvHelper.Configuration;
using System;
using System.IO;
namespace Test
{
class Program
{
static void Main(string[] args)
{
var record = new { DbId = 1, Data1 = "aaa", Data2 = "bbb" };
using (var sw = new StreamWriter(#"c:/temp/testt.csv"))
{
using (var csvWriter = new CsvHelper.CsvWriter(sw))
{
csvWriter.Configuration.RegisterClassMap<MyCsvClassMap>();
csvWriter.WriteHeader<MyCsvClass>();
csvWriter.NextRecord();
csvWriter.WriteRecord(record);
}
}
}
}
public class MyCsvClassMap : ClassMap<MyCsvClass>
{
public MyCsvClassMap()
{
AutoMap();
Map(m => m.DbId).Ignore();
}
}
public class MyCsvClass
{
public int DbId { get; set; }
public string Data1 { get; set; }
public string Data2 { get; set; }
}
}
Output is
Data1, Data2
1, "aaa", "bbb"
when I expect
Data1, Data2
"aaa", "bbb"
The issue with your code is that you create an instance of anonymous type
var record = new { DbId = 1, Data1 = "aaa", Data2 = "bbb" };
instead of
var record = new MyCsvClass { DbId = 1, Data1 = "aaa", Data2 = "bbb" };
The header is fine, because you pass the correct class to type parameter of the generic method.
csvWriter.WriteHeader<MyCsvClass>();
Edit
To export DB entities to CSV you don't need any intermediate class. You can write entities directly to CSV and ClassMap<T> helps you control what values and how get serialized to CSV. If your entity class is MyDbEntity, then just register custom mapping ClassMap<MyDbEntity> where you auto-map all fields ignoring some fields as you did in your MyCsvClassMap.
I think the ClassMap is not correctly configured. No idea why your example is working because it should not compile.
Can you modify the following line of code to register the ClassMap (MyCsvClassMap instead of MyCsvClass):
csvWriter.Configuration.RegisterClassMap<MyCsvClassMap>();
The rest of your example works fine.
Try the following console app:
class Program
{
static void Main(string[] args)
{
var allRecords = new List<MyCsvClass>()
{
new MyCsvClass { DbId = "1", Data1 = "data1", Data2 = "data2"}
};
using (var sw = new StreamWriter("C:\\temp\\test.csv"))
{
using (var csvWriter = new CsvHelper.CsvWriter(sw))
{
csvWriter.Configuration.RegisterClassMap<MyCsvClassMap>();
csvWriter.WriteHeader<MyCsvClass>();
csvWriter.NextRecord();
csvWriter.WriteRecords(allRecords);
}
}
}
public class MyCsvClassMap : ClassMap<MyCsvClass>
{
public MyCsvClassMap()
{
AutoMap();
Map(m => m.DbId).Ignore();
}
}
public class MyCsvClass
{
public string DbId { get; set; }
public string Data1 { get; set; }
public string Data2 { get; set; }
}
}
If this is working there is maybe some other code which causes this behavior.
I am using c#,along with MongoDB.
I have a class that can be resembled by this.
Its a sample, that represents something, please dont comment on the class design
[CollectionName("Venues")]
public class Venue
{
public string Name { get; set; }
public dictionary<string,object> Properties { get; set; }
}
var venue = new Venue
{
Name = "Venue 1",
Properties = new Dictionary<string,object>
{
{ "Chairs", "18" },
{ "Tables", "4" },
{ "HasWaterfall", true }
}
}
Assuming I have an object in a collection, that looks like that.
I would like to find out of it is possible to do two things.
1: Load from the database, only a single item from the dictionary,
currently I can only see how this can be done, by loading the entire
record from the database and then manually getting the value by key.
2: Determine the average of a single item within the database.
For example, across all records I would like to work out the average
chairs, again without loading all records and then doing it in memory with
linq etc....
Basically your sample document gets stored as a below JSON:
{
"_id" : ObjectId("..."),
"Name" : "Venue 1",
"Properties" : {
"Chairs" : "18",
"Tables" : "4",
"HasWaterfall" : true
}
}
This gives you a possibility to define a projection using dot notation:
var filter = Builders<Venue>.Filter.Eq(f => f.Name, "Venue 1");
var projection = Builders<Venue>.Projection.Include("Properties.Chairs");
List<BsonDocument> data = Col.Find(filter).Project(projection).ToList();
which returns below following BsonDocument:
{ "_id" : ObjectId("..."), "Properties" : { "Chairs" : "18" } }
To get the average you need to use $toInt operator introduced in MongoDB 4.0 to convert your values from string to int. Try:
var project = new BsonDocument()
{
{ "chairs", new BsonDocument() { { "$toInt", "$Properties.Chairs" } } }
};
var group = new BsonDocument()
{
{ "_id", "null" },
{ "avg", new BsonDocument() { { "$avg", "$chairs" } } }
};
var avg = Col.Aggregate().Project(project).Group(group).First();
here's an alternative way of doing it using MongoDB.Entities convenience library.
using System.Collections.Generic;
using System.Linq;
using MongoDB.Entities;
namespace StackOverflow
{
class Program
{
[Name("Venues")]
public class Venue : Entity
{
public string Name { get; set; }
public Dictionary<string, object> Properties { get; set; }
}
static void Main(string[] args)
{
new DB("test");
var venue1 = new Venue
{
Name = "Venue 1",
Properties = new Dictionary<string, object> {
{ "Chairs", 28 },
{ "Tables", 4 },
{ "HasWaterfall", true }
}
};
venue1.Save();
var venue2 = new Venue
{
Name = "Venue 2",
Properties = new Dictionary<string, object> {
{ "Chairs", 38 },
{ "Tables", 4 },
{ "HasWaterfall", true }
}
};
venue2.Save();
var chairs = DB.Find<Venue, object>()
.Match(v => v.Name == "Venue 1")
.Project(v => new { ChairCount = v.Properties["Chairs"] })
.Execute();
var avgChairs = DB.Collection<Venue>()
.Average(v => (int)v.Properties["Chairs"]);
}
}
}
results in the following queries being made to the database:
getting chairs in venue 1:
db.runCommand({
"find": "Venues",
"filter": {
"Name": "Venue 1"
},
"projection": {
"Properties.Chairs": NumberInt("1"),
"_id": NumberInt("0")
},
"$db": "test"
})
getting average chair count across all venues:
db.Venues.aggregate([
{
"$group": {
"_id": NumberInt("1"),
"__result": {
"$avg": "$Properties.Chairs"
}
}
}
])
I have s Student class where each student record has a list of Results.
I need to export there results to CSV and I'm using CsvHelper.
public class Student
{
public string Id { get; set; }
public string Name { get; set; }
public Result[] Grades { get; set; }
}
public class Result
{
public string Subject { get; set; }
public decimal? Marks { get; set; }
}
I'm using Reference Maps to map the list of Results, but when exporting to CSV it throws and error.
Mapping Code
public sealed class StudentResultExportMap : ClassMap<Student>
{
public StudentResultExportMap ()
{
AutoMap();
References<GradesMap>(m => m.Grades);
}
}
public sealed class GradesMap: ClassMap<Result>
{
public GradesMap()
{
Map(m => m.Subject);
Map(m => m.Marks);
}
}
Error
Property 'System.String Subject' is not defined for type
'{namespace}.GetStudentResults+Result[]' Parameter name: property
Unfortunately References<GradesMap>(m => m.Grades); doesn't work for an array of Result. It would work for an individual result. I have one solution, which overrides the ToString() method of Result to flatten the grades. It might work for you, depending on what you need.
public class Result
{
public string Subject { get; set; }
public decimal? Marks { get; set; }
public override string ToString()
{
return $"{Subject} = {Marks}";
}
}
Make a slight change to your StudentResultExportMap. You can set the 2nd number on .Index(2, 7) to handle the max number of grades you think a student might have.
public sealed class StudentResultExportMap : ClassMap<Student>
{
public StudentResultExportMap()
{
AutoMap();
Map(m => m.Grades).Name("Grade").Index(2, 7);
}
}
You will then get Id, Name, Grade1, Grade2, Grade3, Grade4, Grade5, Grade6 with the toString() value of Result for each grade.
var records = new List<Student>
{
new Student{ Id = "1", Name = "First", Grades = new [] {
new Result { Subject = "Subject1", Marks = (decimal)2.5 } ,
new Result { Subject = "Subject2", Marks = (decimal)3.5 } }},
new Student{ Id = "2", Name = "Second", Grades = new [] {
new Result { Subject = "Subject1", Marks = (decimal)3.5 } ,
new Result { Subject = "Subject2", Marks = (decimal)4.0 } }}
};
using (var writer = new StreamWriter("path\\to\\StudentResults.csv"))
using (var csv = new CsvWriter(writer))
{
csv.Configuration.RegisterClassMap<StudentResultExportMap>();
csv.WriteRecords(records);
}
I have a mongo collection: Fieldset . Fieldset has List if embedded object Field.
Field has a Dictionary<string, object> Config, I want to be free about the value of this dictionary.
public class Fieldset
{
[BsonId]
[BsonRepresentation(BsonType.ObjectId)]
public string Id { get; set; }
public string Slug { get; set; }
[BsonElement("Fields")]
public List<Field> Fields { get; set; }
}
public class Field
{
[BsonElement("Config")]
public Dictionary<string, object> Config { get; set; }
}
this is an example of value i want to put in. some sort of key value collection:
When I save, this is the value I have in the mongo.
Hmm. It worked like supposed to work>
static void Main(string[] args)
{
var client = new MongoClient();
var database = client.GetDatabase("SO3");
var collection = database.GetCollection<Fieldset>("jobject");
var id = new BsonObjectId(ObjectId.GenerateNewId()).ToString();
var field =
new Field
{
Config = new Dictionary<string, object> {{"value", "item1key"}, {"lable", "item1value"}}
};
var field2 =
new Field
{
Config = new Dictionary<string, object> {{"value", "item2key"}, {"lable", "item2value"}}
};
var fieldset = new Fieldset
{
Id = id,
Slug = "aa",
Fields = new List<Field>
{
field,
field2
}
};
collection.InsertOne(fieldset);
}
{ "_id" : ObjectId("5c55b972ceb9443d385de936"), "Slug" : "aa", "Fields" : [ { "Config" : { "value" : "item1key", "lable" : "item1value" } }, { "Config" : { "value" : "item2key", "lable" : "item2value" } } ] }