I have a csv file that contains integer, string, bool, and float types of data. The problem is, that some fields are empty. When I try to read the csv file with csvhelper, I get TypeConversion.TypeConverterException error. I checked the details, and looks like it's because it can't convert the empty float values to string. How can I handle it?
namespace Smh
{
class Program
{
static void Main(string[] args)
{
List<E> EList = new List<E>();
List<E2> EList2 = new List<E2>();
string filePath = #"D:\Visual Studio\Projects\Example\boo.csv";
using (var reader = new StreamReader(filePath))
using (var csv = new CsvReader(reader, CultureInfo.InvariantCulture))
{
var example= new E();
EList= csv.GetRecords<E>().ToList();
}
.
.
.
A part of my "E" class:
.
.
public float weight { get; set; }
.
.
and the error I get:
https://i.postimg.cc/dVPkYDCm/image.png
A short part of the csv file:
id | speed | type | weight
1 40 type1 5.4
2 43 type2
3 16 type3 5.2
The simplest solution with out using a ClassMap Custom mapper,
is to change the type to float?.
public class Test{
public int id{get;set;}
public int speed{get;set;}
public string type{get;set;}
public float? weight{get;set;}
}
Then If you need to change the default value you can add a Getter Setter that will return Single.NaN
Example:
Live Demo
public static void Main()
{
var result = new List<Test>();
using (var stream = new MemoryStream())
using (var writer = new StreamWriter(stream))
using (var reader = new StreamReader(stream))
using (var csv = new CsvReader(reader, CultureInfo.InvariantCulture))
{
//Csv Creation
writer.WriteLine("id,speed,type,weight");
writer.WriteLine("1,40,type1,5.4");
writer.WriteLine("2,43,type2,");
writer.WriteLine("3,16,type3,5.2");
writer.Flush();
stream.Position = 0;
csv.Configuration.HasHeaderRecord = true;
csv.Configuration.Delimiter = ",";
result = csv.GetRecords<Test>().ToList();
}
result.Dump();
}
Related
How I can directly jump to a row of a csv file ? Please see the below shown structure of csv. I want to jump directly to the row 3 (skip 0,1,2 rows) using csvhelper class ? RowId is a column in the csv file.
RowId Name
----- ----
0 Raju
1 Sabu
2 Ravi
3 Lal
4 Babu
Here is how I read csv file :
CsvConfiguration csvConfiguration = new CsvConfiguration(CultureInfo.InvariantCulture)
{
HasHeaderRecord = true,
Delimiter = ",",
PrepareHeaderForMatch = args => args.Header.ToUpper(),
IgnoreBlankLines = true,
IgnoreReferences = true,
MissingFieldFound = null,
UseNewObjectForNullReferenceMembers = true
};
CsvReader csv = new CsvReader(File.OpenText(FileNameWithPath), csvConfiguration);
csv.Read();
csv.ReadHeader();
while (csv.Read())
{
string d0 = csv.GetField<string>("Name");
var d5 = csv.GetField<int>("Class");
}
And thiscsv file contains more than 2000 records. So If I want to jump to a row (eg: 250 rowid) by skipping all the rows just above the rowid=230, how i can do that ?
After finding this rowid = 230, i also want to start reading from row id 230 onwards.
How we can do that using csvhelper with c#?
One way is to use ShouldSkipRecords.
void Main()
{
var s = new StringBuilder();
s.Append("RowId,Name\r\n");
s.Append("0,zero\r\n");
s.Append("1,one\r\n");
s.Append("2,two\r\n");
s.Append("3,three\r\n");
s.Append("4,four\r\n");
s.Append("5,five\r\n");
s.Append("6,six\r\n");
var config = new CsvConfiguration(CultureInfo.InvariantCulture)
{
ShouldSkipRecord = args => args.Row.Parser.Row > 1 && args.Row.GetField<int>("RowId") < 3,
};
using (var reader = new StringReader(s.ToString()))
using (var csv = new CsvReader(reader, config))
{
csv.GetRecords<Foo>().ToList().Dump();
}
}
private class Foo
{
public int RowId { get; set; }
public string Name { get; set; }
}
Another is to read manually.
void Main()
{
var s = new StringBuilder();
s.Append("RowId,Name\r\n");
s.Append("0,zero\r\n");
s.Append("1,one\r\n");
s.Append("2,two\r\n");
s.Append("3,three\r\n");
s.Append("4,four\r\n");
s.Append("5,five\r\n");
s.Append("6,six\r\n");
var config = new CsvConfiguration(CultureInfo.InvariantCulture)
{
};
using (var reader = new StringReader(s.ToString()))
using (var csv = new CsvReader(reader, config))
{
var records = new List<Foo>();
csv.Read();
csv.ReadHeader();
while (csv.Read())
{
var rowId = csv.GetField<int>("RowId");
if (rowId < 3)
{
continue;
}
records.Add(csv.GetRecord<Foo>());
}
records.Dump();
}
}
private class Foo
{
public int RowId { get; set; }
public string Name { get; set; }
}
I am trying to use CSVHelper library to write data into a MemoryStream and then generate a CSV file using that MemoryStream.
The problem is Double values get converted to weird string values when the double values have decimal points. The expected output and weird output are there at the bottom.
Is anyone know how to overcome this issue? or Is there any mistake in the below code?
public class Foo
{
public Foo()
{
}
public double valOne { get; set; }
public double valTwo { get; set; }
}
public class FooMap : ClassMap<Foo>
{
public FooMap()
{
Map(m => m.valOne).Index(0).Name("Val One");
Map(m => m.valTwo).Index(1).Name("Val Two");
}
}
var records = new List<Foo> {
new Foo{valOne = 3224.12, valTwo = 4122},
new Foo{valOne = 2030.20, valTwo = 5555},
};
var config = new CsvConfiguration(CultureInfo.CurrentCulture) { Delimiter = ",", HasHeaderRecord = true };
using (var memoryStream = new MemoryStream())
using (var writer = new StreamWriter(memoryStream))
using (var csv = new CsvWriter(writer, config))
{
csv.Context.RegisterClassMap<FooMap>();
csv.WriteHeader<Foo>();
csv.NextRecord();
foreach (var record in records)
{
csv.WriteRecord(record);
csv.NextRecord();
}
writer.Flush();
var result = Encoding.UTF8.GetString(memoryStream.ToArray());
byte[] bytes = Encoding.ASCII.GetBytes(result);
return new FileContentResult(bytes, "text/csv")
{
FileDownloadName = "Sample_Report_Name"
};
}
Expected Output:
Val One, Val Two
3224.12,4122
2030.20,5555
Weird Output:
Val One, Val Two
"3224,12",4122
"2030,20",5555
The issue is the CurrentCulture of the computer running the code uses commas instead of periods to indicate the decimal point. Using CultureInfo.InvariantCulture instead of CultureInfo.CurrentCulture should fix the formatting issue.
Also, you can simplify your code by using csv.WriteRecords(records).
var records = new List<Foo> {
new Foo{valOne = 3224.12, valTwo = 4122},
new Foo{valOne = 2030.20, valTwo = 5555},
};
var config = new CsvConfiguration(CultureInfo.CurrentCulture) { Delimiter = ",", HasHeaderRecord = true };
using (var memoryStream = new MemoryStream())
using (var writer = new StreamWriter(memoryStream))
using (var csv = new CsvWriter(writer, config))
{
csv.Context.RegisterClassMap<FooMap>();
csv.WriteRecords(records);
writer.Flush();
var result = Encoding.UTF8.GetString(memoryStream.ToArray());
byte[] bytes = Encoding.ASCII.GetBytes(result);
return new FileContentResult(bytes, "text/csv")
{
FileDownloadName = "Sample_Report_Name"
};
}
I'm a newbie. I want to get data from the CSV file-the Id and Name fields, but when I run the reading method, I get only 100 lines of an incomprehensible type: "CsvHelper.CsvReaderd__87`1[Program+Product]". I do not know how to get data from CSV, I also cannot understand where the error is.
Although the documentation says that having the same names of properties and CSV headers, you do not need to write additional configurations. However, I get the result specified above. The CSV names match the classes. Link to the documentation:https://joshclose.github.io/CsvHelper/getting-started/
reading method:
{
using (var reader = new StreamReader("C:\\Users\\Saint\\Desktop\\TaskRetail\\file.csv", Encoding.UTF8))
using (var csv = new CsvReader(reader, CultureInfo.InvariantCulture))
{
var records = csv.GetRecords<Product>();
Console.WriteLine($"{records}");
}
}
CSV is created without problems, there are two columns with Id and Name with filled rows, there are 100 rows in total:
method for creating a csv with the Id and Name fields:
using (var writer = new StreamWriter("C:\\Users\\Saint\\Desktop\\TaskRetail\\file.csv", false, Encoding.UTF8))
using (var csv = new CsvWriter(writer, CultureInfo.InvariantCulture))
{
csv.WriteRecords(products);
}
the entire code:
using CsvHelper;
using System;
using System.Collections.Generic;
using System.Globalization;
using System.IO;
using System.Net;
using System.Text;
using System.Xml;
public class Program
{
public class Product
{
public int Id { get; set; }
public string Name { get; set; }
public Product(int id, string name)
{
Id = id;
Name = name;
}
}
public const string PathToDoc = "C:/Users/Saint/Desktop/TaskRetail/yml.xml";
public static void Main(string[] args)
{
string url = "https://www.googleapis.com/drive/v3/files/1sSR9kWifwjIP5qFWcyxGCxN0-MoEd_oo?alt=media&key=AIzaSyBsW_sj1GCItGBK0vl8hr9zu1I1vTI1Meo";
string savePath = #"C:\Users\Saint\Desktop\TaskRetail\yml.xml";
WebClient client = new WebClient();
client.DownloadFile(url, savePath);
Research();
}
public static void Research()
{
Encoding.RegisterProvider(CodePagesEncodingProvider.Instance);
var document = new XmlDocument();
document.Load(PathToDoc);
var xmlDoc = document.SelectNodes("/yml_catalog/shop/offers/offer");
var count = xmlDoc.Count;
var products = new List<Product>();
Console.WriteLine($"Offers count: {count}");
for (var i = 0; i < count; i++)
{
var element = xmlDoc.Item(i);
var id = int.Parse(element.Attributes.GetNamedItem("id").Value);
var name = element.SelectSingleNode("name").InnerText;
var product = new Product(id, name);
//Console.WriteLine($"Id: {id}, name: {name}");
products.Add(product);
using (var writer = new StreamWriter("C:\\Users\\Saint\\Desktop\\TaskRetail\\file.csv", false, Encoding.UTF8))
using (var csv = new CsvWriter(writer, CultureInfo.InvariantCulture))
{
csv.WriteRecords(products);
}
var config = new CsvConfiguration(CultureInfo.InvariantCulture) { Delimiter = ",", PrepareHeaderForMatch = header => header.Header.ToLower() };
using (var reader = new StreamReader("C:\\Users\\Saint\\Desktop\\TaskRetail\\file.csv", Encoding.UTF8))
using (var csv = new CsvReader(reader, config))
{
var records = csv.GetRecords<Product>();
foreach (var record in records)
{
Console.WriteLine($"{record.Id} {record.Name}");
}
}
}
}
}
Because GetRecords() does return an object of type IEnumerable,
you have to iterate over your records to print each one of them:
foreach(var record in records)
{
Console.WriteLine($"{record.Id} {record.Name}");
}
Furthermore you have to access each property you want to print individually.
Another option would be to override the ToString() method in your Product class.
EDIT
The initial problem wasn't the correct printing of the values but the parsing of the file as I learned from this comment:
CsvHelper.HeaderValidationException: 'Header with name 'id'[0] was not found. Header with name 'name'[0] was not found.
To tackle this problem one have to make sure that the delimiter character is set correctly. This can be enforced in the config object of the CsvHelper. Furthermore to avoid casing errors the configuration can be set to ignore the casing of the headers:
var config = new CsvConfiguration(CultureInfo.InvariantCulture)
{
Delimiter = ",", // Enforce ',' as delimiter
PrepareHeaderForMatch = header => header.Header.ToLower() // Ignore casing
};
using (var csv = new CsvReader(reader, config))
{
...
}
i need to get the product code,price and stock for calculation on the txt file that contain
//product code;product name;price;stock;
c01;cappuccino;3500;12;
c02;mocaccino;4000;15;
c03;black coffe;3000;10;
c04;chocolate milk;5000;19;
c05;vanilla milk;5000;12;
c06;strawberry milk;5000;13;
c07;coke;3000;13;
c08;fanta;3000;15;
c09;sprite;3000;9;
c10;orange juice;4500;10;
c11;apple juice;4500;9;
c12;mango juice;4500;18;
i've tried
if(line.Contains(""))
but the line in line.contains give error red underline instead, am i missing using namespace for this?
also i've tried
FileStream fs = new FileStream("product.txt", FileMode.OpenOrCreate, FileAccess.Read);
StreamReader sr = new StreamReader(fs);
sr.BaseStream.Seek(0, SeekOrigin.Begin);
string str = sr.ReadLine();
while (str != null)
{
Console.WriteLine("{0}", str);
str = sr.ReadLine();
}
sr.Close();
fs.Close();
as practice to get a word, but it return all contents from txt instead
Using the following class and code, you will be able to extract the data you need in order to do calculations and other manipulations.
It basically reads the file line by line and parses them to the object.
The first line is being skipped by checking for "//".
Hope this suits your needs.
Product Class
public class Product
{
public string Code { get; set; }
public string Name { get; set; }
public decimal Price { get; set; }
public int Stock { get; set; }
}
File Parsing
var products = new List<Product>();
using (var fileStream = new FileStream("product.txt", FileMode.OpenOrCreate, FileAccess.Read))
using (var streamReader = new StreamReader(fileStream))
{
string line;
while (!String.IsNullOrWhiteSpace((line = streamReader.ReadLine())))
{
if (!line.StartsWith("//"))
{
var lineSplit = line.Split(';');
products.Add(new Product
{
Code = lineSplit[0],
Name = lineSplit[1],
Price = Decimal.Parse(lineSplit[2]),
Stock = Int32.Parse(lineSplit[3])
});
}
}
}
if you want to iterate throught those values you can make it like this:
var streamReader = new StreamReader(new FileStream("c:\\file.txt"));
while(!streamReader.EndOfStream)
{
var line = streamReader.ReadLine()
var values = line.Split(';');
for(var i = 0; i < line.Length; i++)
Console.WriteLine(values[i]); //example usage
}
I need to read data from a .csv file and store the header and the content in my object in the following format. A list of the below mentioned class.
public class MappingData
{
private string ColumnName { get; set; }
private List<string> Data { get; set; }
}
So for example say I have a table like as shown below,
| Name | Phone | City |
|:-----------|------------:|:------------:|
| Yassser | 32342342234 | Mumbai
| Sachin | 32342342234 | Surat
| Will | 32342342234 | London
So for the above data my class should have 3 objects, first object will have the following details
ColumnName : 'Name'
Data: ['Yasser', 'Sachin', 'Will']
so, this is what I am trying to do, Below is what I have started with. I am reading the file using stream reader and spliting each line with comma seperator.
private List<MappingData> GetData(string filename)
{
var data = new List<MappingData>();
string fullPath = GetFilePath(filename);
StreamReader reader = new StreamReader(fullPath);
while (!reader.EndOfStream)
{
string line = reader.ReadLine();
if (!String.IsNullOrWhiteSpace(line))
{
string[] values = line.Split(',');
}
}
return data;
}
can some one please help me mold this data into the required format. Thanks.
You should create a small int variable (or bool if you prefer) to determine whether the first row has been completed :
And create each of the list objects you will need (mpName, mpPhone, mpCity), on the first row you set the ColumnName property and on the subsequent rows you add to the MappingData's data list.
Then you add each of the lists (mpName, mpPhone, mpCity) to the data list for the method and return this.
private List<MappingData> GetData(string filename) {
List<MappingData> data = new List<MappingData>();
int NumRow = 0;
MappingData mpName = new MappingData();
MappingData mpPhone = new MappingData();
MappingData mpCity = new MappingData();
string fullPath = GetFilePath(filename);
StreamReader reader = new StreamReader(fullPath);
while (!reader.EndOfStream) {
string line = reader.ReadLine();
if (!String.IsNullOrWhiteSpace(line)) {
string[] values = line.Split(',');
if (NumRow == 0) {
mpName.ColumnName = values[0];
mpPhone.ColumnName = values[1];
mpCity.ColumnName = values[2];
NumRow = 1;
} else {
mpName.Data.Add(values[0]);
mpPhone.Data.Add(values[1]);
mpCity.Data.Add(values[2]);
}
}
}
data.Add(mpName);
data.Add(mpPhone);
data.Add(mpCity);
return data;
}
Hope this helps.
There's an excellent library for processing the CSV files.
KentBoogard
It is very easy with the above third party library to read content from the CSV files.
I would suggest this because you just seem to be starting and don't re invent the wheel.
Still, if you want to process the file in your own way, here's one working implementation. Enjoy
var csvData = File.ReadAllLines("d:\\test.csv");
var dataRows = csvData.Skip(1).ToList();
var csvHeaderColumns = csvData.First().Split(',').ToList();
var outputList = new List<MappingData>();
foreach (var columnName in csvHeaderColumns)
{
var obj = new MappingData { columnName = columnName, data = new List<string>() };
foreach (var rowStrings in dataRows.Select(dataRow => dataRow.Split(',').ToList()))
{
obj.data.Add(rowStrings[csvHeaderColumns.IndexOf(columnName)]);
}
outputList.Add(obj);
}
This will populate your MappingData class.
Assuming the rest of your method is correct then try this:
private List<MappingData> GetData(string filename)
{
var raw = new List<string[]>();
var data = new List<MappingData>();
string fullPath = GetFilePath(filename);
using(var reader = new StreamReader(fullPath))
{
while (!reader.EndOfStream)
{
string line = reader.ReadLine();
if (!String.IsNullOrWhiteSpace(line))
{
raw.Add(line.Split(','));
}
}
}
Func<int, MappingData> extract =
n => new MappingData()
{
ColumnName = raw[0][n],
Data = raw.Skip(1).Select(x => x[n]).ToList(),
};
data.Add(extract(0));
data.Add(extract(1));
data.Add(extract(2));
return data;
}
You'd have to make you MappingData properties accessible though.