Convert csv file to json using C# with no header - c#

I would like to convert CSV file to JSON using C#. I know that there are a lot of similar questions but I couldnĀ“t find something that could help me.
Source file looks like this:
2019-12-01T00:00:00.000Z;2019-12-10T23:59:59.999Z
50;false;2019-12-03T15:00:12.077Z;005033971003;48;141;2019-12-03T00:00:00.000Z;2019-12-03T23:59:59.999Z
100;false;2019-12-02T12:38:05.989Z;005740784001;80;311;2019-12-02T00:00:00.000Z;2019-12-02T23:59:59.999Z
First line is not header (actually I don't know how to call it - header usually have names of each property).
The result should look like this
{
"transactionsFrom": "2019-12-01T00:00:00.000Z","transactionsTo": "2019-12-10T23:59:59.999Z",
"transactions": [{
"logisticCode": "005033971003",
"siteId": "48",
"userId":"141",
"dateOfTransaction": "2019-12-03T15:00:12.077Z",
"price": 50
},
{
"logisticCode": "005729283002",
"siteId": "80",
"userId":"311",
"dateOfTransaction": "2019-12-02T12:38:05.989Z",
"price": 100
}]
}
I would like to use POCO - maybe something like this:
public class Headers
{
public string TransactionFrom { get; set; }
public string TransactionTo { get; set; }
}
public class Results
{
public string logisticCode { get; set; }
public string siteId { get; set; }
public string userId { get; set; }
public string dateOfTransaction { get; set; }
public string price { get; set; }
public string packSale { get; set; }
}
But the problem is I don't know how to continue. Maybe some example would help. I know I can use ChoETL, CsvHelper but I don't how.

This code might help you
Step1 - Create model class
public class Headers
{
public string TransactionFrom { get; set; }
public string TransactionTo { get; set; }
public List<Transaction> Transactions { get; set; }
}
public class Transaction
{
public string logisticCode { get; set; }
public string siteId { get; set; }
public string userId { get; set; }
public string dateOfTransaction { get; set; }
public string price { get; set; }
public string packSale { get; set; }
}
Step 2 - Split the file and read the records
string strInput = #"2019-12-01T00:00:00.000Z;2019-12-10T23:59:59.999Z
50;false;2019-12-03T15:00:12.077Z;005033971003;48;141;2019-12-03T00:00:00.000Z;2019-12-03T23:59:59.999Z
100;false;2019-12-02T12:38:05.989Z;005740784001;80;311;2019-12-02T00:00:00.000Z;2019-12-02T23:59:59.999Z";
var headers = new Headers();
var transactions = new List<Transaction>();
var csvrecords = strInput.Split(new[] { '\r', '\n' },StringSplitOptions.RemoveEmptyEntries);
int count = 1;
foreach(var record in csvrecords)
{
var values = record.Split(';');
if (count == 1)
{
headers.TransactionFrom = values[0];
headers.TransactionTo = values[1];
}
else
{
var transaction = new Transaction();
transaction.logisticCode = values[3].Trim();
transaction.siteId = values[4].Trim();
transaction.userId = values[5].Trim();
transaction.dateOfTransaction = values[2].Trim();
transaction.price = values[0].Trim();
transactions.Add(transaction);
}
count++;
}
headers.Transactions = transactions;
var jsonString = JsonConvert.SerializeObject(headers);
Console.WriteLine(jsonString);
Output -
{
"TransactionFrom": "2019-12-01T00:00:00.000Z",
"TransactionTo": "2019-12-10T23:59:59.999Z",
"Transactions": [
{
"logisticCode": "005033971003",
"siteId": "48",
"userId": "141",
"dateOfTransaction": "2019-12-03T15:00:12.077Z",
"price": "50",
"packSale": null
},
{
"logisticCode": "005740784001",
"siteId": "80",
"userId": "311",
"dateOfTransaction": "2019-12-02T12:38:05.989Z",
"price": "100",
"packSale": null
}
]
}

With Cinchoo ETL, you can do it as follows
Define class structures as below
public class Headers
{
public string TransactionFrom { get; set; }
public string TransactionTo { get; set; }
public List<Transaction1> Transactions { get; set; }
}
public class Transaction
{
[ChoFieldPosition(4)]
public string logisticCode { get; set; }
[ChoFieldPosition(5)]
public string siteId { get; set; }
[ChoFieldPosition(6)]
public string userId { get; set; }
[ChoFieldPosition(2)]
public string dateOfTransaction { get; set; }
[ChoFieldPosition(1)]
public string price { get; set; }
}
Parse the CSV, generate JSON as below
string csv = #"2019-12-01T00:00:00.000Z;2019-12-10T23:59:59.999Z
50;false;2019-12-03T15:00:12.077Z;005033971003;48;141;2019-12-03T00:00:00.000Z;2019-12-03T23:59:59.999Z
100;false;2019-12-02T12:38:05.989Z;005740784001;80;311;2019-12-02T00:00:00.000Z;2019-12-02T23:59:59.999Z";
string csvSeparator = ";";
using (var r = ChoCSVReader.LoadText(csv)
.WithDelimiter(csvSeparator)
.ThrowAndStopOnMissingField(false)
.WithCustomRecordSelector(o =>
{
string line = ((Tuple<long, string>)o).Item2;
if (line.SplitNTrim(csvSeparator).Length == 2)
return typeof(Headers);
else
return typeof(Transaction);
})
)
{
var json = ChoJSONWriter.ToTextAll(r.GroupWhile(r1 => r1.GetType() != typeof(Headers))
.Select(g =>
{
Headers master = (Headers)g.First();
master.Transactions = g.Skip(1).Cast<Transaction1>().ToList();
return master;
}));
Console.WriteLine(json);
}
JSON Output:
[
{
"TransactionFrom": "2019-12-01T00:00:00.000Z",
"TransactionTo": "2019-12-10T23:59:59.999Z",
"Transactions": [
{
"logisticCode": "005033971003",
"siteId": "48",
"userId": "141",
"dateOfTransaction": "false",
"price": "50"
}
{
"logisticCode": "005740784001",
"siteId": "80",
"userId": "311",
"dateOfTransaction": "false",
"price": "100"
}
]
}
]

I am not sure if i can help you with any codes as your source CSV is very confusing, but i'll try to give you some ideas that might work out.
Firstly, you don't need a model class. I mean, you can use it if you want, but seems unnecessary here.
Next up is reading the CSV file. As you haven't posted any codes related to that and also didn't mention any problem with reading the file, i assume you are reading the file properly. Reading the CSV and writing a JSON from it is relatively easy. However, the CSV file itself looks very confusing. How are you reading it tho? Are you reading it as plain text? Do you have column headers or atleast columns?
If you are reading the file as plain text, then i guess you only have one way. And that is splitting the string and construct a new string with the splitted values. Splitting should be relatively easy as you have ;(semi-colon) which is separating each column/data. So the basic idea is splitting the string and storing it in an array or list, something like this :
string[] values = myCSV.split(";");
Now all you need to do is, simply use the strings inside values to construct a new string. You can use the StringBuilder for that, or an easy way(not feasible tho) would be string concatenation. I personally would recommend you to go with the StringBuilder.
Guidelines:
StringBuilder in C#
Creating a new line in StringBuilder
Double quotes inside string
Hopefully this gives you some ideas.

Related

How to work with nested Arrays and Objects using LINQ

so im working on a small project where i am consuming and deserialising json string into objects on C#. i set myself a business logic where i want to search for a team and return the number of goals they have scored (https://raw.githubusercontent.com/openfootball/football.json/master/2014-15/en.1.json)
The issue is i want to return the number of goals using LINQ instead of a loop (my original method). However, i do not know how i can retrieve the score. e.g
namespace ConsoleApp
{
class Program
{
private static string jsonUrl { get; set; } = "https://raw.githubusercontent.com/openfootball/football.json/master/2014-15/en.1.json";
private static string teamKey { get; set; } = "swansea";
static void Main()
{
var goal = Run(teamKey.ToLower());
Console.WriteLine(goal);
Console.ReadKey();
}
public static int Run(string team)
{
using (var webclient = new WebClient())
{
var rawJson = webclient.DownloadString(jsonUrl);
var jsonModel = JsonConvert.DeserializeObject<RootObject>(rawJson);
foreach (var rounds in jsonModel.rounds)
{
foreach (var match in rounds.matches)
{
var goal = match.team1.key.Equals(teamKey) ? match.score1 : 0;
if (goal == 0)
{
goal = match.team2.key.Equals(teamKey) ? match.score2 : 0;
}
return goal;
}
}
return 0;
}
}
}
public class Team1
{
public string key { get; set; }
public string name { get; set; }
public string code { get; set; }
}
public class Team2
{
public string key { get; set; }
public string name { get; set; }
public string code { get; set; }
}
public class Match
{
public string date { get; set; }
public Team1 team1 { get; set; }
public Team2 team2 { get; set; }
public int score1 { get; set; }
public int score2 { get; set; }
}
public class Round
{
public string name { get; set; }
public List<Match> matches { get; set; }
}
public class RootObject
{
public string name { get; set; }
public List<Round> rounds { get; set; }
}
}
The code above successfully executes and returns the correct number of goals based on the football team. but i dont think for performance this is the best way. (input: "swansea" expected result: 2 , Actual result: 2)
the array is represented as follows:
"rounds": [
{
"name": "Matchday 1",
"matches": [
{
"date": "2014-08-16",
"team1": {
"key": "manutd",
"name": "Manchester United",
"code": "MUN"
},
"team2": {
"key": "swansea",
"name": "Swansea",
"code": "SWA"
},
"score1": 1,
"score2": 2
},
{
"date": "2014-08-16",
"team1": {
"key": "leicester",
"name": "Leicester City",
"code": "LEI"
},
"team2": {
"key": "everton",
"name": "Everton",
"code": "EVE"
},
"score1": 3,
"score2": 5
}}]
This seems to require the from x in y ... select z syntax to obtain readable LINQ. Replace the part that starts with foreach (var rounds and ends with return 0; by the following code:
return (from round in jsonModel.rounds
from match in round.matches
let goal = match.team1.key.Equals(teamKey) ? match.score1 : 0
select goal == 0 ? (match.team2.key.Equals(teamKey) ? match.score2 : 0) : goal).FirstOrDefault();
I kept this as similar to the code in the question, for clarity. It would be better to extract helper methods to make the LINQ more readable: One helper for the expression match.team1.key.Equals(teamKey) ? match.score1 : 0, and one for the expression goal == 0 ? (match.team2.key.Equals(teamKey) ? match.score2 : 0) : goal.
The from x in y ... select z can be turned into a LINQ method chain. (E.g. the ReSharper tool can do that automatically.) But the result is so ugly there's no point in showing it.

Issue with accessing json nodes from C#

I have a json file that needs to be saved as sql server table. This is test.json that has Student details with coursework.
[{
"Studentid": "001006360",
"Grade": "2",
"ExtraWork": {
"TopRecommended": ["000133692",
"102067155",
"887273865"],
"OtherCourses": ["228963647",
"138909237",
"899791144",
"216165613",
"113239563"]
},
"Courses": [{
"smalldesc": "this is a test ",
"Details": {
"description": "Summary of the course",
"collegeCode": "32466"
}
},
{
"smalldesc": "Second test",
"Details": {
"description": "Business- Course Summary",
"collegeCode": "32469"
}
}]
}]
Below is the C# program.
I do not know how to access "smalldesc" and "collegeCode".
var jsonText = File.ReadAllText(#"C:\test.json");
var ser = JsonConvert.DeserializeObject<List<RootObject>>(jsonText);
for (int i = 0; i < ser.Count; i++)
{
string Studentid = ser[i].Studentid;
string Grade = ser[i].Grade;
var result = JsonConvert.DeserializeObject<List<Course>>(jsonText);
for (int k = 0; k < result.Count; k++)
{
string smalldesc = result[k].smalldesc;
string collegeCode = result[k].Details.collegeCode;
}
}
Json object class definition:
public class ExtraWork
{
public List<string> TopRecommended { get; set; }
public List<string> OtherCourses { get; set; }
}
public class Details
{
public string description { get; set; }
public string collegeCode { get; set; }
}
public class Course
{
public string smalldesc { get; set; }
public Details Details { get; set; }
}
public class RootObject
{
public string Studentid { get; set; }
public string Grade { get; set; }
public ExtraWork ExtraWork { get; set; }
public List<Course> Courses { get; set; }
}
what's the best way to save to sql server tables.
This line:
var ser = JsonConvert.DeserializeObject<List<RootObject>>(jsonText);
Is already doing all the deserialization for you, there is no need to call it again inside a loop.
Your code can be as simple as this:
var ser = JsonConvert.DeserializeObject<List<RootObject>>(jsonText);
foreach (var s in ser)
{
string Studentid = s.Studentid;
string Grade = s.Grade;
foreach(var course in ser.Courses)
{
string smalldesc = course .smalldesc;
string details = course .Details.collegeCode;
}
}
FYI: using a foreach loop is much simpler to work with when iterating a collection (assuming your collection type implements IEnumerable which most of the included collections will).

Extract array from JSON object containing multiple types

(Just a heads up, I'm very new to C#)
(See sample code and JSON structure below)
I can't figure out how to pull "data" out of the JSON reponse and put it into a data table. The variable "response" is just raw JSON data. So far I've figured out how to parse the JSON into a JObject...so now it has two members (data, meta). Now I'm trying to figure out how to get joTest["data"] into a DataTable. The handful of attempts I've made, keep giving me an error when it sees the "meta" member. Maybe I shouldn't be using a Data Table?
Also, in case it changes anything, I don't need the "links" from the "data" members.
I've tried searching for "Converting JObject into Data Table" But I'm not finding a lot of useful results.
public void PerformFeed()
{
string response;
response = Blah.SendMessage().Result;
JObject joTest = JsonConvert.DeserializeObject<JObject>(response);
}
Json Data Structure
{
"data": [
{
"Val1": "1234",
"Val2": "foo1",
"Val3": "bar1",
"links": [
{
"rel": "self",
"uri": "/blah/1234"
},
{
"rel": "pricing_data",
"uri": "/blah/1234/pricing_data"
}
]
},
{
"Val1": "5678",
"Val2": "foo2",
"Val3": "bar2",
"links": [
{
"rel": "self",
"uri": "/blah/5678"
},
{
"rel": "pricing_data",
"uri": "/blah/5678/pricing_data"
}
]
}
],
"meta": {
"pagination": {
"total": 2,
"count": 2,
"per_page": 25,
"current_page": 1,
"total_pages": 1,
"links": []
}
}
}
UPDATE: I've figured out a "solution" but I really don't think it's a good solution. I built a datatable and then used a foreach statement on the JObject to populate the data table that way. It seems very inefficient...but for now it works. Hopefully I'll find a better way.
public void PerformFeed()
{
DataTable Items = new DataTable();
Items.Columns.Add("Val1");
Items.Columns.Add("Val2");
Items.Columns.Add("Val3");
string response = Blah.SendMessage().Result;
JObject Data = JObject.Parse(response);
foreach (JObject jo in Data["data"])
{
Items.Rows.Add(jo["Val1"], jo["Val2"], jo["Val3"]);
}
}
There is this really nice online utility that helps extracting C# classes from JSON objects. I think the problem here is with your JSON, you're missing a comma ",". You would easily be able to spot the error with some online JSON formatter / validator. Rest the deserialization is pretty straightforward. Try the following:
JObject obtainedObject = JObject.Parse(JsonString);
Following would be the structure of your obtained object:
public class RequiredClass
{
public IList<Datum> data { get; set; }
public Meta meta { get; set; }
}
public class Datum
{
public string Val1 { get; set; }
public string Val2 { get; set; }
public string Val3 { get; set; }
public IList<Link> links { get; set; }
}
public class Link
{
public string rel { get; set; }
public string uri { get; set; }
}
public class Pagination
{
public int total { get; set; }
public int count { get; set; }
public int per_page { get; set; }
public int current_page { get; set; }
public int total_pages { get; set; }
public IList<object> links { get; set; }
}
public class Meta
{
public Pagination pagination { get; set; }
}
Update:
Here's is how you extract your array and convert that to a DataTable:
JObject jObject = JObject.Parse(json);
JToken dataArray = jObject["data"];
DataTable dt = (DataTable) JsonConvert.DeserializeObject(dataArray.ToString(), (typeof(DataTable)));
To avoid the surplus casting, you can try the following using the class structure already mentioned above:
JObject jObject = JObject.Parse(json);
JToken dataArray = jObject["data"];
List<Datum> requiredList = new List<Datum>();
foreach (var item in dataArray)
{
Datum obj = new Datum();
obj.Val1 = (string) item["Val1"] ?? "";
obj.Val2 = (string) item["Val2"] ?? "";
obj.Val3 = (string) item["Val3"] ?? "";
obj.links = new List<Link>();
foreach(var subItem in item["links"])
{
Link lnk = new Link();
lnk.rel = (string) subItem["rel"] ?? "";
lnk.uri = (string) subItem["uri"] ?? "";
obj.links.Add(lnk);
}
requiredList.Add(obj);
}

Error while de-serializing json array [duplicate]

This question already has answers here:
Serialize and Deserialize Json and Json Array in Unity
(9 answers)
Closed 4 years ago.
I'm trying to deserialize a Json string.
This is my code:
[System.Serializable]
public class SharedWorlds
{
public int worldId { get; set; }
public System.DateTime uploaded { get; set; }
public string username { get; set; }
public string levelName { get; set; }
public string gameVersion { get; set; }
public string description { get; set; }
public string filename { get; set; }
public string screenshot1 { get; set; }
public string screenshot2 { get; set; }
public string userTag { get; set; }
public string userURL { get; set; }
public double price { get; set; }
public int nrDownload { get; set; }
public int votes { get; set; }
}
[System.Serializable]
public class Record {
public List<SharedWorlds> record;
}
try {
SDE3D _webService = new SDE3D();
result= _webService.GetMassiveWorldsList ();
var records = JsonUtility.FromJson<Record>(result);
}
catch(System.Exception ex) {
Debug.Log (ex.Message.ToString ());
}
And this is my valid jSon (here two records, but I want to send many records per time).
[
{
"worldId": 5,
"uploaded": "/Date(1524875719000)/",
"username": "quik",
"levelName": "Station",
"gameVersion": "1.0.1",
"description": "iwoeijksf",
"filename": "0000003.dat",
"screenshot1": "0000003a.png",
"screenshot2": "0000003b.png",
"userTag": "",
"userURL": "",
"price": 0,
"nrDownload": 5,
"votes": 5
},
{
"worldId": 4,
"uploaded": "/Date(1524875659000)/",
"username": "aksio",
"levelName": "Garage",
"gameVersion": "1.0.1",
"description": "Adlkld",
"filename": "0000003.dat",
"screenshot1": "0000003a.png",
"screenshot2": "0000003b.png",
"userTag": "",
"userURL": "",
"price": 0,
"nrDownload": 4,
"votes": 4
}
]
I'm getting error:
"ArgumentException: JSON must represent an object type."
I'm pretty sure the error is in this code line:
var records = JsonUtility.FromJson<Record>(result);
How to deserialize an array of json object ?
Thanks
Because your JSON data is not a Record. It's a collection of SharedWorlds. So something like this:
var sharedWorlds = JsonUtility.FromJson<SharedWorld[]>(result);
Or perhaps:
var sharedWorlds = JsonUtility.FromJson<List<SharedWorld>>(result);
From which you could create a Record:
var record = new Record { record = sharedWorlds };
If the JSON needs to deserialize into a Record then it would need to be in the format of a Record object:
{
"record":
[
/* the rest of your JSON within the square brackets */
]
}
Then it would be a Record:
var record = JsonUtility.FromJson<Record>(result);
*Side note: Your class and variable names and the pluralizations you're using are really confusing. The semantics of which is probably not making your debugging any easier for you.
You get to script collection of object no one single object in your JSON

Json parse sub-collection in c#

How can I do JSON in C# like the data below ?
{
"Aliases": [ "teddy", "freddy", "eddy", "Betty" ],
"Name":"reacher gilt",
"Address":"100 East Way",
"Age":74,
"Bars": {
"items": [
{
"Sub_Property1":"beep",
"Sub_Property2":"boop"
},
{
"Sub_Property1":"meep",
"Sub_Property2":"moop"
},
{
"Sub_Property1":"feep",
"Sub_Property2":"foop"
}
]
}
}
Actually my problem is inside the sub-collection. I saw someone did something
like this
person.Bars.Add("items",
new List<BarClass>(new[]{
new BarClass("beep","boop"),
new BarClass("meep","moop"),
new BarClass("feep","foop"),
}));
So, I have to add new BarClass("beep","boop"), but I need to do something
like this
String [] no1 = {1,2,3}
String [] no2 = {4,5,6}
person.Bars.Add("items",
new List<BarClass>(new[]{
for ()
{
new BarClass(no1[i],no2[i])
}
}));
How can i do this? Thanks and please help..
To read the JSON
The best way to read the whole JSON is to Deserialize it to a native C# object. If you do not already have the classes with your, you can create it in Visual Studio as
Copy your JSON text
Create a new empty class file in VS
Edit > Paste Special > Paste JSON As Classes
Here are the classes
public class Person
{
public string[] Aliases { get; set; }
public string Name { get; set; }
public string Address { get; set; }
public int Age { get; set; }
public Bars Bars { get; set; }
}
public class Bars
{
public Item[] items { get; set; }
}
public class Item
{
public string Sub_Property1 { get; set; }
public string Sub_Property2 { get; set; }
}
Now you can use some .NET JSON library to deserialize. JSON.Net aka Newtonsoft JSON is a great library. You get get it from NuGet as well.
Then it's pretty easy to get the C# object from the JSON
//using Newtonsoft.Json;
var jsonString = File.ReadAllText(#"C:\YourDirectory\person.json");
var person = JsonConvert.DeserializeObject<Person>(jsonString);
If you want to read the sub-collection only, you can rather use Linq-to-JSON to read the items directly, like this
//using Newtonsoft.Json.Linq;
var jObject = JObject.Parse(jsonString);
List<Item> details = jObject["Bars"]["items"].ToObject<List<Item>>();
To create the JSON
You first need to create the object, then Serialize to JSON
string[] subProperties1 = new string[] { "1", "2", "3" };
string[] subProperties2 = new string[] { "4", "5", "6" };
Person person = new Person { Name = "Johny", Age = 7, Address = "Earth", Aliases = new string[] { "Sony", "Monty" } };
person.Bars = new Bars {
items = subProperties1.Zip(subProperties2,
(prop1, prop2) => new Item { Sub_Property1 = prop1, Sub_Property2 = prop2 })
.ToArray() };
var json = JsonConvert.SerializeObject(person);
To create the items from your existing string arrays, I have used IEnumerable.Zip function from Linq. You can read about them here.
This is the created JSON data
{
"Aliases": [ "Sony", "Monty" ],
"Name": "Johny",
"Address": "Earth",
"Age": 7,
"Bars": {
"items": [
{
"Sub_Property1": "1",
"Sub_Property2": "4"
},
{
"Sub_Property1": "2",
"Sub_Property2": "5"
},
{
"Sub_Property1": "3",
"Sub_Property2": "6"
}
]
}
}
You should create some classes
public class Person
{
public string Name {get;set;}
public string Address{get;set;}
public int Age {get;set;}
public Header {get;set;}
}
public class Header
{
public Detail[] Details {get;set;}
}
public class Detail
{
public string Sub1 {get;set;}
public string Sub2 {get;set;}
}
Create instance from Person class and initialize to instance after than
JavaScriptSerializer serializer =new JavaScriptSerializer();
var result=serializer.Serialize(instanceOfPerson);
"result" is json data
Assuming that you mean you want to create JSON string, you need to create those classes and use something like Newtonsoft JSON.net:
public class Item
{
public string Sub_Property1 { get; set; }
public string Sub_Property2 { get; set; }
}
public class Bars
{
public List<Item> items { get; set; }
}
public class Person
{
public List<string> Aliases { get; set; }
public string Name { get; set; }
public string Address { get; set; }
public int Age { get; set; }
public Bars Bars { get; set; }
}
Please read the documentation here: http://www.newtonsoft.com/json/help/html/Introduction.htm

Categories

Resources