Develop Reference

Trying to deserialize json array

I am trying to deserialize a local file in my project which is a json file. However I am getting this error with the current code:
"Unexpected character encountered while parsing value: G. Path '', line 0, position 0"
C# code
string filepath = Application.StartupPath + #"\city.list.json";
for(int i = 0; i< 40; i++)
{
foreach (string x in File.ReadLines(filepath))
{
if(x.Contains("id") || x.Contains("name"))
{
var data = JsonConvert.DeserializeObject<City.values>(filepath);
//City city = JsonConvert.DeserializeObject<City>(File.ReadAllText(filepath));
//cityList.Add(data.name, data.id);
}
else
{
}
}
}
class City
{
[JsonObject(MemberSerialization = MemberSerialization.OptIn)]
public class values
{
[JsonProperty(PropertyName = "id")]
public string id { get; set; }
[JsonProperty(PropertyName = "name")]
public string name { get; set; }
}
}
Json file I am trying to deserialize from. This is just a quick sample taken out from the file. It is quite large ^^
[
{
"id":707860,
"name":"Hurzuf",
"country":"UA",
"coord":{
"lon":34.283333,
"lat":44.549999
}
},
{
"id":519188,
"name":"Novinki",
"country":"RU",
"coord":{
"lon":37.666668,
"lat":55.683334
}
},

You seem to have some huge misconceptions of how JSON deserializing works. First thing to address is you shouldn't be iterating through the lines of the json file. As pointed out in the comments, your JSON file is very large (~1.8 million lines) so your best bet is to use the JsonReader overload of DeserializeObject(), see Json.NET performance tips:
List<City.values> cities = new List<City.values>();
string filepath = Application.StartupPath + #"\city.list.json";
using (StreamReader sr = new StreamReader(filepath))
using (JsonReader reader = new JsonTextReader(sr))
{
JsonSerializer serializer = new JsonSerializer();
// read the json from a stream
// json size doesn't matter because only a small piece is read at a time from the HTTP request
cities = JsonConvert.DeserializeObject<List<City.values>>(reader);
}
Draw your attention to this line:
cities = JsonConvert.DeserializeObject<List<City.values>>(reader);
Here we leverage JSON.NET to deserialize. The difference between your code and the code I included here is that your JSON is a collection of objects, what this means is you need to deserialize into a collection of your City.values objects, in this case I used a List<T>.
Now we have a variable cities that is a collection of City.values objects that are included in your JSON.

Write to CSV file using CsvHelper in C#

I tried to write to CSV file using CsvHelper in C#.
This is the link to the library http://joshclose.github.io/CsvHelper/
Nothing is sent to the csv file. I tried doing "exportCsv.WriteField("Hello");" but still nothing happened.
List<string> ColumnOne = new List<string>();
List<string> ColumnTwo = new List<string>();
var csvTextWriter = new
StreamWriter(#"C:\Users\Public\Documents\ExportTest.csv");
var exportCsv = new CsvWriter(csvTextWriter);
//creating a list to store workflows then adding name and description to the myWorkflowsList list
if (myWorkflows.WorkFlowCollection.Any())
{
foreach (var Workflow in myWorkflows.WorkFlowCollection)
{
ColumnOne.Add(Workflow.WorkflowName);
ColumnTwo.Add(Workflow.WorkflowDescription);
}
exportCsv.WriteField(ColumnOne);
//exportCsv.WriteField(ColumnTwo);
exportCsv.NextRecord();
exportCsv.Flush();
Console.WriteLine("File is saved:
C:\\Users\\Public\\Documents\\ExportTest.csv");
Console.ReadLine();
}

Your code doesn't add any records. It doesn't have any calls to WriteRecords or WriteRecord. It looks like it's trying to write an entire list of strings into a single field instead.
To write two columns out to a file you can use `WriteRecords, eg :
var data = from flow in myWorkflows.WorkFlowCollection
select new { flow.WorkflowName,flow.WorkflowDescription};
using (var writer = new StreamWriter("test.csv"))
using (var csv = new CsvWriter(writer))
{
csv.WriteRecords(data);
}
This will write a file with field names WorkflowName and WorkflowDescription
You can change how the fields are written by creating a small class that accepts only the fields you want and sets names etc through attributes :
class Flow
{
[NameAttribute("Workflow Name")]
public string WorkflowName { get; set; }
[NameAttribute("Workflow Description")]
public string WorkflowDescription { get; set; }
public Flow(string workflowName, string workflowDescription)
{
WorkflowName = workflowName;
WorkflowDescription = workflowDescription;
}
}
//...
var data = from flow in myWorkflows.WorkFlowCollection
select new Flow(flow.WorkflowName,flow.WorkflowDescription);
using (var writer = new StreamWriter("test.csv"))
using (var csv = new CsvWriter(writer))
{
csv.WriteRecords(data);
}

Creating JSON from a CSV file in C#

First of all my apologies because this is going to be a "How to" question rather than a technical question. I have a CSV file as follows-
London,Dubai,4
Dubai,Mumbai,8
Dubai,Dhaka,4
Now my plan is to create a JSON object from that CSV in the following format-
[
{
"From": "London",
"To": "Dubai",
"Duration": 4
},
{
"From": "Dubai",
"To": "Mumbai",
"Duration": 8
},
{
"From": "Dubai",
"To": "Dhaka",
"Duration": 4
},
]
How do I go about and do that? Currently I can load the CSV using OpenFileDialog but no idea what else I should do to get it done? Use Model Classes? JSON.Net? Please advice me and some code samples would be appreciated!

You can add csv records to a List<T> and then serialize it with Newtonsoft.Json to get your required JSON object. See the example below:
class Program
{
static void Main(string[] args)
{
string[] csv = new[] { "London,Dubai,4", "Dubai,Mumbai,8", "Dubai,Dhaka,4" };
List<model> list = new List<model>();
foreach (var item in csv)
{
string[] fields = item.Split(',');
list.Add(new model
{
From = fields[0],
To = fields[1],
Duration = fields[2]
});
}
var json = JsonConvert.SerializeObject(list);
Console.WriteLine(json);
Console.ReadLine();
}
}
public class model
{
public string From { get; set; }
public string To { get; set; }
public string Duration { get; set; }
}

You can use TextFieldParser from the Microsoft.VisualBasic.FileIO namespace and Microsoft.VisualBasic.dll assembly to parse CSV files. Despite the VisualBasic name the class is perfectly usable in c#.
First, add the following extension method:
public static class TextFieldParserExtensions
{
public static IEnumerable<string []> ReadAllFields(this TextFieldParser parser)
{
if (parser == null)
throw new ArgumentNullException();
while (!parser.EndOfData)
yield return parser.ReadFields();
}
}
Now you can use LINQ to transform each CSV line into an anonymous or named type for serialization, like so:
var csv = #"London,Dubai,4
Dubai,Mumbai,8
Dubai,Dhaka,4";
string json;
using (var stream = new StringReader(csv))
using (TextFieldParser parser = new TextFieldParser(stream))
{
parser.SetDelimiters(new string[] { "," });
var query = parser.ReadAllFields()
.Select(a => new { From = a[0], To = a[1], Duration = int.Parse(a[2]) });
json = new JavaScriptSerializer().Serialize(query);
}
Here I am using JavaScriptSerializer but the same code can be used with json.net
json = JsonConvert.SerializeObject(query, Formatting.Indented);
Be sure to evaluate the query before closing the TextFieldParser.

I Believe this should work for all different kinds of .csv files
Comments are in the code
public class Program
{
public static void Main(string[] args)
{
var list = new List<Dictionary<string, string>>();
Console.WriteLine("Put in the path to your .csv file");
var response1 = Console.ReadLine();
Console.WriteLine("Initializing...");
// Read All of the lines in the .csv file
var csvFile = File.ReadAllLines(response1);
// Get The First Row and Make Those You Field Names
var fieldNamesArray = csvFile.First().Split(',');
// Get The Amount Of Columns In The .csv
// Do the -1 so you can use it for the indexer below
var fieldNamesIndex = fieldNamesArray.Count() - 1;
// Skip The First Row And Create An IEnumerable Without The Field Names
var csvValues = csvFile.Skip(1);
// Iterate Through All Of The Records
foreach (var item in csvValues)
{
var newDiction = new Dictionary<string, string>();
for (int i = 0; i < fieldNamesIndex;)
{
foreach (var field in item.Split(','))
{
// Think Of It Like This
// Each Record Is Technically A List Of Dictionary<string, string>
// Because When You Split(',') you have a string[]
// Then you iterate through that string[]
// So there is your value but now you need the field name to show up
// That is where the Index will come into play demonstrated below
// The Index starting at 0 is why I did the -1 on the fieldNamesIndex variable above
// Because technically if you count the fields below its actually 6 elements
//
// 0,1,2,3,4,5 These Are The Field Names
// 0,1,2,3,4,5 These Are The Values
// 0,1,2,3,4,5
//
// So what this is doing is int i is starting at 0 for each record
// As long as i is less than fieldNamesIndex
// Then split the record so you have all of the values
// i is used to find the fieldName in the fieldNamesArray
// Add that to the Dictionary
// Then i is incremented by 1
// Add that Dictionary to the list once all of the values have been added to the dictionary
//
// Add the field name at the specified index and the field value
newDiction.Add(fieldNamesArray.ElementAt(i++), field);
}
list.Add(newDiction);
}
}
Console.WriteLine("Would You Like To Convert To Json Now?");
Console.WriteLine("[y] or [n]");
var response = Console.ReadLine();
if (response == "y")
{
Console.WriteLine("Where Do You Want The New File?");
var response2 = Console.ReadLine();
// Serialize the list into your Json
var json = JsonConvert.SerializeObject(list);
File.Create(response2).Dispose();
File.AppendAllText(response2, json);
Console.WriteLine(json);
Console.ReadLine();
}
else
{
Console.WriteLine("Ok See You Later");
Console.ReadLine();
}
}
}

How to keep json file format valid while using Json.net library to write new record to existing to json file?

my json file
[
{ "name" : "student1",
"grade" : "A"
},
{ "name" : "student2",
"grade" : "B"
}
]
My write code
using (FileStream fs = File.Open(WebConfigurationManager.AppSettings["JsonFileLocation"], FileMode.Append))
using (StreamWriter sw = new StreamWriter(fs))
using (JsonWriter jw = new JsonTextWriter(sw))
{
var student = new Students{name = name, grade = grade};
jw.Formatting = Formatting.Indented;
JsonSerializer serializer = new JsonSerializer();
serializer.Serialize(jw, student);
}
After I write new student
[
{ "name" : "student1",
"grade" : "A"
},
{ "name" : "student2",
"grade" : "B"
}
]
{ "name" : "student3",
"grade" : "C"
}
Notice how it appended at the end of the file rather than after the last record and within square brackets. This is making my json file invalid. How do I make it so it will append new record within square bracket and with comma.

You're appending a single student object JSON into an array of Student object Json. Notice the [] to denote the array.
You need to read the array, append the existing student object into the array and write back the array as JSON.
The below code reads the JSON contents from the file into a list of Students object (you might want to rename the class name to Student), appends the new Students object to this list and then writes back the complete list of Students into the file. The file will have 3 students objects in the proper JSON format.
var fileName = WebConfigurationManager.AppSettings["JsonFileLocation"];
List<Students> studentsList = new List<Students>();
using (var sr = new StreamReader(fileName))
{
var jsonStudentsText = sr.ReadAllText();
studentsList = JsonConvert.DeserializeObject<List<Students>>(jsonStudentsText);
}
studentsList.Add(new Students{name = name, grade = grade});
using (var sw = new StreamWriter(fileName))
{
using (JsonWriter jw = new JsonTextWriter(sw))
{
jw.Formatting = Formatting.Indented;
JsonSerializer serializer = new JsonSerializer();
serializer.Serialize(jw, studentsList);
}
}

try dessiarilize json to list then append a new object to list and serrialize it back to json
List<Students> studentlist = JsonConvert.DeserializeObject<List<Students>>(your json);
studentlist.Add(new Students{name = name, grade = grade});
JsonSerializer serializer = new JsonSerializer();
using (StreamWriter sw = new StreamWriter(#"c:\json.txt"))
using (JsonWriter writer = new JsonTextWriter(sw))
{
serializer.Serialize(writer, studentlist);
}

C# Serializing datacontracts from file

I have a list of Xml messages specifically DataContract messages that i record to a file. And i am trying to deserialize them from file one by one. I do not want to read the whole file into memory at once because i expect it to be very big.
I have an implementation of this serialization and that works. I did this by serializing using a FileStream and reading the bytes and using regular expression to determine the end of element. Then taking the element and using DataContractSerializer to get the actual object.
But i was told I should be using higher level code to do this task and it seems like that should be possible. I have the following code that i think should work but it doesn't.
FileStream readStream = File.OpenRead(filename);
DataContractSerializer ds = new DataContractSerializer(typeof(MessageType));
MessageType msg;
while ((msg = (MessageType)ds.ReadObject(readStream)) != null)
{
Console.WriteLine("Test " + msg.Property1);
}
The above code is fed with an input file containing something along the following lines:
<MessageType>....</MessageType>
<MessageType>....</MessageType>
<MessageType>....</MessageType>
It appears that i can read and deserialize the first element correctly but after that it fails saying:
System.Runtime.Serialization.SerializationException was unhandled
Message=There was an error deserializing the object of type MessageType. The data at the root level is invalid. Line 1, position 1.
Source=System.Runtime.Serialization
I have read somewhere that it is due to the way DataContractSerializer works with padded '\0''s to the end - but i couldn't figure out how to fix this problem when reading from a stream without figuring out the end of MessageType tag in some other way. Is there another Serialization class that i should be using? or perhaps a way around this problem?
Thanks!

When you're deserializing the data from the file, WCF uses by default a reader which can only consume proper XML documents. The document which you're reading isn't - it contains multiple root elements, so it's effectively a fragment. You can change the reader the serializer is using by using another overload of ReadObject, as shown in the example below, to one which accepts fragments (by using the XmlReaderSettings object). Or you can have some sort of wrapping element around the <MessageType> elements, and you'd read until the reader were positioned at the end element for the wrapper.
public class StackOverflow_7760551
{
[DataContract]
public class Person
{
[DataMember]
public string Name { get; set; }
[DataMember]
public int Age { get; set; }
public override string ToString()
{
return string.Format("Person[Name={0},Age={1}]", this.Name, this.Age);
}
}
public static void Test()
{
const string fileName = "test.xml";
using (FileStream fs = File.Create(fileName))
{
Person[] people = new Person[]
{
new Person { Name = "John", Age = 33 },
new Person { Name = "Jane", Age = 28 },
new Person { Name = "Jack", Age = 23 }
};
foreach (Person p in people)
{
XmlWriterSettings ws = new XmlWriterSettings
{
Indent = true,
IndentChars = " ",
OmitXmlDeclaration = true,
Encoding = new UTF8Encoding(false),
CloseOutput = false,
};
using (XmlWriter w = XmlWriter.Create(fs, ws))
{
DataContractSerializer dcs = new DataContractSerializer(typeof(Person));
dcs.WriteObject(w, p);
}
}
}
Console.WriteLine(File.ReadAllText(fileName));
using (FileStream fs = File.OpenRead(fileName))
{
XmlReaderSettings rs = new XmlReaderSettings
{
ConformanceLevel = ConformanceLevel.Fragment,
};
XmlReader r = XmlReader.Create(fs, rs);
while (!r.EOF)
{
Person p = new DataContractSerializer(typeof(Person)).ReadObject(r) as Person;
Console.WriteLine(p);
}
}
File.Delete(fileName);
}
}

Maybe your file contains BOM
It's common for UTF-8 encoding

XmlSerializer xml = new XmlSerializer(typeof(MessageType));
XmlDocument xdoc = new XmlDocument();
xdoc.Load(stream);
foreach(XmlElement elm in xdoc.GetElementsByTagName("MessageType"))
{
MessageType mt = (MessageType)xml.Deserialize(new StringReader(elm.OuterXml));
}