Deserialize List<BsonDocument> from JSON string - c#

I'm trying to import the sample dataset in this mongo tutorial, but I am trying to do it using C# code. Here is what I have:
var reader = new StreamReader("C:\\primer-dataset.json");
string line;
var sb = new StringBuilder();
while ((line = reader.ReadLine()) != null)
{
sb.Append(line);
}
var documents = BsonSerializer.Deserialize<List<BsonDocument>>(sb.ToString());
var collection = _database.GetCollection<BsonDocument>("restaurants");
collection.InsertMany(documents);
I get the following error when I try to Deserialize the json string:
Cannot deserialize a 'List' from BsonType 'Document'
I can't find anything online to tell what I'm doing incorrectly.


If someone ever has the same problem. The solution is to add the brackets to json string before the parsing:
var json = await File.ReadAllTextAsync(path);
json = "[" + json + "]";
var bsonDocuments = BsonSerializer.Deserialize<BsonDocument[]>(json);
if (bsonDocuments.Length > 0)
{
await collection.InsertManyAsync(bsonDocuments);
}


In the end, I had to modify the JSON document based on Quantic's comment. The sample doc assumes the user is going to use mongoimport to import the data. Once I added the necessary brackets and commas to make it a standard array, I was able to import it using the following C# code:
var collection = _database.GetCollection<BsonDocument>("restaurants");
var reader = new StreamReader("C:\\primer-dataset.json");
string line;
var sb = new StringBuilder();
while ((line = reader.ReadLine()) != null)
{
sb.Append(line);
}
var arr = JArray.Parse(sb.ToString());
foreach(JObject o in arr)
{
var d = BsonDocument.Parse(o.ToString());
collection.InsertOne(d);
}

Related

Serialize dynamic list to CSV without header in Servicestack.Text

I'm trying to generate a csv file using CsvSerializer.SerializeToCsv(data), but I want to omit the headers.
I read this question, but this is not working as I'm using a list of dynamic objects.
I've tried:
IEnumerable<dynamic> data = ...;
CsvConfig<object>.OmitHeaders = true;
string csvFile = CsvSerializer.SerializeToCsv(data);
And
IEnumerable<dynamic> data = ...;
CsvConfig<dynamic>.OmitHeaders = true;
string csvFile = CsvSerializer.SerializeToCsv(data);
Both options are serializing the csvFile with headers, which I don't need.

Since I didn't find a way with a library, I opt to do this manually. Something like this worked for me:
var parsedData = new List<string>();
// parse data into comma separated objects
parsedData.AddRange(data.Select(d =>
{
var dProperties = (IDictionary<string, object>)d;
var valuesFixed = dProperties.Values.Select(v => v.ToString().ToRFC4180String());
return string.Join(",", valuesFixed);
}));
var file = string.Join("\r\n", parsedData);
Where FillInnerQuotes is just an extensor method to manage special characters based on rfc4180 standard.
public static string ToRFC4180String(this string value)
{
if(value.Contains("\""))
value = value.Replace("\"", "\"\"");
if(value.Contains("\"")
|| value.Contains("\n")
|| value.Contains("\r")
|| value.Contains("\r\n")
|| value.Contains(","))
return $"\"{value}\"";
return value;
}

How to split CSV file

"0.0.0.0,""0.255.255.255"",""ZZ"""
"1.0.0.0,""1.0.0.255"",""AU"""
"1.0.1.0,""1.0.3.255"",""CN"""
"1.0.4.0,""1.0.7.255"",""AU"""
"1.0.8.0,""1.0.15.255"",""CN"""
"1.0.16.0,""1.0.31.255"",""JP"""
"1.0.32.0,""1.0.63.255"",""CN"""
"1.0.64.0,""1.0.127.255"",""JP"""
"1.0.128.0,""1.0.255.255"",""TH"""
"1.1.0.0,""1.1.0.255"",""CN"""
"1.1.1.0,""1.1.1.255"",""AU"""
"1.1.2.0,""1.1.63.255"",""CN"""
"1.1.64.0,""1.1.127.255"",""JP"""
"1.1.128.0,""1.1.255.255"",""TH"""
İN EXCEL
0.0.0.0,"0.255.255.255","ZZ"
1.0.0.0,"1.0.0.255","AU"
1.0.1.0,"1.0.3.255","CN"
1.0.4.0,"1.0.7.255","AU"
1.0.8.0,"1.0.15.255","CN"
1.0.16.0,"1.0.31.255","JP"
1.0.32.0,"1.0.63.255","CN"
1.0.64.0,"1.0.127.255","JP"
1.0.128.0,"1.0.255.255","TH"
1.1.0.0,"1.1.0.255","CN"
1.1.1.0,"1.1.1.255","AU"
1.1.2.0,"1.1.63.255","CN"
1.1.64.0,"1.1.127.255","JP"
1.1.128.0,"1.1.255.255","TH"
1.2.0.0,"1.2.2.255","CN"
1.2.3.0,"1.2.3.255","AU"
1.2.4.0,"1.2.127.255","CN"
1.2.128.0,"1.2.255.255","TH"
1.3.0.0,"1.3.255.255","CN"
1.4.0.0,"1.4.0.255","AU"
1.4.1.0,"1.4.127.255","CN"
1.4.128.0,"1.4.255.255","TH"
How can split this CSV file.
For example 0.0.0.0 0.255.255.255 ZZ for first row and how can add datagridview with 3columns

You can do it via the following way..
using System.IO;
static void Main(string[] args)
{
using(var reader = new StreamReader(#"C:\test.csv"))
{
List<string> listA = new List<string>();
List<string> listB = new List<string>();
while (!reader.EndOfStream)
{
var line = reader.ReadLine();
var values = line.Split(','); // or whatever yur get by reading that file
listA.Add(values[0]);
listB.Add(values[1]);
}
}
}

A CSV file is either a Tab delimited or a Comma delimited file. That said; you have to read the file line by line and then separate the values available in a line based on the delimiter character. The first line usually appears in a CSV file is usually the headers which you can use in order to produce a KeyValue pair to make your collection more efficient. For example:
Dictionary<int, Dictionary<String, String>> values = new Dictionary<int, Dictionary<String,String>>();
using(FileStream fileStream = new FileStream(#"D:\MyCSV.csv", FileMode.Open, FileAccess.Read, FileShare.Read)) {
using(StreamReader streamReader = new StreamReader(fileStream)){
//You can skip this line if there is no header
// Then instead of Dictionary<String,String> you use List<String>
var headers = streamReader.ReadLine().Split(',');
String line = null;
int lineNumber = 1;
while(!streamReader.EndOfStream){
line = streamReader.ReadLine().split(',');
if(line.Length == headers.Length){
var temp = new Dictionary<String, String>();
for(int i = 0; i < headers.Length; i++){
// You can remove '"' character by line[i].Replace("\"", "") or through using the Substring method
temp.Add(headers[i], line[i]);
}
values.Add(lineNumber, temp);
}
lineNumber++;
}
}
In case the data structure of your CSV is constant and it will not change in the future, you can develop a strongly typed data model and get rid of the Dictionary type. This approach will be more elegant and more efficient.

First of all, your CSV lines are surrounded by quotes. Is it copy/paste mistake? If not, you will need to sanitize the file to a valid CSV file.
You can try Cinchoo ETL - an open source library to load the CSV file to datatable, then you can assign it to your DataGridView source.
I'll show you both approach, how to handle
Valid CSV: (test.csv)
0.0.0.0,"0.255.255.255","ZZ"
1.0.0.0,"1.0.0.255","AU"
1.0.1.0,"1.0.3.255","CN"
1.0.4.0,"1.0.7.255","AU"
1.0.8.0,"1.0.15.255","CN"
1.0.16.0,"1.0.31.255","JP"
1.0.32.0,"1.0.63.255","CN"
1.0.64.0,"1.0.127.255","JP"
1.0.128.0,"1.0.255.255","TH"
1.1.0.0,"1.1.0.255","CN"
1.1.1.0,"1.1.1.255","AU"
1.1.2.0,"1.1.63.255","CN"
1.1.64.0,"1.1.127.255","JP"
1.1.128.0,"1.1.255.255","TH"
Read CSV:
using (var p = new ChoCSVReader("test.csv"))
{
var dt = p.AsDataTable();
//Assign dt to DataGridView
}
Next approach
Invalid CSV: (test.csv)
"0.0.0.0,""0.255.255.255"",""ZZ"""
"1.0.0.0,""1.0.0.255"",""AU"""
"1.0.1.0,""1.0.3.255"",""CN"""
"1.0.4.0,""1.0.7.255"",""AU"""
"1.0.8.0,""1.0.15.255"",""CN"""
"1.0.16.0,""1.0.31.255"",""JP"""
"1.0.32.0,""1.0.63.255"",""CN"""
"1.0.64.0,""1.0.127.255"",""JP"""
"1.0.128.0,""1.0.255.255"",""TH"""
"1.1.0.0,""1.1.0.255"",""CN"""
"1.1.1.0,""1.1.1.255"",""AU"""
"1.1.2.0,""1.1.63.255"",""CN"""
"1.1.64.0,""1.1.127.255"",""JP"""
"1.1.128.0,""1.1.255.255"",""TH"""
Read CSV:
using (var p = new ChoCSVReader("Sample6.csv"))
{
p.SanitizeLine += (o, e) =>
{
string line = e.Line as string;
if (line != null)
{
line = line.Substring(1, line.Length - 2);
line = line.Replace(#"""""", #"""");
}
e.Line - line;
};
var dt = p.AsDataTable();
//Assign dt to DataGridView
}
Hope it helps.

Convert Variable Header to JSON JArray

I have code that gets data from JSON:
dynamic myJSON = Newtonsoft.Json.JsonConvert.DeserializeObject(sourceString);
foreach (var item in myJSON["data"]["results"])
{ ... }
Now, I'd like to make this part myJSON["data"]["results"] for versatile, ideally by passing in a variable such as:
dynamic myJSON = Newtonsoft.Json.JsonConvert.DeserializeObject(sourceString);
var variableResults = ((Newtonsoft.Json.Linq.JArray) string.Format("myJSON[\"{0}\"][\"{1}\"]", myData, myResults);
foreach (var item in variableResults)
{ ... }
I've tried different conversions, and looked here, but no question comes close, and the conversions always comes out string, not a JArray. Any ideas?

You can use JObject to parse it:
var json = "{asd: {qwe: [1,2,3,4]}}";
var obj = JObject.Parse(json);
// just plug in your variables here
var myData = "asd";
var myResult = "qwe";
var path = $"{myData}.{myResult}";
foreach (var item in obj.SelectToken(path).ToObject<int[]>())
{
// ...
}

My solution ended up being:
// User would use a textbox with space separated values like "data results"
var parameterList = textBox1.Text.Trim().Replace(" ", ".");
var jsonObject = JObject.Parse(sourceString);
var jsonList = jsonObject.SelectToken(parameterList);
if (jsonList == null) { ... }
foreach (var item in jsonList)
{ ... }

C# Convert Json to CSV

I have a json file in the following format:
{
"HDRDTL":["SRNO","STK_IDN","CERTIMG"],
"PKTDTL":[
{"SRNO":"2814","STK_IDN":"1001101259","CERTIMG":"6262941723"},
{"SRNO":"2815","STK_IDN":"1001101269","CERTIMG":"6262941726"},
{"SRNO":"2816","STK_IDN":"1001101279","CERTIMG":"6262941729"}
],
"IMGTTL":
["CERTIMG","ARRIMG"],
"IMGDTL":{"CERTIMG":"CRd6z2uq3gvx7kk","ARRIMG":"ASd6z2uq3gvx7kk"}
}
The "PKTDTL" array is the part I need to convert to csv.
How can I achieve this in C# ?
Note, as an example, I only have 3 items in the "PKTDTL", the real json file has thousands of lines of ({"SRNO" ...}) and is huge.
I have written following code which does not work. see error message commented in the code.
Question
Anyone knows why it failed ?
Do you have a better/alternative method
public static void ConvertJsonToCSV(string InFile)
{
string OutFile=InFile.Replace("json","csv");
StreamReader sInFile = new StreamReader(InFile);
StreamWriter sOutFile = new StreamWriter(OutFile);
using (sInFile)
{
string sJson = sInFile.ReadToEnd();
//*************************************************************
// the below line failed, error message: unexpected json token
// when reading datatable, expected startArray, got StartObject.
//**************************************************************
DataTable dt = JsonConvert.DeserializeObject<DataTable>(sJson);
//DataTable dt = JsonConvert.DeserializeAnonymousType(sJson, new { Makes = default(DataTable) }).Makes;
var sb = new StringBuilder();
string[] columnNames = (from dc in dt.Columns.Cast<DataColumn>()
select dc.ColumnName).ToArray();
sb.AppendLine(string.Join(",", columnNames));
foreach (DataRow dr in dt.Rows)
{
foreach (object field in dr.ItemArray)
{
sb.Append(field.ToString().Replace(",", "") + ",");
}
sb.Replace(",", "\n", sb.Length - 1, 1);
}
sOutFile.Write(sb.ToString());
sOutFile.Close();
sInFile.Close();
}
}

With Cinchoo ETL - an open source library, you can do do the conversion easily with few lines of code
string json = #"{
""HDRDTL"":[""SRNO"",""STK_IDN"",""CERTIMG""],
""PKTDTL"":[
{""SRNO"":""2814"",""STK_IDN"":""1001101259"",""CERTIMG"":""6262941723""},
{""SRNO"":""2815"",""STK_IDN"":""1001101269"",""CERTIMG"":""6262941726""},
{""SRNO"":""2816"",""STK_IDN"":""1001101279"",""CERTIMG"":""6262941729""}
],
""IMGTTL"":
[""CERTIMG"",""ARRIMG""],
""IMGDTL"":{""CERTIMG"":""CRd6z2uq3gvx7kk"",""ARRIMG"":""ASd6z2uq3gvx7kk""}
}";
StringBuilder sb = new StringBuilder();
using (var p = ChoJSONReader.LoadText(json).WithJSONPath("$..PKTDTL")
)
{
using (var w = new ChoCSVWriter(sb)
.WithFirstLineHeader()
)
w.Write(p);
}
Console.WriteLine(sb.ToString());
Output:
SRNO,STK_IDN,CERTIMG
2814,1001101259,6262941723
2815,1001101269,6262941726
2816,1001101279,6262941729
Checkout CodeProject article for some additional help.
Disclaimer: I'm the author of this library.

I found myself in similar situation. this is what i did. first I created a JObject (using Newtonsoft.Json.Linq) let us consider that your json is in a string variable called "strJSON".
JObject jsonObj= JObject.parse(strJSON);
JObject PKTDL=jsonObj["PKTDTL"] as JObject;
IList<string> keys = PKTDL.Properties().Select(p => p.Name).ToList(); // this gives column names
StringBuilder sb=new StringBuilder();
string headers="";
foreach(string key in keys)
{
headers+=","+key;
}
sb.AppendLine(headers.TrimStart(','));
foreach(JObject j in jsonObj["PKTDTL"]) //if jobject doesnt work try "JToken j"
{
string values="";
foreach(string key in keys)
{
values+=","+jsonObj["PKTDTL"][key];
}
sb.AppendLine(values.TrimStart(','));
}
File.WriteAllText(filePath, sb.ToString());
then save sb as a csv

Similar question to duplicate (see the JsonToCsv(string, string) method there). If you are already using NewtonSoft Json.NET and CsvHelper you can do similar conversion with the help of ExpandoObject.
var jsonText = #"{
""HDRDTL"":[""SRNO"",""STK_IDN"",""CERTIMG""],
""PKTDTL"":[
{""SRNO"":""2814"",""STK_IDN"":""1001101259"",""CERTIMG"":""6262941723""},
{""SRNO"":""2815"",""STK_IDN"":""1001101269"",""CERTIMG"":""6262941726""},
{""SRNO"":""2816"",""STK_IDN"":""1001101279"",""CERTIMG"":""6262941729""}
],
""IMGTTL"":
[""CERTIMG"",""ARRIMG""],
""IMGDTL"":{""CERTIMG"":""CRd6z2uq3gvx7kk"",""ARRIMG"":""ASd6z2uq3gvx7kk""}
}";
var jtoken = JObject.Parse(jsonText).SelectToken("$..PKTDTL");
var expandos = jtoken.ToObject<ExpandoObject[]>();
string csvText;
using (var writer = new StringWriter())
{
using (var csv = new CsvWriter(writer))
{
csv.WriteRecords(expandos as IEnumerable<dynamic>);
}
csvText = writer.ToString();
}
Console.WriteLine(csvText);

Convert Comma separated string to XML using C#

I want to convert the string to XML.
I have a string like below. It contains the Programming language names.
string lang = "java,php,c#,asp.net,spring,hibernate";
I want to convert this string to XML formal like below:
<Languages>
<lang Name="java"/>
<lang Name="php"/>
<lang Name="c#"/>
<lang Name="asp.net"/>
<lang Name="spring"/>
<lang Name="hibernate"/>
</Languages>
I want to store this XML data in a variable to store later in a database.

It can also be done using Linq-to-XML:
using System.Xml.Linq; // required namespace
XDocument xmlDoc = new XDocument();
XElement xElm = new XElement("Languages",
from l in lang.Split(',')
select new XElement("lang", new XAttribute("Name", l)
)
);
xmlDoc.Add(xElm);

string lang = "java,php,c#,asp.net,spring,hibernate";
string[] langs = lang.Split(',');
XmlDocument document = new XmlDocument();
XmlElement root = document.CreateElement("Languages");
document.AppendChild(root);
for (int i = 0; i < langs.Length; i++)
{
XmlElement langElement = document.CreateElement("lang");
XmlAttribute nameAttr = document.CreateAttribute("Name");
nameAttr.Value = langs[i];
langElement.Attributes.Append(nameAttr);
root.AppendChild(langElement);
}
document.WriteTo(new XmlTextWriter(Console.Out) {
Formatting = Formatting.Indented
});

A short version of what you have done, using Linq and the string manipulation functions
var vales = lang.Split(','); //Splits the CSV
var xmlBody = vales.Select(v => string.Format("<lang Name=\"{0}\"/>",v));
var xml = string.Join(string.Empty, xmlBody); //Potentially add a new line as a seperator
xml = string.Format("<Languages>{0}</Languages>", xml);
The other option is to convert your csv into a model that implements ISerialize and then use the xml serializer. That is more code and not necessarily bad. If you would like to see an example, feel free to ask and I will post an example.

This is working,
class Program
{
static void Main(string[] args)
{
string lang = "java,php,c#,asp.net,spring,hibernate";
StringBuilder sb = new StringBuilder();
sb.AppendFormat("<Languages>");
foreach (string s in lang.Split(','))
{
sb.AppendFormat("<lang Name=\"{0}\"/>", s);
}
sb.AppendFormat("</Languages>");
Console.WriteLine(sb.ToString());
Console.ReadLine();
}
}

Categories

Resources