Do not say there is built in Unity Json or to download the Lindq for C# I am using SimpleJSON
I am trying to read the json file and add the elements to a list so I can call them easier. I cannot figure out how to add all of "Heads" values to list.
So I could possibly call
listname[index].key
listname[key].value
to return somthing where I can get example.("Small_Cube_Head":{"Colors":{"Default"})
I basically want to be able to call the "Small_Cube_Head" and all color values
JSON FILE
{
"Heads":{
"Large_Cube_Head":["Default","e97451"],
"Medium_Cube_Head":["Default","ffffff","76d7ea","e97451","5fa777","f4c430","d0ff14","0047ab","e32636","fc419a","720b98","ff8b00","000000","848482"],
"Small_Cube_Head":["Default"]
}
}
}
code
/**
* json: Returns the whole file as a String
**/
private void LoadJson()
{
using (StreamReader r = new StreamReader("Assets/JSON/PlayerPartsList.json"))
{
json = r.ReadToEnd();
//Debug.Log(json);
JSONNode node = JSON.Parse(json);
Debug.Log(node["Heads"].Count); //returns 3
for (int i = 0; i < node["Heads"].Count; i++) {
//headParts.Add(node["Heads"].);
//Debug.Log(node["Heads"][i].Value.ToString());
//Debug.Log(node["Heads"]["Small_Cube_Head"].Value);
}
}
}
Use Keys like an enumerator to get each head name, and then you can loop through the count of that headName as the index
KeyEnumerator headNameEnum = node["Heads"].Keys;
while (headNameEnum.MoveNext())
{
String headName = headNameEnum.Current().Value;
Debug.Log("headName: " + headName);
for (int i=0; i < node["Heads"][headName].Count; i++) {
String valueName = node["Heads"][headName][i].Value;
Debug.Log("valueName: " + valueName);
}
}
Related
I'm trying to get covid-19 results (only information about Iran) from an Api and show it on a textbox.
and the full result (all countries) that i get from the Api is a json format.
so to get only Iran section i made a Function that loops through lines of the string one by one and check if in that line there is a "{" and if yes get index of that and continue checking if in another line there is a "}" and get index of that too then check if between these, there is "Iran" then add this text (from "{" to "}") in a string:
private string getBetween(string strSourceText, string strStartingPosition, string strEndingPosition)
{
int Starting_CurlyBracket_Index = 0;
int Ending_CurlyBracket_Index = 0;
string FinalText = null;
bool isTurnTo_firstIf = true;
foreach (var line in strSourceText.Split('\r', '\n'))
{
if (isTurnTo_firstIf == true)
{
if (line.Contains(strStartingPosition))
{
Starting_CurlyBracket_Index = line.IndexOf(strStartingPosition); //i think problem is here
isTurnTo_firstIf = false;
}
}
else if (isTurnTo_firstIf == false)
{
if (line.Contains(strEndingPosition))
{
Ending_CurlyBracket_Index = line.IndexOf(strEndingPosition); //i think problem is here
if (strSourceText.Substring(Starting_CurlyBracket_Index, Ending_CurlyBracket_Index - Starting_CurlyBracket_Index).Contains("Iran")) //error here
{
FinalText = strSourceText.Substring(Starting_CurlyBracket_Index, Ending_CurlyBracket_Index - Starting_CurlyBracket_Index);
break;
}
else
{
isTurnTo_firstIf = true;
}
}
}
}
return FinalText;
}
and i call the function like this:
string OnlyIranSection = getBetween(Sorted_Covid19_Result, "{", "}"); //Sorted_Covid19_Result is the full result in json format that converted to string
textBox1.Text = OnlyIranSection;
but i get this Error:
and i know.. its because it gets indexes in the current line but what i need is getting that index in the strSourceText so i can show only this section of the whole result:
USING JSON
As per the comments I read it was really needed to use JSON utility to achieve your needs easier.
You can start with this basic example:
static void Main(string[] args)
{
string jsonString = #"{
""results"": [
{""continent"":""Asia"",""country"":""Indonesia""},
{""continent"":""Asia"",""country"":""Iran""},
{""continent"":""Asia"",""country"":""Philippines""}
]
}";
var result = JsonConvert.DeserializeObject<JsonResult>(jsonString);
var iranInfo = result.InfoList.Where(i => i.Country.ToString() == "Iran").FirstOrDefault();
}
public class JsonResult
{
[JsonProperty("results")]
public List<Info> InfoList { get; set; }
}
public class Info
{
public object Continent { get; set; }
public object Country { get; set; }
}
UPDATE: USING INDEX
As long as the structure of the JSON is consistent always then this kind of sample solution can give you hint.
Console.WriteLine("Original JSON:");
Console.WriteLine(jsonString);
Console.WriteLine();
Console.WriteLine("Step1: Make the json as single line,");
jsonString = jsonString.Replace(" ", "").Replace(Environment.NewLine, " ");
Console.WriteLine(jsonString);
Console.WriteLine();
Console.WriteLine("Step2: Get index of country Iran. And use that index to get the below output using substring.");
var iranIndex = jsonString.ToLower().IndexOf(#"""country"":""iran""");
var iranInitialInfo = jsonString.Substring(iranIndex);
Console.WriteLine(iranInitialInfo);
Console.WriteLine();
Console.WriteLine("Step3: Get inedx of continent. And use that index to get below output using substring.");
var continentIndex = iranInitialInfo.IndexOf(#"""continent"":");
iranInitialInfo = iranInitialInfo.Substring(0, continentIndex-3);
Console.WriteLine(iranInitialInfo);
Console.WriteLine();
Console.WriteLine("Step4: Get the first part of the info by using. And combine it with the initialInfo to bring the output below.");
var beginningIranInfo = jsonString.Substring(0, iranIndex);
var lastOpenCurlyBraceIndex = beginningIranInfo.LastIndexOf("{");
beginningIranInfo = beginningIranInfo.Substring(lastOpenCurlyBraceIndex);
var iranInfo = beginningIranInfo + iranInitialInfo;
Console.WriteLine(iranInfo);
OUTPUT USING INDEX:
I've been working on a webscraper as a Windows Forms application in C#. The user enter a search term and the term and the program will then split the search string for each individual words and look up the amount of search results through Yahoo and Google.
My issue lies with the orientation of the huge HTML document. I've tried multiple approaches such as
iterating recursively and comparing ids aswell as with lamba and the Where statements. Both results in null. I also manually looked into the html document to make sure the id of the div I want exist in the document.
The id I'm looking for is "resultStats" but it is suuuuuper nested. My code looks like this:
using HtmlAgilityPack;
using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.Threading.Tasks;
namespace WebScraper2._0
{
public class Webscraper
{
private string Google = "http://google.com/#q=";
private string Yahoo = "http://search.yahoo.com/search?p=";
private HtmlWeb web = new HtmlWeb();
private HtmlDocument GoogleDoc = new HtmlDocument();
private HtmlDocument YahooDoc = new HtmlDocument();
public Webscraper()
{
Console.WriteLine("Init");
}
public int WebScrape(string searchterms)
{
//Console.WriteLine(searchterms);
string[] ssize = searchterms.Split(new char[0]);
int YahooMatches = 0;
int GoogleMatches = 0;
foreach (var term in ssize)
{
//Console.WriteLine(term);
var y = web.Load(Yahoo + term);
var g = web.Load(Google + term + "&cad=h");
YahooMatches += YahooFilter(y);
GoogleMatches += GoogleFilter(g);
}
Console.WriteLine("Yahoo found " + YahooMatches.ToString() + " matches");
Console.WriteLine("Google found " + GoogleMatches.ToString() + " matches");
return YahooMatches + GoogleMatches;
}
//Parse to get correct info
public int YahooFilter(HtmlDocument doc)
{
//Look for node with correct ID
IEnumerable<HtmlNode> nodes = doc.DocumentNode.Descendants().Where(n => n.HasClass("mw-jump-link"));
foreach (var item in nodes)
{
// displaying final output
Console.WriteLine(item.InnerText);
}
//TODO: Return search resultamount.
return 0;
}
int testCounter = 0;
string toReturn = "";
bool foundMatch = false;
//Parse to get correct info
public int GoogleFilter(HtmlDocument doc)
{
if (doc == null)
{
Console.WriteLine("Null");
}
foreach (var node in doc.DocumentNode.ChildNodes)
{
toReturn += Looper(node, testCounter, toReturn, foundMatch);
}
Console.WriteLine(toReturn);
/*
var stuff = doc.DocumentNode.Descendants("div")
.Where(node => node.GetAttributeValue("id", "")
.Equals("extabar")).ToList();
IEnumerable<HtmlNode> nodes = doc.DocumentNode.Descendants().Where(n => n.HasClass("appbar"));
*/
return 0;
}
public string Looper(HtmlNode node, int counter, string returnstring, bool foundMatch)
{
Console.WriteLine("Loop started" + counter.ToString());
counter++;
Console.WriteLine(node.Id);
if (node.Id == "resultStats")
{
returnstring += node.InnerText;
}
foreach (HtmlNode n in node.Descendants())
{
Looper(n, counter, returnstring, foundMatch);
}
return returnstring;
}
}
}
I made an google HTML Scraper a few weeks ago, a few things to consider
First: Google don't like when you try to Scrape their Search HTML, while i was running a list of companies trying to get their addresses and phone number, Google block my IP from accessing their website for a little bit (Which cause a hilarious panic in the office)
Second: Google will change the HTML (Id names and etc) of the page so using ID's won't work, on my case i used the combination of HTML Tags and specific information to parse the response and extract the information that i wanted.
Third: It's better to just use their API to grab the information you need, just make sure you respect their free tier query limit and you should be golden.
Here is the Code i used.
public static string getBetween(string strSource, string strStart, string strEnd)
{
int Start, End;
if (strSource.Contains(strStart) && strSource.Contains(strEnd))
{
Start = strSource.IndexOf(strStart, 0) + strStart.Length;
End = strSource.IndexOf(strEnd, Start);
return strSource.Substring(Start, End - Start);
}
else
{
return "";
}
}
public void SearchResult()
{
//Run a Google Search
string uriString = "http://www.google.com/search";
string keywordString = "Search String";
WebClient webClient = new WebClient();
NameValueCollection nameValueCollection = new NameValueCollection();
nameValueCollection.Add("q", keywordString);
webClient.QueryString.Add(nameValueCollection);
string result = webClient.DownloadString(uriString);
string search = getBetween(result, "Address", "Hours");
rtbHtml.Text = getBetween(search, "\">", "<");
}
On my case i used the String Address and Hours to limit what information i wanted to extract.
Edit: Fixed the Logic and added the Code i used.
Edit2: forgot to add the GetBetween Class. (sorry it's my first Answer)
I have data in tab-separated values (TSV) text files that I want to read and (eventually) store in database tables. With the TSV files, each line contains one record, but in one file the record can have 2 fields, in another file 4 fields, etc. I wrote working code to handle the 2-field records, but I thought this might be a good case for a generic method (or two) rather than writing new methods for each kind of record. However, I have not been able to code this because of 2 problems: I can't create a new object for holding the record data, and I don't know how to use reflection to generically fill the instance variables of my objects.
I looked at several other similar posts, including Datatable to object by using reflection and linq
Below is the code that works (this is in Windows, if that matters) and also the code that doesn't work.
public class TSVFile
{
public class TSVRec
{
public string item1;
public string item2;
}
private string fileName = "";
public TSVFile(string _fileName)
{
fileName = _fileName;
}
public TSVRec GetTSVRec(string Line)
{
TSVRec rec = new TSVRec();
try
{
string[] fields = Line.Split(new char[1] { '\t' });
rec.item1 = fields[0];
rec.item2 = fields[1];
}
catch (Exception ex)
{
System.Windows.Forms.MessageBox.Show("Bad import data on line: " +
Line + "\n" + ex.Message, "Error",
System.Windows.Forms.MessageBoxButtons.OK,
System.Windows.Forms.MessageBoxIcon.Error);
}
return rec;
}
public List<TSVRec> ImportTSVRec()
{
List<TSVRec> loadedData = new List<TSVRec>();
using (StreamReader sr = File.OpenText(fileName))
{
string Line = null;
while ((Line = sr.ReadLine()) != null)
{
loadedData.Add(GetTSVRec(Line));
}
}
return loadedData;
}
// *** Attempted generic methods ***
public T GetRec<T>(string Line)
{
T rec = new T(); // compile error!
Type t = typeof(T);
FieldInfo[] instanceVars = t.GetFields();
string[] fields = Line.Split(new char[1] { '\t' });
for (int i = 0; i < instanceVars.Length - 1; i++)
{
rec. ??? = fields[i]; // how do I finish this line???
}
return rec;
}
public List<T> Import<T>(Type t)
{
List<T> loadedData = new List<T>();
using (StreamReader sr = File.OpenText(fileName))
{
string Line = null;
while ((Line = sr.ReadLine()) != null)
{
loadedData.Add(GetRec<T>(Line));
}
}
return loadedData;
}
}
I saw the line
T rec = new T();
in the above-mentioned post, but it doesn't work for me...
I would appreciate any suggestions for how to make this work, if possible. I want to learn more about using reflection with generics, so I don't only want to understand how, but also why.
I wish #EdPlunkett had posted his suggestion as an answer, rather than a comment, so I could mark it as the answer...
To summarize: to do what I want to do, there is no need for "Assigning instance variables obtained through reflection in generic method". In fact, I can have a generic solution without using a generic method:
public class GenRec
{
public List<string> items = new List<string>();
}
public GenRec GetRec(string Line)
{
GenRec rec = new GenRec();
try
{
string[] fields = Line.Split(new char[1] { '\t' });
for (int i = 0; i < fields.Length; i++)
rec.items.Add(fields[i]);
}
catch (Exception ex)
{
System.Windows.Forms.MessageBox.Show("Bad import data on line: " + Line + "\n" + ex.Message, "Error",
System.Windows.Forms.MessageBoxButtons.OK,
System.Windows.Forms.MessageBoxIcon.Error);
}
return rec;
}
public List<GenRec> Import()
{
List<GenRec> loadedData = new List<GenRec>();
using (StreamReader sr = File.OpenText(fileName))
{
string Line = null;
while ((Line = sr.ReadLine()) != null)
loadedData.Add(GetRec(Line));
}
return loadedData;
}
I just tested this, and it works like a charm!
Of course, this isn't helping me to learn how to write generic methods or use reflection, but I'll take it...
I'm working on a utility to read through a JSON file I've been given and to transform it into SQL Server. My weapon of choice is a .NET Core Console App (I'm trying to do all of my new work with .NET Core unless there is a compelling reason not to). I have the whole thing "working" but there is clearly a problem somewhere because the performance is truly horrifying almost to the point of being unusable.
The JSON file is approximately 27MB and contains a main array of 214 elements and each of those contains a couple of fields along with an array of from 150-350 records (that array has several fields and potentially a small <5 record array or two). Total records are approximately 35,000.
In the code below I've changed some names and stripped out a few of the fields to keep it more readable but all of the logic and code that does actual work is unchanged.
Keep in mind, I've done a lot of testing with the placement and number of calls to SaveChanges() think initially that number of trips to the Db was the problem. Although the version below is calling SaveChanges() once for each iteration of the 214-record loop, I've tried moving it outside of the entire looping structure and there is no discernible change in performance. In other words, with zero trips to the Db, this is still SLOW. How slow you ask, how does > 24 hours to run hit you? I'm willing to try anything at this point and am even considering moving the whole process into SQL Server but would much reather work in C# than TSQL.
static void Main(string[] args)
{
string statusMsg = String.Empty;
JArray sets = JArray.Parse(File.ReadAllText(#"C:\Users\Public\Downloads\ImportFile.json"));
try
{
using (var _db = new WidgetDb())
{
for (int s = 0; s < sets.Count; s++)
{
Console.WriteLine($"{s.ToString()}: {sets[s]["name"]}");
// First we create the Set
Set eSet = new Set()
{
SetCode = (string)sets[s]["code"],
SetName = (string)sets[s]["name"],
Type = (string)sets[s]["type"],
Block = (string)sets[s]["block"] ?? ""
};
_db.Entry(eSet).State = Microsoft.EntityFrameworkCore.EntityState.Added;
JArray widgets = sets[s]["widgets"].ToObject<JArray>();
for (int c = 0; c < widgets.Count; c++)
{
Widget eWidget = new Widget()
{
WidgetId = (string)widgets[c]["id"],
Layout = (string)widgets[c]["layout"] ?? "",
WidgetName = (string)widgets[c]["name"],
WidgetNames = "",
ReleaseDate = releaseDate,
SetCode = (string)sets[s]["code"]
};
// WidgetColors
if (widgets[c]["colors"] != null)
{
JArray widgetColors = widgets[c]["colors"].ToObject<JArray>();
for (int cc = 0; cc < widgetColors.Count; cc++)
{
WidgetColor eWidgetColor = new WidgetColor()
{
WidgetId = eWidget.WidgetId,
Color = (string)widgets[c]["colors"][cc]
};
_db.Entry(eWidgetColor).State = Microsoft.EntityFrameworkCore.EntityState.Added;
}
}
// WidgetTypes
if (widgets[c]["types"] != null)
{
JArray widgetTypes = widgets[c]["types"].ToObject<JArray>();
for (int ct = 0; ct < widgetTypes.Count; ct++)
{
WidgetType eWidgetType = new WidgetType()
{
WidgetId = eWidget.WidgetId,
Type = (string)widgets[c]["types"][ct]
};
_db.Entry(eWidgetType).State = Microsoft.EntityFrameworkCore.EntityState.Added;
}
}
// WidgetVariations
if (widgets[c]["variations"] != null)
{
JArray widgetVariations = widgets[c]["variations"].ToObject<JArray>();
for (int cv = 0; cv < widgetVariations.Count; cv++)
{
WidgetVariation eWidgetVariation = new WidgetVariation()
{
WidgetId = eWidget.WidgetId,
Variation = (string)widgets[c]["variations"][cv]
};
_db.Entry(eWidgetVariation).State = Microsoft.EntityFrameworkCore.EntityState.Added;
}
}
}
_db.SaveChanges();
}
}
statusMsg = "Import Complete";
}
catch (Exception ex)
{
statusMsg = ex.Message + " (" + ex.InnerException + ")";
}
Console.WriteLine(statusMsg);
Console.ReadKey();
}
I had an issue with that kind of code, lots of loops and tons of changing state.
Any change / manipulation you make in _db context, will generate a "trace" of it. And it making your context slower each time. Read more here.
The fix for me was to create new EF context(_db) at some key points. It saved me a few hours per run!
You could try to create a new instance of _db each iteration in this loop
contains a main array of 214 elements
If it make no change, try to add some stopwatch to get a best idea of what/where is taking so long.
If you're making thousands of updates then EF is not really the way to go. Something like SQLBulkCopy will do the trick.
You could try the bulkwriter library.
IEnumerable<string> ReadFile(string path)
{
using (var stream = File.OpenRead(path))
using (var reader = new StreamReader(stream))
{
while (reader.Peek() >= 0)
{
yield return reader.ReadLine();
}
}
}
var items =
from line in ReadFile(#"C:\products.csv")
let values = line.Split(',')
select new Product {Sku = values[0], Name = values[1]};
then
using (var bulkWriter = new BulkWriter<Product>(connectionString)) {
bulkWriter.WriteToDatabase(items);
}
I'm Getting this error when accessing data from a json file.
I'm trying to follow the following tutorial: http://wiki.unity3d.com/index.php/SimpleJSON
and created a test.json file, that I want to extract data from containing:
{
"version": "1.0",
"data": {
"sampleArray": [
"string value",
5,
{
"name": "sub object"
}
]
}
}
using the following code in Unity:
void LoadFiles()
{
FileInfo f = m_info[0]; //Array of Files in Folder
// I had a foreach loop here, but wanted to specify the file for testing before I tried to parse through one of my own
print("I Found : " + f);
var N = JSONNode.LoadFromFile(f.FullName);
var versionString = N["version"].Value; // versionString will be a string containing "1.0"
var versionNumber = N["version"].AsFloat; // versionNumber will be a float containing 1.0
var name = N["data"]["sampleArray"][2]["name"];// name will be a string containing "sub object"
print("vs=" + versionString + " vn=" + versionNumber + " name=" + name);
}
and all i get is Unknown tags, from what I gather from the source :
public static JSONNode Deserialize(System.IO.BinaryReader aReader)
{
JSONBinaryTag type = (JSONBinaryTag)aReader.ReadByte();
switch(type)
{
case JSONBinaryTag.Array:
{
int count = aReader.ReadInt32();
JSONArray tmp = new JSONArray();
for(int i = 0; i < count; i++)
tmp.Add(Deserialize(aReader));
return tmp;
}
case JSONBinaryTag.Class:
{
int count = aReader.ReadInt32();
JSONClass tmp = new JSONClass();
for(int i = 0; i < count; i++)
{
string key = aReader.ReadString();
var val = Deserialize(aReader);
tmp.Add(key, val);
}
return tmp;
}
case JSONBinaryTag.Value:
{
return new JSONData(aReader.ReadString());
}
case JSONBinaryTag.IntValue:
{
return new JSONData(aReader.ReadInt32());
}
case JSONBinaryTag.DoubleValue:
{
return new JSONData(aReader.ReadDouble());
}
case JSONBinaryTag.BoolValue:
{
return new JSONData(aReader.ReadBoolean());
}
case JSONBinaryTag.FloatValue:
{
return new JSONData(aReader.ReadSingle());
}
default:
{
throw new Exception("Error deserializing JSON. Unknown tag: " + type);
}
}
}
I'm falling all the way through the Switch, but with .Value or .AsFloat I should hit those case statements. Any Idea what's going on, is this code to old for Unity 5.0 ?
It seems like I was under false pretenses, that the JSON file was like XML, text, well it seems it is not, or at least assumed to be binary by SimpleJson when reading files. I tried writing the "Text" to a file, and then reading it and it worked fine. So this is an error where I assumed the data in the examples to be text.
JSONNode.LoadFromFile function instead to JSONNode.Parse(string) convert stream to string
System.IO.FileStream fs = System.IO.File.OpenRead(f.FullName);
long length = fs.Length;
byte[] stream = new byte[length];
fs.Read(stream, 0, (int)length);
string json = System.Text.Encoding.UTF8.GetString(stream);
var N = JSONNode.Parse(json);