I have a php file which reads a .txt file and sends it via a php server to a c# unity script. Below is a snippet of the text file showing the first 3 lines:
{ "lemma" : "aljotta", "gloss" : "Fisħ soup" }
{ "lemma" : "arguzin", "gloss" : "Slave driver" }
{ "lemma" : "armunjaka", "gloss" : "Armunjaka" }
This is the php script:
<?php
$file = fopen("lemmas.txt", "r");
echo fread($file, filesize("lemmas.txt"));
fclose($file);
?>
In a c# script, the text is returned and each line is separated into an array (string[] lines) slot as seen below:
IEnumerator GetTextFromFile()
{
bool succcessful = true;
WWWForm form = new WWWForm();
WWW www = new WWW("http://localhost:9000/tounity.php", form);
yield return www;
if(www.error != null)
{
succcessful = false;
}
else
{
succcessful = true;
}
if (succcessful)
{
populateWordList(www.text);
}
}
void populateWordList(string text)
{
string[] textArray = text.Split('\n');
wordsList = gameDatabase.GetWords(textArray);
}
The array is then passed to a method which deserializes each line into an object of class GameDatabase as seen in the image below:
public string lemma { get; set; }
public string gloss { get; set; }
public GameDatabase(string lemma, string gloss)
{
this.lemma = lemma;
this.gloss = gloss;
}
public ArrayList GetWords(string[] lines)
{
foreach (string line in lines)
{
GameDatabase gd = JsonConvert.DeserializeObject<GameDatabase>(line);
lemmasAndGlossesList.Add(new GameDatabase(gd.lemma, gd.gloss));
}
foreach(GameDatabase line in lemmasAndGlossesList)
{
Debug.Log(line.lemma + "------" + line.gloss);
}
return lemmasAndGlossesList;
}
The error occurs in GameDatabase gd = JsonConvert.DeserializeObject<GameDatabase>(line); and returns
JsonReaderException: Unexpected character encountered while parsing value: . Path '', line 0, position 0.
I have searched extensively, however, haven't found anything that works. Any help would be greatly appreciated. It is worth noting that this problem doesn't happen when loading the text file directly into unity without using php.
EDIT
When using the vs debugger this is the value in the line to be deserialized:
The JSON visualiser of Visual Studio 2019 however reports this:
Thanks to Jonathon K's comment and your reply we can see the data returned by the PHP script starts with a BOM: the first three bytes. This nice article explains how to handle such data properly. In short: use a StreamReader to read the data.
This little program demonstrates how it could work with your data:
using System;
using Newtonsoft.Json;
using System.IO;
public class Program
{
public static void Main()
{
var bytes = new byte[] {
0xEF,0xBB,0xBF,0x7B,0x20,0x22,0x6C,0x65,0x6D,0x6D,0x61,0x22,
0x20,0x3A,0x20,0x22,0x61,0x72,0x67,0x75,0x7A,0x69,0x6E,0x22,
0x2C,0x20,0x22,0x67,0x6C,0x6F,0x73,0x73,0x22,0x20,0x3A,0x20,
0x22,0x53,0x6C,0x61,0x76,0x65,0x20,0x64,0x72,0x69,0x76,0x65,
0x72,0x22,0x20,0x7D};
string json;
using(var ms = new MemoryStream(bytes))
using(var sr = new StreamReader(ms))
{
json = sr.ReadToEnd();
Console.WriteLine(json);
}
// I'm using dynamic here. In your case you can use GameDatabase
dynamic obj = JsonConvert.DeserializeObject(json);
Console.WriteLine(obj.lemma);
}
}
Output:
{ "lemma" : "arguzin", "gloss" : "Slave driver" }
arguzin
I dont know the c# sintax but this will work.
change your JSON file.
[
{ "lemma" : "aljotta", "gloss" : "Fisħ soup" },
{ "lemma" : "arguzin", "gloss" : "Slave driver" },
{ "lemma" : "armunjaka", "gloss" : "Armunjaka" }
]
apply JsonConvert.DeserializeObject to www.text
for (GameDatabase line in JsonConvert.DeserializeObject<GameDatabase[]>(www.text)){
Debug.Log(line.lemma + "------" + line.gloss);
}
Maybe my c# syntax is wrong but i would that u understand my idea
I think it's possible that you're going about deserialization wrong by using JsonConvert.
Instead, read up on this documentation and try to use the Unity functions:
https://docs.unity3d.com/Manual/JSONSerialization.html
For starters, you're defining lemma and gloss incorrectly if you're looking to use them for deserializing JSON in Unity. See this answer for more info:
Serialize and Deserialize Json and Json Array in Unity
Related
I have a JSON file that is given as:
{
"Person": "true",
"Age": "true",
"Location": "false",
"Phone": "true"
}
I am able to read it in Unity by using the code below. I am using SimpleJSON library.
using System.Collections;
using System.Collections.Generic;
using System.IO;
using SimpleJSON;
using UnityEngine;
using UnityEngine.UI;
public class ReadWriteScene : MonoBehaviour {
public string jsonFile;
JSONNode itemsData;
string path;
// Start is called before the first frame update
void Start () {
path = Path.Combine (Application.streamingAssetsPath, "Settings.json");
if (File.Exists (path)) {
jsonFile = File.ReadAllText (path);
DeserializePages ();
}
}
void Update () {
}
public void DeserializePages () {
itemsData = JSON.Parse (jsonFile);
var parseJSON = JSON.Parse (jsonFile);
Debug.Log(parseJSON["Phone"].Value);
}
}
But I do not know how to write or make changes to the JSON via code? For example, how do I change the attribute "Age" to "false"?
It's very easy.
itemsData["Age"] = "false";
Check https://docs.unity3d.com/Packages/com.unity.systemgraph#2.0/api/SimpleJSON.JSONNode.html.
itemsData["Age"] = "false";
File.WriteAllTextAsync(path, itemsData.ToString());
It was quite easy. I used JSONObject and did the following:
public void SaveData(){
JSONObject json = new JSONObject();
json.Add("Person", "true");
json.Add("Age", "false");
json.Add("Location", "true");
json.Add("Phone", "true");
File.WriteAllText(path, json.ToString());
}
Know that, I did not create a new file, it replaces the already created file but make sure you add all the given attributes or else it will get erased.
I'm trying to create a list of notifications in Unity, which are delivered by a JSON API. I'm using the SimpleJson plugin, which I already used in another scene perfectly,
Here is the JSON:
[
{
"_id": {
"$oid": "5d30eccda6e0712cfd0832c3"
},
"titulo": "Primera Notificacion",
"texto": "Prueba de notificacion"
},
{
"_id": {
"$oid": "5d336c36a6e07114ac728cc2"
},
"titulo": "Segunda notificacion",
"texto": "Prueba de notificacion 2"
}
]
Here is the error:
Exception: JSON Parse: Quotation marks seems to be messed up.
SimpleJSON.JSONNode.Parse (System.String aJSON) (at Assets/QRcode/Scripts/SimpleJSON.cs:735)
SimpleJSON.JSON.Parse (System.String aJSON) (at Assets/QRcode/Scripts/SimpleJSON.cs:1421)
DataLoaderNot+d__5.MoveNext () (at Assets/QRcode/Scripts/DataLoaderNot.cs:29)
UnityEngine.SetupCoroutine.InvokeMoveNext (System.Collections.IEnumerator enumerator, System.IntPtr returnValueAddress) (at C:/buildslave/unity/build/Runtime/Export/Scripting/Coroutines.cs:17)
I'm using this code to call JSON:
using UnityEngine;
using System.Collections;
using UnityEngine.UI;
using SimpleJSON;
public class DataLoader : MonoBehaviour
{
string JsonDataString;
string JsonDataString2;
static public string OriginalJsonSite = "http://(web service name)/API/testnot.php";
public Text TituloNot;
public Text TextoNot;
IEnumerator Start ()
{
WWW readingsite = new WWW (OriginalJsonSite);
Debug.Log(OriginalJsonSite);
yield return readingsite;
if (string.IsNullOrEmpty (readingsite.error)) {
JsonDataString = readingsite.text;
JsonDataString2 = JsonDataString.Substring(3, JsonDataString.Length - 4);
}
JSONNode jsonNode = SimpleJSON.JSON.Parse(JsonDataString2);
JSONArray array = jsonNode.AsArray;
Debug.Log(JsonDataString2);
TituloNot.text = array[0]["titulo"].ToString();
Debug.Log(jsonNode["titulo"]);
TextoNot.text = array[0]["texto"].ToString();
Debug.Log(jsonNode["texto"]);
}
}
The lines
Debug.Log(jsonNode["titulo"]);
Debug.Log(jsonNode["texto"]);
won't work for sure, you should remove them.
Also, if the returned JSON is really exactly like the one you posted, you should also remove the line
JsonDataString2 = JsonDataString.Substring(3, JsonDataString.Length - 4);
The parse will take care of finding matching brackets [] and braces {} so this line will break it.
Edit: I didn't test it but this code should work in my opinion:
using UnityEngine;
using System.Collections;
using UnityEngine.UI;
using SimpleJSON;
public class DataLoader : MonoBehaviour
{
string jsonDataString;
static public string originalJsonSite = "http://(web service name)/API/testnot.php";
public Text tituloNot;
public Text textoNot;
IEnumerator Start()
{
WWW readingsite = new WWW (originalJsonSite);
yield return readingsite;
if (string.IsNullOrEmpty(readingsite.error))
{
jsonDataString = readingsite.text;
}
JSONNode jsonNode = SimpleJSON.JSON.Parse(jsonDataString);
JSONArray array = jsonNode.AsArray;
tituloNot.text = array[0]["titulo"].ToString();
textoNot.text = array[0]["texto"].ToString();
}
}
I've been having a problem with my ml.net console app. This is my first time using ml.net in Visual Studio so I was following this tutorial from microsoft.com, which is a sentiment analysis using binary classification.
I'm trying to process some test data in the form of tsv files to get a positive or negative sentiment analysis, but in debugging I'm receiving warnings there being 1 format error and 2 bad values.
I decided to ask all you great devs here on Stack to see if anyone can help me find a solution.
Here's an image of the debugging below:
Here's the link to my test data:
wiki-data
wiki-test-data
Finally, here's my code for those who what to reproduce the problem:
There's 2 c# files: SentimentData.cs & Program.cs.
1 - SentimentData.cs:
using System;
using System.Collections.Generic;
using System.Text;
using Microsoft.ML.Runtime.Api;
namespace MachineLearningTut
{
public class SentimentData
{
[Column(ordinal: "0")]
public string SentimentText;
[Column(ordinal: "1", name: "Label")]
public float Sentiment;
}
public class SentimentPrediction
{
[ColumnName("PredictedLabel")]
public bool Sentiment;
}
}
2 - Program.cs:
using System;
using Microsoft.ML.Models;
using Microsoft.ML.Runtime;
using Microsoft.ML.Runtime.Api;
using Microsoft.ML.Trainers;
using Microsoft.ML.Transforms;
using System.Collections.Generic;
using System.Linq;
using Microsoft.ML;
using Microsoft.ML.Data;
using System.Threading.Tasks;
namespace MachineLearningTut
{
class Program
{
const string _dataPath = #".\Data\wikipedia-detox-250-line-data.tsv";
const string _testDataPath = #".\Data\wikipedia-detox-250-line-test.tsv";
const string _modelpath = #".\Data\Model.zip";
static async Task Main(string[] args)
{
var model = await TrainAsync();
Evaluate(model);
Predict(model);
}
public static async Task<PredictionModel<SentimentData, SentimentPrediction>> TrainAsync()
{
var pipeline = new LearningPipeline();
pipeline.Add(new TextLoader (_dataPath).CreateFrom<SentimentData>());
pipeline.Add(new TextFeaturizer("Features", "SentimentText"));
pipeline.Add(new FastForestBinaryClassifier() { NumLeaves = 5, NumTrees = 5, MinDocumentsInLeafs = 2 });
PredictionModel<SentimentData, SentimentPrediction> model = pipeline.Train<SentimentData, SentimentPrediction>();
await model.WriteAsync(path: _modelpath);
return model;
}
public static void Evaluate(PredictionModel<SentimentData, SentimentPrediction> model)
{
var testData = new TextLoader(_testDataPath).CreateFrom<SentimentData>();
var evaluator = new BinaryClassificationEvaluator();
BinaryClassificationMetrics metrics = evaluator.Evaluate(model, testData);
Console.WriteLine();
Console.WriteLine("PredictionModel quality metrics evaluation");
Console.WriteLine("-------------------------------------");
Console.WriteLine($"Accuracy: {metrics.Accuracy:P2}");
Console.WriteLine($"Auc: {metrics.Auc:P2}");
Console.WriteLine($"F1Score: {metrics.F1Score:P2}");
}
public static void Predict(PredictionModel<SentimentData, SentimentPrediction> model)
{
IEnumerable<SentimentData> sentiments = new[]
{
new SentimentData
{
SentimentText = "Please refrain from adding nonsense to Wikipedia."
},
new SentimentData
{
SentimentText = "He is the best, and the article should say that."
}
};
IEnumerable<SentimentPrediction> predictions = model.Predict(sentiments);
Console.WriteLine();
Console.WriteLine("Sentiment Predictions");
Console.WriteLine("---------------------");
var sentimentsAndPredictions = sentiments.Zip(predictions, (sentiment, prediction) => (sentiment, prediction));
foreach (var item in sentimentsAndPredictions)
{
Console.WriteLine($"Sentiment: {item.sentiment.SentimentText} | Prediction: {(item.prediction.Sentiment ? "Positive" : "Negative")}");
}
Console.WriteLine();
}
}
}
If anyone would like to see the code or more details on the solution, ask me on the chat and I'll send it. Thanks in advance!!! [Throws a Thumbs Up]
I think I got a fix for you. A couple of things to update:
First, I think you got your SentimentData properties switched to what the data has. Try changing it to
[Column(ordinal: "0", name: "Label")]
public float Sentiment;
[Column(ordinal: "1")]
public string SentimentText;
Second, use the useHeader parameter in the TextLoader.CreateFrom method. Don't forget to add that to the other one for the validation data, as well.
pipeline.Add(new TextLoader(_dataPath).CreateFrom<SentimentData>(useHeader: true));
With those two updates, I got the below output. Looks like a nice model with an AUC of 85%!
Another thing that helps with text type datasets is indicating that the text has quotes:
TextLoader("someFile.txt").CreateFrom<Input>(useHeader: true, allowQuotedStrings: true)
There is bad formated value on 252 and 253 rows. May me there fields wich contains the delimiter charachter.
If you post code or sample data we can be more precise.
I'm trying to load a json file downloaded and saved in the persistentDataPath of an iOS device. The download and save works great. However I think I have big issues to load the file. In fact when I try to compile the project in Xcode I have some errors messages.
First here is my C# code :
using UnityEngine;
using System.Collections;
using System.IO;
using System.Net;
using UnityEngine.UI;
public class ReadJson : MonoBehaviour
{
public Text City;
public Text Temperature;
public Image Weather;
public Text WeatherUrl;
[System.Serializable]
public class CityInfo
{
public string name;
}
[System.Serializable]
public class CurrentCondition
{
public string date;
public string hour;
public int tmp;
public int wnd_spd;
public int wnd_gust;
public string wnd_dir;
public double pressure;
public int humidity;
public string condition;
public string icon;
public string icon_big;
}
[System.Serializable]
public class RootObject
{
public CityInfo city_info;
public CurrentCondition current_condition;
}
void Start () {
WebClient client = new WebClient();
File.Delete(Path.Combine (Application.persistentDataPath, "myjson.json"));
client.DownloadFile ("http://www.myjsonurl", Path.Combine (Application.persistentDataPath, "myjson.json"));
TextAsset asset = Resources.Load (Path.Combine (Application.dataPath + "/Documents", "myjson")) as TextAsset;
RootObject m = JsonUtility.FromJson<RootObject> (asset.text);
City.text = m.city_info.name;
Temperature.text = (m.current_condition.tmp).ToString();
}
}
And now the xcode console :
2016-10-21 17:01:20.766001 json[1404:516674] [DYMTLInitPlatform] platform initialization successful
2016-10-21 17:01:20.929950 json[1404:516508] -> registered mono modules 0xb95f70
2016-10-21 17:01:21.356590 json[1404:516508] You've implemented -[<UIApplicationDelegate> application:didReceiveRemoteNotification:fetchCompletionHandler:], but you still need to add "remote-notification" to the list of your supported UIBackgroundModes in your Info.plist.
-> applicationDidFinishLaunching()
2016-10-21 17:01:21.452967 json[1404:516508] Metal GPU Frame Capture Enabled
2016-10-21 17:01:21.453369 json[1404:516508] Metal API Validation Enabled
-> applicationDidBecomeActive()
Init: screen size 750x1334
Initializing Metal device caps
Initialize engine version: 5.3.4f1 (fdbb5133b820)
UnloadTime: 1.705875 ms
Salut/var/mobile/Containers/Data/Application/51A2490E-94EB-4904-9F2E-112AD5632A98/Documents
(Filename: /Users/builduser/buildslave/unity/build/artifacts/generated/common/runtime/UnityEngineDebugBindings.gen.cpp Line: 37)
NullReferenceException: A null value was found where an object instance was required.
(Filename: currently not available on il2cpp Line: -1)
Setting up 1 worker threads for Enlighten.
Thread -> id: 19f17000 -> priority: 1
If someone has an idea. Thanks a lot in advance.
The problem with your code is on this line:
TextAsset asset = Resources.Load(Path.Combine(Application.dataPath + "/Documents", "myjson")) as TextAsset;
I don't think you understand how and when to use Resources.Load. If you want to use the Resources.Load function, you must create a special folder from the Editor. This folder should be at <ProjectDirectory>/Assets/Resources. You can only put/write stuff into this folder from the Editor. You can't do it after build on iOS. It becomes read only after you build it. This special folder can then be read with the Resources.Load function.
Replacing:
TextAsset asset = Resources.Load(Path.Combine(Application.dataPath + "/Documents", "myjson")) as TextAsset;
with
string text = File.ReadAllText(Path.Combine(Application.persistentDataPath, "myjson.json"));
RootObject m = JsonUtility.FromJson<RootObject>(text);
City.text = m.city_info.name;
Temperature.text = (m.current_condition.tmp).ToString();
Should solve your problem.
Another API that should not be using is WebClient. Unless you have a good reason to use that, you shouldn't. You should either use the WWW or UnityWebRequest(newer and recommended) API.
This is what your code should look like without WebClient:
void Start()
{
StartCoroutine(DownloadAndloadJson());
}
IEnumerator DownloadAndloadJson()
{
string url = "http://www.myjsonurl";
UnityWebRequest www = UnityWebRequest.Get(url);
yield return www.Send();
if (www.isError)
{
Debug.Log(www.error);
}
else
{
//Sucessfully downloaded the Json File
Debug.Log("Json: " + www.downloadHandler.text);
string jsonFile = www.downloadHandler.text;
//Delete old json file. Don't know why but this was in your original code
File.Delete(Path.Combine(Application.persistentDataPath, "myjson.json"));
//Save the downloaded data as File
File.WriteAllText(Path.Combine(Application.persistentDataPath, "myjson.json"), jsonFile);
//Load the downloaded data
string text = File.ReadAllText(Path.Combine(Application.persistentDataPath, "myjson.json"));
Debug.Log(text);
RootObject m = JsonUtility.FromJson<RootObject>(text);
City.text = m.city_info.name;
Temperature.text = (m.current_condition.tmp).ToString();
}
}
If you have problems please update to Unity 5.4. The UnityWebRequest API should be used for this and it was introduced in 5.3 but bugs were fixed in 5.4. You need using UnityEngine.Networking; to use UnityWebRequest.
I had asked this question before. The solution using Newtonsoft worked great until I deployed the website on web hosting server, where its giving me nightmares. Therefore, I am planning to use some System.Web libraries so that I don't have to deal with *.dlls and such and I can easily deploy my website.
Can someone help me parse the same json output using System.Web.Script.Serialization or any System library? Thanks a lot.
I hope your real json string is valid since the one you posted is corrupted but Json.Net tolerates it.
The only trick is deserializing to this funny type List<Dictionary<string,object>>
Here is an example with corrected json(removed trailing ,s)
string json = #"[ { ""ew"" : ""bharat"", ""hws"" : [ ""\u092D\u093E\u0930\u0924"",""\u092D\u0930\u0924"",""\u092D\u0930\u093E\u0924"",""\u092D\u093E\u0930\u093E\u0924"",""\u092C\u0939\u0930\u0924"" ] }, { ""ew"" : ""india"", ""hws"" : [ ""\u0907\u0902\u0921\u093F\u092F\u093E"",""\u0907\u0928\u094D\u0921\u093F\u092F\u093E"",""\u0907\u0923\u094D\u0921\u093F\u092F\u093E"",""\u0908\u0928\u094D\u0921\u093F\u092F\u093E"",""\u0907\u0928\u0921\u093F\u092F\u093E"" ] } ]";
var list = new JavaScriptSerializer().Deserialize<List<Dictionary<string,object>>>(json);
foreach (var dict in list)
{
var ew = (string)dict["ew"];
var firstValOfHws = ((ArrayList)dict["hws"])[0];
}
--EDIT--
OK, This should work
var serializer = new DataContractJsonSerializer(typeof(List<Result>));
var result = (List<Result>)serializer.ReadObject(new MemoryStream(Encoding.UTF8.GetBytes(json)));
public class Result
{
public string ew;
public List<string> hws;
}