I am working on a method to asynchronously read the contents of a json file ( containing an array of json objects) and insert it into a mongodb collection but I cannot figure out what the issue is. There is no error when debugging, but my collection is still empty.
public async void InsertDocumentsInCollection(string File)
{
string text = System.IO.File.ReadAllText(File);
IEnumerable<BsonDocument> doc = BsonSerializer.Deserialize<BsonArray>(text).Select(p => p.AsBsonDocument);
//Name of the collection is Cars
var collection = _database.GetCollection<BsonDocument>("Cars");
await collection.InsertManyAsync(doc);
}
i tried to reproduce the issue with the following, but it works just fine. maybe it's something wrong with the contents of the text file. can you post a sample of the file?
using MongoDB.Bson;
using MongoDB.Bson.Serialization;
using MongoDB.Driver;
using System.IO;
using System.Linq;
using System.Threading.Tasks;
namespace StackOverflow
{
public class Program
{
private static async Task Main(string[] args)
{
var content = File.ReadAllText("E:\\Downloads\\cars.txt"); //https://mongoplayground.net/p/LY0W7vjDuvp
var docs = BsonSerializer.Deserialize<BsonArray>(content).Select(p => p.AsBsonDocument);
var collection = new MongoClient("mongodb://localhost")
.GetDatabase("test")
.GetCollection<BsonDocument>("Cars");
await collection.InsertManyAsync(docs);
}
}
}
I have a php file which reads a .txt file and sends it via a php server to a c# unity script. Below is a snippet of the text file showing the first 3 lines:
{ "lemma" : "aljotta", "gloss" : "Fisħ soup" }
{ "lemma" : "arguzin", "gloss" : "Slave driver" }
{ "lemma" : "armunjaka", "gloss" : "Armunjaka" }
This is the php script:
<?php
$file = fopen("lemmas.txt", "r");
echo fread($file, filesize("lemmas.txt"));
fclose($file);
?>
In a c# script, the text is returned and each line is separated into an array (string[] lines) slot as seen below:
IEnumerator GetTextFromFile()
{
bool succcessful = true;
WWWForm form = new WWWForm();
WWW www = new WWW("http://localhost:9000/tounity.php", form);
yield return www;
if(www.error != null)
{
succcessful = false;
}
else
{
succcessful = true;
}
if (succcessful)
{
populateWordList(www.text);
}
}
void populateWordList(string text)
{
string[] textArray = text.Split('\n');
wordsList = gameDatabase.GetWords(textArray);
}
The array is then passed to a method which deserializes each line into an object of class GameDatabase as seen in the image below:
public string lemma { get; set; }
public string gloss { get; set; }
public GameDatabase(string lemma, string gloss)
{
this.lemma = lemma;
this.gloss = gloss;
}
public ArrayList GetWords(string[] lines)
{
foreach (string line in lines)
{
GameDatabase gd = JsonConvert.DeserializeObject<GameDatabase>(line);
lemmasAndGlossesList.Add(new GameDatabase(gd.lemma, gd.gloss));
}
foreach(GameDatabase line in lemmasAndGlossesList)
{
Debug.Log(line.lemma + "------" + line.gloss);
}
return lemmasAndGlossesList;
}
The error occurs in GameDatabase gd = JsonConvert.DeserializeObject<GameDatabase>(line); and returns
JsonReaderException: Unexpected character encountered while parsing value: . Path '', line 0, position 0.
I have searched extensively, however, haven't found anything that works. Any help would be greatly appreciated. It is worth noting that this problem doesn't happen when loading the text file directly into unity without using php.
EDIT
When using the vs debugger this is the value in the line to be deserialized:
The JSON visualiser of Visual Studio 2019 however reports this:
Thanks to Jonathon K's comment and your reply we can see the data returned by the PHP script starts with a BOM: the first three bytes. This nice article explains how to handle such data properly. In short: use a StreamReader to read the data.
This little program demonstrates how it could work with your data:
using System;
using Newtonsoft.Json;
using System.IO;
public class Program
{
public static void Main()
{
var bytes = new byte[] {
0xEF,0xBB,0xBF,0x7B,0x20,0x22,0x6C,0x65,0x6D,0x6D,0x61,0x22,
0x20,0x3A,0x20,0x22,0x61,0x72,0x67,0x75,0x7A,0x69,0x6E,0x22,
0x2C,0x20,0x22,0x67,0x6C,0x6F,0x73,0x73,0x22,0x20,0x3A,0x20,
0x22,0x53,0x6C,0x61,0x76,0x65,0x20,0x64,0x72,0x69,0x76,0x65,
0x72,0x22,0x20,0x7D};
string json;
using(var ms = new MemoryStream(bytes))
using(var sr = new StreamReader(ms))
{
json = sr.ReadToEnd();
Console.WriteLine(json);
}
// I'm using dynamic here. In your case you can use GameDatabase
dynamic obj = JsonConvert.DeserializeObject(json);
Console.WriteLine(obj.lemma);
}
}
Output:
{ "lemma" : "arguzin", "gloss" : "Slave driver" }
arguzin
I dont know the c# sintax but this will work.
change your JSON file.
[
{ "lemma" : "aljotta", "gloss" : "Fisħ soup" },
{ "lemma" : "arguzin", "gloss" : "Slave driver" },
{ "lemma" : "armunjaka", "gloss" : "Armunjaka" }
]
apply JsonConvert.DeserializeObject to www.text
for (GameDatabase line in JsonConvert.DeserializeObject<GameDatabase[]>(www.text)){
Debug.Log(line.lemma + "------" + line.gloss);
}
Maybe my c# syntax is wrong but i would that u understand my idea
I think it's possible that you're going about deserialization wrong by using JsonConvert.
Instead, read up on this documentation and try to use the Unity functions:
https://docs.unity3d.com/Manual/JSONSerialization.html
For starters, you're defining lemma and gloss incorrectly if you're looking to use them for deserializing JSON in Unity. See this answer for more info:
Serialize and Deserialize Json and Json Array in Unity
I'm trying to create a list of notifications in Unity, which are delivered by a JSON API. I'm using the SimpleJson plugin, which I already used in another scene perfectly,
Here is the JSON:
[
{
"_id": {
"$oid": "5d30eccda6e0712cfd0832c3"
},
"titulo": "Primera Notificacion",
"texto": "Prueba de notificacion"
},
{
"_id": {
"$oid": "5d336c36a6e07114ac728cc2"
},
"titulo": "Segunda notificacion",
"texto": "Prueba de notificacion 2"
}
]
Here is the error:
Exception: JSON Parse: Quotation marks seems to be messed up.
SimpleJSON.JSONNode.Parse (System.String aJSON) (at Assets/QRcode/Scripts/SimpleJSON.cs:735)
SimpleJSON.JSON.Parse (System.String aJSON) (at Assets/QRcode/Scripts/SimpleJSON.cs:1421)
DataLoaderNot+d__5.MoveNext () (at Assets/QRcode/Scripts/DataLoaderNot.cs:29)
UnityEngine.SetupCoroutine.InvokeMoveNext (System.Collections.IEnumerator enumerator, System.IntPtr returnValueAddress) (at C:/buildslave/unity/build/Runtime/Export/Scripting/Coroutines.cs:17)
I'm using this code to call JSON:
using UnityEngine;
using System.Collections;
using UnityEngine.UI;
using SimpleJSON;
public class DataLoader : MonoBehaviour
{
string JsonDataString;
string JsonDataString2;
static public string OriginalJsonSite = "http://(web service name)/API/testnot.php";
public Text TituloNot;
public Text TextoNot;
IEnumerator Start ()
{
WWW readingsite = new WWW (OriginalJsonSite);
Debug.Log(OriginalJsonSite);
yield return readingsite;
if (string.IsNullOrEmpty (readingsite.error)) {
JsonDataString = readingsite.text;
JsonDataString2 = JsonDataString.Substring(3, JsonDataString.Length - 4);
}
JSONNode jsonNode = SimpleJSON.JSON.Parse(JsonDataString2);
JSONArray array = jsonNode.AsArray;
Debug.Log(JsonDataString2);
TituloNot.text = array[0]["titulo"].ToString();
Debug.Log(jsonNode["titulo"]);
TextoNot.text = array[0]["texto"].ToString();
Debug.Log(jsonNode["texto"]);
}
}
The lines
Debug.Log(jsonNode["titulo"]);
Debug.Log(jsonNode["texto"]);
won't work for sure, you should remove them.
Also, if the returned JSON is really exactly like the one you posted, you should also remove the line
JsonDataString2 = JsonDataString.Substring(3, JsonDataString.Length - 4);
The parse will take care of finding matching brackets [] and braces {} so this line will break it.
Edit: I didn't test it but this code should work in my opinion:
using UnityEngine;
using System.Collections;
using UnityEngine.UI;
using SimpleJSON;
public class DataLoader : MonoBehaviour
{
string jsonDataString;
static public string originalJsonSite = "http://(web service name)/API/testnot.php";
public Text tituloNot;
public Text textoNot;
IEnumerator Start()
{
WWW readingsite = new WWW (originalJsonSite);
yield return readingsite;
if (string.IsNullOrEmpty(readingsite.error))
{
jsonDataString = readingsite.text;
}
JSONNode jsonNode = SimpleJSON.JSON.Parse(jsonDataString);
JSONArray array = jsonNode.AsArray;
tituloNot.text = array[0]["titulo"].ToString();
textoNot.text = array[0]["texto"].ToString();
}
}
I've been having a problem with my ml.net console app. This is my first time using ml.net in Visual Studio so I was following this tutorial from microsoft.com, which is a sentiment analysis using binary classification.
I'm trying to process some test data in the form of tsv files to get a positive or negative sentiment analysis, but in debugging I'm receiving warnings there being 1 format error and 2 bad values.
I decided to ask all you great devs here on Stack to see if anyone can help me find a solution.
Here's an image of the debugging below:
Here's the link to my test data:
wiki-data
wiki-test-data
Finally, here's my code for those who what to reproduce the problem:
There's 2 c# files: SentimentData.cs & Program.cs.
1 - SentimentData.cs:
using System;
using System.Collections.Generic;
using System.Text;
using Microsoft.ML.Runtime.Api;
namespace MachineLearningTut
{
public class SentimentData
{
[Column(ordinal: "0")]
public string SentimentText;
[Column(ordinal: "1", name: "Label")]
public float Sentiment;
}
public class SentimentPrediction
{
[ColumnName("PredictedLabel")]
public bool Sentiment;
}
}
2 - Program.cs:
using System;
using Microsoft.ML.Models;
using Microsoft.ML.Runtime;
using Microsoft.ML.Runtime.Api;
using Microsoft.ML.Trainers;
using Microsoft.ML.Transforms;
using System.Collections.Generic;
using System.Linq;
using Microsoft.ML;
using Microsoft.ML.Data;
using System.Threading.Tasks;
namespace MachineLearningTut
{
class Program
{
const string _dataPath = #".\Data\wikipedia-detox-250-line-data.tsv";
const string _testDataPath = #".\Data\wikipedia-detox-250-line-test.tsv";
const string _modelpath = #".\Data\Model.zip";
static async Task Main(string[] args)
{
var model = await TrainAsync();
Evaluate(model);
Predict(model);
}
public static async Task<PredictionModel<SentimentData, SentimentPrediction>> TrainAsync()
{
var pipeline = new LearningPipeline();
pipeline.Add(new TextLoader (_dataPath).CreateFrom<SentimentData>());
pipeline.Add(new TextFeaturizer("Features", "SentimentText"));
pipeline.Add(new FastForestBinaryClassifier() { NumLeaves = 5, NumTrees = 5, MinDocumentsInLeafs = 2 });
PredictionModel<SentimentData, SentimentPrediction> model = pipeline.Train<SentimentData, SentimentPrediction>();
await model.WriteAsync(path: _modelpath);
return model;
}
public static void Evaluate(PredictionModel<SentimentData, SentimentPrediction> model)
{
var testData = new TextLoader(_testDataPath).CreateFrom<SentimentData>();
var evaluator = new BinaryClassificationEvaluator();
BinaryClassificationMetrics metrics = evaluator.Evaluate(model, testData);
Console.WriteLine();
Console.WriteLine("PredictionModel quality metrics evaluation");
Console.WriteLine("-------------------------------------");
Console.WriteLine($"Accuracy: {metrics.Accuracy:P2}");
Console.WriteLine($"Auc: {metrics.Auc:P2}");
Console.WriteLine($"F1Score: {metrics.F1Score:P2}");
}
public static void Predict(PredictionModel<SentimentData, SentimentPrediction> model)
{
IEnumerable<SentimentData> sentiments = new[]
{
new SentimentData
{
SentimentText = "Please refrain from adding nonsense to Wikipedia."
},
new SentimentData
{
SentimentText = "He is the best, and the article should say that."
}
};
IEnumerable<SentimentPrediction> predictions = model.Predict(sentiments);
Console.WriteLine();
Console.WriteLine("Sentiment Predictions");
Console.WriteLine("---------------------");
var sentimentsAndPredictions = sentiments.Zip(predictions, (sentiment, prediction) => (sentiment, prediction));
foreach (var item in sentimentsAndPredictions)
{
Console.WriteLine($"Sentiment: {item.sentiment.SentimentText} | Prediction: {(item.prediction.Sentiment ? "Positive" : "Negative")}");
}
Console.WriteLine();
}
}
}
If anyone would like to see the code or more details on the solution, ask me on the chat and I'll send it. Thanks in advance!!! [Throws a Thumbs Up]
I think I got a fix for you. A couple of things to update:
First, I think you got your SentimentData properties switched to what the data has. Try changing it to
[Column(ordinal: "0", name: "Label")]
public float Sentiment;
[Column(ordinal: "1")]
public string SentimentText;
Second, use the useHeader parameter in the TextLoader.CreateFrom method. Don't forget to add that to the other one for the validation data, as well.
pipeline.Add(new TextLoader(_dataPath).CreateFrom<SentimentData>(useHeader: true));
With those two updates, I got the below output. Looks like a nice model with an AUC of 85%!
Another thing that helps with text type datasets is indicating that the text has quotes:
TextLoader("someFile.txt").CreateFrom<Input>(useHeader: true, allowQuotedStrings: true)
There is bad formated value on 252 and 253 rows. May me there fields wich contains the delimiter charachter.
If you post code or sample data we can be more precise.
I am trying to serialize an object with Newtonsoft Json converter like this:
using System;
using System.Collections.Generic;
using System.IO;
using System.Net;
using Newtonsoft.Json;
string json = new Newtonsoft.Json.JsonConvert.SerializeObject(new
{
jlpt = "5"
});
but it gives me an error saying that
SerializeObject does not exist.
However, when I click to check the reference I see it.
Can anyone point me to what I am doing wrong?
Remove new instance creation of Newtonsoft.Json.JsonConvert, because SerializeObject is a static method you don't need create a instance of the Newtonsoft.Json.JsonConvert to use it
string json = Newtonsoft.Json.JsonConvert.SerializeObject(new
{
jlpt = "5"
});
also if you add using Newtonsoft.Json; to the program then you simply use like this
string json = JsonConvert.SerializeObject(new
{
jlpt = "5"
});