Develop Reference

Mapping JSON from Yahoo! Finance API to C# objects [duplicate]

This question already has answers here:
Deserialize JSON data with C#
(4 answers)
Closed 1 year ago.
I am a student working on a project. I am trying to use the Yahoo! Finance API as a source for my data https://www.yahoofinanceapi.com . I can use HttpWebRequests to call the API and get the "OK" code, see the code below on how I did it:
string BaseURL = "https://yfapi.net/v6/finance/quote?symbols=AAPL";
string addSymbol = "%2C";
string URL = BaseURL;
foreach (string stock in stocks)
{
URL += addSymbol + stock;
}
HttpWebRequest request = (HttpWebRequest)WebRequest.Create(URL);
request.Headers.Add("X-API-KEY", "[My API key]");
HttpWebResponse response = (HttpWebResponse)request.GetResponse();
Console.WriteLine(response.ContentType);
Console.WriteLine(response.StatusCode);
response.ContentType gives back "application/json".
response.StatusCode gives back "OK".
Since the response is a JSON I tried to parse it into a string using .ToString() but this obviously doesn't work. When I print it, it just says "System.Net.HttpWebResponse" instead of the showing the actual data in the JSON.
After that I tried to deserialize it using newtonsoft
Results result = JsonConvert.DeserializeObject<Results>(request.GetResponse().ToString());
where Results is a class I made where there is a list of stocks, Stock is also a class I made with some variables in it with the same names of variables in the JSON response.
I got a JSON response from PostMan when I tested the API, opened the response to see what kind of data it contained.
When I ran my code I got the following error message:
Newtonsoft.Json.JsonReaderException: 'Unexpected character encountered while parsing value: S. Path '', line 0, position 0.'
This was as far as I got, I tested a few other methods trying to get this working but this one worked the "best".
My biggest issue at the moment is mapping the response into a c# object.
Anything that can help me understand is appreciated :D

You're trying to serialise the HttpWebResponse object into Results, which means deserialisation won't work.
The data is still wrapped & won't be in the format of the Results object.
The GetResponseStream() method can be used to get the contents of the HTTP response as a stream, which can then be deserialised directly, in this case, using Json.NET.
Replace this section:
HttpWebResponse response = (HttpWebResponse)request.GetResponse();
Console.WriteLine(response.ContentType);
Console.WriteLine(response.StatusCode);
With this:
var serializer = new JsonSerializer();
using (var response = (HttpWebResponse)request.GetResponse())
{
var encoding = Encoding.GetEncoding(response.CharacterSet);
using var responseStream = response.GetResponseStream();
using var reader = new StreamReader(responseStream, encoding);
using (var jsonTextReader = new JsonTextReader(reader))
{
Console.WriteLine(response.ContentType);
Console.WriteLine(response.StatusCode);
Results result = serializer.Deserialize<Results>(jsonTextReader);
}
}
Alternatively, a much better solution if you're using .NET 4.5 <= would be to use HttpClient like below:
private static readonly HttpClient httpClient = new HttpClient();
...
string BaseURL = "https://yfapi.net/v6/finance/quote?symbols=AAPL";
string addSymbol = "%2C";
string URL = BaseURL;
foreach(string stock in stocks) {
URL += addSymbol + stock;
}
client.DefaultRequestHeaders.Add("X-API-KEY", "[My API key]");
var data = await httpClient.GetStringAsync(address);
Results result = JsonConvert.DeserializeObject<Results>(data);

SendGrid - Loading Exported Contacts

I'm trying to process the resulting URLs from the marketing/contacts/exports/{id} call I used the following code however the data is dataresponse is zipped/encrypted.
SendGridClient client = new SendGridClient(AppConfig.ReadSetting("SENDGRID_API_KEY"));
SingleSendContactList allListsSingleSend = new SingleSendContactList();
var response = await client.RequestAsync(method: SendGridClient.Method.GET, urlPath: "marketing/lists?page_size=500");
allListsSingleSend = JsonConvert.DeserializeObject<SingleSendContactList>(response.Body.ReadAsStringAsync().Result);
SingleSendExportSetup SSExportSetup = new SingleSendExportSetup();
SSExportSetup.file_type = "json";
SSExportSetup.max_file_size = 5000;
// marketing/contacts/exports, POST returns JSON with id as main field
SingleSendExportID ExportContactsSingleSend = new SingleSendExportID();
response = await client.RequestAsync(method: SendGridClient.Method.POST, urlPath: "marketing/contacts/exports", requestBody: JsonConvert.SerializeObject(SSExportSetup));
ExportContactsSingleSend = JsonConvert.DeserializeObject<SingleSendExportID>(response.Body.ReadAsStringAsync().Result);
//Use this id to then call
// marketing/contacts/exports/id, GET returns JSON with urls as a property, use urls[0]
SingleSendExportURL ExportURLSingleSend = new SingleSendExportURL();
response = await client.RequestAsync(method: SendGridClient.Method.GET, urlPath: "marketing/contacts/exports/" + ExportContactsSingleSend.id);
ExportURLSingleSend = JsonConvert.DeserializeObject<SingleSendExportURL>(response.Body.ReadAsStringAsync().Result);
// Call the URL to get the returned JSON
List<SingleSendExportData> ExportDataSingleSend = new List<SingleSendExportData>();
var dataresponse = await client.RequestAsync(method: SendGridClient.Method.GET, urlPath: ExportURLSingleSend.urls[0]);
ExportDataSingleSend = JsonConvert.DeserializeObject<List<SingleSendExportData>>(dataresponse.Body.ReadAsStringAsync().Result);
I can take the URL in ExportURLSingleSend.urls[0] and drop it in a browser and it will force the download of the file and the data is fine. How do I get this data to convert to my class List<SingleSendExportData>?
I posted this question to SendGrid support and they sent me here.
Thanks.
UPDATE:
This post solved my issue:
https://weblog.west-wind.com/posts/2007/jun/29/httpwebrequest-and-gzip-http-responses
However, now I am realizing that the JSON coming from SendGrid is not properly formatted. There is no comma between rows of data.
UPDATE 2:
This post solved my handling of line delimited JSON files;
Line delimited json serializing and de-serializing
Now I am able to read these zipped JSON files without saving them and then handle the JSON format to load into my list List ExportDataSingleSend.

Unexpected character encountered while parsing value line 1, position 1

I'm building .Net Core 3.1 Web API. To send requests to Web API. I'm using System.Net.Http library. I put JSON into POST request body. This is Web API side handler:
using Microsoft.AspNetCore.Mvc;
[ApiController]
public class MyController : ControllerBase
{
[HttpPost]
public int HandleRequest([FromBody] string jsonString)
{
return 1;
}
}
The code below sends empty ArrayList to Web API but it always returns "Unexpected character encountered while parsing value line 1, position 1.".
var serializedData = new JavaScriptSerializer().Serialize(new ArrayList());
var content = new StringContent(serializedData, Encoding.UTF8, "application/json");
var response = client.PostAsync(url, content);
return response.Result.Content.ReadAsStringAsync();
I've tried to use JsonConvert and played around the object to serialize but it always returns the same error. But then I've found that if I make serialization twice then it starts working. So code below works - sends the request without any errors and the data on Web API side is correct.
var serializer = new JavaScriptSerializer();
var serializedData = serializer.Serialize(serializer.Serialize(new ArrayList()));
var content = new StringContent(serializedData, Encoding.UTF8, "application/json");
var response = client.PostAsync(url, content);
return response.Result.Content.ReadAsStringAsync();
Why do I need to do double serialization to make it work?

First, when you are using async methods you should add await keyword:
var response = await client.PostAsync(url, content);
return await response.Result.Content.ReadAsStringAsync();
Secondly, you are sending an array, so why not just try to send it and get as object not string:
var updateData = new ArrayList();
var jsonObject = JsonConvert.SerializeObject(updateData);
var content = new StringContent(jsonObject, Encoding.UTF8, "application/json");
var res = await apiClient.PutAsync(url, content);
And then in WebAPI:
[HttpPost]
public int HandleRequest([FromBody] ArrayList list)
{
return 1;
}

var serializedData = new JavaScriptSerializer().Serialize(new ArrayList());
Console.WriteLine(serializedData);
// Output: []
var serializer = new JavaScriptSerializer();
var serializedData = serializer.Serialize(serializer.Serialize(new ArrayList()));
Console.WriteLine(serializedData);
// Output: "[]"
You have an error because your endpoint expects string but in first serialization you are sending an array.
Second serialization works, because first you serialize ArrayList to string and then that string to string. In the end you by accident send a serialized string and endpoint can deserialize it.
Error "Unexpected character encountered while parsing value line 1, position 1." means that server receives at first character [ but it expects " because it tries deserialize JSON to string - but it was array.

AWS sagemaker invokeEndpoint model internal error

I am trying to send a request on a model on sagemaker using .NET. The code I am using is:
var data = File.ReadAllBytes(#"C:\path\file.csv");
var credentials = new Amazon.Runtime.BasicAWSCredentials("","");
var awsClient = new AmazonSageMakerRuntimeClient(credentials, RegionEndpoint.EUCentral1);
var request = new Amazon.SageMakerRuntime.Model.InvokeEndpointRequest
{
EndpointName = "EndpointName",
ContentType = "text/csv",
Body = new MemoryStream(data),
};
var response = awsClient.InvokeEndpoint(request);
var predictions = Encoding.UTF8.GetString(response.Body.ToArray());
the error that I am getting on awsClient.InvokeEndpoint(request)
is:
Amazon.SageMakerRuntime.Model.ModelErrorException: 'The service
returned an error with Error Code ModelError and HTTP Body:
{"ErrorCode":"INTERNAL_FAILURE_FROM_MODEL","LogStreamArn":"arn:aws:logs:eu-central-1:xxxxxxxx:log-group:/aws/sagemaker/Endpoints/myEndpoint","Message":"Received
server error (500) from model with message \"\". See
"https:// url_to_logs_on_amazon"
in account xxxxxxxxxxx for more
information.","OriginalMessage":"","OriginalStatusCode":500}'
the url that the error message suggests for more information does not help at all.
I believe that it is a data format issue but I was not able to find a solution.
Does anyone has encountered this behavior before?

The problem relied on the data format as suspected. In my case all I had to do is send the data as a json serialized string array and use ContentType = application/json because the python function running on the endpoint which is responsible for sending the data to the predictor was only accepting json strings.
Another way to solve this issues is to modify the python function which is responsible for the input handling to accept all content types and modify the data in a way that the predictor will understand.
example of working code for my case:
var data = new string[] { "this movie was extremely good .", "the plot was very boring ." };
var serializedData = JsonConvert.SerializeObject(data);
var credentials = new Amazon.Runtime.BasicAWSCredentials("","");
var awsClient = new AmazonSageMakerRuntimeClient(credentials, RegionEndpoint.EUCentral1);
var request = new Amazon.SageMakerRuntime.Model.InvokeEndpointRequest
{
EndpointName = "endpoint",
ContentType = "application/json",
Body = new MemoryStream(Encoding.ASCII.GetBytes(serializedData)),
};
var response = awsClient.InvokeEndpoint(request);
var predictions = Encoding.UTF8.GetString(response.Body.ToArray());

C# retrieve JSON from URL

I am creating a program that gets the steam market prices for various CS:GO items, and compares them to one another. I am having trouble getting the steam market JSON into my program. Market JSON
Here is my code:
using (WebClient webClient = new System.Net.WebClient())
{
WebClient n = new WebClient(); // <-- error on this line
string json = n.DownloadString("http://steamcommunity.com/market/priceoverview/?currency=1&appid=730&market_hash_name=P250%20%7C%20Valence%20(Factory%20New");
dynamic array = JsonConvert.DeserializeObject(json);
test.Text = array.lowest_price.ToString(); ;
}
I am getting this error when instanciating WebClient():
An unhandled exception of type 'System.Net.WebException' occurred in
System.dll
Additional information: The remote server returned an error: (500)
Internal Server Error.
Is this even a valid JSON? If not, is there an alternative? I heard backpack.tf has an api as well. Would this be better?
Thanks

Looks like the URL is malformed. I.just tried http://steamcommunity.com/market/priceoverview/?currency=1&appid=730&market_hash_name=Shadow%20Case and it returned a good response.
Of particular note is the enclosed parenthesis in your URL.

Fixed code:
using (var webClient = new System.Net.WebClient())
{
var json = webClient.DownloadString("http://steamcommunity.com/market/priceoverview/?currency=1&appid=730&market_hash_name=P250%20%7C%20Valence%20(Factory%20New)");
dynamic array = JsonConvert.DeserializeObject(json);
var price = array.lowest_price.ToString(); ;
}