I'm trying to deserialize some JSON in C#, but when I run my program I'm getting this error message:
I've looked through all my code, and I can't find a "<" anywhere there shouldn't be one, and I went to the web address that the json is coming from:
http://forecast.weather.gov/MapClick.php?lat=47.1211&lon=-88.5694&FcstType=json,
and there isn't a "<" character. I used json2csharp.com to translate to C# classes, and everything there seems fine as well. Any thoughts? Here is the part of my code where I try to do all of this:
var http = new HttpClient();
var url = "http://forecast.weather.gov/MapClick.php?lat=47.1211&lon=-88.5694&FcstType=json";
var response = await http.GetAsync(url);
var result = await response.Content.ReadAsStringAsync();
var serializer = new DataContractJsonSerializer(typeof(RootObject2));
var ms = new MemoryStream(Encoding.UTF8.GetBytes(result));
var data = (RootObject2)serializer.ReadObject(ms);
return data;
Your call is failing because you are not setting a header the API is expecting. Add a user agent and check for success prior to attempting to read the response.
var http = new HttpClient();
var url = "http://forecast.weather.gov/MapClick.php?lat=47.1211&lon=-88.5694&FcstType=json";
//Supply the same header as chrome
http.DefaultRequestHeaders.Add("User-Agent", "Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.84 Safari/537.36");
var response = await http.GetAsync(url);
if (response.IsSuccessStatusCode)
{
var result = await response.Content.ReadAsStringAsync();
var ms = new MemoryStream(Encoding.UTF8.GetBytes(result));
var serializer = new DataContractJsonSerializer(typeof(RootObject2));
var data = (RootObject2)serializer.ReadObject(ms);
}
check that answer, it says some issue with the connection, that he was not receiving the full response from the API
Unexpected character encountered while parsing value:
Related
I am experiencing a known bug in the HttpClient. Anytime the server response contains "UTF-8" (including quotes), an exception is triggered:
The character set provided in ContentType is invalid. Cannot read content as string using an invalid character set. ---> System.ArgumentException: '"utf-8"' is not a supported encoding name.
Example code:
HttpClient _client = new HttpClient();
HttpRequestMessage requestMessage = new HttpRequestMessage(HttpMethod.Get, "https://www.facebook.com");
requestMessage.Headers.Add("User-Agent", "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/80.0.4044.55 Safari/537.36");
HttpResponseMessage response = _client.SendAsync(requestMessage).GetAwaiter().GetResult();
What is the usual workaroud? I am using .NETFramework 4.6.1.
To workaround the referenced issue:
using (var client = new HttpClient())
{
HttpRequestMessage requestMessage = new HttpRequestMessage(HttpMethod.Get,
"https://www.facebook.com");
HttpResponseMessage response = await client.SendAsync(requestMessage);
byte[] buf = await response.Content.ReadAsByteArrayAsync();
string content = Encoding.UTF8.GetString(buf);
}
I know that my question looks like a duplicated question, but I could not find a helpful solution for my issue.
So I am trying to scrape data from a cargo ships data providing website Link (It's a Korean website. The black button on the right is the search button)
but in order to obtain data from it, some radio buttons have to be set up then hit search.
I thought I would be able to just pass parameters values through FormUrlEncodedContent then simply use PostAsync, but somehow I could not be able to get them pass through.
Here is my codes so far
using (var client = new HttpClient())
{
client.DefaultRequestHeaders.TryAddWithoutValidation("User-Agent", "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/54.0.2840.99 Safari/537.36");
client.DefaultRequestHeaders.TryAddWithoutValidation("Content-Type", "application/x-www-form-urlencoded");
var doc = new HtmlAgilityPack.HtmlDocument();
var content = new FormUrlEncodedContent(structInfo.ScriptValues);
var response = await client.PostAsync(structInfo.PageURL, content);
var responseString = await response.Content.ReadAsStringAsync();
Console.WriteLine(responseString);
}
using (WebClient client = new WebClient())
{
var reqparm = new System.Collections.Specialized.NameValueCollection();
reqparm.Add("v_time", "month");
reqparm.Add("ROCD", "ALL");
reqparm.Add("ORDER", "item2");
reqparm.Add("v_gu", "S");
byte[] responsebytes = client.UploadValues("http://info.bptc.co.kr:9084/content/sw/frame/berth_status_text_frame_sw_kr.jsp", "POST", reqparm);
string responsebody = Encoding.UTF8.GetString(responsebytes);
Console.WriteLine(responsebody);
}
Values I put in the StructInfo Class
PageURL = "http://info.bptc.co.kr:9084/content/sw/frame/berth_status_text_frame_sw_kr.jsp",
ScriptValues = new Dictionary<string, string>
{
{"v_time", "month"},
{"ROCD", "ALL"},
{"ORDER", "item2"},
{"v_gu", "S"}
},
What I have tried so far are HttpClient, WebClient, WebBrowser but I had no luck.
But a strange thing is when I try to send a post with Burp Suite, data comes out just fine like the way in I wanted.
I've been searching a solution for last 4 hours, didn't have any luck.
Would you guys mind help me?
Thanks
Generated code for C# - RestSharp by Postman
var client = new RestClient("http://info.bptc.co.kr:9084/Berth_status_text_servlet_sw_kr");
client.Timeout = -1;
var request = new RestRequest(Method.POST);
request.AddHeader("Content-Type", "application/x-www-form-urlencoded");
request.AddParameter("v_time", "3days");
request.AddParameter("ROCD", "ALL");
request.AddParameter("ORDER", "item2");
request.AddParameter("v_gu", "S");
IRestResponse response = client.Execute(request);
Console.WriteLine(response.Content);
HttpClient version
using var client = new HttpClient();
var content = new FormUrlEncodedContent(new[]
{
new KeyValuePair<string, string>("v_time", "3days"),
new KeyValuePair<string, string>("ROCD", "ALL"),
new KeyValuePair<string, string>("ORDER", "item2"),
new KeyValuePair<string, string>("v_gu", "S"),
});
string url = "http://info.bptc.co.kr:9084/Berth_status_text_servlet_sw_kr";
var response = await client.PostAsync(url, content);
var bytes = await response.Content.ReadAsByteArrayAsync();
string responseString = Encoding.UTF8.GetString(bytes);
Console.WriteLine(responseString);
The issue
If we talk about the HttpClient version, assuming you are using .net core.
The exception is thrown on ReadAsStringAsync call.
More specifically down below:
https://github.com/microsoft/referencesource/blob/aaca53b025f41ab638466b1efe569df314f689ea/System/net/System/Net/Http/HttpContent.cs#L95
The response has ContentType: text/html; charset=euc-kr.
And the problem is .net core is not supporting Korean charset out of the box.
My workaround is using ReadAsByteArrayAsync instead and then using supported UTF8 encoder later. It screws Korean characters though.
The better way would be to reference the System.Text.Encoding.CodePages package and then use Encoding.RegisterProvider.
Something like this Encoding.GetEncoding can't work in UWP app
Trying to use the HttpClient to get a json response from an API but keep getting html response. In the browser and in Postman I get the result in json just by typing in the url. When using RestSharp I also get the response in json. What do I need to add to get the response in json? The variable responseString in a html string, not a json string.
I use .net core 3.1.
Here's the code:
class Program
{
static async Task Main(string[] args)
{
var response = await GetResponse();
System.Console.ReadKey();
}
public static async Task<string> GetResponse()
{
var client = new HttpClient();
client.BaseAddress = new Uri("https://musicbrainz.org/ws/2/");
client.DefaultRequestHeaders.Add("Accept",
"application/json");
using var response = await client.GetAsync((
"/artist/5b11f4ce-a62d-471e-81fc-a69a8278c7da?fmt=json&inc=url-rels+release-groups"));
response.EnsureSuccessStatusCode();
var responseString = await response.Content.ReadAsStringAsync();
return responseString;
}
}
I think the API is looking for a User-Agent.
Try to Add
client.DefaultRequestHeaders.Add("User-Agent", "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/79.0.3945.130 Safari/537.36")
As a side note, you might want to consider declaring your HttpClient as static. It is recommended in its documentation. Also if you want json, you need to use DeserializeObject method:
var result = JsonConvert.DeserializeObject<RootObject>(responseString);
You just need to make sure that you have already installed the Newtonsoft.Json from Nuget packages and also added the following to your using directives:
using Newtonsoft.Json;
So I am trying to call a api in my C# method and am not getting the response that I would be getting, if I called the api using postman or insomia.
public static async Task GetObject()
{
var httpClient = new HttpClient();
//var t = await httpClient.GetStringAsync("https://ipapi.co/json/");
var y = await httpClient.GetAsync("https://ipapi.co/json/");
var o = await y.Content.ReadAsStringAsync();
var j = y.Content.ReadAsStringAsync();
}
The api is getting my request but its not returning the right result. You should be getting this result. Click on this https://ipapi.co/json/
What am getting is this
{
"error":true,
"reason":"RateLimited",
"message":"Sign up for IP Address Location API # https://ipapi.co"
}
But I don't get it, when am using postman
You need to add the user-agent header.
httpClient.DefaultRequestHeaders.Add("User-Agent", "Mozilla/5.0 (Windows NT 6.1) AppleWebKit/535.1 (KHTML, like Gecko) Chrome/14.0.835.202 Safari/535.1");
Better to Use xNet.dll or Leaf.xNet.dll libraries
Or restsharp one
Google them
I'm also new If ur interesting to contact me
Discord Cyx.#4260
I have following C# code:
Uri url = new Uri("http://lu32kap.typo3.lrz.de/mensaapp/exportDB.php?mensa_id=all");
HttpClient client = new HttpClient();
client.DefaultRequestHeaders.UserAgent.TryParseAdd("Mozilla/5.0 (compatible; MSIE 10.0; Windows NT 6.2; WOW64; Trident/6.0)");
HttpResponseMessage response = await client.GetAsync(url);
response.EnsureSuccessStatusCode();
var content = response.Content;
if(content != null)
{
string result = await content.ReadAsStringAsync();
if (result != null)
{
tblock.Text = result;
}
}
Every time I run it, I get a COMException "HRESULT E_FAIL".
I was able to track it down partially. It's caused by the website I'm trying to get my data from because if I'm changing it to "https://www.google.de/" it works.
It's crashing at:
string result = await content.ReadAsStringAsync();
Nevertheless I need to get it to work with this website because it returns a, with PHP generated, json object.
Is there a way to fix this?
The image behind this link shows the crash in VS2015
I ran this code locally and I end up getting this Exception
The character set provided in ContentType is invalid. Cannot read
content as string using an invalid character set.
And it looks like it is returning UTF8
'utf8' is not a supported encoding name. For information on defining a
custom encoding, see the documentation for the
Encoding.RegisterProvider method.
Can you ensure that the output on the server is in the correct format? Perhaps try this answer:
Parsing UTF8 JSON response from server
Solution:
It was a problem with the UTF8 encoding. I was able to build a small workaround.
Thanks to Glitch100!
Uri url = new Uri("http://lu32kap.typo3.lrz.de/mensaapp/exportDB.php?mensa_id=all");
HttpClient client = new HttpClient();
client.DefaultRequestHeaders.UserAgent.TryParseAdd("Mozilla/5.0 (compatible; MSIE 10.0; Windows NT 6.2; WOW64; Trident/6.0)");
HttpResponseMessage response = await client.GetAsync(url);
response.EnsureSuccessStatusCode();
IHttpContent content = response.Content;
if(content != null)
{
IBuffer buffer = await content.ReadAsBufferAsync();
using (DataReader dataReader = DataReader.FromBuffer(buffer))
{
string result = dataReader.ReadString(buffer.Length);
if (result != null)
{
tblock.Text = result;
}
}
}