I come from an iOS (Swift) background. In one of my Swift apps, I have this class that calls an API. I'm trying to port it to C# (Windows Form application) but I'm hitting several snags. First here's the Swift code. Nothing fancy. One method does a POST request to login to the API and the other function executes a GET method to retrieve the JSON response for a user profile. Both these methods are asynchronous.
import Foundation
class API {
private let session = NSURLSession.sharedSession()
private let baseURL = "https://www.example.com/api/"
func login(userID userID: String, password: String, completion: (error: NSError?) -> ()) {
let url = NSURL(string: baseURL + "login")!
let params = ["username": userID, "password": password]
let request = NSMutableURLRequest(URL: url)
request.HTTPMethod = "POST"
request.encodeParameters(params) // encodeParameters is an extension method
session.dataTaskWithRequest(request, completionHandler: { data, response, error in
if let httpResponse = response as? NSHTTPURLResponse {
if httpResponse.statusCode != 200 {
completion(error: error)
} else {
completion(error: nil)
}
}
}).resume()
}
func fetchUser(completion: (user: User?, error: NSError?) -> ()) {
let url = NSURL(string: baseURL + "profile")!
let request = NSURLRequest(URL: url)
session.dataTaskWithRequest(request, completionHandler: { data, response, error in
if let error = error {
completion(user: nil, error: error)
} else {
// Parsing JSON
var jsonDict = [String: AnyObject]()
do {
jsonDict = try NSJSONSerialization.JSONObjectWithData(data, options: []) as! [String: AnyObject]
} catch {
print("Error occurred parsing data: \(error)")
completion(user: nil, error: error)
}
let user = User()
user.name = jsonDict["name"] as! String
user.age = jsonDict["age"] as! Int
completion(user: user, error: nil)
}
}).resume()
}
}
Here's my attempt to convert this to C#.
using System;
using System.Collections.Generic;
using System.ComponentModel;
using System.Drawing;
using System.Threading.Tasks;
using System.Windows.Forms;
using System.Net.Http;
using System.Runtime.Serialization.Json;
using System.Text;
using System.Xml.Linq;
using System.Xml.XPath;
namespace MyTrayApp
{
public partial class Form1 : Form
{
private string baseURL = "https://www.example.com/api/";
public Form1()
{
InitializeComponent();
}
private async void Form1_Load(object sender, EventArgs e)
{
await login("myusername", "mypassword");
await fetchUser();
}
async Task login(string userID, string password)
{
using (var client = new HttpClient())
{
client.BaseAddress = new Uri(baseURL);
var parameters = new Dictionary<string, string>
{
{ "username", userID },
{ "password", password }
};
var encodedParameters = new FormUrlEncodedContent(parameters);
var response = await client.PostAsync("login", encodedParameters);
string responseString = await response.Content.ReadAsStringAsync();
//Console.WriteLine(responseString);
}
}
async Task fetchUser()
{
using (var client = new HttpClient())
{
client.BaseAddress = new Uri(baseURL);
var response = await client.GetAsync("profile");
response.EnsureSuccessStatusCode();
var responseString = await response.Content.ReadAsStringAsync();
var jsonReader = JsonReaderWriterFactory.CreateJsonReader(Encoding.UTF8.GetBytes(responseString.ToCharArray()), new System.Xml.XmlDictionaryReaderQuotas());
var root = XElement.Load(jsonReader);
Console.WriteLine(root.XPathSelectElement("//name").Value);
//Console.WriteLine(responseString);
}
}
}
}
These are the problems I'm having.
In my Swift methods, they have completion handlers. How can I do the same in C#?
In Swift, you get an NSData object and you can pass it to NSJSONSerialization to create a JSON object. In my current implementation, I get an XML exception at XElement.Load(jsonReader);. I'm not sure if this is the correct way to do this even. I found tons of different solutions here on SO. But some are for Metro apps, some are for web it's all too overwhelming. Also most solutions are on using third-party libraries like JSON.NET. I'm trying to achieve this without third-party libraries.
In my Swift methods, they have completion handlers. How can I do the
same in C#?
The point of wiring up a completion handler is so that you don't tie up a thread while waiting for the HTTP call to complete. The beauty of async/await is that you don't have to do this in C#. The await keyword instructs the compiler to literally rewrite the rest of the method as a callback. The current thread is freed as soon as await is encountered, preventing your UI from freezing up. You have written your async code correctly; it will behave asynchronously even though it looks synchronous.
Your second question is a bit broad, but I will make 2 suggestions:
Don't use XElement when dealing with JSON data. That part of an Microsoft's XML parsing library (one of them) and has nothing to do with JSON.
I'm not sure why achieving this without a 3rd-party library is important. I know people have their reasons, but Json.NET in particular has become so popular and ubiquitous that Microsoft itself has baked it into their ASP.NET MVC and Web API frameworks. That said, if you must avoid it, here is how you would deserialize JSON using only Microsoft libraries.
Related
I have a site on localhost created to learn how to send http requests, I send a post request to it, trying to simulate sending data via forms, as a response I expect that the data I sent will be added to the database, as it happens when sending via forms, but this does not happen. I assume that I am sending the post request incorrectly and something is missing in it
The site: localhost site
Post request: Pose request headers and data
My C# code:
using System;
using System.Collections.Generic;
using System.Net.Http;
using System.Threading.Tasks;
namespace AirParsingScript
{
class Program
{
private static readonly HttpClient Client = new HttpClient();
static async Task Main()
{
var values = new Dictionary<string, string>
{
{ "Person.DocId_pre", "421" },
{ "Person.DocId", "ADGAGSA" },
{ "Person.Email", "4124421" },
{ "Person.TeleNumber", "4444" }
};
var content = new FormUrlEncodedContent(values);
var response = await Client.PostAsync("https://localhost:44391", content);
var responseString = await Client.GetStringAsync("https://localhost:44391");
// var responseString = await response.Content.ReadAsStringAsync();
Console.WriteLine(responseString);
}
}
}
I wrote a little tool to check the availability of a product (yes, the PS5) by checking the products shop page:
var client = new HttpClient();
HttpResponseMessage response = await client.GetAsync("https://www.mediamarkt.de/de/product/_sony-playstation®5-2661938.html");
HttpContent responseContent = response.Content;
using (var reader = new StreamReader(await responseContent.ReadAsStreamAsync()))
{
var output = reader.ReadToEndAsync();
Console.WriteLine(output.Result);
}
For some reason the result page is requesting me to do a captcha while calling the exact same URL in my browser giving me the correct page without captcha.
What is the reason of this behaviour and how do I avoid it?
This is not a direct answer but a workaround
This website is protected by Cloudflare, which shows you recaptcha that only solvable in javascript environment. Obviously, HttpClient does not have such. While there are some solutions for this in other languages, I could not find any for C#. Will show an example in Selenium, web testing framework, that uses web browser driver (in my case Chrome).
using OpenQA.Selenium;
using OpenQA.Selenium.Chrome;
using OpenQA.Selenium.Support.UI;
using System;
class Program
{
public static void Main(string[] args)
{
using (var driver = new ChromeDriver())
{
driver.Url = "https://www.mediamarkt.de/de/product/_sony-playstation®5-2661938.html";
// selenium does not behave well when element you are looking for is not visible,
// this method helps us to close cookie banner that blocks the view
CloseCookieBannerIfAppears(driver);
var buyButton = By.XPath("//div[contains(#class, \"Badge\")]").FindElement(driver);
Console.WriteLine(buyButton.Text); // Ausverkauft
}
}
private static void CloseCookieBannerIfAppears(IWebDriver driver)
{
var buttonInAcceptCookieBannerSelector = By.XPath("//button[#id=\"privacy-layer-accept-all-button\"]");
var waitForCookieBanner = new WebDriverWait(driver, TimeSpan.FromSeconds(5));
if (waitForCookieBanner.Until(x => x.FindElements(buttonInAcceptCookieBannerSelector).Count > 0))
{
driver.FindElement(buttonInAcceptCookieBannerSelector)
.Click();
}
}
}
Also looks like they have unprotected API, so you should be able to get this data directly as well. You can see that there is id parameter both in your link and in api call - _sony-playstation®5-2661938.html vs productId=2661938
using Newtonsoft.Json.Linq;
using System;
using System.Net.Http;
using System.Threading.Tasks;
class Program
{
public static async Task Main(string[] args)
{
var httpClient = new HttpClient();
var response = await httpClient.GetAsync("https://delivery-prod-teasermanagement.cloud.mmst.eu/api/teaser/find?productId=2661938");
var content = await response.Content.ReadAsStringAsync();
var status = JArray.Parse(content)[0]["promotionData"]["badge"];
Console.WriteLine(status); // Ausverkauft
}
}
Maybe there are some other edge cases, but you should be able to get the point.
I have an existing and functioning API, which now have to be able to get data from another external API. How do i do that best?
I have tried with using HTTPClient, but i can't seem to get it to work. The error i get:
"No MediaTypeFormatter is available to read an object of type 'IList`1' from content with media type 'text/html'." -> I get this error on line 37. Can you spot it and/or tell me how I can do this differently, taking into account that all i want is the data (From the external API) and not to display it using a view, as this is an API?
Code below. I have also created a Pastebin: https://pastebin.com/MuKjEVys
using System;
using System.Collections.Generic;
using System.Linq;
using Microsoft.AspNetCore.Mvc;
using Newtonsoft.Json;
using System.Net.Http;
using System.Net.Http.Headers;
using System.Net;
namespace API.Controllers
{
[ApiVersion("1.0")]
[Route("api/v{version:apiVersion}/[controller]")]
[ApiController]
public class ExternalApiController : Controller
{
private string ExternalApiLink = "https://blablabla.com/api";
private string ExternalApiLinkGet = "/module/1/";
[HttpGet("getdata")]
public ActionResult<ExternalApi> GetDataFromExternal()
{
using (var client = new HttpClient())
{
client.BaseAddress = new Uri(ExternalApiLink);
var requestApi = client.GetAsync(ExternalApiLinkGet);
client.DefaultRequestHeaders.Authorization = new AuthenticationHeaderValue("Bearer", "XXXX");
requestApi.Wait();
var resultFromApi = requestApi.Result;
if (resultFromApi.IsSuccessStatusCode)
{
var readResponse = resultFromApi.Content.ReadAsAsync<IList<ExternalApi>>();
readResponse.Wait();
var data = readResponse.Result;
return Json(data);
}else
{
return NotFound();
}
}
}
}
}
Your response content seems to be json, while the content-type is text/html. If that is the case, the first thing to do would be to call the party that is exposing the service and have them fix it. In the meantime you could just read the content of the response as a string, and deserialize that string:
// Note that I made this method async.
public async Task<IActionResult> GetDataFromExternal()
{
using (var client = new HttpClient())
{
client.BaseAddress = new Uri(ExternalApiLink);
// note that I moved this line above the GetAsync method.
client.DefaultRequestHeaders.Authorization = new AuthenticationHeaderValue("Bearer", "XXXX");
// note that I'm disposing the response
using (var response = await client.GetAsync(ExternalApiLinkGet))
{
if (response.IsSuccessStatusCode)
{
// Since the response content is json, but the content-type
// is text/html, simply read the content as a string.
string content = await response.ReadAsStringAsync();
// You can return the actual received content like below,
// or you may deserialize the content before returning to make
// sure it is correct, using JsonConvert.DeserializeObject<List<ExternalApi>>()
// var data = JsonConvert.DeserializeObject<List<ExternalApi>>(content);
// return Json(data);
return Content(content, "application/json");
}
else
{
return NotFound();
}
}
}
}
I have created a web API in visual studio 2015 using a MySQL database. The API is working perfect.
So I decided to make a console client application in which I can consume my web-service (web API). The client code is based on HttpClient, and in the API I have used HttpResponse. Now when I run my console application code, I get nothing. Below is my code:
Class
class meters_info_dev
{
public int id { get; set; }
public string meter_msn { get; set; }
public string meter_kwh { get; set; }
}
This class is same as in my web API model class:
Model in web API
namespace WebServiceMySQL.Models
{
using System;
using System.Collections.Generic;
public partial class meters_info_dev
{
public int id { get; set; }
public string meter_msn { get; set; }
public string meter_kwh { get; set; }
}
Console application code
static HttpClient client = new HttpClient();
static void ShowAllProducts(meters_info_dev mi)
{
Console.WriteLine($"Meter Serial Number:{mi.meter_msn}\t Meter_kwh: {mi.meter_kwh}", "\n");
}
static async Task<List<meters_info_dev>> GetAllRecordsAsync(string path)
{
List<meters_info_dev> mID = new List<meters_info_dev>();
HttpResponseMessage response = await client.GetAsync(path);
if (response.IsSuccessStatusCode)
{
mID = await response.Content.ReadAsAsync<List<meters_info_dev>>();
}
else
{
Console.WriteLine("No Record Found");
}
return mID;
}
static void Main()
{
RunAsync().Wait();
}
static async Task RunAsync()
{
client.BaseAddress = new Uri("http://localhost:2813/");
client.DefaultRequestHeaders.Accept.Clear();
client.DefaultRequestHeaders.Accept.Add(new MediaTypeWithQualityHeaderValue("application/json"));
var m = await GetAllRecordsAsync("api/metersinfo/");
foreach(var b in m)
{
ShowAllProducts(b);
}
}
In my API I have 3 GET methods under a single controller, so I have created different routes for them. Also the URL for them is different.
http://localhost:2813/api/metersinfo/ will return all records
While debugging the code, I found that List<meters_info_dev> mID = new List<meters_info_dev>(); is empty:
While the response is 302 Found, the URL is also correct:
Update 1
After a suggestion I have done the following:
using (var client = new HttpClient())
{
List<meters_info_dev> mID = new List<meters_info_dev>();
HttpResponseMessage response = await client.GetAsync(path);
if (response.IsSuccessStatusCode)
{
mID = await response.Content.ReadAsAsync<List<meters_info_dev>>();
}
else
{
Console.WriteLine("No Record Found");
}
return mID;
}
When I run the application, I get the exception "An invalid request URI was provided. The request URI must either be an absolute URI or BaseAddress must be set."
Update 2
I have added a new piece of code:
using (var cl = new HttpClient())
{
var res = await cl.GetAsync("http://localhost:2813/api/metersinfo");
var resp = await res.Content.ReadAsStringAsync();
}
And in the response I am getting all the records:
I don't know why it's not working with the other logic and what the problem is. I have also read the questions Httpclient consume web api via console app C# and Consuming Api in Console Application.
Any help would be highly appreciated.
The code needs quite a bit of work.
The line you highlighted will always be empty because that's where you initialise the variable. What you want is run thorugh the code until you get the result back form the call.
First, make sure your api actually works, you can call the GET method you want in the browser and you see results.
using (var client = new HttpClient())
{
var result = await client.GetAsync("bla");
return await result.Content.ReadAsStringAsync();
}
that's an example of course, so replace that with your particular data and methods.
now, when you check the results just because your response.IsSuccessStatusCode is false that doesn't mean there are no records. What it means is that the call failed completely. Success result with an empty list is not the same thing as complete failure.
If you want to see what you get back you can alter your code a little bit:
if(response.IsSuccessStatusCode)
{
var responseData = await response.Content.ReadAsStringAsync();
//more stuff
}
put a breakpoint on this line and see what you actually get back, then you worry about casting the result to your list of objects. Just make sure you get back the same thing you get when you test the call in the browser.
<------------------------------->
More details after edit.
Why don't you simplify your code a little bit.
for example just set the URL of the request in one go :
using (var client = new HttpClient())
{
var result = await client.GetAsync("http://localhost:2813/api/metersinfo");
var response = await result.Content.ReadAsStringAsync();
//set debug point here and check to see if you get the correct data in the response object
}
Your first order of the day is to see if you can hit the url and get the data.
You can worry about the base address once you get a correct response. Start simple and work your way up from there, once you have a working sample.
<----------------- new edit ---------------->
Ok, now that you are getting a response back, you can serialise the string back to the list of objects using something like Newtonsoft.Json. This is a NuGet package, you might either have it already installed, if not just add it.
Add a using statement at the top of the file.
using Newtonsoft.Json;
then your code becomes something like :
using (var client = new HttpClient())
{
var result = await client.GetAsync("bla");
var response = await result.Content.ReadAsStringAsync();
var mID = JsonConvert.DeserializeObject<List<meters_info_dev>>(response);
}
At this point you should have your list of objects and you can do whatever else you need.
My team maintains a tool that is responsible for doing rapid verification of over 1000 different client websites. The tool is a Windows Service (.NET 4.5.2, C#) that reads requests from a queue, and executes a 'health check' for each request. It usually handles over 500 requests a minute, but can be responsible for more. Each request takes a second or two to execute.
A request contains a Uri and credentials needed for doing the health check. A health check is a POST against the AUTH page with the credentials (the app has custom auth, it's not header based auth), and then a GET to the home page, with a quick verification that it's the home page we expect. It then goes to a status page in the application, and does some quick checks against that. The GET requests have to use the cookies from the Set-Cookie header in the auth post.
We've been having performance problems with the tool as it scales. It currently creates a new HttpWebRequest object for each post and get in the process. There is a shared CookieContainer that is populated by the first post, so that we can get to the home page and then the status page.
I want to change this service to use the HttpClient object available in .NET 4.5. The problem is everywhere I read online says you want to avoid rapid creation and destruction of HttpClients. You'd rather keep one instance alive for the lifetime of the application. The problem I have is that HttpClient seems to work really well against one endpoint, not many.
I have looked into several options, and am not sure which is best to proceed:
Create a new HttpClient for each request, and use it for the duration of that request. That means it will live for a couple seconds, and be used for 3 calls. This would not be easy to implement, but I'm concerned about the overhead of creating and destroying hundreds of HttpClients a minute.
Figure out if it's possible to use one HttpClient instance for different endpoints by avoiding usage of a BaseAddress, and using the client to pass HttpRequestMessages using SendAsync. I haven't been able to figure out cookies with this method yet. To avoid having the HttpClient store the cookies, I set UseCookies to false in the HttpClientHandler, and tried managing cookies via headers in the HttpRequest/ResponseMessages themselves, but it looks like HttpClient simply strips cookies when UseCookies is set to false, so I was unable to pass cookies between request. edit: cookies work fine because they are stored per domain.
Store several hundred different HttpClient instances in some sort of dictionary, and pull the appropriate one for each Uri as the requests come in. I'm not sure about the memory overhead on this though. Also each unique Uri is only verified once every 5 minutes, so I'm not sure if having an HttpClient used once every 5 minutes keeps an unnecessary number of ports open.
Keep using HttpWebRequests. Maybe this older method still performs better in this situation.
If anyone has faced a similar issue, I'd love some input on where to proceed on this.
Thanks!
The problem with creating new HttpClients for each request is that HttpClientHandler will close the underlying TCP/IP connection. However, if you are using each HttpClient for the 3 requests to one host and then hitting a different host, then keeping the connection open doesn't help when you move to a new host. So, you probably will not see perf problem with one client per host. HttpClient itself is a very lightweight object. It isn't going to cost much to create one.
However, HttpClient simply delegates the real work to HttpClientHandler which uses HttpWebRequest under the covers, therefore will be unlikely to have any better performance than directly using HttpWebRequest.
If you are looking for better performance, then I suggest looking into replacing HttpClientHandler with the new WinHttpHandler which bypasses HttpWebRequest and goes directly to the Win32 API to make calls.
The full source is available for WinHttpHandler on GitHub so you can see exactly how it handles cookies and credentials.
And I would be really interested to hear if you do get much better perf with WinHttpHandler.
To start with, what part of this would you need modified to suit your needs?
var urisToCheck = new List<Uri>(); //get these somehow
//basic auth work?
var credentials = new NetworkCredential("user", "pass");
var handler = new HttpClientHandler { Credentials = credentials };
var client = new HttpClient(handler);
Parallel.ForEach(urisToCheck,
async uri =>
{
var response = await client.GetAsync(uri.AbsoluteUri);
//check for whatever you want here
}
);
here's my basic API client that uses the same HttpClient object for every request.
using System;
using System.Net;
using System.Net.Http;
using System.Net.Http.Headers;
using System.Text;
using System.Threading.Tasks;
//You need to install package Newtonsoft.Json > https://www.nuget.org/packages/Newtonsoft.Json/
using Newtonsoft.Json;
using Newtonsoft.Json.Serialization;
public class MyApiClient : IDisposable
{
private readonly TimeSpan _timeout;
private HttpClient _httpClient;
private HttpClientHandler _httpClientHandler;
private readonly string _baseUrl;
private const string ClientUserAgent = "my-api-client-v1";
private const string MediaTypeJson = "application/json";
public MyApiClient(string baseUrl, TimeSpan? timeout = null)
{
_baseUrl = NormalizeBaseUrl(baseUrl);
_timeout = timeout ?? TimeSpan.FromSeconds(90);
}
public async Task<string> PostAsync(string url, object input)
{
EnsureHttpClientCreated();
using (var requestContent = new StringContent(ConvertToJsonString(input), Encoding.UTF8, MediaTypeJson))
{
using (var response = await _httpClient.PostAsync(url, requestContent))
{
response.EnsureSuccessStatusCode();
return await response.Content.ReadAsStringAsync();
}
}
}
public async Task<TResult> PostAsync<TResult>(string url, object input) where TResult : class, new()
{
var strResponse = await PostAsync(url, input);
return JsonConvert.DeserializeObject<TResult>(strResponse, new JsonSerializerSettings
{
ContractResolver = new CamelCasePropertyNamesContractResolver()
});
}
public async Task<TResult> GetAsync<TResult>(string url) where TResult : class, new()
{
var strResponse = await GetAsync(url);
return JsonConvert.DeserializeObject<TResult>(strResponse, new JsonSerializerSettings
{
ContractResolver = new CamelCasePropertyNamesContractResolver()
});
}
public async Task<string> GetAsync(string url)
{
EnsureHttpClientCreated();
using (var response = await _httpClient.GetAsync(url))
{
response.EnsureSuccessStatusCode();
return await response.Content.ReadAsStringAsync();
}
}
public async Task<string> PutAsync(string url, object input)
{
return await PutAsync(url, new StringContent(JsonConvert.SerializeObject(input), Encoding.UTF8, MediaTypeJson));
}
public async Task<string> PutAsync(string url, HttpContent content)
{
EnsureHttpClientCreated();
using (var response = await _httpClient.PutAsync(url, content))
{
response.EnsureSuccessStatusCode();
return await response.Content.ReadAsStringAsync();
}
}
public async Task<string> DeleteAsync(string url)
{
EnsureHttpClientCreated();
using (var response = await _httpClient.DeleteAsync(url))
{
response.EnsureSuccessStatusCode();
return await response.Content.ReadAsStringAsync();
}
}
public void Dispose()
{
_httpClientHandler?.Dispose();
_httpClient?.Dispose();
}
private void CreateHttpClient()
{
_httpClientHandler = new HttpClientHandler
{
AutomaticDecompression = DecompressionMethods.Deflate | DecompressionMethods.GZip
};
_httpClient = new HttpClient(_httpClientHandler, false)
{
Timeout = _timeout
};
_httpClient.DefaultRequestHeaders.UserAgent.ParseAdd(ClientUserAgent);
if (!string.IsNullOrWhiteSpace(_baseUrl))
{
_httpClient.BaseAddress = new Uri(_baseUrl);
}
_httpClient.DefaultRequestHeaders.Accept.Add(new MediaTypeWithQualityHeaderValue(MediaTypeJson));
}
private void EnsureHttpClientCreated()
{
if (_httpClient == null)
{
CreateHttpClient();
}
}
private static string ConvertToJsonString(object obj)
{
if (obj == null)
{
return string.Empty;
}
return JsonConvert.SerializeObject(obj, new JsonSerializerSettings
{
ContractResolver = new CamelCasePropertyNamesContractResolver()
});
}
private static string NormalizeBaseUrl(string url)
{
return url.EndsWith("/") ? url : url + "/";
}
}
Usage
using ( var client = new MyApiClient("http://localhost:8080"))
{
var response = client.GetAsync("api/users/findByUsername?username=alper").Result;
var userResponse = client.GetAsync<MyUser>("api/users/findByUsername?username=alper").Result;
}
Register this object as singleton to your dependency injection library. It's safe to reuse because it's stateless.
Do NOT recreate HTTPClient for each request.
Reuse Httpclient as much as possible