Develop Reference

How to consume an API via OData protocol in .Net Core?

I'm trying to consume a public endpoint to get a list of objects in json. However, the API implements the OData protocol, and it always returns an invalid request. I think it's due to the way of calling this API. What should I adapt to my code to consume this API?
This is my code:
[HttpGet]
public async Task<List<TaxaSelic>> ObterTaxaSelic()
{
List<TaxaSelic> taxasSelic = new List<TaxaSelic>();
string apiResponse = string.Empty;
Uri uri = new Uri("https://api.bcb.gov.br/dados/serie/bcdata.sgs.4390/dados/ultimos/10/");
using (HttpClient httpClient = new HttpClient())
{
using (HttpResponseMessage response = await httpClient.GetAsync(uri))
{
if (response.IsSuccessStatusCode)
apiResponse = await response.Content.ReadAsStringAsync();
taxasSelic = JsonConvert.DeserializeObject<List<TaxaSelic>>(apiResponse);
}
}
return taxasSelic;
}
and...
public class TaxaSelic : Entity
{
[JsonProperty("data")]
public string Data { get; set; }
[JsonProperty("valor")]
public string Valor { get; set; }
}

The website's server in some way checks the caller's user-agent. When I set a user-agent as following, it works like a charm.
using (HttpClient httpClient = new HttpClient())
{
httpClient.DefaultRequestHeaders.UserAgent.ParseAdd("Mozilla/5.0 (Linux; Android 9; SM-G973U Build/PPR1.180610.011) " +
"AppleWebKit/537.36 (KHTML, like Gecko) Chrome/69.0.3497.100 Mobile " +
"Safari/537.36");
using (HttpResponseMessage response = await httpClient.GetAsync(uri))
{
... rest of your code
.
.
.
Example output I got,
HTTP/1.1 200 OK
ETag: W/"173-8/kblablablaMdY/QZ9jQ"
Access-Control-Allow-Origin: *
Date: Mon, 18 Jul 2022 11:48:19 GMT
Set-Cookie: cookie_p=!ptxb5qIxn7aCp0blablablablaTOUoak=; path=/; Httponly; Secure
Set-Cookie: TS01799025=01blablablacae675f; Path=/
Strict-Transport-Security: max-age=16070400; includeSubDomains
Content-Type: application/json; charset=utf-8
Content-Length: 371
[{"data":"01/10/2021","valor":"0.49"},{"data":"01/11/2021","valor":"0.59"},{"data":"01/12/2021","valor":"0.77"},{"data":"01/01/2022","valor":"0.73"},{"data":"01/02/2022","valor":"0.76"},{"data":"01/03/2022","valor":"0.93"},{"data":"01/04/2022","valor":"0.83"},{"data":"01/05/2022","valor":"1.03"},{"data":"01/06/2022","valor":"1.02"},{"data":"01/07/2022","valor":"0.54"}]

How to get verbose info of the HTTP request from HttpClient like those from libcurl?

I am writing a WinForms program (C#) using System.Net.Http.HttpClient to download webpages.
I'd like to get verbose info of the HTTP(S) requests like the following from libcurl:
* Trying xxx.xxx.xxx.xxx...
* TCP_NODELAY set
* Connected to xxx.xxx.xxx.xxx (xxx.xxx.xxx.xxx) port xxxx (#3)
* allocate connect buffer!
* Establish HTTP proxy tunnel to www.amazon.com:443
-> CONNECT www.amazon.com:443 HTTP/1.1
-> Host: www.amazon.com:443
-> User-Agent: libcurl/7.64.1 r-curl/4.3.2 httr/1.4.2
-> Proxy-Connection: Keep-Alive
->
<- HTTP/1.1 200 Connection established
<-
* Proxy replied 200 to CONNECT request
* CONNECT phase completed!
* CONNECT phase completed!
* CONNECT phase completed!
-> GET /xxxxxxx HTTP/1.1
-> Host: www.amazon.com
-> User-Agent: libcurl/7.64.1 r-curl/4.3.2 httr/1.4.2
-> Accept-Encoding: deflate, gzip
-> Cookie: session-id-time=xxxxxx; i18n-prefs=USD; session-id=xxxx; sp-cdn="L5Z9:SG"
-> Accept: application/json, text/xml, application/xml, */*
->
<- HTTP/1.1 200 OK
<- Server: Server
<- Content-Type: text/html;charset=UTF-8
<- x-amz-rid: xxxx
<- X-Content-Type-Options: nosniff
<- Expires: -1
<- X-XSS-Protection: 1;
<- Content-Language: en-US
<- Pragma: no-cache
<- Accept-CH-Lifetime: 86400
<- Cache-Control: no-cache
<- Content-Encoding: gzip
<- Accept-CH: ect,rtt,downlink
<- Strict-Transport-Security: max-age=47474747; includeSubDomains; preload
<- Vary: Content-Type,X-Amazon-Wtm-Tag-SP-Search-Secured-Port-Enabled,Accept-Encoding,X-Amzn-CDN-Cache,X-Amzn-AX-Treatment,User-Agent
<- X-Frame-Options: SAMEORIGIN
<- Permissions-Policy: interest-cohort=()
<- Date: Tue, 04 Jan 2022 05:06:02 GMT
<- Transfer-Encoding: chunked
<- Connection: keep-alive
<- Connection: Transfer-Encoding
* Added cookie ubid-main="xxxx" for domain amazon.com, path /, expire 1672808762
<- Set-Cookie: ubid-main=xxx; Domain=.amazon.com; Expires=Wed, 04-Jan-2023 05:06:02 GMT; Path=/; Secure
<-
* Connection #3 to host xxx.xxx.xxx.xxx left intact
How do I get this kind of message in real time? I am not looking for capturing exceptions.
I understand there are answers using the app.config to create logs, but this kind of solution is not useful here because the log file is locked while the program is running. So I can't really read the log file of the program and show the content in the same program.

From the moment you can't use the logs, a good approach in this case, would be to create a function which is going to print all the data that you need just like libcurl. You may not get the info in real time just like curl or the logging but you'll have the data available to be used in your code without reading another file.
There is also CurlSharp but it doesn't receive a commit since 2017, although it contains many features used in the official libcurl. Beside the commits, you would also need to update the .NET version and the code.
Let's say that you want to do a Get Request, you would do something like this:
HttpClient client = new HttpClient();
HttpResponseMessage response = await client.GetAsync("http://www.contoso.com/");
response.EnsureSuccessStatusCode();
With the HttpResponseMessage you can print many properties related to the request, in particular with the Headers property, you can use the HttpResponseHeaders class to print other properties such as the Connection, the Pragma or the Server. Here's an example:
try
{
HttpClient client = new HttpClient();
HttpResponseMessage response = await client.GetAsync("http://www.contoso.com/");
response.EnsureSuccessStatusCode();
//if you need it
//string responseBody = await response.Content.ReadAsStringAsync();
PrintHttpGetResponse(response);
}
catch (HttpRequestException e)
{
Console.WriteLine("\nException Caught!");
Console.WriteLine("Message :{0} ", e.Message);
}
void PrintHttpGetResponse(HttpResponseMessage response) {
Console.WriteLine($"Content Type ---> {response.Content.Headers.ContentType}");
Console.WriteLine($"Status Code ---> {response.StatusCode}");
Console.WriteLine($"Headers ---> {response.Headers}");
Console.WriteLine($"Connection ---> {response.Headers.Connection}");
Console.WriteLine($"IsSuccessStatusCode ---> {response.IsSuccessStatusCode}");
}
You could also use some kind of Model with all the properties that you need, to save the response content and share it with other layers or classes easily in your code. For example:
using System.Net;
using System.Net.Http.Headers;
public class ResponseContent {
MediaTypeHeaderValue ContentType { get; set; }
HttpStatusCode StatusCode { get; set; }
HttpResponseHeaders Headers { get; set; }
HttpHeaderValueCollection<string> Connection { get; set; }
bool IsSuccessStatusCode { get; set; }
}

Strict ordering of HTTP headers in HttpWebrequest

In spite of the RFC stating that the order of uniquely-named headers shouldn't matter, the website I'm sending this request to does implement a check on the order of headers.
This works:
GET https://www.thewebsite.com HTTP/1.1
Host: www.thewebsite.com
Connection: keep-alive
Accept: */*
User-Agent: Mozilla/5.0 etc
This doesn't work:
GET https://www.thewebsite.com HTTP/1.1
Accept: */*
User-Agent: Mozilla/5.0 etc
Host: www.thewebsite.com
Connection: keep-alive
The default HttpWebRequest seems to put the Host and Connection headers at the end, before the blank line, rather than just after the url.
Is there any way (using a fork of HttpWebRequest or some other library in Nuget even) to specify the order of headers in a HttpWebRequest?
If possible, I'd rather not start going down the route of implementing a proxy to sort them or having to code the whole thing up using a TcpClient.
I'd appreciate any hints at all on this.
Update: With Fiddler running, header order in HttpWebrequest can be re-shuffled in CustomRules.cs. Still no closer to a solution without a proxy though.

Some server implement header ordering as a precaution for any attacks or spam, an article explaining Why ordering HTTP headers is important.
But the standard is, the order in which header fields with differing field names are received is not significant.
HttpWebRequest, there is no easy way to order the headers and the Connection and Host is added internally.
If ordering is really important, use the HttpClient instead, it can easily arrange the Headers based on the example of #Jason.
If you will be using HttpClient, you can create a custom HttpClientHandler and you can arrange your header from there. It can be something like this.
HANDLER
public class CustomHttpClientHandler : HttpClientHandler
{
protected override async Task<HttpResponseMessage> SendAsync(HttpRequestMessage request, CancellationToken cancellationToken)
{
request.Headers.Clear();
request.Headers.Add("Host", $"{request.RequestUri.Authority}");
request.Headers.Add("Connection", "keep-alive");
request.Headers.Add("Accept", "*/*");
request.Headers.Add("User-Agent", "Mozilla/5.0 etc");
return await base.SendAsync(request, cancellationToken);
}
}
IMPLEMENTATION
HttpClient clientRequest = new HttpClient(new CustomHttpClientHandler());
await clientRequest.GetAsync(url);

.Net Core
If you set the headers yourself, you can specify the order. When the common headers are added it will find the existing headers instead of appending them:
using System.Net;
namespace ConsoleApp2
{
class Program
{
static void Main(string[] args)
{
var request = WebRequest.Create("http://www.google.com");
request.Headers.Add("Host", "www.google.com");
// this will be set within GetResponse.
request.Headers.Add("Connection", "");
request.Headers.Add("Accept", "*/*");
request.Headers.Add("User-Agent", "Mozilla/5.0 etc");
request.GetResponse();
}
}
}
Here is an example with HttpClient:
using System.Net.Http;
using System.Threading.Tasks;
namespace ConsoleApp3
{
class Program
{
static async Task Main(string[] args)
{
var client = new HttpClient();
client.DefaultRequestHeaders.Add("Host", "www.google.com");
client.DefaultRequestHeaders.Add("Connection", "keep-alive");
client.DefaultRequestHeaders.Add("Accept", "*/*");
client.DefaultRequestHeaders.Add("User-Agent", "Mozilla/5.0 etc");
await client.GetAsync("http://www.google.com");
await client.PostAsync("http://www.google.com", new StringContent(""));
}
}
}
Edit
The above code did not work on .Net Framework only .Net Core
.Net Framework
On .Net Framework the headers are reserved so they cannot be set like this, see Cannot set some HTTP headers when using System.Net.WebRequest.
One work around is to use reflection to modify the behavior of the framework class, but be warned this could break if the libraries are updated so it's not recommended!.
Essentially, HttpWebRequest calls ToString on WebHeaderCollection to serialize.
See https://referencesource.microsoft.com/#System/net/System/Net/HttpWebRequest.cs,5079
So a custom class can be made to override ToString. Unfortunately reflection is needed to set the headers as WebRequest copies the collection on assignment to Headers, instead of taking the new reference.
WARNING, THE FOLLOWING CODE CAN BREAK IF FRAMEWORK CHANGES
If you use this, write some unit tests that verify the behavior still stays consistent after updates to .NET Framework
using System.Collections.Generic;
using System.Linq;
using System.Net;
using System.Reflection;
namespace ConsoleApp2
{
class Program
{
static void Main(string[] args)
{
// WARNING, CODE CAN BREAK IF FRAMEWORK CHANGES
// If you use this, write some unit tests that verify the behavior still stays consistent after updates to .NET Framework
var request = (HttpWebRequest)WebRequest.Create("http://www.google.com");
var field = typeof(HttpWebRequest).GetField("_HttpRequestHeaders", BindingFlags.Instance | BindingFlags.NonPublic);
var headers = new CustomWebHeaderCollection(new Dictionary<string, string>
{
["Host"] = "www.google.com",
["Connection"] = "keep-alive",
["Accept"] = "*/*",
["User-Agent"] = "Mozilla/5.0 etc"
});
field.SetValue(request, headers);
request.GetResponse();
}
}
internal class CustomWebHeaderCollection : WebHeaderCollection
{
private readonly Dictionary<string, string> _customHeaders;
public CustomWebHeaderCollection(Dictionary<string, string> customHeaders)
{
_customHeaders = customHeaders;
}
public override string ToString()
{
// Could call base.ToString() split on Newline and sort as needed
var lines = _customHeaders
.Select(kvp => $"{kvp.Key}: {kvp.Value}")
// These two new lines are needed after the HTTP header
.Concat(new [] { string.Empty, string.Empty });
var headers = string.Join("\r\n", lines);
return headers;
}
}
}

WPF App and Neo4j Client always throws me a 404

Im using the neo4j Client v4.0.0.1 and want to create just a single node from a class but Im always getting the following 404 error.
Error Code:
System.Exception: 'Received an unexpected HTTP status when executing the request.
The response status was: 404 Not Found
The response from Neo4j (which might include useful detail!) was: '
If I use curl to check if all is fine:
curl -I localhost:7474
HTTP/1.1 200 OK
Date: Tue, 25 Feb 2020 14:23:18 GMT
Access-Control-Allow-Origin: *
Content-Type: application/json;charset=utf-8
Content-Length: 223
And:
curl -I http://localhost:7474/db/data
HTTP/1.1 302 Found
Date: Tue, 25 Feb 2020 14:23:26 GMT
Location: http://localhost:7474/db/data/
Content-Length: 0
all seems to be fine. Thats my try Id like to start with:
public class Person
{
public string Name { get; set; }
public int Id { get; set; }
public int Age { get; set; }
}
private void createNode(){
var client = new GraphClient(new Uri("http://localhost:7474/db/data/"), "neo4j", "neo4j");
client.Connect();
var results = client.Cypher
.Create("(n:Person)")
.Return((a) => new {
Person = a.As<Person>()
}).Results;
}

I recommend using Neo4j driver. I'm not sure that neo4j client .net library has support with Neo4j 4.0. Also, you can return to 3.5 version

API - HttpResponseMessage: (407) proxy authentication required

Edit 1: Other Controller
public class identityController : ApiController
{
[HttpGet]
public async Task<IHttpActionResult> getfullname(string firstName)
{
string name = firstName;
return Ok(name);
}
}
I have created a controller which uses an API from another solution.
Method that i use in the controller looks like below:
public class GetNameController : ApiController
{
[HttpGet]
public async Task<IHttpActionResult> CalculatePrice(string firstName)
{
string _apiUrl = String.Format("api/identity/getfullname?firstName={0}", firstName);
string _baseAddress = "http://testApp.azurewebsites.net/";
using (var client = new HttpClient())
{
client.BaseAddress = new Uri(_baseAddress);
client.DefaultRequestHeaders.Accept.Clear();
client.DefaultRequestHeaders.Accept.Add(new MediaTypeWithQualityHeaderValue("application/json"));
HttpResponseMessage response = await client.GetAsync(_apiUrl);
if (response.IsSuccessStatusCode)
{
return Ok(response);
}
}
return NotFound();
}
The result of response.IsSuccessStatusCode is always false. When i check the response values i see this result:
{
StatusCode: 400, ReasonPhrase: 'Bad Request', Version: 1.1, Content: System.Net.Http.StreamContent, Headers:
{
Connection: close
Date: Thu, 21 Jul 2016 12:28:21 GMT
Server: Microsoft-HTTPAPI/2.0
Content-Length: 334
Content-Type: text/html; charset=us-ascii
}
}
What could i be missing?

string _apiUrl = String.Format("api/identity/{0}", firstName);
This is assuming that your url is correct, and your testapp is up and running. Even though when I hit it azure tells me your app is stopped. You will need to get your app started first, then change the string _apiUrl to the suggestion above.
http://testapp.azurewebsites.net/api/identity/getfullname?firstName=steve
Gives me this message
Error 403 - This web app is stopped.
The web app you have attempted to reach is currently stopped and does
not accept any requests. Please try to reload the page or visit it
again soon.
If you are the web app administrator, please find the common 403 error
scenarios and resolution here. For further troubleshooting tools and
recommendations, please visit Azure Portal.
So there are several things in your identity controller that are going on.
the functions name is getFullName. Since the word get is in the name of the function. Any httpget request will be routed to the function automagically. Thus making the [HttpGet] redundant. This only works if there is 1 and only 1 httpget request in your controller. If there are multiple you will need to fully qualify the url like you have done
Since youa re using the [httpget] method attribute I can assume you are using webapi2. That being the case and you are using a
primitive in your controller argument you can do notneed to fully
qualify the parameter name on your call. ?firstname={0} changes to
/{0}