Develop Reference

How to fix Unable to send GET request - 403 error

I am new to the C# world, and can't for the life of me figure out how to get around this error. I am trying to make a simple get request using a platform API key. I have built out the API connection in Google App Script on the same laptop, and it works fine, but when trying to build out the same API in C#, it is returning:
{StatusCode: 403, ReasonPhrase: 'Forbidden', Version: 1.1, Content: System.Net.Http.HttpConnectionResponseContent, Headers:
{
Date: Sun, 13 Mar 2022 02:41:29 GMT
Transfer-Encoding: chunked
Connection: close
CF-Chl-Bypass: 1
Permissions-Policy: accelerometer=(),autoplay=(),camera=(),clipboard-read=(),clipboard-write=(),fullscreen=(),geolocation=(),gyroscope=(),hid=(),interest-cohort=(),magnetometer=(),microphone=(),payment=(),publickey-credentials-get=(),screen-wake-lock=(),serial=(),sync-xhr=(),usb=()
Cache-Control: private, max-age=0, no-store, no-cache, must-revalidate, post-check=0, pre-check=0
X-Frame-Options: SAMEORIGIN
Expect-CT: max-age=604800, report-uri="https://report-uri.cloudflare.com/cdn-cgi/beacon/expect-ct"
Strict-Transport-Security: max-age=2592000
Server: cloudflare
CF-RAY: 6eb1692f8bd776c3-LHR
Content-Type: text/html; charset=utf-8
Expires: Thu, 01 Jan 1970 00:00:01 GMT
}}
The API documentation says:
"To authenticate against the API, include your API key in the 'Authorization' header, prefixed with 'Key ', in every request. Example: 'Authorization: Key yourapikey'"
And so, I have tried adding this to
a) the HttpClient via HttpClient.DefaultRequestHeaders.Authorization
b) the HttpClient via HttpClient.DefaultHeaders.Add
c) the HttpRequestMessage via HttpRequestMessage.Headers.Add
In each instance, the request URI looks good, as well as the headers, but still returning 403.
My current structure is:
// services
builder.Services.AddHttpClient("myplatform", c =>
{
c.BaseAddress = new Uri("https://seller-api.myplatform.com/v2/");
c.DefaultRequestHeaders.Accept.Add(new MediaTypeWithQualityHeaderValue("application/json"));
});
// controller
// constructor uses IHttpClientFactory
this._httpClient = clientFactory.CreateClient("myplatform");
// service
httpClient.DefaultRequestHeaders.Authorization = new AuthenticationHeaderValue("Key", platformKey);
string endPoint = "offers" + CreateQueryString(pageNumber, pageSize);
// example endPoint: offers?page_number=1&page_size=100
var requestMsg = new HttpRequestMessage()
{
Method = HttpMethod.Get,
RequestUri = new Uri(httpClient.BaseAddress + endPoint)
};
try
{
var result = await httpClient.SendAsync(requestMsg);
}
catch (Exception ex)
{
}
Again, the content of the call works when calling through GoogleAppScript. What am I doing wrong in C#, and how can I correct this?
Not sure if this is all the information needed - let me know otherwise! Important to note, the target framework is .NET 6.0.
EDIT
As suggested by a comment around duplicate clients, I have removed the client factory. I am now creating a new HttpClient in the controller constructor, and passing this client to my service to do the GET request.
this._httpClient = new HttpClient();
Again, the client and the request message look well formed at time of request but still returning 403 error. Is there an issue with my VS22 client, or web client etc.?
Also, the call I am making successfully via Google AppScript is using UrlFetchApp. Not sure what is the issue here with the C# side..
EDIT2
Adding current GAS code for reference:
var url = 'https://seller-api.platform.com/v2';
var end_point = '/offers?';
var header = {
'Authorization': api_key
}
var params = {
'method': 'GET',
'headers': header
}
// call API
var page_query = 'page_number=' + page + '&page_size=' + maxItemsPerPage;
var full_url = url + end_point + page_query;
var response = UrlFetchApp.fetch(full_url, params);

The HTTP 403 Forbidden response status code indicates that the server understands the request but refuses to authorize it.
There are two ways add request headers when using HttpClient:
Add headers for all requests using HttpClient.DefaultRequestHeaders.
HttpClient = new HttpClient();
HttpClient.DefaultRequestHeaders.Add("Key", platformKey);
var response = await HttpClient.GetAsync(GetRandomNumberUrl);
response.EnsureSuccessStatusCode();
await response.Content.ReadAsStringAsync();
Add headers per request using HttpRequestMessage.Headers.
HttpClient = new HttpClient();
using (var request = new HttpRequestMessage(HttpMethod.Get, randomNumberUrl))
{
request.Headers.Authorization = new AuthenticationHeaderValue("Bearer", platformKey);
var response = await HttpClient.SendAsync(request);
response.EnsureSuccessStatusCode();
await response.Content.ReadAsStringAsync();
}
Your problem is here in this line:
httpClient.DefaultRequestHeaders.Authorization = new AuthenticationHeaderValue("Key", platformKey);
your _httpClient and Authorization httpClient is not same instance.
Try this way:
HttpClient HttpClient = new HttpClient();
string url = "https://seller-api.platform.com/v2";
string end_point = "/offers?";
string api_key = "key here";
string page_query = "page_number=" + 10 + "&page_size=" + 20;
string full_url = url + end_point + page_query;
using (var request = new HttpRequestMessage(HttpMethod.Get, url))
{
request.Headers.Add("Authorization", api_key);
var response = await HttpClient.SendAsync(request);
response.EnsureSuccessStatusCode();
var m = await response.Content.ReadAsStringAsync();
}
Solution 2:
Try calling it like a browser :
HttpClient httpClient = new HttpClient();
string url = "https://gatewayt.whatever.com/chkt/request/request.php";
string end_point = "/offers?";
string api_key = "key here";
string page_query = "page_number=" + 10 + "&page_size=" + 20;
string full_url = url + end_point + page_query;
httpClient.DefaultRequestHeaders.TryAddWithoutValidation("Accept", "text/html,application/xhtml+xml,application/xml");
httpClient.DefaultRequestHeaders.TryAddWithoutValidation("Accept-Encoding", "gzip, deflate");
httpClient.DefaultRequestHeaders.TryAddWithoutValidation("User-Agent", "Mozilla/5.0 (Windows NT 6.2; WOW64; rv:19.0) Gecko/20100101 Firefox/19.0");
httpClient.DefaultRequestHeaders.TryAddWithoutValidation("Accept-Charset", "ISO-8859-1");
httpClient.DefaultRequestHeaders.Authorization = new AuthenticationHeaderValue("Key", api_key);
var response = await httpClient.GetAsync(url);
response.EnsureSuccessStatusCode();
await response.Content.ReadAsStringAsync();
source: Request like browser Link1 Link2 Link3

Difficulty with SendAsync HttpPost with header and body

Having difficulty getting a 200 response back from SendAsync for an HttpPost. This is for a multi-phase authentication using a basic authentication token to get a bearer token which is then used to get an accessKeyId and accessKeySecret which is then used to get a usable accessToken.
This is the format that is expected (provided by the vendor). I believe the code I have written handles the Header data correctly, but I am not sure about the body. When testing with Restlet or Postman the body must be entered exactly as shown. I think when the quoted tag and value are placed in C# the quotes are being lost in translation; see code below this section.
*************************From the Vendor*****************************
With a bearer token an access key can now be created using endpoint
https://xxxxxxxxxxxx/x-keys`
a.POST Request:
i. Header:
1. Authorization: bearer eyJ0eXAiOiJKV1QiLCJhbGciOiJSUzI1NiJ9….
2. Content-Type: application/json
ii. Body:
1. {
2. "userId": "xxxxxxxxx-xxxxx-xxxx-xxxxx-xxxxxxxxxxxx",
3. "tenantId": "xxxxxxxxx-xxxxx-xxxx-xxxxx-xxxxxxxxxxxx"
4. }
>String newBearerToken = "bearer " + bearerToken;
>var cl = new HttpClient();
>cl.DefaultRequestHeaders.TryAddWithoutValidation("Content-Type", "application/json; charset=utf-8");
>cl.DefaultRequestHeaders.Authorization = new AuthenticationHeaderValue("Authorization", newBearerToken);
>var nvc = new List<KeyValuePair<string, string>>();
>nvc.Add(new KeyValuePair<string, string>("\"" + "userId" + "\"", "\"" + "xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx" + "\""));
>nvc.Add(new KeyValuePair<string, string>("\"" + "tenantId" + "\"", "\"" + "xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx" + "\""));
>var req = new HttpRequestMessage(HttpMethod.Post, "https://xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx/-keys") { Content = new FormUrlEncodedContent(nvc) };
>var result = await cl.SendAsync(req);
I expect a response body with new access tokens and an HttpResponse of 200 but this is what is returned:
"result {StatusCode: 302, ReasonPhrase: '', Version: 1.1, Content:
System.Net.Http.StreamContent, Headers: { Content-Security-Policy:
default-src https: data: 'unsafe-inline' 'unsafe-eval';
connect-src...."

The authentication header should be the following...
new AuthenticationHeaderValue("Bearer", newBearerToken);
not "Authorization" if you're using bearer tokens
Your current implementation is doing the following
Authorization: Authorization bearer eyJ0eXAiOiJKV1QiLCJhbGciOiJSUzI1NiJ9
Remove the bearer from the token too
String newBearerToken = bearerToken;

Batch request - Dynamics CRM

All,
I am trying to implement a batch request to Dynamics CRM with the following source code:
public async Task<HttpResponseMessage> HttpPatchCrmApi(string resource, string data)
{
string uniq = Guid.NewGuid().ToString();
MultipartContent content = new MultipartContent("mixed", "batch_" + uniq);
HttpRequestMessage batchRequest = new HttpRequestMessage(HttpMethod.Post, CrmBaseUrl + "/api/data/v8.0/$batch");
HttpRequestMessage request = new HttpRequestMessage(HttpMethod.Post, CrmBaseUrl + resource);
request.Content = new StringContent(data, Encoding.UTF8, "application/json");
HttpMessageContent query = new HttpMessageContent(request);
content.Add(query);
batchRequest.Content = content;
HttpResponseMessage response = await RbWebApi.SendAsync(batchRequest);
return response;
}
The problem is that I am getting "400 Bad request"
EDIT:
As suggested in the comments here is the stack trace of the request from fiddler:
POST https://Hidden.api.crm4.dynamics.com/api/data/v8.0/$batch HTTP/1.1
Authorization: Bearer eyJ0eXAiOiJKV.... very long string
Content-Type: multipart/mixed; boundary="batch_7b6e3c60-1284-4958-a39a-4653af21833c"
Host: Hidden.api.crm4.dynamics.com
Content-Length: 313
Expect: 100-continue
--batch_7b6e3c60-1284-4958-a39a-4653af21833c
Content-Type: application/http; msgtype=request
POST /api/data/v8.0/my_recurringgifts HTTP/1.1
Host: Hidden.api.crm4.dynamics.com
Content-Type: application/json; charset=utf-8
{"my_name":"slavi"}
--batch_7b6e3c60-1284-4958-a39a-4653af21833c--
While writing the code I was inspiring myself from here and here

I think your request is wrong.
You must build the request Body EXACTLY like defined by Microsoft
This means the Blank lines must be there at the right place all the attributes must exist in the body (like "--changeset_XXX" for example) and as I see you dont meet this requirements.
I just build a Request in Postman against my CRM and it worked:
URL
https://yourTenant.api.crm.dynamics.com/api/data/v8.0/$batch
Headers
OData-MaxVersion:4.0
OData-Version:4.0
Accept:application/json
Authorization:Bearer aVeryLongStringTokenHere
Content-Type: multipart/mixed;boundary=batch_1234567
Body
--batch_1234567
Content-Type:multipart/mixed;boundary=changeset_555666
--changeset_555666
Content-Type:application/http
Content-Transfer-Encoding:binary
Content-ID:1
POST https://yourTenant.api.crm.dynamics.com/api/data/v8.0/accounts HTTP/1.1
Content-Type:application/json;type=entry
{name: 'BatchJobTest788'}
--changeset_555666
Content-Type:application/http
Content-Transfer-Encoding:binary
Content-ID:2
POST https://yourTenant.api.crm.dynamics.com/api/data/v8.0/accounts HTTP/1.1
Content-Type:application/json;type=entry
{new_name: 'BatchJobTest348'}
--changeset_555666--
--batch_1234567--
Additional Remarks:
The Content-Type of your Header holds your BatchId
The Content-Type of your Batch holds your ChangesetId (if it is a change to data)
Before starting to programm REST calls try to define them in a REST tool like POSTMAN and make them work. Then build the working request in your code.
Here a good explanation-source for the batching in CRM

Curl request written in C# doesn't work

I'm trying to write a specific curl request in C#, and I keep getting a 500 server error response from the server. This curl request essentially makes a post request to an API by the company Highwinds. This request sends json data, and sets the Auth Bearer token header.
This is the curl request that works fine (note that I've replaced my actual bearer token with {token} and my actual account id with {accountId} to obfuscate that info):
curl -H "Authorization: Bearer {token}" -H "Content-Type: application/json" -d "#data.json" "https://striketracker.highwinds.com/api/accounts/{accountId}/purge"
Here's the C# code that gives me a generic 500 server error from the Highwinds API (note that I've replaced my actual bearer token with {token}, my actual account id with {accountId}, and the url in the json string with {url}, in order to obfuscate that personal info):
var accountId = "{accountId}";
var purgeURI = string.Format("https://striketracker.highwinds.com/api/accounts/{0}/purge", {accountId});
var query =
#"{""list"": [{""url"": ""{url}"",""recursive"": true}]}";
var token = {token};
using (var httpClient = new HttpClient())
{
var url = new Uri(purgeURI);
using (var httpRequestMessage = new HttpRequestMessage(HttpMethod.Post, url))
{
httpRequestMessage.Headers.Add(System.Net.HttpRequestHeader.Authorization.ToString(),
string.Format("Bearer {0}", token));
httpRequestMessage.Content = new StringContent(query,
Encoding.UTF8,
"application/json");
await httpClient.SendAsync(httpRequestMessage).ContinueWith(task =>
{
var response = task.Result;
var blah = response.Content.ReadAsStringAsync().Result;
Console.WriteLine(response.Content.ReadAsStringAsync().Result);
});
}
}
Thanks!
*Update: The following line of code was added to remove the Expect header that HttpRequest adds to a request by default. After removing this header I was able to get Highwinds API to accept the request without bombing.
"request.ServicePoint.Expect100Continue = false;"

My best recommendation would be to proxy both requests through something like tcpmon http://archive.apache.org/dist/ws/tcpmon/1.0/ (Basically run the server and point to local host and have tcpmon redirect the request to striketracker.highwinds.com). Try it from curl and from your source and you should be able to see what's different between the requests.

How to use OAuth2 in RestSharp

After a couple of days sorting out OAuth2 at the server-end (Spring java) I started working on the client written in C#. I am using RestSharp to call my web API but I am having real difficulty with the OAuth2. There is hardly any documentation and the few examples I found online do not work. Can someone provide me a code sample that is up to date and that I can use?
So far I have the following:
var client = new RestClient("http://example.com/myapi/oauth/token");
RestRequest request = new RestRequest() { Method = Method.POST };
request.AddHeader("Content-Type", "application/json");
request.AddParameter("grant_type", "client_credentials");
request.AddParameter("client_id", "client-app");
request.AddParameter("client_secret", "secret");
var response = client.Execute(request);
I am simply running this code in debug mode and when I look into the response I get unauthorized.
When I do curl on the console with the same parameters it works fine but it seems I can't make this to work in C#. Here is the curl command:
curl -H "Accept: application/json" client-app:secret#example.com/myapi/oauth/token -d grant_type=client_credentials
By the way, I have replaced my true API urls and other information with placeholders.

See RFC 6749 - 4.4.2. Client Credentials - Access Token Request
Here is the basic format of the request
POST /token HTTP/1.1
Host: server.example.com
Authorization: Basic czZCaGRSa3F0MzpnWDFmQmF0M2JW
Content-Type: application/x-www-form-urlencoded
grant_type=client_credentials
Your cURL request
curl -H "Accept: application/json" \
-d grant_type=client_credentials \
client-app:secret#example.com/myapi/oauth/token
The reason your cURL command works
Default Content-Type (if not specified) with POST (default when you use -d switch) is application/x-www-form-urlencoded
Default authentication type, if not specified, is Basic. The username and password are passed either through the -u option or in the URL
-u username:password (client-app:secret)
-- or put it in the url --
client-app:secret#example.com/myapi/oauth/token
You could also specify the auth type with --basic or --digest
You can use the -v switch in your cURL command to see all the headers involved in the request.
RestSharp fix:
Set the Content-Type to application/x-www-form-urlencoded
Add the Basic authentication
client.Authenticator = new HttpBasicAuthenticator("client-app", "secret");
Get rid of
request.AddParameter("client_id", "client-app");
request.AddParameter("client_secret", "secret");
Set the Accept header to application/json

I am able to get both of the following functions worked.
public RestClient getClient2(string user, string token)
{
RestClient client = new RestClient();
client.BaseUrl = new Uri(baseUrl);
client.Authenticator = new HttpBasicAuthenticator(user, token);
//client.Authenticator = new OAuth2UriQueryParameterAuthenticator(token); //works
//client.Authenticator = new OAuth2AuthorizationRequestHeaderAuthenticator(token); // doesn't work
return client;
}
public GitHubUser GetGitHubUser2()
{
RestRequest request = new RestRequest();
request.Resource = "/users/huj";
request.RootElement = "GitHubUser";
RestClient client = getClient2(myUser, myToken);
return Execute<GitHubUser>(client, request);
}
/// <summary>
/// http://stackoverflow.com/questions/30133937/how-to-use-oauth2-in-restsharp
/// </summary>
/// <returns>GitHubUser</returns>
public GitHubUser GetGitHubUser3()
{
//RestRequest request = new RestRequest(Method.POST); //empty data
RestRequest request = new RestRequest();
request.AddHeader("Content-Type", "application/x-www-form-urlencoded");
request.AddHeader("Accept", "application/json");
request.AddParameter("grant_type", "client_credentials");
request.Resource = "/users/huj";
request.RootElement = "GitHubUser";
RestClient client = getClient2(myUser, myToken);
return Execute<GitHubUser>(client, request);
}