Mocking HttpMessageHandler depending on the Request Url c# - c#

I have the following method to create a Http client with a mocked response:
public static HttpClient GetMockedHttpClient(string responseContent, HttpStatusCode httpStatusCode)
{
var handlerMock = new Mock<HttpMessageHandler>(MockBehavior.Strict);
handlerMock
.Protected()
.Setup<Task<HttpResponseMessage>>(
"SendAsync",
ItExpr.IsAny<HttpRequestMessage>(),
ItExpr.IsAny<CancellationToken>()
)
.ReturnsAsync(new HttpResponseMessage()
{
StatusCode = httpStatusCode,
Content = new StringContent(responseContent),
})
.Verifiable();
return new HttpClient(handlerMock.Object);
}
How can I make the response dependent on a specific url, so that the setup only takes effect when the HttpClient is called with a given address?

What you need to change is the line:
ItExpr.IsAny<HttpRequestMessage>(),
IsAny<HttpRequestMessage>() means the mock will catch any property of that type, if you want it to catch a specific uri you use Is<HttpRequestMessage>()
ItExpr.Is<HttpRequestMessage>(m => m.RequestUri == [your uri goes here]),

I would suggest using https://github.com/justeat/httpclient-interception.
I had to run similar tests and it did the job, although I used it to mock whole client.

Related

Polly: Retry request with StreamContent and MemoryStream - no content on second try

I'm using Polly in combination with Microsoft.Extensions.Http.Polly to handle communication with an external API which has rate-limiting (N requests / second).I'm also using .NET 6.
The policy itself works fine for most requests, however it doesn't work properly for sending (stream) data. The API Client requires the usage of MemoryStream. When the Polly policy handles the requests and retries it, the stream data is not sent.
I verified this behavior stems from .NET itself with this minimal example:
using var fileStream = File.OpenRead(#"C:\myfile.pdf");
using var memoryStream = new MemoryStream();
await fileStream.CopyToAsync(memoryStream);
var response = await httpClient.SendAsync(
new HttpRequestMessage
{
// The endpoint will fail the request on the first request
RequestUri = new Uri("https://localhost:7186/api/test"),
Content = new StreamContent(memoryStream),
Method = HttpMethod.Post
}
);
Inspecting the request I see that Request.ContentLength is the length of the file on the first try. On the second try it's 0.
However if I change the example to use the FileStream directly it works:
using var fileStream = File.OpenRead(#"C:\myfile.pdf");
var response = await httpClient.SendAsync(
new HttpRequestMessage
{
// The endpoint will fail the request on the first request
RequestUri = new Uri("https://localhost:7186/api/test"),
Content = new StreamContent(fileStream ),
Method = HttpMethod.Post
}
);
And this is my Polly policy that I add to the chain of AddHttpClient.
public static IAsyncPolicy<HttpResponseMessage> GetRetryPolicy()
{
return Policy
.HandleResult<HttpResponseMessage>(response =>
{
return response.StatusCode == System.Net.HttpStatusCode.Forbidden;
})
.WaitAndRetryAsync(4, (retry) => TimeSpan.FromSeconds(1));
}
My question:
How do I properly retry requests where StreamContent with a stream of type MemoryStream is involved, similar to the behavior of FileStream?
Edit for clarification:
I'm using an external API Client library (Egnyte) which accepts an instance of HttpClient
public class EgnyteClient {
public EgnyteClient(string apiKey, string domain, HttpClient? httpClient = null){
...
}
}
I pass an instance which I injected via the HttpContextFactory pattern. This instance uses the retry policy from above.
This is my method for writing a file using EgnyteClient
public async Task UploadFile(string path, MemoryStream stream){
// _egnyteClient is assigned in the constructor
await _egnyteClient.Files.CreateOrUpdateFile(path, stream);
}
This method call works (doesn't throw an exception) even when the API sometimes returns a 403 statucode because the internal HttpClient uses the Polly retry policy. HOWEVER the data isn't always properly transferred since it just works if it was the first attempt.
The root cause of your problem could be the following: once you have sent out a request then the MemoryStream's Position is at the end of the stream. So, any further requests needs to rewind the stream to be able to copy it again into the StreamContent (memoryStream.Position = 0;).
Here is how you can do that with retry:
private StreamContent GetContent(MemoryStream ms)
{
ms.Position = 0;
return new StreamContent(ms);
}
var response = await httpClient.SendAsync(
new HttpRequestMessage
{
RequestUri = new Uri("https://localhost:7186/api/test"),
Content = GetContent(memoryStream),
Method = HttpMethod.Post
}
);
This ensures that the memoryStream has been rewinded for each each retry attempt.
UPDATE #1
After receiving some clarification and digging in the source code of the Egnyte I think I know understand the problem scope better.
A 3rd party library receives an HttpClient instance which is decorated with a retry policy (related source code)
A MemoryStream is passed to a library which is passed forward as a StreamContent as a part of an HttpRequestMessage (related source code)
HRM is passed directly to the HttpClient and the response is wrapped into a ServiceResponse (related source code)
Based on the source code you can receive one of the followings:
An HttpRequestException thrown by the HttpClient
An EgnyteApiException or QPSLimitExceededException or RateLimitExceededException thrown by the ExceptionHelper
An EgnyteApiException thrown by the SendRequestAsync if there was a problem related to the deserialization
A ServiceResponse from SendRequestAsync
As far as I can see you can access the StatusCode only if you receive an HttpRequestException or an EgnyteApiException.
Because you can't rewind the MemoryStream whenever an HttpClient performs a retry I would suggest to decorate the UploadFile with retry. Inside the method you can always set the stream parameter's Position to 0.
public async Task UploadFile(string path, MemoryStream stream){
stream.Position = 0;
await _egnyteClient.Files.CreateOrUpdateFile(path, stream);
}
So rather than decorating the entire HttpClient you should decorate your UploadFile method with retry. Because of this you need to alter the policy definition to something like this:
public static IAsyncPolicy GetRetryPolicy()
=> Policy
.Handle<EgnyteApiException>(ex => ex.StatusCode == HttpStatusCode.Forbidden)
.Or<HttpRequestException>(ex => ex.StatusCode == HttpStatusCode.Forbidden)
.WaitAndRetryAsync(4, _ => TimeSpan.FromSeconds(1));
Maybe the Or builder clause is not needed because I haven't seen any EnsureSuccessStatusCode call anywhere, but for safety I would build the policy like that.

Moq verify using async match (to match HttpContent)

I'm trying to verify the HttpContent of an HttpRequestMessage, but reading the content requires an async operation. How can I do this using Moq?
Illustrative example:
[TestMethod]
public async Task Example()
{
var mockHttpMessageHandler = new Mock<HttpMessageHandler>(MockBehavior.Strict);
mockHttpMessageHandler
.Protected()
.Setup<Task<HttpResponseMessage>>(
"SendAsync",
ItExpr.IsAny<HttpRequestMessage>(),
ItExpr.IsAny<CancellationToken>()
)
.ReturnsAsync(new HttpResponseMessage()
{
StatusCode = HttpStatusCode.OK,
Content = new StringContent(""),
})
.Verifiable();
// call some business component that should send the expected JSON via HTTP Post
await ExecuteSomeBusinessComponentThatPostsViaHttp(httpClient);
this.MockHttpMessageHandler.Protected().Verify(
"SendAsync",
Times.Once(),
ItExpr.Is<HttpRequestMessage>(request =>
JToken.DeepEquals(
// the next line does not compile
JToken.Parse(await request.Content.ReadAsStringAsync()),
JObject.FromObject(new { Result = new { Foo = "Bar" } }))),
ItExpr.IsAny<CancellationToken>());
}
How can I get this test to compile? I.e., how can I use await within an It.Is(...) expression?
Alternatively, is there some other way to test the http request content?

Proper way to mock HttpClient and send/get cookies

I am using moq to mock a wrapper I created for HttpClient class:
public interface IHttpClientWrapper
{
Task<HttpResponseMessage> PostAsync(Uri uri,
HttpContent content,
CookieContainer cookies = null);
}
and in my "normal" implementation of PostAsync, I just delegate the call to HttpClient
public Task<HttpResponseMessage> PostAsync(Uri uri, HttpContent content, CookieContainer cookies = null)
{
var client = cookies == null ? new HttpClient()
: new HttpClient(new HttpClientHandler { CookieContainer = cookies });
return client.PostAsync(uri, content);
}
So, in my application, everything works fine and I get the cookies set by the server (cookies.Count is not 0)
For my test, I have a Mock<IHttpClientWrapper>, and I have set up its PostAsync method to return a new HttpResponseMessage. I also call HttpResponseMessage.Headers.AddCookies method to add 2 cookies to this response.
But when I call my mocked object in a way like this:
/* I setup url and content */
var mock = new Mock<IHttpClientHelper>();
mock.Setup(/* setup PostAsync to return the response I create */)...
var cookies = new CookieContainer();
var response = await mock.PostAsync(url, content, cookies);
then, cookies.Count is always 0.
So, I was wondering what is different than calling the actual server? Do I need to have additional headers? How can I set the cookies here?
CookieContainer passed to PostAsync method as a parameter. The fact that PostAsync adds cookies to CookiesContainer is a side effect of this method, a detail of particular IHttpClientHelper implementation. new Mock<IHttpClientHelper> creates another implementation which doesn't add cookies.
So, if you want mock to add cookies to a container it need an additional setup
mock.Setup(_ => _.PostAsync(It.IsAny<Uri>(), It.IsAny<HttpContent>(), It.IsAny<CookieContainer>()))
.Callback<Uri, HttpContent, CookieContainer>((u, c, cookieContainer) =>
{
// Add required cookies here
cookieContainer.Add(...);
});
Callback is a method of Mock to setup side effects.

C#: HttpClient with POST parameters

I use codes below to send POST request to a server:
string url = "http://myserver/method?param1=1&param2=2"
HttpClientHandler handler = new HttpClientHandler();
HttpClient httpClient = new HttpClient(handler);
HttpRequestMessage request = new HttpRequestMessage(HttpMethod.Post, url);
HttpResponseMessage response = await httpClient.SendAsync(request);
I don't have access to the server to debug but I want to know, is this request sent as POST or GET?
If it is GET, How can I change my code to send param1 & param2 as POST data (not in the URL)?
A cleaner alternative would be to use a Dictionary to handle parameters. They are key-value pairs after all.
private static readonly HttpClient httpclient;
static MyClassName()
{
// HttpClient is intended to be instantiated once and re-used throughout the life of an application.
// Instantiating an HttpClient class for every request will exhaust the number of sockets available under heavy loads.
// This will result in SocketException errors.
// https://learn.microsoft.com/en-us/dotnet/api/system.net.http.httpclient?view=netframework-4.7.1
httpclient = new HttpClient();
}
var url = "http://myserver/method";
var parameters = new Dictionary<string, string> { { "param1", "1" }, { "param2", "2" } };
var encodedContent = new FormUrlEncodedContent (parameters);
var response = await httpclient.PostAsync (url, encodedContent).ConfigureAwait (false);
if (response.StatusCode == HttpStatusCode.OK) {
// Do something with response. Example get content:
// var responseContent = await response.Content.ReadAsStringAsync ().ConfigureAwait (false);
}
Also dont forget to Dispose() httpclient, if you dont use the keyword using
As stated in the Remarks section of the HttpClient class in the Microsoft docs, HttpClient should be instantiated once and re-used.
Edit:
You may want to look into response.EnsureSuccessStatusCode(); instead of if (response.StatusCode == HttpStatusCode.OK).
You may want to keep your httpclient and dont Dispose() it. See: Do HttpClient and HttpClientHandler have to be disposed?
Edit:
Do not worry about using .ConfigureAwait(false) in .NET Core. For more details look at https://blog.stephencleary.com/2017/03/aspnetcore-synchronization-context.html
This is how I use it for DI:
using HttpClient httpClient = clientFactory.CreateClient("name set in builder host");
// httpClient.DefaultRequestHeaders.TryAddWithoutValidation("Authorization", $"Token {token}");
HttpResponseMessage? res = await httpClient!.PostAsync(url, content);
try
{
res.EnsureSuccessStatusCode();
return res;
}
catch (Exception)
{
// Add error handling
}
content is:
List<KeyValuePair<string, string>> values = new()
{
new KeyValuePair<string, string>("data", "value")
};
FormUrlEncodedContent requestContent = new(values);
and clientFactory is the interface:
IHttpClientFactory
msdn interface
As Ben said, you are POSTing your request ( HttpMethod.Post specified in your code )
The querystring (get) parameters included in your url probably will not do anything.
Try this:
string url = "http://myserver/method";
string content = "param1=1&param2=2";
HttpClientHandler handler = new HttpClientHandler();
HttpClient httpClient = new HttpClient(handler);
HttpRequestMessage request = new HttpRequestMessage(HttpMethod.Post, url);
HttpResponseMessage response = await httpClient.SendAsync(request,content);
HTH,
bovako

Example of RestSharp ASYNC client.ExecuteAsync<T> () works

Could someone please help me modify the code below:
client.ExecuteAsync(request, response => {
Console.WriteLine(response.Content);
});
Basically I want to use ExecuteAsync method above but don't want to print but return response.Content to the caller.
Is there any easy way to achieve this?
I tried this but doesnt' work:
public T Execute<T>(RestRequest request) where T : new()
{
var client = new RestClient();
client.BaseUrl = BaseUrl;
client.Authenticator = new HttpBasicAuthenticator(_accountSid, _secretKey);
request.AddParameter("AccountSid", _accountSid, ParameterType.UrlSegment); // used on every request
var response = client.ExecuteAsync(request, response => {
return response.data);
});
}
The above code is from
https://github.com/restsharp/RestSharp
There's the thing... you can't return an asynchronously delivered value, because your calling method will already have returned. Blocking the caller until you have a result defeats the point of using ExecuteAsync. In this case, I'd return a Task<string> (assuming response.Content is a string):
Task<string> GetResponseContentAsync(...)
{
var tcs=new TaskCompletionSource<string>();
client.ExecuteAsync(request, response => {
tcs.SetResult(response.Content);
});
return tcs.Task;
}
Now, when the task completes, you have a value. As we move to c#5 async/await, you should get used to stating asynchrony in terms of Task<T> as it's pretty core.
http://msdn.microsoft.com/en-us/library/dd537609.aspx
http://msdn.microsoft.com/en-us/library/hh191443.aspx
With the help of #spender, this is what i got:
You can add new file in RestSharp project, and add this code:
public partial class RestClient
{
public Task<IRestResponse<T>> ExecuteAsync<T>(IRestRequest request)
{
var tcs=new TaskCompletionSource<IRestResponse<T>>();
this.ExecuteAsync(request, response =>
{
tcs.SetResult(
Deserialize<T>(request, response)
);
});
return tcs.Task;
}
}
This will practically return the full response, with status code and everything, so you can check if the status of the response is OK before getting the content, and you can get the content with:
response.Content
From reading the code it looks like you want to use ExecuteAsGet or ExecuteAsPost instead of the async implementation.
Or maybe just Execute- not sure exactly what type Client is.
At some point, there was a bunch of overloads introduced that support System.Threading.Tasks.Task and had the word "Task" in them. They can be awaited (like HttpClient's methods).
For instance:
ExecuteTaskAsync(request) (returns Task<IRestResponse>)
ExecuteGetTaskAsync(request) (returns Task<IRestResponse>)
ExecuteGetTaskAsync<T>(request) (returns Task<IRestResponse<T>>)
Those then became deprecated in later v106 versions in favour of Task-support being the default I believe, e.g. the first one became client.ExecuteAsync(request).
So these days you can just do:
var response = await client.ExecuteAsync(request);
return response.Content;

Categories

Resources