Develop Reference

How to specify an input file to DOM of <input type='file'> using AngleSharp?

Using AngleSharp, how do I specify file to fill in <input type="file" name="myInputFile">? I've read this StackOverflow question, but it seems like different than my intended case. I'm trying to fill a form programmatically while uploading a file of my choice.

Every IHtmlInputElement has a Files property that can be used to add files.
var input = document.QuerySelector<IHtmlInputElement>("input[type=file][name=myInputFile]");
input?.Files.Add(file);
In the previously used example the file variable refers to any IFile instance. AngleSharp is a PCL does not come with a proper implementation out of the box, however, a simple one may look like:
class FileEntry : IFile
{
private readonly String _fileName;
private readonly Stream _content;
private readonly String _type;
private readonly DateTime _modified;
public FileEntry(String fileName, String type, Stream content)
{
_fileName = fileName;
_type = type;
_content = content;
_modified = DateTime.Now;
}
public Stream Body
{
get { return _content; }
}
public Boolean IsClosed
{
get { return _content.CanRead == false; }
}
public DateTime LastModified
{
get { return _modified; }
}
public Int32 Length
{
get
{
return (Int32)_content.Length;
}
}
public String Name
{
get { return _fileName; }
}
public String Type
{
get { return _type; }
}
public void Close()
{
_content.Close();
}
public void Dispose()
{
_content.Dispose();
}
public IBlob Slice(Int32 start = 0, Int32 end = Int32.MaxValue, String contentType = null)
{
var ms = new MemoryStream();
_content.Position = start;
var buffer = new Byte[Math.Max(0, Math.Min(end, _content.Length) - start)];
_content.Read(buffer, 0, buffer.Length);
ms.Write(buffer, 0, buffer.Length);
_content.Position = 0;
return new FileEntry(_fileName, _type, ms);
}
}
A more sophisticated one would auto-determine the MIME type and have constructor overloads to allow passing in (local) file paths etc.
Hope this helps!

How do Test HttpWebRequest Depended method?

I have this class:
public class RestClient
{
public RestClient()
{ }
protected virtual HttpWebRequest CreateHttpWebRequest(Uri uri)
{
return (HttpWebRequest)HttpWebRequest.Create(uri);
}
/// <summary>
/// Perform a http POST request in order to push data to server
/// </summary>
/// <param name="uri">End Point Uri</param>
/// <param name="data">Data to be transmitted</param>
/// <returns></returns>
///
public long PostRequest(Uri uri,string data)
{
try
{
HttpWebRequest request = CreateHttpWebRequest(uri); //(HttpWebRequest)HttpWebRequest.Create(uri);
request.Method = "POST";
request.ContentType = "application/json";
System.Text.UTF8Encoding encoding = new UTF8Encoding();
byte[] bytes = encoding.GetBytes(data);
using (Stream requestStream = request.GetRequestStream())
{
//Transmit data
requestStream.Write(bytes, 0, bytes.Length);
requestStream.Flush();
requestStream.Close();
}
//Get the Response from the server
using (HttpWebResponse response = (HttpWebResponse)request.GetResponse())
{
if (response.StatusCode != HttpStatusCode.OK && response.StatusCode != HttpStatusCode.NoContent)
{
throw new Exception(String.Format(
"Server error (HTTP {0}: {1}).",
response.StatusCode,
response.StatusDescription));
}
}
return request.ContentLength;
}
catch (Exception e)
{
throw e;
}
}
}
And would like to Unit test (using nunit) the PostRequest Method.
Doing some research, I could found some way to mock the HttpWebRequest in this post (Is it possible to mock out a .NET HttpWebResponse?) and a way to inject it into the class in this post (How to unit test a method with HttpWebRequest/Response dependencies).
However, when I tried to test my method I got this error:
System.InvalidCastException : Unable to cast object of type 'Castle.Proxies.IHttpWebRequestProxy' to type 'System.Net.HttpWebRequest'.
in this line of my test
client.HttpWebRequestFake = (HttpWebRequest)factory.Object.Create("http://127.0.0.1");
That is my test code:
public class TesableRestClient : RestClient
{
public HttpWebRequest HttpWebRequestFake { get; set; }
protected override HttpWebRequest CreateHttpWebRequest(Uri url)
{
if (HttpWebRequestFake != null)
return HttpWebRequestFake;
return base.CreateHttpWebRequest(url);
}
}
[TestFixture]
public class TransferWebRequestTest
{
[Test]
public void TestPostResquest()
{
string expectedContent = "Content";
var expectedBytes = Encoding.UTF8.GetBytes(expectedContent);
var responseStream = new MemoryStream();
responseStream.Write(expectedBytes, 0, expectedBytes.Length);
responseStream.Seek(0, SeekOrigin.Begin);
var response = new Mock<IHttpWebResponse>();
response.Setup(c => c.GetResponseStream()).Returns(responseStream);
var request = new Mock<IHttpWebRequest>();
request.Setup(c => c.GetResponse()).Returns(response.Object);
var factory = new Mock<IHttpWebRequestFactory>();
factory.Setup(c => c.Create(It.IsAny<string>()))
.Returns(request.Object);
TesableRestClient client = new TesableRestClient();
client.HttpWebRequestFake = (HttpWebRequest)factory.Object.Create("http://127.0.0.1");
// DoStuff call the url with a request and then processes the
long bytesSent = client.PostRequest(new Uri("http://127.0.0.1"), expectedContent);
Assert.AreEqual(expectedBytes, bytesSent);
}
The HttpWebRequest/Response is this:
public interface IHttpWebRequest
{
// expose the members you need
string Method { get; set; }
string ContentType { get; set; }
long ContentLength { get; set; }
IHttpWebResponse GetResponse();
}
public interface IHttpWebResponse : IDisposable
{
// expose the members you need
HttpStatusCode StatusCode { get; }
string StatusDescription { get;}
Stream GetResponseStream();
}
public interface IHttpWebRequestFactory
{
IHttpWebRequest Create(string uri);
}
// barebones implementation
public class HttpWebRequestFactory : IHttpWebRequestFactory
{
public IHttpWebRequest Create(string uri)
{
return new WrapHttpWebRequest((HttpWebRequest)WebRequest.Create(uri));
}
}
public class WrapHttpWebRequest : IHttpWebRequest
{
private readonly HttpWebRequest _request;
public WrapHttpWebRequest(HttpWebRequest request)
{
_request = request;
}
public string Method
{
get { return _request.Method; }
set { _request.Method = value; }
}
public string ContentType
{
get { return _request.ContentType; }
set { _request.ContentType = value; }
}
public long ContentLength
{
get { return _request.ContentLength; }
set { _request.ContentLength = value; }
}
public IHttpWebResponse GetResponse()
{
return new WrapHttpWebResponse((HttpWebResponse)_request.GetResponse());
}
}
public class WrapHttpWebResponse : IHttpWebResponse
{
private HttpWebResponse _response;
public WrapHttpWebResponse(HttpWebResponse response)
{
_response = response;
}
public void Dispose()
{
Dispose(true);
GC.SuppressFinalize(this);
}
private void Dispose(bool disposing)
{
if (disposing)
{
if (_response != null)
{
((IDisposable)_response).Dispose();
_response = null;
}
}
}
public Stream GetResponseStream()
{
return _response.GetResponseStream();
}
public HttpStatusCode StatusCode
{
get { return _response.StatusCode; }
}
public string StatusDescription
{
get { return _response.StatusDescription; }
}
}
Any idea of how I could solve this?
Thank you

I solved my issue doing the follow:
First, created a interface IHttpWebRequestFactory
public interface IHttpWebRequestFactory
{
HttpWebRequest Create(string uri);
}
In the class that I want to test, I created the following methods:
protected virtual HttpWebRequest CreateHttpWebRequest(Uri uri)
{
return (HttpWebRequest)HttpWebRequest.Create(uri);
}
protected virtual HttpWebResponse GetHttpWebResponse(HttpWebRequest request)
{
return (HttpWebResponse)request.GetResponse();
}
In my test file, I created a "Testable " class, that inherits from the class I really want to test and overrides the virtual methods:
//Class Created to test the PostRequestMethod
public class TestableRestClient : RestClient
{
public HttpWebRequest HttpWebRequestFake { get; set; }
public string responseValue;
protected override HttpWebRequest CreateHttpWebRequest(Uri url)
{
if (HttpWebRequestFake != null)
return HttpWebRequestFake;
return base.CreateHttpWebRequest(url);
}
protected override HttpWebResponse GetHttpWebResponse(HttpWebRequest request)
{
HttpWebResponse response = (HttpWebResponse)request.GetResponse();
using (var streamReader = new StreamReader(request.GetResponse().GetResponseStream()))
{
responseValue = streamReader.ReadToEnd();
}
return base.GetHttpWebResponse(request);
}
}
Then I used Moq to mock the behavior of methods I'm using in my class
[TestFixture]
public class DMSTransferWebRequestTest
{
[Test]
public void TestPostResquest()
{
string expected = "Content";
//Prepare the Mocked Response Stream
byte [] expectedBytes = Encoding.UTF8.GetBytes(expected);
Stream responseStream = new MemoryStream();
responseStream.Write(expectedBytes, 0, expectedBytes.Length);
responseStream.Seek(0, SeekOrigin.Begin);
//Prepare the Mocked Request Stream
Stream requestStream = new MemoryStream();
requestStream.Write(expectedBytes, 0, expectedBytes.Length);
requestStream.Seek(0, SeekOrigin.Begin);
//Mock the HttpWebResponse
Mock<HttpWebResponse> response = new Mock<HttpWebResponse>();
//Set the method GetResponseStream to return the Response Stream mocked
response.Setup(c => c.GetResponseStream()).Returns(responseStream);
response.Setup(c => c.StatusCode).Returns(HttpStatusCode.OK);
//Set the method GetRequestStream to return the Request Stream mocked
Mock<HttpWebRequest> request = new Mock<HttpWebRequest>();
request.Setup(c => c.GetResponse()).Returns(response.Object);
request.Setup(c => c.GetRequestStream()).Returns(requestStream);
//Create a Object to mock the HttpWebRequest Create Method
Mock<IHttpWebRequestFactory> factory = new Mock<IHttpWebRequestFactory>();
factory.Setup(c => c.Create(It.IsAny<string>()))
.Returns(request.Object);
TestableRestClient client = new TestableRestClient();
client.HttpWebRequestFake = factory.Object.Create("http://mytest");
long actualBytes = client.PostRequest(new Uri("http://mytest"), expected);
string actual = client.responseValue;
Assert.AreEqual(expected, actual);
}
}

Not really sure what you want to achieve, but this may help. The error message tells you the exact problem and gives you a hint what to do.
If you check your code, the class WrapHttpWebRequest is not of type HttpWebRequest. However, it holds an HttpWebRequest. These two steps solve the direct issue, but you may run into another issue. First, provide a property to the class WrapHttpWebRequest:
public HttpWebRequest HttpWebRequest { get { return _request; } }
Then change you failing code line to this:
client.HttpWebRequestFake = factory.Object.Create("http://127.0.0.1").HttpWebRequest;
I reckon though that it would be better to change the class WrapHttpWebRequest and inherit from 'HttpWebRequest' like this ...
public class WrapHttpWebRequest: HttpWebRequest, IHttpWebRequest
... and change its implementation accordingly.

How to send multiple images from android to a WCF Rest Service as a stream to write to a network drive?

After much googling and searching, I managed to send an image using multiparsers from android to my WCF service, but ideally, I'd like to send several images at once, instead of calling the method over and over again, since it'd take a lot longer, and add a bunch more overhead.
This is my current code
Android (Taken from code found on here somewhere):
public static String postFile(Bitmap bitmap, String urlString) throws Exception {
HttpClient client = new DefaultHttpClient();
HttpPost post = new HttpPost(urlString);
MultipartEntityBuilder builder = MultipartEntityBuilder.create();
builder.setMode(HttpMultipartMode.BROWSER_COMPATIBLE);
ByteArrayOutputStream bao = new ByteArrayOutputStream();
bitmap.compress(Bitmap.CompressFormat.PNG, 30, bao);
byte[] data = bao.toByteArray();
//filename
String fileName = String.format("File_%d.png",new Date().getTime());
ByteArrayBody bab = new ByteArrayBody(data, fileName);
builder.addPart("image", bab);
final HttpEntity yourEntity = builder.build();
class ProgressiveEntity implements HttpEntity {
#Override
public void consumeContent() throws IOException {
yourEntity.consumeContent();
}
#Override
public InputStream getContent() throws IOException,
IllegalStateException {
return yourEntity.getContent();
}
#Override
public Header getContentEncoding() {
return yourEntity.getContentEncoding();
}
#Override
public long getContentLength() {
return yourEntity.getContentLength();
}
#Override
public Header getContentType() {
return yourEntity.getContentType();
}
#Override
public boolean isChunked() {
return yourEntity.isChunked();
}
#Override
public boolean isRepeatable() {
return yourEntity.isRepeatable();
}
#Override
public boolean isStreaming() {
return yourEntity.isStreaming();
} // CONSIDER put a _real_ delegator into here!
#Override
public void writeTo(OutputStream outstream) throws IOException {
class ProxyOutputStream extends FilterOutputStream {
/**
* #author Stephen Colebourne
*/
public ProxyOutputStream(OutputStream proxy) {
super(proxy);
}
public void write(int idx) throws IOException {
out.write(idx);
}
public void write(byte[] bts) throws IOException {
out.write(bts);
}
public void write(byte[] bts, int st, int end) throws IOException {
out.write(bts, st, end);
}
public void flush() throws IOException {
out.flush();
}
public void close() throws IOException {
out.close();
}
} // CONSIDER import this class (and risk more Jar File Hell)
class ProgressiveOutputStream extends ProxyOutputStream {
public ProgressiveOutputStream(OutputStream proxy) {
super(proxy);
}
public void write(byte[] bts, int st, int end) throws IOException {
// FIXME Put your progress bar stuff here!
out.write(bts, st, end);
}
}
yourEntity.writeTo(new ProgressiveOutputStream(outstream));
}
};
ProgressiveEntity myEntity = new ProgressiveEntity();
post.setEntity(myEntity);
HttpResponse response = client.execute(post);
return getContent(response);
}
public static String getContent(HttpResponse response) throws IOException {
BufferedReader rd = new BufferedReader(new InputStreamReader(response.getEntity().getContent()));
String body = "";
String content = "";
while ((body = rd.readLine()) != null)
{
content += body + "\n";
}
return content.trim();
}
C# WCF Service method to take it
[WebInvoke(UriTemplate = "UploadPicture/{filename}", Method = "POST", BodyStyle = WebMessageBodyStyle.Bare, ResponseFormat = WebMessageFormat.Json, RequestFormat = WebMessageFormat.Json)]
public String UploadPicture(string filename, Stream fileStream)
{
WriteLog("Uploading picture...");
try
{
MultipartParser parser = new MultipartParser(fileStream);
if (parser.Success)
{
string fileName = parser.Filename;
string contentType = parser.ContentType;
byte[] fileContent = parser.FileContents;
FileStream fileToupload = new FileStream("\\\\OHS-SUN\\Tracker\\robbie\\" + filename, FileMode.Create);
fileToupload.Write(fileContent, 0, fileContent.Length);
fileToupload.Close();
fileToupload.Dispose();
fileStream.Close();
return "Success !!!";
}
else
{
return "Exception!!!";
}
}
catch (Exception ex)
{
WriteLog("Uploading picture exception: " + ex.Message);
}
return "Picture uploaded!";
}
I'd like to go from sending one image, to sending several, each with 2 text attributes; the filename, and the project number they're associated with. Essentially, both is what I need it for. At the moment, I'm just trying to do put another addPart on to the android bit, but then I don't know how to add metadata to that and I wouldn't know how to parse it based on the name. I'm fine with using any third party libraries, the one I'm using on C# at the moment is already one.
Thanks a lot!

Instead of sending multiple images in one thing, I just stuck it in an asynchronous class and sent them concurrently with a max of 10 at a time until all the images are sent in that particular session. Seems to work fine, the implementation's the same, so I've not had to change any of that, which is good. If anyone would like me to post the code I did to do that, I'd be happy to. It's just small snippets here and there added to the above code, though.
Well, the main bit I added was:
public static class FileUploader extends AsyncTask<UploadFile , Void , String> implements Future<String>
{
#Override
protected void onPreExecute() {
filesUploading ++;
}
#Override
protected String doInBackground(UploadFile... uploadFile)
{
try
{
return postFile(uploadFile[0].file, uploadFile[0].projectNo, uploadFile[0].filename);
} catch (Exception e)
{
e.printStackTrace();
}
return null;
}
#Override
protected void onPostExecute(String result) {
super.onPostExecute(result);
filesUploading --;
}
#Override
public boolean isDone() {
return AsyncTask.Status.FINISHED == getStatus();
}
}
This allows me to send each image separately, and handle them separately.

How to use ETag in Web API using action filter along with HttpResponseMessage

I have a ASP.Net Web API controller which simply returns the list of users.
public sealed class UserController : ApiController
{
[EnableTag]
public HttpResponseMessage Get()
{
var userList= this.RetrieveUserList(); // This will return list of users
this.responseMessage = new HttpResponseMessage(HttpStatusCode.OK)
{
Content = new ObjectContent<List<UserViewModel>>(userList, new JsonMediaTypeFormatter())
};
return this.responseMessage;
}
}
and an action filter attribute class EnableTag which is responsible to manage ETag and cache:
public class EnableTag : System.Web.Http.Filters.ActionFilterAttribute
{
private static ConcurrentDictionary<string, EntityTagHeaderValue> etags = new ConcurrentDictionary<string, EntityTagHeaderValue>();
public override void OnActionExecuting(HttpActionContext context)
{
if (context != null)
{
var request = context.Request;
if (request.Method == HttpMethod.Get)
{
var key = GetKey(request);
ICollection<EntityTagHeaderValue> etagsFromClient = request.Headers.IfNoneMatch;
if (etagsFromClient.Count > 0)
{
EntityTagHeaderValue etag = null;
if (etags.TryGetValue(key, out etag) && etagsFromClient.Any(t => t.Tag == etag.Tag))
{
context.Response = new HttpResponseMessage(HttpStatusCode.NotModified);
SetCacheControl(context.Response);
}
}
}
}
}
public override void OnActionExecuted(HttpActionExecutedContext context)
{
var request = context.Request;
var key = GetKey(request);
EntityTagHeaderValue etag;
if (!etags.TryGetValue(key, out etag) || request.Method == HttpMethod.Put || request.Method == HttpMethod.Post)
{
etag = new EntityTagHeaderValue("\"" + Guid.NewGuid().ToString() + "\"");
etags.AddOrUpdate(key, etag, (k, val) => etag);
}
context.Response.Headers.ETag = etag;
SetCacheControl(context.Response);
}
private static void SetCacheControl(HttpResponseMessage response)
{
response.Headers.CacheControl = new CacheControlHeaderValue()
{
MaxAge = TimeSpan.FromSeconds(60),
MustRevalidate = true,
Private = true
};
}
private static string GetKey(HttpRequestMessage request)
{
return request.RequestUri.ToString();
}
}
The above code create an attribute class to manage ETag. So on the first request, it will create a new E-Tag and for the subsequent request it will check whether any ETag is existed. If so, it will generate Not Modified HTTP Status and return back to client.
My problem is, I want to create a new ETag if there are changes in my user list, ex. a new user is added, or an existing user is deleted. and append it with the response. This can be tracked by the userList variable.
Currently, the ETag received from client and server are same from every second request, so in this case it will always generate Not Modified status, while I want it when actually nothing changed.
Can anyone guide me in this direction?

My requirement was to cache my web api JSON responses... And all the solutions provided don't have an easy "link" to where the data is generated - ie in the Controller...
So my solution was to create a wrapper "CacheableJsonResult" which generated a Response, and then added the ETag to the header. This allows a etag to be passed in when the controller method is generated and wants to return the content...
public class CacheableJsonResult<T> : JsonResult<T>
{
private readonly string _eTag;
private const int MaxAge = 10; //10 seconds between requests so it doesn't even check the eTag!
public CacheableJsonResult(T content, JsonSerializerSettings serializerSettings, Encoding encoding, HttpRequestMessage request, string eTag)
:base(content, serializerSettings, encoding, request)
{
_eTag = eTag;
}
public override Task<HttpResponseMessage> ExecuteAsync(System.Threading.CancellationToken cancellationToken)
{
Task<HttpResponseMessage> response = base.ExecuteAsync(cancellationToken);
return response.ContinueWith<HttpResponseMessage>((prior) =>
{
HttpResponseMessage message = prior.Result;
message.Headers.ETag = new EntityTagHeaderValue(String.Format("\"{0}\"", _eTag));
message.Headers.CacheControl = new CacheControlHeaderValue
{
Public = true,
MaxAge = TimeSpan.FromSeconds(MaxAge)
};
return message;
}, cancellationToken);
}
}
And then, in your controller - return this object:
[HttpGet]
[Route("results/{runId}")]
public async Task<IHttpActionResult> GetRunResults(int runId)
{
//Is the current cache key in our cache?
//Yes - return 304
//No - get data - and update CacheKeys
string tag = GetETag(Request);
string cacheTag = GetCacheTag("GetRunResults"); //you need to implement this map - or use Redis if multiple web servers
if (tag == cacheTag )
return new StatusCodeResult(HttpStatusCode.NotModified, Request);
//Build data, and update Cache...
string newTag = "123"; //however you define this - I have a DB auto-inc ID on my messages
//Call our new CacheableJsonResult - and assign the new cache tag
return new CacheableJsonResult<WebsiteRunResults>(results, GlobalConfiguration.Configuration.Formatters.JsonFormatter.SerializerSettings, System.Text.UTF8Encoding.Default, Request, newTag);
}
}
private static string GetETag(HttpRequestMessage request)
{
IEnumerable<string> values = null;
if (request.Headers.TryGetValues("If-None-Match", out values))
return new EntityTagHeaderValue(values.FirstOrDefault()).Tag;
return null;
}
You need to define how granular to make your tags; my data is user-specific, so I include the UserId in the CacheKey (etag)

a good solution for ETag and in ASP.NET Web API is to use CacheCow . A good article is here.
It's easy to use and you don't have to create a custom Attribute.
Have fun
.u

I found CacheCow very bloated for what it does, if the only reason is, to lower the amount of data transfered, you might want to use something like this:
public class EntityTagContentHashAttribute : ActionFilterAttribute
{
private IEnumerable<string> _receivedEntityTags;
private readonly HttpMethod[] _supportedRequestMethods = {
HttpMethod.Get,
HttpMethod.Head
};
public override void OnActionExecuting(HttpActionContext context) {
if (!_supportedRequestMethods.Contains(context.Request.Method))
throw new HttpResponseException(context.Request.CreateErrorResponse(HttpStatusCode.PreconditionFailed,
"This request method is not supported in combination with ETag."));
var conditions = context.Request.Headers.IfNoneMatch;
if (conditions != null) {
_receivedEntityTags = conditions.Select(t => t.Tag.Trim('"'));
}
}
public override void OnActionExecuted(HttpActionExecutedContext context)
{
var objectContent = context.Response.Content as ObjectContent;
if (objectContent == null) return;
var computedEntityTag = ComputeHash(objectContent.Value);
if (_receivedEntityTags.Contains(computedEntityTag))
{
context.Response.StatusCode = HttpStatusCode.NotModified;
context.Response.Content = null;
}
context.Response.Headers.ETag = new EntityTagHeaderValue("\"" + computedEntityTag + "\"", true);
}
private static string ComputeHash(object instance) {
var cryptoServiceProvider = new MD5CryptoServiceProvider();
var serializer = new DataContractSerializer(instance.GetType());
using (var memoryStream = new MemoryStream())
{
serializer.WriteObject(memoryStream, instance);
cryptoServiceProvider.ComputeHash(memoryStream.ToArray());
return String.Join("", cryptoServiceProvider.Hash.Select(c => c.ToString("x2")));
}
}
}
No need for setting up anything, set and forget. The way i like it. :)

I like the answer which was provided by #Viezevingertjes. It is the most elegant and "No need for setting up anything" approach is very convenient. I like it too :)
However I think it has a few drawbacks:
The whole OnActionExecuting() method and storing ETags in _receivedEntityTags is unnecessary because the Request is available inside the OnActionExecuted method as well.
Only works with ObjectContent response types.
Extra work load because of the serialization.
Also it was not part of the question and nobody mentioned it. But ETag should be used for Cache validation. Therefore it should be used with Cache-Control header so clients don't even have to call the server until the cache expires (it can be very short period of time depends on your resource). When the cache expired then client makes a request with ETag and validate it. For more details about caching see this article.
So that's why I decided to pimp it up a little but. Simplified filter no need for OnActionExecuting method, works with Any response types, no Serialization. And most importantly adds CacheControl header as well. It can be improved e.g. with Public cache enabled, etc... However I strongly advise you to understand caching and modify it carefully. If you use HTTPS and the endpoints are secured then this setup should be fine.
/// <summary>
/// Enables HTTP Response CacheControl management with ETag values.
/// </summary>
public class ClientCacheWithEtagAttribute : ActionFilterAttribute
{
private readonly TimeSpan _clientCache;
private readonly HttpMethod[] _supportedRequestMethods = {
HttpMethod.Get,
HttpMethod.Head
};
/// <summary>
/// Default constructor
/// </summary>
/// <param name="clientCacheInSeconds">Indicates for how long the client should cache the response. The value is in seconds</param>
public ClientCacheWithEtagAttribute(int clientCacheInSeconds)
{
_clientCache = TimeSpan.FromSeconds(clientCacheInSeconds);
}
public override async Task OnActionExecutedAsync(HttpActionExecutedContext actionExecutedContext, CancellationToken cancellationToken)
{
if (!_supportedRequestMethods.Contains(actionExecutedContext.Request.Method))
{
return;
}
if (actionExecutedContext.Response?.Content == null)
{
return;
}
var body = await actionExecutedContext.Response.Content.ReadAsStringAsync();
if (body == null)
{
return;
}
var computedEntityTag = GetETag(Encoding.UTF8.GetBytes(body));
if (actionExecutedContext.Request.Headers.IfNoneMatch.Any()
&& actionExecutedContext.Request.Headers.IfNoneMatch.First().Tag.Trim('"').Equals(computedEntityTag, StringComparison.InvariantCultureIgnoreCase))
{
actionExecutedContext.Response.StatusCode = HttpStatusCode.NotModified;
actionExecutedContext.Response.Content = null;
}
var cacheControlHeader = new CacheControlHeaderValue
{
Private = true,
MaxAge = _clientCache
};
actionExecutedContext.Response.Headers.ETag = new EntityTagHeaderValue($"\"{computedEntityTag}\"", false);
actionExecutedContext.Response.Headers.CacheControl = cacheControlHeader;
}
private static string GetETag(byte[] contentBytes)
{
using (var md5 = MD5.Create())
{
var hash = md5.ComputeHash(contentBytes);
string hex = BitConverter.ToString(hash);
return hex.Replace("-", "");
}
}
}
Usage e.g: with 1 min client side caching:
[ClientCacheWithEtag(60)]

Seems to be a nice way to do it:
public class CacheControlAttribute : System.Web.Http.Filters.ActionFilterAttribute
{
public int MaxAge { get; set; }
public CacheControlAttribute()
{
MaxAge = 3600;
}
public override void OnActionExecuted(HttpActionExecutedContext context)
{
if (context.Response != null)
{
context.Response.Headers.CacheControl = new CacheControlHeaderValue
{
Public = true,
MaxAge = TimeSpan.FromSeconds(MaxAge)
};
context.Response.Headers.ETag = new EntityTagHeaderValue(string.Concat("\"", context.Response.Content.ReadAsStringAsync().Result.GetHashCode(), "\""),true);
}
base.OnActionExecuted(context);
}
}

Use WebClient with socks proxy

Is there any way to use a socks proxy with WebClient? Specifically with the DownloadString method that it provides?
I don't want to use any third party stuff like privoxy, freecap whatever and I can't use commercial libraries like those from Chilkat. I tried using stuff from http://www.mentalis.org/ in fact I used their WebRequest implementation but they don't seem to have something similar for WebClient.

SOCKS is not supported directly by the WebRequest/WebResponse classes and by extension, the WebClient class (it relies on WebRequest to do its work).
It really can't, as it works on the transport layer (TCP/IP) and not through a simple redirect to a server that forwards HTTP requests (which is the level that the WebRequest/WebResponse classes work on).
You can create a specialized derivation of WebRequest/WebResponse (that uses ProxySocket to handle the low-level handshaking and then) and then create a specialized WebClient class which overrides the GetWebRequest and GetWebResponse methods.
Once you have that class substituted for your WebClient instances, it should work as normal (you might have to set up the proxy in each case where you use it though).

Here is how I ended up doing it, thank you casperOne for the answer
public class SocksWebClient : WebClient
{
public IProxyDetails ProxyDetails { get; set; }
public string UserAgent { get; set; }
protected override WebRequest GetWebRequest(Uri address)
{
WebRequest result = null;
if (ProxyDetails != null)
{
if (ProxyDetails.ProxyType == ProxyType.Proxy)
{
result = (HttpWebRequest)WebRequest.Create(address);
result.Proxy = new WebProxy(ProxyDetails.FullProxyAddress);
if (!string.IsNullOrEmpty(UserAgent))
((HttpWebRequest)result).UserAgent = UserAgent;
}
else if (ProxyDetails.ProxyType == ProxyType.Socks)
{
result = SocksHttpWebRequest.Create(address);
result.Proxy = new WebProxy(ProxyDetails.FullProxyAddress);
//TODO: implement user and password
}
else if (ProxyDetails.ProxyType == ProxyType.None)
{
result = (HttpWebRequest)WebRequest.Create(address);
if (!string.IsNullOrEmpty(UserAgent))
((HttpWebRequest)result).UserAgent = UserAgent;
}
}
else
{
result = (HttpWebRequest)WebRequest.Create(address);
if (!string.IsNullOrEmpty(UserAgent))
((HttpWebRequest)result).UserAgent = UserAgent;
}
return result;
}
}
The SocksHttpWebRequest class is taken from the blog linked to by #casperOne, the code for which is as follows:
using System;
using System.Collections.Specialized;
using System.IO;
using System.Net;
using System.Net.Sockets;
using System.Text;
using Org.Mentalis.Network.ProxySocket;
namespace Ditrans
{
public class SocksHttpWebRequest : WebRequest
{
#region Member Variables
private readonly Uri _requestUri;
private WebHeaderCollection _requestHeaders;
private string _method;
private SocksHttpWebResponse _response;
private string _requestMessage;
private byte[] _requestContentBuffer;
// darn MS for making everything internal (yeah, I'm talking about you, System.net.KnownHttpVerb)
static readonly StringCollection validHttpVerbs =
new StringCollection { "GET", "HEAD", "POST", "PUT", "DELETE", "TRACE", "OPTIONS" };
#endregion
#region Constructor
private SocksHttpWebRequest(Uri requestUri)
{
_requestUri = requestUri;
}
#endregion
#region WebRequest Members
public override WebResponse GetResponse()
{
if (Proxy == null)
{
throw new InvalidOperationException("Proxy property cannot be null.");
}
if (String.IsNullOrEmpty(Method))
{
throw new InvalidOperationException("Method has not been set.");
}
if (RequestSubmitted)
{
return _response;
}
_response = InternalGetResponse();
RequestSubmitted = true;
return _response;
}
public override Uri RequestUri
{
get { return _requestUri; }
}
public override IWebProxy Proxy { get; set; }
public override WebHeaderCollection Headers
{
get
{
if (_requestHeaders == null)
{
_requestHeaders = new WebHeaderCollection();
}
return _requestHeaders;
}
set
{
if (RequestSubmitted)
{
throw new InvalidOperationException("This operation cannot be performed after the request has been submitted.");
}
_requestHeaders = value;
}
}
public bool RequestSubmitted { get; private set; }
public override string Method
{
get
{
return _method ?? "GET";
}
set
{
if (validHttpVerbs.Contains(value))
{
_method = value;
}
else
{
throw new ArgumentOutOfRangeException("value", string.Format("'{0}' is not a known HTTP verb.", value));
}
}
}
public override long ContentLength { get; set; }
public override string ContentType { get; set; }
public override Stream GetRequestStream()
{
if (RequestSubmitted)
{
throw new InvalidOperationException("This operation cannot be performed after the request has been submitted.");
}
if (_requestContentBuffer == null)
{
_requestContentBuffer = new byte[ContentLength];
}
else if (ContentLength == default(long))
{
_requestContentBuffer = new byte[int.MaxValue];
}
else if (_requestContentBuffer.Length != ContentLength)
{
Array.Resize(ref _requestContentBuffer, (int) ContentLength);
}
return new MemoryStream(_requestContentBuffer);
}
#endregion
#region Methods
public static new WebRequest Create(string requestUri)
{
return new SocksHttpWebRequest(new Uri(requestUri));
}
public static new WebRequest Create(Uri requestUri)
{
return new SocksHttpWebRequest(requestUri);
}
private string BuildHttpRequestMessage()
{
if (RequestSubmitted)
{
throw new InvalidOperationException("This operation cannot be performed after the request has been submitted.");
}
var message = new StringBuilder();
message.AppendFormat("{0} {1} HTTP/1.0\r\nHost: {2}\r\n", Method, RequestUri.PathAndQuery, RequestUri.Host);
// add the headers
foreach (var key in Headers.Keys)
{
message.AppendFormat("{0}: {1}\r\n", key, Headers[key.ToString()]);
}
if (!string.IsNullOrEmpty(ContentType))
{
message.AppendFormat("Content-Type: {0}\r\n", ContentType);
}
if (ContentLength > 0)
{
message.AppendFormat("Content-Length: {0}\r\n", ContentLength);
}
// add a blank line to indicate the end of the headers
message.Append("\r\n");
// add content
if(_requestContentBuffer != null && _requestContentBuffer.Length > 0)
{
using (var stream = new MemoryStream(_requestContentBuffer, false))
{
using (var reader = new StreamReader(stream))
{
message.Append(reader.ReadToEnd());
}
}
}
return message.ToString();
}
private SocksHttpWebResponse InternalGetResponse()
{
var response = new StringBuilder();
using (var _socksConnection =
new ProxySocket(AddressFamily.InterNetwork, SocketType.Stream, ProtocolType.Tcp))
{
var proxyUri = Proxy.GetProxy(RequestUri);
var ipAddress = GetProxyIpAddress(proxyUri);
_socksConnection.ProxyEndPoint = new IPEndPoint(ipAddress, proxyUri.Port);
_socksConnection.ProxyType = ProxyTypes.Socks5;
// open connection
_socksConnection.Connect(RequestUri.Host, 80);
// send an HTTP request
_socksConnection.Send(Encoding.ASCII.GetBytes(RequestMessage));
// read the HTTP reply
var buffer = new byte[1024];
var bytesReceived = _socksConnection.Receive(buffer);
while (bytesReceived > 0)
{
response.Append(Encoding.ASCII.GetString(buffer, 0, bytesReceived));
bytesReceived = _socksConnection.Receive(buffer);
}
}
return new SocksHttpWebResponse(response.ToString());
}
private static IPAddress GetProxyIpAddress(Uri proxyUri)
{
IPAddress ipAddress;
if (!IPAddress.TryParse(proxyUri.Host, out ipAddress))
{
try
{
return Dns.GetHostEntry(proxyUri.Host).AddressList[0];
}
catch (Exception e)
{
throw new InvalidOperationException(
string.Format("Unable to resolve proxy hostname '{0}' to a valid IP address.", proxyUri.Host), e);
}
}
return ipAddress;
}
#endregion
#region Properties
public string RequestMessage
{
get
{
if (string.IsNullOrEmpty(_requestMessage))
{
_requestMessage = BuildHttpRequestMessage();
}
return _requestMessage;
}
}
#endregion
}
}
Note that, as #casperOne pointed out, this makes use of a (free) third party library called ProxySocket.

I came across this aswell and found the nice BetterHttpClient
It derives from WebClient and allows you to specify a socks proxy:
BetterHttpClient.HttpClient client = new BetterHttpClient.HttpClient(new BetterHttpClient.Proxy("IP address", port, BetterHttpClient.ProxyTypeEnum.Socks));

I was looking for library in order to do this. finally i found the MihaZupan/HttpToSocks5Proxy it's a real lifesaver. just like this you can use it:
using MihaZupan;
var proxy = new HttpToSocks5Proxy("127.0.0.1", 1080,
"username", "password" // optional
);
var handler = new HttpClientHandler { Proxy = proxy };
HttpClient httpClient = new HttpClient(handler, true);
var result = await httpClient.SendAsync(
new HttpRequestMessage(HttpMethod.Get, "https://httpbin.org/ip"));
Console.WriteLine("HTTPS GET: " + await result.Content.ReadAsStringAsync());
If you want a more convenient, WebClient-esque API for reading the content, you can wrap the response-read calls into an extension method:
public static class ExtensionMethods
{
public static async Task<string> DownloadStringAsync(this HttpClient client, string url)
{
var response = await client.SendAsync(new HttpRequestMessage(
HttpMethod.Get, url));
return await response.Content.ReadAsStringAsync();
}
}

For future readers:
Since .NET 6, socks proxies are supported natively with HttpClient.
var handler = new HttpClientHandler
{
Proxy = new WebProxy("socks5://127.0.0.1", 9050)
};
var httpClient = new HttpClient(handler);
Note:
In time of writing this answer, WebClient is considered as obsolete. HttpClient should be used instead.

Try Yove.Proxy. Example:
using (var w = new WebClient())
{
using (var proxy = new ProxyClient("67.201.33.10", 25283, ProxyType.Socks5))
{
w.Proxy = proxy;
Console.WriteLine(w.DownloadString("https://api.ipify.org"));
}
}