I have a method I am using to to work with a Json response. Since this is part of a harness and I am going to be creating a number of different data objects is it possible to make this more generic so I don't have to create same code for each different data object I create.
[DataContract]
class Stats
{
[DataMember]
public string StatusCode {get;set;}
[DataMember]
public int ProspectCount {get;set;}
[DataMember]
public int MessageCount {get;set;}
[DataMember]
public int NewListingCount {get;set;}
[DataMember]
public int ReminderCount {get;set;}
[DataMember]
public int MyListingCount {get;set;}
[DataMember]
public int OfficListingCount {get;set;}
}
public static Stats SendRequest(string requestUrl)
{
try
{
HttpWebRequest request = WebRequest.Create(requestUrl) as HttpWebRequest;
using (HttpWebResponse response = request.GetResponse() as HttpWebResponse)
{
if (response.StatusCode != HttpStatusCode.OK)
throw new Exception(String.Format(
"Server error (HTTP {0}: {1}).",
response.StatusCode,
response.StatusDescription));
DataContractJsonSerializer jsonSerializer = new DataContractJsonSerializer(typeof(Stats));
object objResponse = jsonSerializer.ReadObject(response.GetResponseStream());
Stats jsonResponse
= objResponse as Stats;
return jsonResponse;
}
}
catch (Exception e)
{
Console.WriteLine(e.Message);
return null;
}
}
I essentially want to be able to create a second and third data contract without having to recreate the SendRequest method.
Two options:
1: Make the SendRequest method the only static method in another class and have every object that needs it call it like this:
RequestSender.SendRequest(requestUrl).
2: Make this and the other classes you have to write extend a superclass that contains the SendRequest(String requestUrl) method so that they have access to that same method via inheritance.
Would a generic method work? The signature might look like this:
public static T SendRequest<T>(string requestUrl) where T : class
In the method body you would replace Stats with T.
Usage:
Stats response = RequestSender.SendRequest<Stats>("some URL");
OtherClass anotherResponse = RequestSender.SendRequest<OtherClass>("some other URL");
I ended up utilizing Json.NET for the serialization piece. This will do everything I needed it to. So now my method looks like:
public static string ProcessRequest(string requestUrl)
{
try
{
HttpWebRequest request = WebRequest.Create(requestUrl) as HttpWebRequest;
using (HttpWebResponse response = request.GetResponse() as HttpWebResponse)
{
string responseContents;
if (response.StatusCode != HttpStatusCode.OK)
throw new Exception(String.Format(
"Server error (HTTP {0}: {1}).",
response.StatusCode,
response.StatusDescription));
Stream responseStream = response.GetResponseStream();
using(StreamReader readStream = new StreamReader(responseStream))
{
responseContents = readStream.ReadToEnd();
}
return responseContents;
}
}
catch (Exception e)
{
Console.WriteLine(e.Message);
return null;
}
}
And the serialization component in another method is a simple one liner that takes the response as a string.:
Stats results = JsonConvert.DeserializeObject<Stats>(response);
I'm using Moq to mock HttpWebResponse for unit testing. I created a mock class that inherits from the class I'm trying to test and I'm overriding the method that returns the HttpWebResponse. I want to write a JSON string to the response stream from the unit test but I get an ArgumentException when trying to read the stream saying "Stream was not readable".
Here is what I have:
public class ParentClass
{
private string ProcessResponse(HttpWebResponse response)
{
string json = String.Empty;
using (var streamReader = new StreamReader(response.GetResponseStream())) //exception here
{
json = streamReader.ReadToEnd();
}
return json;
}
//other stuff
}
public class MockClass : ParentClass
{
public Stream responseStream = new MemoryStream();
public void WriteJson(string json)
{
using(var writer = new StreamWriter(responseStream))
{
writer.Write(json);
}
}
protected override HttpWebResponse GetResponse(HttpWebRequest request)
{
var response = new Mock<HttpWebResponse>();
response.Setup(r => r.GetResponseStream()).Returns(responseStream);
return response.Object;
}
}
The problem seemed to be with the using statement. Here are the changes I made:
public void WriteJson(string json)
{
var jsonBytes = Encoding.UTF8.GetBytes(json);
responseStream.Write(jsonBytes, 0, jsonBytes.Length);
responseStream.Seek(0, SeekOrigin.Begin);
}
It's safer to use an overload of StreamWriter which leaves the underlying stream open:
public StreamWriter(Stream stream, Encoding encoding, int bufferSize, bool leaveOpen)
I have a data: URI that I need to “download” (read: load as a stream or byte array) using the normal .Net WebClient/WebRequest. How can I do that?
I need this because I want to display a XAML file generated from SVG, which includes some images using data: URIs. I don't want to always parse the XAML, save the images to disk and then change the XAML to point to the files. I believe WPF uses WebRequest internally to get those images.
You can use WebRequest.RegisterPrefix() to do that. You will need to implement IWebRequestCreate that returns a custom WebRequest that returns a custom WebResponse, which can finally be used to get the data from the URI. It could look like this:
public class DataWebRequestFactory : IWebRequestCreate
{
class DataWebRequest : WebRequest
{
private readonly Uri m_uri;
public DataWebRequest(Uri uri)
{
m_uri = uri;
}
public override WebResponse GetResponse()
{
return new DataWebResponse(m_uri);
}
}
class DataWebResponse : WebResponse
{
private readonly string m_contentType;
private readonly byte[] m_data;
public DataWebResponse(Uri uri)
{
string uriString = uri.AbsoluteUri;
int commaIndex = uriString.IndexOf(',');
var headers = uriString.Substring(0, commaIndex).Split(';');
m_contentType = headers[0];
string dataString = uriString.Substring(commaIndex + 1);
m_data = Convert.FromBase64String(dataString);
}
public override string ContentType
{
get { return m_contentType; }
set
{
throw new NotSupportedException();
}
}
public override long ContentLength
{
get { return m_data.Length; }
set
{
throw new NotSupportedException();
}
}
public override Stream GetResponseStream()
{
return new MemoryStream(m_data);
}
}
public WebRequest Create(Uri uri)
{
return new DataWebRequest(uri);
}
}
This supports only base64 encoding, but support for URI encoding could be easily added.
You then register it like this:
WebRequest.RegisterPrefix("data", new DataWebRequestFactory());
And yes, this does work for retrieving data: images in XAML files.
I wrote helper class to retrieve data from web-service to use in Silverlight :
public static class RequestMethod
{
public static string Get = "GET";
public static string Post = "POST";
}
public static class RequestDataType
{
public static string Xml = "Xml";
public static string Json = "application/json";
}
public class HttpWebRequestHelper<T>
{
private readonly SynchronizationContext _context;
private readonly ObservableCollection<T> _collection =
new ObservableCollection<T>();
public ObservableCollection<T> Collection
{
get { return _collection; }
}
public HttpWebRequestHelper()
{
_context = SynchronizationContext.Current;
}
public ObservableCollection<T> GetCollection ()
{
if (HttpWebRequest == null) throw new ArgumentNullException("request");
HttpWebRequest.BeginGetResponse((result) =>
{
var response = HttpWebRequest.EndGetResponse(result);
var stream = response.GetResponseStream();
if (stream == null) return;
var reader = new StreamReader(stream, Encoding.UTF8);
var responseString = reader.ReadToEnd();
var desirilizedObject = JsonConvert.DeserializeObject<IEnumerable<T>>(responseString);
_context.Post((state) =>
{
Collection.Clear();
foreach (var item in desirilizedObject)
{
Collection.Add(item);
}
}, null);
}, null);
return Collection;
}
private HttpWebRequest HttpWebRequest { get; set; }
public void CreateRequest (string url,string requestMethod, string requestDataType)
{
var uri = new Uri(url, UriKind.Absolute);
var request = (HttpWebRequest)WebRequest.Create(uri);
request.Method = requestMethod;
request.Accept = requestDataType;
HttpWebRequest= request;
}
}
This is sample usage of my class :
var request = new HttpWebRequestHelper<DriverModel>();
request.CreateRequest("http://localhost:11624/api/Drivers/", RequestMethod.Get, RequestDataType.Json);
request.GetCollection();
this.DataContext = request;
I want to Update UI when HttpWebRequest is complete .
How can I do this ?
I suppose you got errors when your request is done ang tyring to change UI element. If this is the case you should join in to the current thread for instance;
Deployment.Current.Dispatcher.BeginInvoke(() => { this.Visibility = Visibility.Collapsed; });
Is there any way to use a socks proxy with WebClient? Specifically with the DownloadString method that it provides?
I don't want to use any third party stuff like privoxy, freecap whatever and I can't use commercial libraries like those from Chilkat. I tried using stuff from http://www.mentalis.org/ in fact I used their WebRequest implementation but they don't seem to have something similar for WebClient.
SOCKS is not supported directly by the WebRequest/WebResponse classes and by extension, the WebClient class (it relies on WebRequest to do its work).
It really can't, as it works on the transport layer (TCP/IP) and not through a simple redirect to a server that forwards HTTP requests (which is the level that the WebRequest/WebResponse classes work on).
You can create a specialized derivation of WebRequest/WebResponse (that uses ProxySocket to handle the low-level handshaking and then) and then create a specialized WebClient class which overrides the GetWebRequest and GetWebResponse methods.
Once you have that class substituted for your WebClient instances, it should work as normal (you might have to set up the proxy in each case where you use it though).
Here is how I ended up doing it, thank you casperOne for the answer
public class SocksWebClient : WebClient
{
public IProxyDetails ProxyDetails { get; set; }
public string UserAgent { get; set; }
protected override WebRequest GetWebRequest(Uri address)
{
WebRequest result = null;
if (ProxyDetails != null)
{
if (ProxyDetails.ProxyType == ProxyType.Proxy)
{
result = (HttpWebRequest)WebRequest.Create(address);
result.Proxy = new WebProxy(ProxyDetails.FullProxyAddress);
if (!string.IsNullOrEmpty(UserAgent))
((HttpWebRequest)result).UserAgent = UserAgent;
}
else if (ProxyDetails.ProxyType == ProxyType.Socks)
{
result = SocksHttpWebRequest.Create(address);
result.Proxy = new WebProxy(ProxyDetails.FullProxyAddress);
//TODO: implement user and password
}
else if (ProxyDetails.ProxyType == ProxyType.None)
{
result = (HttpWebRequest)WebRequest.Create(address);
if (!string.IsNullOrEmpty(UserAgent))
((HttpWebRequest)result).UserAgent = UserAgent;
}
}
else
{
result = (HttpWebRequest)WebRequest.Create(address);
if (!string.IsNullOrEmpty(UserAgent))
((HttpWebRequest)result).UserAgent = UserAgent;
}
return result;
}
}
The SocksHttpWebRequest class is taken from the blog linked to by #casperOne, the code for which is as follows:
using System;
using System.Collections.Specialized;
using System.IO;
using System.Net;
using System.Net.Sockets;
using System.Text;
using Org.Mentalis.Network.ProxySocket;
namespace Ditrans
{
public class SocksHttpWebRequest : WebRequest
{
#region Member Variables
private readonly Uri _requestUri;
private WebHeaderCollection _requestHeaders;
private string _method;
private SocksHttpWebResponse _response;
private string _requestMessage;
private byte[] _requestContentBuffer;
// darn MS for making everything internal (yeah, I'm talking about you, System.net.KnownHttpVerb)
static readonly StringCollection validHttpVerbs =
new StringCollection { "GET", "HEAD", "POST", "PUT", "DELETE", "TRACE", "OPTIONS" };
#endregion
#region Constructor
private SocksHttpWebRequest(Uri requestUri)
{
_requestUri = requestUri;
}
#endregion
#region WebRequest Members
public override WebResponse GetResponse()
{
if (Proxy == null)
{
throw new InvalidOperationException("Proxy property cannot be null.");
}
if (String.IsNullOrEmpty(Method))
{
throw new InvalidOperationException("Method has not been set.");
}
if (RequestSubmitted)
{
return _response;
}
_response = InternalGetResponse();
RequestSubmitted = true;
return _response;
}
public override Uri RequestUri
{
get { return _requestUri; }
}
public override IWebProxy Proxy { get; set; }
public override WebHeaderCollection Headers
{
get
{
if (_requestHeaders == null)
{
_requestHeaders = new WebHeaderCollection();
}
return _requestHeaders;
}
set
{
if (RequestSubmitted)
{
throw new InvalidOperationException("This operation cannot be performed after the request has been submitted.");
}
_requestHeaders = value;
}
}
public bool RequestSubmitted { get; private set; }
public override string Method
{
get
{
return _method ?? "GET";
}
set
{
if (validHttpVerbs.Contains(value))
{
_method = value;
}
else
{
throw new ArgumentOutOfRangeException("value", string.Format("'{0}' is not a known HTTP verb.", value));
}
}
}
public override long ContentLength { get; set; }
public override string ContentType { get; set; }
public override Stream GetRequestStream()
{
if (RequestSubmitted)
{
throw new InvalidOperationException("This operation cannot be performed after the request has been submitted.");
}
if (_requestContentBuffer == null)
{
_requestContentBuffer = new byte[ContentLength];
}
else if (ContentLength == default(long))
{
_requestContentBuffer = new byte[int.MaxValue];
}
else if (_requestContentBuffer.Length != ContentLength)
{
Array.Resize(ref _requestContentBuffer, (int) ContentLength);
}
return new MemoryStream(_requestContentBuffer);
}
#endregion
#region Methods
public static new WebRequest Create(string requestUri)
{
return new SocksHttpWebRequest(new Uri(requestUri));
}
public static new WebRequest Create(Uri requestUri)
{
return new SocksHttpWebRequest(requestUri);
}
private string BuildHttpRequestMessage()
{
if (RequestSubmitted)
{
throw new InvalidOperationException("This operation cannot be performed after the request has been submitted.");
}
var message = new StringBuilder();
message.AppendFormat("{0} {1} HTTP/1.0\r\nHost: {2}\r\n", Method, RequestUri.PathAndQuery, RequestUri.Host);
// add the headers
foreach (var key in Headers.Keys)
{
message.AppendFormat("{0}: {1}\r\n", key, Headers[key.ToString()]);
}
if (!string.IsNullOrEmpty(ContentType))
{
message.AppendFormat("Content-Type: {0}\r\n", ContentType);
}
if (ContentLength > 0)
{
message.AppendFormat("Content-Length: {0}\r\n", ContentLength);
}
// add a blank line to indicate the end of the headers
message.Append("\r\n");
// add content
if(_requestContentBuffer != null && _requestContentBuffer.Length > 0)
{
using (var stream = new MemoryStream(_requestContentBuffer, false))
{
using (var reader = new StreamReader(stream))
{
message.Append(reader.ReadToEnd());
}
}
}
return message.ToString();
}
private SocksHttpWebResponse InternalGetResponse()
{
var response = new StringBuilder();
using (var _socksConnection =
new ProxySocket(AddressFamily.InterNetwork, SocketType.Stream, ProtocolType.Tcp))
{
var proxyUri = Proxy.GetProxy(RequestUri);
var ipAddress = GetProxyIpAddress(proxyUri);
_socksConnection.ProxyEndPoint = new IPEndPoint(ipAddress, proxyUri.Port);
_socksConnection.ProxyType = ProxyTypes.Socks5;
// open connection
_socksConnection.Connect(RequestUri.Host, 80);
// send an HTTP request
_socksConnection.Send(Encoding.ASCII.GetBytes(RequestMessage));
// read the HTTP reply
var buffer = new byte[1024];
var bytesReceived = _socksConnection.Receive(buffer);
while (bytesReceived > 0)
{
response.Append(Encoding.ASCII.GetString(buffer, 0, bytesReceived));
bytesReceived = _socksConnection.Receive(buffer);
}
}
return new SocksHttpWebResponse(response.ToString());
}
private static IPAddress GetProxyIpAddress(Uri proxyUri)
{
IPAddress ipAddress;
if (!IPAddress.TryParse(proxyUri.Host, out ipAddress))
{
try
{
return Dns.GetHostEntry(proxyUri.Host).AddressList[0];
}
catch (Exception e)
{
throw new InvalidOperationException(
string.Format("Unable to resolve proxy hostname '{0}' to a valid IP address.", proxyUri.Host), e);
}
}
return ipAddress;
}
#endregion
#region Properties
public string RequestMessage
{
get
{
if (string.IsNullOrEmpty(_requestMessage))
{
_requestMessage = BuildHttpRequestMessage();
}
return _requestMessage;
}
}
#endregion
}
}
Note that, as #casperOne pointed out, this makes use of a (free) third party library called ProxySocket.
I came across this aswell and found the nice BetterHttpClient
It derives from WebClient and allows you to specify a socks proxy:
BetterHttpClient.HttpClient client = new BetterHttpClient.HttpClient(new BetterHttpClient.Proxy("IP address", port, BetterHttpClient.ProxyTypeEnum.Socks));
I was looking for library in order to do this. finally i found the MihaZupan/HttpToSocks5Proxy it's a real lifesaver. just like this you can use it:
using MihaZupan;
var proxy = new HttpToSocks5Proxy("127.0.0.1", 1080,
"username", "password" // optional
);
var handler = new HttpClientHandler { Proxy = proxy };
HttpClient httpClient = new HttpClient(handler, true);
var result = await httpClient.SendAsync(
new HttpRequestMessage(HttpMethod.Get, "https://httpbin.org/ip"));
Console.WriteLine("HTTPS GET: " + await result.Content.ReadAsStringAsync());
If you want a more convenient, WebClient-esque API for reading the content, you can wrap the response-read calls into an extension method:
public static class ExtensionMethods
{
public static async Task<string> DownloadStringAsync(this HttpClient client, string url)
{
var response = await client.SendAsync(new HttpRequestMessage(
HttpMethod.Get, url));
return await response.Content.ReadAsStringAsync();
}
}
For future readers:
Since .NET 6, socks proxies are supported natively with HttpClient.
var handler = new HttpClientHandler
{
Proxy = new WebProxy("socks5://127.0.0.1", 9050)
};
var httpClient = new HttpClient(handler);
Note:
In time of writing this answer, WebClient is considered as obsolete. HttpClient should be used instead.
Try Yove.Proxy. Example:
using (var w = new WebClient())
{
using (var proxy = new ProxyClient("67.201.33.10", 25283, ProxyType.Socks5))
{
w.Proxy = proxy;
Console.WriteLine(w.DownloadString("https://api.ipify.org"));
}
}