Develop Reference

Kestrel modify incoming http headers

I have an IP camera that I'm trying to receive events from. For example, if it detects motion in a specific part of the frame, I want to know about it and get the images captured with the event.
The problem is that it uses HTTP/1.0 POST to send this information. According to the spec (https://www.w3.org/Protocols/HTTP/1.0/draft-ietf-http-spec.html#BodyLength), the body length can either be communicated by the Content-Length header or by closing the connection when everything has been sent to the server. The IP camera does the latter of the two methods.
Kestrel's web server doesn't support this as seen here Microsoft.AspNetCore.Server.Kestrel.Core.Internal.Http.Http1MessageBody.For() contains the following (https://github.com/aspnet/KestrelHttpServer/blob/2191327b59f87f23f69fff2ac0dba9e58b67141b/src/Kestrel.Core/Internal/Http/Http1MessageBody.cs, Lines 308-318):
// Avoid slowing down most common case
if (!object.ReferenceEquals(context.Method, HttpMethods.Get))
{
// If we got here, request contains no Content-Length or Transfer-Encoding header.
// Reject with 411 Length Required.
if (context.Method == HttpMethod.Post || context.Method == HttpMethod.Put)
{
var requestRejectionReason = httpVersion == HttpVersion.Http11 ? RequestRejectionReason.LengthRequired : RequestRejectionReason.LengthRequiredHttp10;
BadHttpRequestException.Throw(requestRejectionReason, context.Method);
}
}
As a result, Kestrel throws a BadHttpRequestException every time.
Any ideas on how to work around this issue?
TCP Server that can run next to Kestrel?
A low-level hook to add the content-length?

Override Host in HttpClient

I'm using HttpClient but it has problems with DNS resolve (it is using the sync method for this) So I use another lib for doing DNS queries and now I'm tryging to get custom urls by IP but I need to replace Host header. For example I have url http://fb.com but I need to get http://1.1.1.1 with Host set to fb.com I've tryied:
_req = new HttpRequestMessage(HttpMethod.Get, newUri.ToString());
_req.Headers.Host = uri.Host;
_httpClient.DefaultRequestHeaders.Host = uri.Host;
but this doesn't work. Is there any way to set own Host header like in HttpWebRequest?

It's work. The problem was with Fiddler which override Host header based on url. When Fiddler is off everything is going fine.

C# How to spoof IP address for WebRequest

I have asp.net website hosted and I am making WebRequest to post data and get response. The website is having IP filtering. I want to spoof sender IP address for testing purpose. Is it possible to do it programmatically or I have to use any tool.
public string GetResponse(string request)
{
lock (Obj)
{
request = request + _dataControlInfo.SendEndingWith;
Logger.Info(request);
var req = (HttpWebRequest)WebRequest.Create(_serviceUrl);
req.Headers.Add("SOAPAction", "\"\"");
req.ContentType = "text/xml;charset=\"utf-8\"";
req.Accept = "text/xml";
req.Method = "POST";
var stm = req.GetRequestStream();
var bytes = UtfEncoding.StringToUtf8ByteArray(request);
stm.Write(bytes, 0, bytes.Length);
stm.Close();
var resp = req.GetResponse();
var stmr = new StreamReader(resp.GetResponseStream());
var strResponseXml = stmr.ReadToEnd();
Logger.Info(strResponseXml);
return strResponseXml;
}
}
Please specify any possibilities.

What your looking for is SharpPCap which is a .NET port of WinPCap.. it allows you to do IP Spoofing, which is what your talking about. The only problem with your idea is that you wont be able to get a response back. You can send requests out, but if you dont have a proper return address then the request will be lost in the interwebs.
Edit
To do this yoruself w/out the help of a library you will need to construct the raw packets yourself. This has been answered here.

If you're expecting to get a response, then no. Without the correct IP address, the server won't send the response to the correct destination.
If you insist on trying anyway, see this article for programmatically setting the client's IP address.

Or you can use Web Performance Tests and a load test with IP Switching enabled

You can try to use a proxy, as documented here.
( http://msdn.microsoft.com/en-us/library/system.net.httpwebrequest.proxy.aspx ).
Setting up a proxy on a different computer, then configuring that computer as your requests proxy server should make the request appear as if it came from the proxy's IP, not yours.

Some servers can also consider X-Forwarded-For and X-Real-IP headers.
So if server checks for these headers you can add them to your Web request.
But it depends on server implementation.

Use the Spoof class found in the System.Security namespace...

HTTPS Redirect Causing Error "Server cannot append header after HTTP headers have been sent"

I need to check that our visitors are using HTTPS. In BasePage I check if the request is coming via HTTPS. If it's not, I redirect back with HTTPS. However, when someone comes to the site and this function is used, I get the error:
System.Web.HttpException: Server
cannot append header after HTTP
headers have been sent. at
System.Web.HttpResponse.AppendHeader(String
name, String value) at
System.Web.HttpResponse.AddHeader(String
name, String value) at
Premier.Payment.Website.Generic.BasePage..ctor()
Here is the code I started with:
// If page not currently SSL
if (HttpContext.Current.Request.ServerVariables["HTTPS"].Equals("off"))
{
// If SSL is required
if (GetConfigSetting("SSLRequired").ToUpper().Equals("TRUE"))
{
string redi = "https://" +
HttpContext.Current.Request.ServerVariables["SERVER_NAME"].ToString() +
HttpContext.Current.Request.ServerVariables["SCRIPT_NAME"].ToString() +
"?" + HttpContext.Current.Request.ServerVariables["QUERY_STRING"].ToString();
HttpContext.Current.Response.Redirect(redi.ToString());
}
}
I also tried adding this above it (a bit I used in another site for a similar problem):
// Wait until page is copletely loaded before sending anything since we re-build
HttpContext.Current.Response.BufferOutput = true;
I am using c# in .NET 3.5 on IIS 6.

Chad,
Did you try ending the output when you redirect? There is a second parameter that you'd set to true to tell the output to stop when the redirect header is issued. Or, if you are buffering the output then maybe you need to clear the buffer before doing the redirect so the headers are not sent out along with the redirect header.
Brian

This error usually means that something has bee written to the response stream before a redirection is initiated. So you should make sure that the test for https is done fairly high up in the page load function.

HttpWebRequest long URI workaround?

I've encountered an issue with HttpWebRequest that if the URI is over 2048 characters long the request fails and returns a 404 error even though the server is perfectly capable of servicing a request with a URI that long. I know this since the same URI that causes an error if submitted via HttpWebRequest works fine when pasted directly into a browser address bar.
My current workaround is to allow users to set a compatability flag to say that it's safe to send the parameters as a POST request instead in the case where the URI would be too long but this is not ideal since the protocol I'm using is RESTful and GET should be used for queries. Plus there is no guarentee that other implementors of the protocol will accept POSTed queries
Is there another class in .Net that has equivalent functionality to HttpWebRequest that doesn't suffer from the URI length limit that I could use?
I'm aware of WebClient but I don't really want to use that as I need to be able to fully control the HTTP Headers which WebClient restricts the ability to do.
Edit
Because Shoban asked for it:
http://localhost/BBCDemo/sparql/?query=PREFIX+rdf%3A+%3Chttp%3A%2F%2Fwww.w3.org%2F1999%2F02%2F22-rdf-syntax-ns%23%3E%0D%0APREFIX+rdfs%3A+%3Chttp%3A%2F%2Fwww.w3.org%2F2000%2F01%2Frdf-schema%23%3E%0D%0APREFIX+xsd%3A+%3Chttp%3A%2F%2Fwww.w3.org%2F2001%2FXMLSchema%23%3E%0D%0APREFIX+skos%3A+%3Chttp%3A%2F%2Fwww.w3.org%2F2004%2F02%2Fskos%2Fcore%23%3E%0D%0APREFIX+dc%3A+%3Chttp%3A%2F%2Fpurl.org%2Fdc%2Felements%2F1.1%2F%3E%0D%0APREFIX+po%3A+%3Chttp%3A%2F%2Fpurl.org%2Fontology%2Fpo%2F%3E%0D%0APREFIX+timeline%3A+%3Chttp%3A%2F%2Fpurl.org%2FNET%2Fc4dm%2Ftimeline.owl%23%3E%0D%0ASELECT+*+WHERE+{%0D%0A++++%3Chttp%3A%2F%2Fwww.bbc.co.uk%2Fprogrammes%2Fb00n4d6y%23programme%3E+dc%3Atitle+%3Ftitle+.%0D%0A++++%3Chttp%3A%2F%2Fwww.bbc.co.uk%2Fprogrammes%2Fb00n4d6y%23programme%3E+po%3Ashort_synopsis+%3Fsynopsis-short+.%0D%0A++++%3Chttp%3A%2F%2Fwww.bbc.co.uk%2Fprogrammes%2Fb00n4d6y%23programme%3E+po%3Amedium_synopsis+%3Fsynopsis-med+.%0D%0A++++%3Chttp%3A%2F%2Fwww.bbc.co.uk%2Fprogrammes%2Fb00n4d6y%23programme%3E+po%3Along_synopsis+%3Fsynopsis-long+.%0D%0A++++%3Chttp%3A%2F%2Fwww.bbc.co.uk%2Fprogrammes%2Fb00n4d6y%23programme%3E+po%3Amasterbrand+%3Fchannel+.%0D%0A++++%3Chttp%3A%2F%2Fwww.bbc.co.uk%2Fprogrammes%2Fb00n4d6y%23programme%3E+po%3Agenre+%3Fgenre+.%0D%0A++++%3Fchannel+dc%3Atitle+%3Fchanneltitle+.%0D%0A++++OPTIONAL+{%0D%0A++++++++%3Chttp%3A%2F%2Fwww.bbc.co.uk%2Fprogrammes%2Fb00n4d6y%23programme%3E+po%3Abrand+%3Fbrand+.%0D%0A++++++++%3Fbrand+dc%3Atitle+%3Fbrandtitle+.%0D%0A++++}%0D%0A++++OPTIONAL+{%0D%0A++++++++%3Chttp%3A%2F%2Fwww.bbc.co.uk%2Fprogrammes%2Fb00n4d6y%23programme%3E+po%3Aversion+%3Fver+.%0D%0A++++++++%3Fver+po%3Atime+%3Finterval+.%0D%0A++++++++%3Finterval+timeline%3Astart+%3Fstart+.%0D%0A++++++++%3Finterval+timeline%3Aend+%3Fend+.%0D%0A++++}%0D%0A}&default-graph-uri=&timeout=30000
Which is the following encoded onto the querystring:
PREFIX rdf: <http://www.w3.org/1999/02/22-rdf-syntax-ns#>
PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#>
PREFIX xsd: <http://www.w3.org/2001/XMLSchema#>
PREFIX skos: <http://www.w3.org/2004/02/skos/core#>
PREFIX dc: <http://purl.org/dc/elements/1.1/>
PREFIX po: <http://purl.org/ontology/po/>
PREFIX timeline: <http://purl.org/NET/c4dm/timeline.owl#>
SELECT * WHERE {
<http://www.bbc.co.uk/programmes/b00n4d6y#programme> dc:title ?title .
<http://www.bbc.co.uk/programmes/b00n4d6y#programme> po:short_synopsis ?synopsis-short .
<http://www.bbc.co.uk/programmes/b00n4d6y#programme> po:medium_synopsis ?synopsis-med .
<http://www.bbc.co.uk/programmes/b00n4d6y#programme> po:long_synopsis ?synopsis-long .
<http://www.bbc.co.uk/programmes/b00n4d6y#programme> po:masterbrand ?channel .
<http://www.bbc.co.uk/programmes/b00n4d6y#programme> po:genre ?genre .
?channel dc:title ?channeltitle .
OPTIONAL {
<http://www.bbc.co.uk/programmes/b00n4d6y#programme> po:brand ?brand .
?brand dc:title ?brandtitle .
}
OPTIONAL {
<http://www.bbc.co.uk/programmes/b00n4d6y#programme> po:version ?ver .
?ver po:time ?interval .
?interval timeline:start ?start .
?interval timeline:end ?end .
}
}

the protocol I'm using is RESTful and GET should be used for queries.
There's no reason POST can't also be used for queries; for really long request data you have to, as very-long-URIs aren't globally supported, and have never been. This is one area where HTTP does not live up to the REST ideal.
The reason POST generally isn't used on a plain-HTML level is to stop the browser prompting for reloads, and promote eg. bookmarking. But for HttpWebRequest you don't have either of those concerns, so go ahead and POST it. Web applications should use a parameter or a URI path part to distinguish write requests from queries, not merely the request method. (Of course a write request from a GET method should still be denied.)

I don't think HttpWebRequest is actually incompatible with GET URLs of the size you are talking about. I say this based on two things:
In my own work I use HttpWebRequest to send HTTP GET requests longer than 2048 characters without trouble. I'm not sure what my longest ones are, but we're talking 10,000+ characters. (This is primarily between a web application and an instance of Solr running under Tomcat.)
.NET does have some limits on GET URL lengths, but the ones I'm aware of are much higher than 2048 characters. For example, I learned today from my profiler that WebRequest.Create(string url) calls the Uri class constructor, and that is documented to throw a UriFormatException if "the length of uriString exceeds 65534 characters."
I'm not sure where your problem might be, if it's not HttpWebRequest itself. Do you know under what conditions your web service will return HTTP 404 (i.e. "not found")? (I assume the 404 is coming from your web service, rather than being faked inside the depths of .NET.) I'd also want to double-check that the address you're pasting into the browser is actually the same one that's being sent by .NET; as feroze suggested, you should use a network sniffing tool for this. If the two addresses are the same, then maybe next compare how the HTTP headers vary between the .NET case and the browser case. (Incidentally, I personally find Fiddler a bit handier than wireshark for HTTP debugging tasks along these lines.)
See also this somewhat related question: How does HttpWebRequest differ (functional) from pasteing a URL into an address bar?

Here's a snippet which constructs HttpWebRequest instances with bigger and bigger url values until an exception gets thrown:
using System.Net;
...
StringBuilder url = new StringBuilder("http://example.com?p=");
try
{
for (int i = 1; i < Int32.MaxValue; i++)
{
url.Append("0");
HttpWebRequest request = HttpWebRequest.CreateHttp(url.ToString());
}
}
catch (Exception ex)
{
Console.Out.WriteLine("Error occurred at url length: " + url.Length);
Console.Out.WriteLine(ex.GetType().ToString() + ": " + ex.Message);
return;
}
Console.Out.WriteLine("Completed without error!");
On my machine (in LINQPad running .Net 4.5), this snippet outputs:
Error occurred at url length: 65520
System.UriFormatException: Invalid URI: The Uri string is too long.

Your query string is wrong according to RFC3986. '{' and '}' characters are not allowed in a URI.