Develop Reference

414. The request URL is too long. asp.net

I'm getting the error "HTTP Error 414. The request URL is too long." From the following article, I understand that this is due to a very long query string:
http://www.mytecbits.com/microsoft/iis/query-string-too-long
In web.config, I have maxQueryStringLength="2097151". Is this the maximum value?
In order to solve this problem, should I set maxUrl in web.config? If so, what's the maximum value supported?
What should I do to fix this error?

The GET request should never be this long. You need to change it to POST method instead since it was designed to transmit block of data such as forms.
An excerpt from the RFC 2616: Hypertext Transfer Protocol -- HTTP/1.1:
The POST method is used to request that the origin server accept the
entity enclosed in the request as a new subordinate of the resource
identified by the Request-URI in the Request-Line. POST is designed to
allow a uniform method to cover the following functions:
Annotation of existing resources;
Posting a message to a bulletinboard, newsgroup, mailing list, or
similar group of articles;
Providing a block of data, such as the result of submitting a
form, to a data-handling process;
Extending a database through an appendoperation.

Prevent unescaping url in outbound request

If I do this in .Net Core 3.1:
await new HttpClient().GetAsync("http://test.com/page?parameter=%2D%2E%5F%2E%2D");
then this happens:
GET http://test.com/page?parameter=-._.- HTTP/1.1
but this is what I want:
GET http://test.com/page?parameter=%2D%2E%5F%2E%2D HTTP/1.1
The background is that I get a signed Url from a third party and I need to use the url as it is, non-unescaped. I manage to find the resource with the unescaped url, but the signature check fails on the other end because the url they see in the request is not the url that was signed.
I can paste the url into any browser and get the resource, but the signature check fails when I do it programatically in .Net Core 3.1.
The unescaping is supposed to happen according to documentation on the Uri Class:
Escaped characters (also known as percent-encoded octets) that don't
have a reserved purpose are decoded (also known as being unescaped).
These unreserved characters include uppercase and lowercase letters
(%41-%5A and %61-%7A), decimal digits (%30-%39), hyphen (%2D), period
(%2E), underscore (%5F), and tilde (%7E).
I have tried solutions listed in these questions:
GETting a URL with an url-encoded slash. But the schemeSetting seems not to work for .Net Core 3.1 and and neither does the workaround ForceCanonicalPathAndQuery.
How to make System.Uri not to unescape %2f (slash) in path?. Again schemeSetting seems not to work for .Net Core 3.1, and neither does the workaround LeaveDotsAndSlashesEscaped.
So, does anyone know how I can use the signed url as is, non-unescaped, on .Net Core 3.1?

So after fiddling around a bit I came up with this:
private static Uri CreateNonUnescapedUri(string url)
{
// Initiate Uri as e.g "http://test.com" so internal flags will indicate that the url does not include characters that needs unescaping
int offset = url.IndexOf("://");
offset = url.IndexOf('/', offset + 4);
var uri = new Uri(url.Substring(0, offset));
// Then replace internal field with complete url that would otherwise be unescaped
typeof(Uri).GetField("_string", BindingFlags.Instance | BindingFlags.NonPublic).SetValue(uri, url);
return uri;
}
I tested it on 300 signed url's.
Offcourse changing the internal state of the Uri which is 5600 lines of pure madness is bound to fail in the future, but I need this working by monday and this is what I've got. Let me know if anyone has a real solution.
Edit April 2022:
In .Net 6 there is a new constructor that will keep the original url as is, using UriCreationOptions:
var uri = new Uri("http://test.com/page?parameter=%2D%2E%5F%2E%2D",
new UriCreationOptions { DangerousDisablePathAndQueryCanonicalization = true });
I have no idea whats supposedly dangerous about it though.
For .Net Core 3.1 I'm still using the hack above, I never did find a better solution for it.

HttpWebRequest maximum length for Get request url

I've been tasked with building a service which pulls information from a 3rd party API into our internal datawarehouse. They have a get request to pull the data I want where you specify the parameters you want via query strings. E.g.
http://www.api.com?parameter=firstname&parameter=surname
In my code the length of the URL is over 3600 characters long as the requirement is for 116 parameters.
My web request is generated using this code:
private HttpWebRequest GetWebRequest(string url, string type, int timeout)
{
var httpWebRequest = (HttpWebRequest) WebRequest.Create(_baseUrl + url);
httpWebRequest.Method = type;
httpWebRequest.Timeout = timeout;
httpWebRequest.ContentType = "application/json";
httpWebRequest.Headers.Add("Authorization", "Bearer " + _token.access_token);
httpWebRequest.ContentLength = 0;
return httpWebRequest;
}
When I run the code I am getting back a web exception with the message "Unable to connect to the remote server" with an internal exception message of "No connection could be made because the target machine actively refused it IP Address"
I have not included the entire URL in this post as I have found that if I copy and paste the url into Postman and run the request I get the response I expect so I know that the URL is formatted correctly. I have also discovered that if I cut down the length of the url to around 2100 characters the request then works.
I have been searching but have not found any definitive documentation to suggest that there is a limit to the length of the URL, but I can not explain why the whole url works in Postman but not in a c# web request and that if I cut the length of the URL it then works in the web request!
If anyone has any ideas about this I'd be greatfull.

An old post suggests that depending on the server and client the maximum request length is somewhere between 2 - 4 and 8 KB, which is consistent with your observations.
Postman is 'another' client, so it is well possible that it works there while it doesn't in your program. Bottom-line is: you should not GET such long requests. You should POST them instead. If you have control over the service, you could change it so it supports POST too, if not already (documented or not).

How to ensure a url is called from my application and not manually from browser

I have an application that contains a button, on click of this button, it will open a browser window using a URL with querystring parameters (the url of a page that i am coding).
Is there a way to ensure that the URL is coming from my application and only from my application - and not just anyone typing the URL manually in a webbrowser?
If not, what is the best way to ensure that a specific URL is coming from a specific application - and not just manually entered in the address bar or a web browser-
Im using asp.net.

You can check if the request was made from one of the pages of your application using:
Request.UrlReferrer.Contains("mywebsite.com")
That's the simple way.
The secure way is to put a cookie on the client containing a value encrypted using a secure key or hashed using a secure salt. If the cookie is set to expire when the page is closed it should be impossible for someone to forge.
Here's an example:
On the pages that would redirect to the page you are trying to protect:
HttpCookie cookie = new HttpCookie("SecureCheck");
//don't set the cookie's expiration so it's deleted when the browser is closed
cookie.Value = System.Web.Security.FormsAuthentication.HashPasswordForStoringInConfigFile(Session.SessionID, "SHA1");
Response.Cookies.Add(cookie);
On the page you are trying to protect:
//check to see if the cookie is there and it has the correct value
if (string.IsNullOrEmpty(Request.Cookies["SecureCheck"]) || System.Web.Security.FormsAuthentication.HashPasswordForStoringInConfigFile(Session.SessionID, "SHA1") != Request.Cookies["SecureCheck"])
throw Exception("Invalid request. Please access this page only from the application.");
//if we got this far the exception was not thrown and we are safe to continue
//insert whatever code here

There's no reliable way to do this for a GET request, nor is their any reason to try for a legitimate user. What you should do instead is ensure that regardless of where the request comes from the user has the proper permissions and access rights and that the session is protected appropriately (HTTP only cookies, SSL, etc.) If the request is changing data, then it should be a POST, not a GET, and it should be accompanied by some suitable cross-site request forgery prevention techniques (such as a cookie containing a nonce that is verified against a matching nonce on the form itself).

There is no way, other than rejecting the request if it doesn't contain a previously generated random one-time token in the parameters (that would be stored in the session, for example).

While there is no 100% secure way to do this, what I am suggesting might at least take care of your basic needs.
This is what you can do .
Client: Add a HTTP header with an encoded string that is like hash (sha256) of some word.
Then make your client always do a POST request instead of GET.
Server: Check the HTTP Header for encoded string. Also make sure it is a POST request.
This is not 100% as ofcourse someone smart enough could figure out and still generate a request, but depending on your need you might find this enough or not

You can check the referer, the user agent, add an additional header to the request, always do post requests to that url. However, considering HTTP is transmitted in plain text, somebody is always able to let wireshark or fiddler run, capture the HTTP packets and recreate the requests with your measures in place.

Pass parameters from your application so that you can verify on the server side.
I suggest you use an encryption algorithm and generate random text using a password(key). Then, decrypt the param on the server side and check if it matches your expectation.
I am not very clear though. sorry about that, If had to do something like this, then, I would do something similar to mentioned above.

You can use to check the header on MVC controller like Request.Headers["Accept"]; if it is coming from your code in angularjs or jquery:
sample angularjs like this:
var url = ServiceServerPath + urlSearchService + '/SearchCustomer?input=' + $scope.strInput;
$http({
method: 'GET',
url: url,
headers: {
'Content-Type': 'application/json'
},.....
And on the MVC [HttpGet] Action method
[HttpGet]
[PreventDirectAccess]//It is my custom filters
// ---> /Index/SearchCustomer?input={input}/
public string SearchCustomer(string input)
{
try
{
var isJsonRequestOnMVC = Request.Headers["Accept"];//TODO: This will check if the request comes from MVC else comes from Browser
if (!isJsonRequestOnMVC.Contains("application/json")) return "Error Request on server!";
var serialize = new JavaScriptSerializer();
ISearch customer = new SearchCustomer();
IEnumerable<ContactInfoResult> returnSearch = customer.GetCustomerDynamic(input);
return serialize.Serialize(returnSearch);
}
catch (Exception err)
{
throw;
}
}

HTTPS Redirect Causing Error "Server cannot append header after HTTP headers have been sent"

I need to check that our visitors are using HTTPS. In BasePage I check if the request is coming via HTTPS. If it's not, I redirect back with HTTPS. However, when someone comes to the site and this function is used, I get the error:
System.Web.HttpException: Server
cannot append header after HTTP
headers have been sent. at
System.Web.HttpResponse.AppendHeader(String
name, String value) at
System.Web.HttpResponse.AddHeader(String
name, String value) at
Premier.Payment.Website.Generic.BasePage..ctor()
Here is the code I started with:
// If page not currently SSL
if (HttpContext.Current.Request.ServerVariables["HTTPS"].Equals("off"))
{
// If SSL is required
if (GetConfigSetting("SSLRequired").ToUpper().Equals("TRUE"))
{
string redi = "https://" +
HttpContext.Current.Request.ServerVariables["SERVER_NAME"].ToString() +
HttpContext.Current.Request.ServerVariables["SCRIPT_NAME"].ToString() +
"?" + HttpContext.Current.Request.ServerVariables["QUERY_STRING"].ToString();
HttpContext.Current.Response.Redirect(redi.ToString());
}
}
I also tried adding this above it (a bit I used in another site for a similar problem):
// Wait until page is copletely loaded before sending anything since we re-build
HttpContext.Current.Response.BufferOutput = true;
I am using c# in .NET 3.5 on IIS 6.

Chad,
Did you try ending the output when you redirect? There is a second parameter that you'd set to true to tell the output to stop when the redirect header is issued. Or, if you are buffering the output then maybe you need to clear the buffer before doing the redirect so the headers are not sent out along with the redirect header.
Brian

This error usually means that something has bee written to the response stream before a redirection is initiated. So you should make sure that the test for https is done fairly high up in the page load function.