I have an ASP.NET MVC application with a route that allows searching for stuff via /search/<searchterm>.
When I supply "search/abc" it works well, but when I supply "/search/a+b+c" (correctly url encoded) then IIS7 rejects the request with HTTP Error 404.11 (The request filtering module is configured to deny a request that contains a double escape sequence). FIrst of all, why does it do this? It only seems to throw the error if it is part of the URL, but not as part of a query string ( /transmit?q=a+b+c works fine).
Now I could enable double escape requests in the security section of my web.config but I'm hesitant to do so as I don't understand the implications, and neither why the server would reject the request "a+b+c" as part of the URL but accept as part of a query string.
Can someone explain and give some advice what to do?
Edit: Added emphasis to relevant sections.
Basically: IIS is being excessively paranoid. You can safely disable this check if you're not doing anything particularly unwise with the uri decoded data (such as generating local filesystem URI's via string concatenation).
To disable the check do the following (from here): (see my comment below for what double escaping entails).
<system.webServer>
<security>
<requestFiltering allowDoubleEscaping="true"/>
</security>
</system.webServer>
If the plus symbol is a valid character in a search input, you will need to enable "allowDoubleEscaping" to permit IIS to process such input from the URI's path.
Finally, a very simple, if limited workaround is simply to avoid '+' and use '%20' instead. In any case, using the '+' symbol to encode a space is not valid url encoding, but specific to a limited set of protocols and probably widely supported for backwards-compatibility reasons. If only for canonicalization purposes, you're better off encoding spaces as '%20' anyhow; and this nicely sidesteps the IIS7 issue (which can still crop up for other sequences, such as %25ab.)
I would just like to add some information to Eamon Nerbonne's answer related to the "what to do" part of your question (not explaining the whys).
You can easily change a particular application's settings too with
opening the console with admin rights (Start - cmd - right click, Run as administrator)
typing in the following (taken from here: http://blogs.iis.net/thomad/archive/2007/12/17/iis7-rejecting-urls-containing.aspx):
%windir%\system32\inetsrv\appcmd set config "YOURSITENAME" -section:system.webServer/security/requestfiltering -allowDoubleEscaping:true
(you can e.g. substitute YOURSITENAME with Default Web Site for applying this rule to the default website)
Enter, ready.
An example:
firstly I had the same problem:
Typing in the text mentioned above:
Now it works as expected:
Have you thought about having the search URL like '/search/a/b/c'?
You'd need to setup a route like
search/{*path}
And then extract the search values from your path string in the action.
I ran into this under IIS 7.5 doing a Server.TransferRequest() in an application.
Encoding the filename caused the double-escape problem, but if I didn't encode it then I'd run into the "potentially dangerous Request.Path" error.
Putting an any protocol, even an empty one, on the URL I pass to Server.TranferRequest() fixed the problem.
Does not work:
context.Server.TransferRequest("/application_name/folder/bar%20bar.jpg");
Works:
context.Server.TransferRequest("://folder/bar%20bar.jpg");
Related
I have problem like
HTTP Error 400.0 - Bad Request
ASP.NET detected invalid characters in the URL.
... when requesting the following URL:
http://hunarmandindia.com/(S(4wuduaje2xivjefwffjv5bgq))/Company/recruiter-login.aspx
If anyone can have any solution about this problem please give me solution...
There are a number of characters that are prevented by IIS/ASP.NET by default in a URL as they are a telltale sign of someone trying to perform certain attacks on your site. You can disable or loosen these checks, as described here (http://dirk.net/2008/06/09/ampersand-the-request-url-in-iis7/) for example, but before you go ahead and do that, I'd first check whether you truly need the invalid characters that are being used (likely ones are ampersands, angle brackets, the word 'script' etc. but it's hard to tell without seeing the URL in question), as loosening these rules obviously opens you up to the attacks this feature is trying to prevent.
I'm attempting to make a web request through a WebClient in a Windows Phone 8.1 Application. The web request contains a set of search terms, which whilst building the request signature are percent encoded as such: OAuthTools.UrlEncodeStrict(searchTerms), this is done because the search terms likely contain spaces. I then add all other parameters, and generate a signature. The request URL will then look something like this:
http://platform.fatsecret.com/rest/server.api?format=json&max_results=10&method=recipes.search&oauth_consumer_key=******&oauth_nonce=ksoordxpd5oahp9e&oauth_signature=Uxq3pmkTkRrASO%2Bmoiujs24BtT4%3D&oauth_signature_method=HMAC-SHA1&oauth_timestamp=1420634671&oauth_version=1.0&page_number=0&search_expression=pumpkin%20pie
As you can see, the final parameter, search_expression=pumpkin%20pie is encoded correctly. However, when putting this URL into a URI object (as the WebClient.DownloadStringAsync method requires), the URL is decoded to:
http://platform.fatsecret.com/rest/server.api?format=json&max_results=10&method=recipes.search&oauth_consumer_key=******&oauth_nonce=ksoordxpd5oahp9e&oauth_signature=Uxq3pmkTkRrASO%2Bmoiujs24BtT4%3D&oauth_signature_method=HMAC-SHA1&oauth_timestamp=1420634671&oauth_version=1.0&page_number=0&search_expression=pumpkin pie
Removing the escaped spacing for the search_expression parameter. This results, in this case, in an invalid OAuth signature (as the parameters change), but using an unescaped search_expression with spaces also results in an invalid OAuth signature (I assume because REST parameters should not contain spaces). Is there a way to ensure that the URI class does not decode my URL string? I have found this answer explaining that a workaround is to include:
<uri>
<schemeSettings>
<add name="http" genericUriParserOptions="DontUnescapePathDotsAndSlashes" />
</schemeSettings>
</uri>
Inside of the Web.config or App.config file. However, there appears to be no suitable config file to place this in inside a Windows Phone 8.1 development environment (I have tried adding it to packages.config and the app manifest, neither of which had any effect). Is there another way around this issue? Or alternatively, is there another way I can send a web request using a raw string, as opposed to through a URI object?
I'm getting the error "HTTP Error 414. The request URL is too long." From the following article, I understand that this is due to a very long query string:
http://www.mytecbits.com/microsoft/iis/query-string-too-long
In web.config, I have maxQueryStringLength="2097151". Is this the maximum value?
In order to solve this problem, should I set maxUrl in web.config? If so, what's the maximum value supported?
What should I do to fix this error?
This error is actually thrown from http.sys, not from IIS. The error gets thrown before the request is passed along to IIS in the request-handling pipeline.
To verify this, you can check the Server header value in the HTTP response headers, as per https://stackoverflow.com/a/32022511/12484.
To get https.sys to accept longer request URLs without throwing the HTTP 414 error, in the Windows Registry on the server PC, at Computer\HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\HTTP\Parameters, create a DWORD-type value with name MaxFieldLength and value sufficiently large, e.g. 65535.
Reference: Http.sys registry settings for Windows
If you decide to make this change, then obviously it’ll need to be made in all environments (including all production server(s)) -- not just on your local dev PC.
Also, whatever script and/or documentation your team uses to set up new server instances will need to be updated to include this registry setting, so that your team doesn’t forget to apply this setting 18 months from now when setting up a new production server.
Finally, be aware making this change could have adverse security consequences for all applications running on your server, as a large HTTP request submitted by an attacker won’t be rejected early in the pipeline as it would normally.
As an alternative to making this change to bypass the http.sys security, consider changing the request to accept HTTP POST instead of HTTP GET, and put the parameters into the POST request body instead of into a long URL. For more discussion on this, see question Design RESTful GET API with a long list of query parameters.
As described in this answer -> What is the maximum length of a URL in different browsers?
The allowed length of a url depends on a combination of browser and server. Hence it's hard to say exactly how long the url can be. The answer recommends to stay below 2000 char in the url. I do not know why your querystring is so long. Can you shorten it? It's hard to give you any recommendations without knowing more about the solution and your query string.
Generally, Url has its own limits in length and if you set this value you may solve the problem for a while, but bear in mind that for a long url situations, best practice is working with forms. To be specific, it is better to use POST actions instead of Get.
just to complement, if you try with massive parameters, using Request ajax and receive de 414 ERROR. change the dataType property to JSON then submit as POST type.
this resolved my problem.
I've got a custom HttpModule to redirect legacy URLs from an old build of the site which checks the incoming request URL against a database table of redirects.
However, when the incoming request URL contains a plus (+) sign, the request doesn't fall through the HttpModule - it works as expected for standard URLs.
For example, these URLs works:
http://www.example.com/sample-url
http://www.example.com/sample url
http://www.example.com/sample%20url
These don't:
http://www.example.com/sample+url
http://www.example.com/sample%2Burl
Here's my module declaration:
<add name="LegacyUrlHttpModule" type="Web.LegacyUrlHttpModule, Framework.Web" preCondition="managedHandler" />
Am I missing a setting here or something?
Scott Hanselmann wrote a nice blog post explaining how you could enable all kind of crap symbols in the Path portion of an url.
His conclusion is the following:
After ALL this effort to get crazy stuff in the Request Path, it's
worth mentioning that simply keeping the values as a part of the Query
String (remember WAY back at the beginning of this post?) is easier,
cleaner, more flexible, and more secure.
So basically if you have such characters in a url, those characters should be passed as query string parameters instead of attempting to pass them in the Path portion.
IIS rejects + in URLs by default. A workaround would be to allowDoubleEscaping
<system.webServer>
<security>
<requestFiltering allowDoubleEscaping="true" />
</security>
</system.webServer>
but beware that this may make your site more vulnerable to malicious URLs.
You can follow the below steps:
in the IIS webserver section, double click the 'Request Filtering' icon
in the 'File Name Extension' right click->Edit Feature Settings...' the file 'web.config'
check the option 'Allow double escaping' (this option is unchecked by default)
repeat all above 3 steps for the 'default website' (or whatever you have given the name to your site)
re-start the IIS
My WebApp is part CMS, and when I serve up an HTML page to the user it typically contains relative paths in a.href and img.src attributes.
I currently have them accessed by urls like: ~/get-data.aspx/instance/user/page.html -- where instance indicates the particular instance for the report and "user/page.html" is a path created by an external application that generates the content.
This works pretty reliably with code in the application's BeginRequest method that translates the text after ".aspx" into a query string, then uses Context.RewritePath().
So far so good, but I've just tripped over something that took me by surprise: it appears that if any of the query string ("instance/user/page.html") happens to contain a plus sign ("+") the BeginRequest method is never called, and a 404 is immediately returned to the user.
So my question is two-fold:
Am I correct in my belief that a "+" would cause the 404, and if so are there other things that could cause similar problems? Is there a way around that problem (perhaps a different method than BeginRequest)?
Is there a better way to preserve relative URL paths for generated content than what I'm using? I'd rather not require site admins to install a 3rd party rewrite tool if I can help it.
Yes this is an IIS security setting. You can get round it by putting this in your web config.
<system.webServer>
<security>
<requestFiltering allowDoubleEscaping="True"/>
</security>
</system.webServer>
However youmight have to be carefull what you are doing. With double escaping enabled it will allow an escaped url to itself contain escape characters that could be used to exploit your url rewriting to pass escaped characters into your querystring to construct a different querystring than you intend.