I have a bit of a strange problem. I have a controller that is supposed to receive a URL as one of the parameters. I am encoding the URL, and trying to call the controller as such:
http://www.mysite.com/dowork/1/http%3a%2f%2fwww.otherurl.com
However, I am getting the error
A potentially dangerous Request.Path value was detected from the client (:).
What gives? Any way to handle this WITHOUT disabling validation?
If that is .NET 4.0, you can edit this configuration setting:
<system.web>
<httpRuntime requestPathInvalidCharacters="<,>,%,&,:,\,?" />
</system.web>
If you don't wish to edit this, you can additionally encode your incoming url - use custom encoding (like replacing dangerous characters) or use base64 encoding or omit protocol part if possible before sending to controller (drop http://, if https:// is possible also, you need to think how to pass that).
IF you pass it as a query string parameter (?path=....) you avoid the problem, the characters are only illegal as part of a path.
If you insist on passing the data as part of the path you will need to encode differently, like convert the string into a range of hexadecimal values or similar.
Related
I'm running a web service on my server using WCF and .Net 4. The contract type is WebGet. Here is my problem, at one point in time, someone was sending data through the querystring that was URL encoded. So I added HttpUtility.UrlDecode to decode the parameters. I think that fixed my issue at the time. Now, I've sent a URL encoded string to it and I see that the string is being URL decoded coming into the method (before even getting to the HttpUtility.UrlDecode).
So now I'm confused, if the .Net code is decoding it before it gets to my method, why would I need to call on decode explicitly? But for a time it wasn't, so is this a recent change to the underlying .Net framework?
My problem now is that my users are sending data (unencoded), where the data looks like this: "abc%1234" and I'm getting "abc34", the decoding is eating 3 characters. However, if I urlencode the % sign to be "abc%251234", the value coming into the method is "abc%1234" (what I expected) and then the call to HttpUtility.UrlDecode is changing it to "abc34" (which is not what I expected).
I'm not sure how to proceed here. Do I rip out the explicit call to URLDecode until it starts coming across encoded again or is there a better way to handle this?
It's a subtle thing in documentation, easily missed:
HttpRequest.QueryString Property
Property Value
NameValueCollection
The query string variables sent by the client. Keys and values are
URL-decoded.
So if you access the query string via HttpRequest.QueryString (or Params) collection they are already decoded.
You can get to the raw string in RawUrl, QueryString.ToString() (manually that is - re: manipulation, split, etc.).
End of day, %:
Because the percent ("%") character serves as the indicator for
percent-encoded octets, it must be percent-encoded as "%25" for that
octet to be used as data within a URI.
REF: RFC3986
Hth
I am receiving the rather self explanatory error:
A potentially dangerous Request.Path value was detected from the client (*).
The issue is due to * in the request URL:
https://stackoverflow.com/Search/test*/0/1/10/1
This url is used to populate a search page where 'test*' is the search term and the rest of the url relates to various other filters.
Is there an easy way to allow these special characters in the URL? I've tried modifying the web.config, to no avail.
Should I manually encode / decode the special characters?
Or is there a best practice for doing this, I would like to avoid using query strings. - but it may be an option.
The application itself is a c# asp.net webforms application that uses routing to produce the nice URL above.
If you're using .NET 4.0 you should be able to allow these urls via the web.config
<system.web>
<httpRuntime
requestPathInvalidCharacters="<,>,%,&,:,\,?" />
</system.web>
Note, I've just removed the asterisk (*), the original default string is:
<httpRuntime
requestPathInvalidCharacters="<,>,*,%,&,:,\,?" />
See this question for more details.
The * character is not allowed in the path of the URL, but there is no problem using it in the query string:
http://localhost:3286/Search/?q=test*
It's not an encoding issue, the * character has no special meaning in an URL, so it doesn't matter if you URL encode it or not. You would need to encode it using a different scheme, and then decode it.
For example using an arbitrary character as escape character:
query = query.Replace("x", "xxx").Replace("y", "xxy").Replace("*", "xyy");
And decoding:
query = query.Replace("xyy", "*").Replace("xxy", "y").Replace("xxx", "x");
For me, I am working on .net 4.5.2 with web api 2.0,
I have the same error, i set it just by adding requestPathInvalidCharacters=""
in the requestPathInvalidCharacters you have to set not allowed characters else you have to remove characters that cause this problem.
<system.web>
<httpRuntime targetFramework="4.5.2" requestPathInvalidCharacters="" />
<pages >
<namespaces>
....
</namespaces>
</pages>
</system.web>
**Note that it is not a good practice, may be a post with this parameter as attribute of an object is better or try to encode the special character.
-- After searching on best practice for designing rest api, i found that in search, sort and paginnation, we have to handle the query parameter like this
/companies?search=Digital%26Mckinsey
and this solve the problem when we encode & and remplace it on the url by %26
any way, on the server we receive the correct parameter Digital&Mckinsey
this link may help on best practice of designing rest web api
https://hackernoon.com/restful-api-designing-guidelines-the-best-practices-60e1d954e7c9
You should encode the route value and then (if required) decode the value before searching.
For me, when typing the url, a user accidentally used a / instead of a ? to start the query parameters
e.g.:
url.com/endpoint/parameter=SomeValue&otherparameter=Another+value
which should have been:
url.com/endpoint?parameter=SomeValue&otherparameter=Another+value
This exception occurred in my application and was rather misleading.
It was thrown when I was calling an .aspx page Web Method using an ajax method call, passing a JSON array object. The Web Page method signature contained an array of a strongly-typed .NET object, OrderDetails.
The Actual_Qty property was defined as an int, and the JSON object Actual_Qty property contained "4 " (extra space character).
After removing the extra space, the conversion was made possible, the Web Page method was successfully reached by the ajax call.
Try to set web project's server propery as Local IIS if it is IIS Express. Be sure if project url is right and create virual directory.
When dealing with Uniform Resource Locator(URL) s there are certain syntax standards, in this particular situation we are dealing with Reserved Characters.
As up to RFC 3986, Reserved Characters may (or may not) be defined as delimiters by the generic syntax, by each scheme-specific syntax, or by the implementation-specific syntax of a URI's dereferencing algorithm; And asterisk(*) is a Reserved Character.
The best practice is to use Unreserved Characters in URLs or you can try encoding it.
Keep digging :
HTML URL Encoding Reference (w3schools)
When to Encode or Decode (RFC 3986)
I had a similar issue in Azure Data Factory with the : character.
I resolved the problem by substituting : with %3A
as shown here.
For example, I substituted
date1=2020-01-25T00:00:00.000Z
with
date1=2020-01-25T00%3A00%3A00.000Z
I know you can apply a wildcard in the route attribute to allow / such as date input for example:
[Route("orders/{*orderdate}")]
The problem with wildcard is only applicable to the last paramter in URI. How do I solve the issue if want to have the following URI:
[Route("orders/{orderdate}/customers")]
Update:
I know there are few options to solve the issue by refactoring the code so please do not offer a solution something like:
change the route template to [Route("orders/customers/{orderdate}")]
change the date to a different format (e.g. "dd-mm-yyyy")
#bet.. I think the genericUriParserOptions is no longer applicable to .net 4.5 or later..
Also as suggested by #JotaBe, you might need to correctly decode the url request. In most case the %2F will be automatically translated to a slash '/'. So if you need to escape it you will need to decode the '%' char in the first place.. so your URL: will look something like: www.domain.com/api/orders/23%252F06%252F2015/customers
Notice the characters '%252F' will be translated to the actual '%2F'
EDIT
Ok here is the complete solution (Tried it and working for me):
Assuming you have an API endpoint like so:
[Route("orders/{date}/customers")]
public HttpResponseMessage Get(string date)
{
}
In the web.config you will need to set the requestPathInvalidCharacters to empty which tells the asp.net to allow all request
<system.web>
<httpRuntime targetFramework="4.5" requestPathInvalidCharacters=""/>
</system.web>
<system.webServer>
<security>
<requestFiltering allowDoubleEscaping="true" />
</security>
</system.webServer>
When the client sending the request to the API you will need to make sure to escape the '%' like so:
www.domain.com/api/orders/23%252F06%252F2015/customers
You then need to decode the request
[Route("orders/{date}/customers")]
public HttpResponseMessage Get(string date)
{
DateTime actualDate = DateTime.Parse(System.Net.WebUtility.UrlDecode(date)); // date is 23/06/2015
}
As noted in the comment by #AlexeiLevenkov, this is wrong:
You cannot have a parameter in the URL which accepts forward slashes, because this is a special symbol which separates each URL fragment. So, whenever you include this symbol in your URL, there will be new fragments, and a single parameter can't include several fragments.
If you want more details, read this, but these are the most relevant excerpts:
the URL path finishes in the first ? or # found in the URL. So, the slashes only create fragments in the section of the URL path before the occurrence or one of those symbols.
From section 3.4: The query component is indicated by the first question mark ("?") character and terminated by a number sign ("#") character or by the end of the URI.
So, the query string can include forward slashes, /, if desired, and they will not define path segments at all.
These are some solutions for the question:
include fragments for day, month and year, like this: [Route("orders/{month}/{day}/{year}/customers")] and then create the date on the server side
require the user to use a different separator, like dash or dot, which won't create problems, receive it at string an parse it yourself (or use your own custom binder to support that format)
use the URL Rewrite extension to change the URL before it reaches the routing system, and parse it as explained in the previous solution (this requires hosting in IIS)
receive it as a query string, i.e. something like this: ´?date=02/03/2015´ (you'd better encode it)
NOTE: your original question said "query string", and my comment about encoding referred to the query string, which is the last segment of an URL after the question mark, if present, like &id=27. I corrected your question so that it doesn't mention "query string", which was not the right name for what you need
C# has its own method who skips the rules of escape sequences
the name of method is
Uri.UnescapeDataString(your querystring parameter)
you can use it while getting the parameters value
You can use the following URI [Route("orders/{DD:int}/{MM:int}/{YY:int}}/customers")]
and then use a custom model binder to take DD/MM/YY and turn them into a date that you can bind in your action method.
You can choose how you want to deal with constraints (go stricter with regex's) or use validation and return 400 if it doesn't match.
The simpler approach is, to take the Day/Month/Year and put it together in code.
Here is a link for dealing with modelbinding.
I wrote an MVC action to receive a post from a service. My problem is that the service is posting multipart data with wrong encoding.
Let me give an example:
The service will post the "á" for the form field "text".
I see (using Wireshark), that the byte written on the packet is 225, which is the right byte for "á" in ISO-8859-1 .
I do Request.Form["text"] and actually get a strange (different) char.
I believe this is cause by .NET attempting to convert the value 225 to a unicode char, when converting to string using the utf-8 encoding, but couldn't, as 225 isn't valid for utf-8.
So my question is: Is there a way to override the parsing of those bytes to string?
You could try to add a HttpModule and try to overwrite the ContentEncoding property of the Request object. Though I'm not sure this will work.
It's possible to set the default encoding in Web.Config's GlobalizationSection. The setting is called RequestEncoding and is taken to effect only if the HTTP request of your service does not contain a ContentType header. See http://msdn.microsoft.com/en-us/library/system.web.configuration.globalizationsection.requestencoding.aspx
You can further use inside Web.config to set the above setting only to a specific directory / MVC controller.
I am receiving the rather self explanatory error:
A potentially dangerous Request.Path value was detected from the client (*).
The issue is due to * in the request URL:
https://stackoverflow.com/Search/test*/0/1/10/1
This url is used to populate a search page where 'test*' is the search term and the rest of the url relates to various other filters.
Is there an easy way to allow these special characters in the URL? I've tried modifying the web.config, to no avail.
Should I manually encode / decode the special characters?
Or is there a best practice for doing this, I would like to avoid using query strings. - but it may be an option.
The application itself is a c# asp.net webforms application that uses routing to produce the nice URL above.
If you're using .NET 4.0 you should be able to allow these urls via the web.config
<system.web>
<httpRuntime
requestPathInvalidCharacters="<,>,%,&,:,\,?" />
</system.web>
Note, I've just removed the asterisk (*), the original default string is:
<httpRuntime
requestPathInvalidCharacters="<,>,*,%,&,:,\,?" />
See this question for more details.
The * character is not allowed in the path of the URL, but there is no problem using it in the query string:
http://localhost:3286/Search/?q=test*
It's not an encoding issue, the * character has no special meaning in an URL, so it doesn't matter if you URL encode it or not. You would need to encode it using a different scheme, and then decode it.
For example using an arbitrary character as escape character:
query = query.Replace("x", "xxx").Replace("y", "xxy").Replace("*", "xyy");
And decoding:
query = query.Replace("xyy", "*").Replace("xxy", "y").Replace("xxx", "x");
For me, I am working on .net 4.5.2 with web api 2.0,
I have the same error, i set it just by adding requestPathInvalidCharacters=""
in the requestPathInvalidCharacters you have to set not allowed characters else you have to remove characters that cause this problem.
<system.web>
<httpRuntime targetFramework="4.5.2" requestPathInvalidCharacters="" />
<pages >
<namespaces>
....
</namespaces>
</pages>
</system.web>
**Note that it is not a good practice, may be a post with this parameter as attribute of an object is better or try to encode the special character.
-- After searching on best practice for designing rest api, i found that in search, sort and paginnation, we have to handle the query parameter like this
/companies?search=Digital%26Mckinsey
and this solve the problem when we encode & and remplace it on the url by %26
any way, on the server we receive the correct parameter Digital&Mckinsey
this link may help on best practice of designing rest web api
https://hackernoon.com/restful-api-designing-guidelines-the-best-practices-60e1d954e7c9
You should encode the route value and then (if required) decode the value before searching.
For me, when typing the url, a user accidentally used a / instead of a ? to start the query parameters
e.g.:
url.com/endpoint/parameter=SomeValue&otherparameter=Another+value
which should have been:
url.com/endpoint?parameter=SomeValue&otherparameter=Another+value
This exception occurred in my application and was rather misleading.
It was thrown when I was calling an .aspx page Web Method using an ajax method call, passing a JSON array object. The Web Page method signature contained an array of a strongly-typed .NET object, OrderDetails.
The Actual_Qty property was defined as an int, and the JSON object Actual_Qty property contained "4 " (extra space character).
After removing the extra space, the conversion was made possible, the Web Page method was successfully reached by the ajax call.
Try to set web project's server propery as Local IIS if it is IIS Express. Be sure if project url is right and create virual directory.
When dealing with Uniform Resource Locator(URL) s there are certain syntax standards, in this particular situation we are dealing with Reserved Characters.
As up to RFC 3986, Reserved Characters may (or may not) be defined as delimiters by the generic syntax, by each scheme-specific syntax, or by the implementation-specific syntax of a URI's dereferencing algorithm; And asterisk(*) is a Reserved Character.
The best practice is to use Unreserved Characters in URLs or you can try encoding it.
Keep digging :
HTML URL Encoding Reference (w3schools)
When to Encode or Decode (RFC 3986)
I had a similar issue in Azure Data Factory with the : character.
I resolved the problem by substituting : with %3A
as shown here.
For example, I substituted
date1=2020-01-25T00:00:00.000Z
with
date1=2020-01-25T00%3A00%3A00.000Z