I'm attempting to make a web request through a WebClient in a Windows Phone 8.1 Application. The web request contains a set of search terms, which whilst building the request signature are percent encoded as such: OAuthTools.UrlEncodeStrict(searchTerms), this is done because the search terms likely contain spaces. I then add all other parameters, and generate a signature. The request URL will then look something like this:
http://platform.fatsecret.com/rest/server.api?format=json&max_results=10&method=recipes.search&oauth_consumer_key=******&oauth_nonce=ksoordxpd5oahp9e&oauth_signature=Uxq3pmkTkRrASO%2Bmoiujs24BtT4%3D&oauth_signature_method=HMAC-SHA1&oauth_timestamp=1420634671&oauth_version=1.0&page_number=0&search_expression=pumpkin%20pie
As you can see, the final parameter, search_expression=pumpkin%20pie is encoded correctly. However, when putting this URL into a URI object (as the WebClient.DownloadStringAsync method requires), the URL is decoded to:
http://platform.fatsecret.com/rest/server.api?format=json&max_results=10&method=recipes.search&oauth_consumer_key=******&oauth_nonce=ksoordxpd5oahp9e&oauth_signature=Uxq3pmkTkRrASO%2Bmoiujs24BtT4%3D&oauth_signature_method=HMAC-SHA1&oauth_timestamp=1420634671&oauth_version=1.0&page_number=0&search_expression=pumpkin pie
Removing the escaped spacing for the search_expression parameter. This results, in this case, in an invalid OAuth signature (as the parameters change), but using an unescaped search_expression with spaces also results in an invalid OAuth signature (I assume because REST parameters should not contain spaces). Is there a way to ensure that the URI class does not decode my URL string? I have found this answer explaining that a workaround is to include:
<uri>
<schemeSettings>
<add name="http" genericUriParserOptions="DontUnescapePathDotsAndSlashes" />
</schemeSettings>
</uri>
Inside of the Web.config or App.config file. However, there appears to be no suitable config file to place this in inside a Windows Phone 8.1 development environment (I have tried adding it to packages.config and the app manifest, neither of which had any effect). Is there another way around this issue? Or alternatively, is there another way I can send a web request using a raw string, as opposed to through a URI object?
Related
So I have a URL encoded path where the last part of the route is generated with WebUtility.UrlEncode (asp.net core 3.1)
blah/blah/MRgeOkgo8fqRVrQbUawTmtP6DbSN42QTXLqH1064Wl9P1iyi3v9%2F%2BmB36VOcz8WD7qSKoyJ3%2B0mZ874WQxAlptzQo6mylIa%2BN%2BKasrdkFXY0whRafA48UknQtP9BYXkg6QSfDGOxAu8Fl%2F%2Bq%2BIftYw%3D%3D
this is url encoded, so on the code at the path /blah/blah I decode it with WebUtility.UrlDecode to get
MRgeOkgo8fqRVrQbUawTmtP6DbSN42QTXLqH1064Wl9P1iyi3v9/ mB36VOcz8WD7qSKoyJ3 0mZ874WQxAlptzQo6mylIa N KasrdkFXY0whRafA48UknQtP9BYXkg6QSfDGOxAu8Fl/ q IftYw==
(which I then run through another encryption algorithmn to get the data out).
The problem is the URL encoded path generates a 404.11 when passed into IIS. IIS rejects the request because of double escaping. I know how to fix this, just add in
<security>
<requestFiltering allowDoubleEscaping="true" />
</security>
to the web config.
However, I don't know why IIS is generating a 404.11 because the URL looks fine.
What gives?
As mentioned by #Dai, I switched to encoding the data to a url part with WebEncoders.Base64UrlEncode (from Microsoft.AspNetCore.WebUtilities) and all good now.
I am stuck at an unexpected issue in my project. The issue is that there is a URL produced on the fly in my code that I have to submit it to a RESTful web service via a GET request. For e.g. the URL to submit looks like this: http://mysampleserver.com:8080/calc/8999/bpaX
The RESTful server accepts URL as its last parameter in the format below:
http://myRestfulAPI.domainname.com/capture/bbbb/http://mysampleserver.com:8080/calc/8999/bpaX
I also used System.Net.HttpUtility.UrlEncode(....) to encode the "URL to submit" first to incorporate it in the RESTful service call.
That resulted in getting the error below:
System.Web.HttpException: A potentially dangerous Request.Path value was detected from the client (:)
To try to resolve it, I followed the steps described per this web page but no luck.
I am using MVC 4 to implement the RESTful API in C#.
Any clue or idea how to get around this showstopper issue?
There are at least two solutions I can think of.
Change your RESTFul service to use post, because you send information to your server, and potentially it will change your resource status, based on HTTP protocol , you should use POST anyway.
You can also encode your url with Base64
The steps that you've tried are the correct steps. See also this question potentially dangerous... which is the same issue.
There are a number of characters that .NET doesn't allow in in a URL by default, and the : is one of them (as a query string, at least). They are 'potentially dangerous'. Making this change to the configuration file allows these characters to be passed through to your application.
You need to Url.Encode the url in the query string (mvc parameters) otherwise it is interpreted as more URL encoding for MVC to decode as parameters. Try something like #Url.Encode(yourStringObject) and pass it as the last value or as a query (i.e. &q=url)
are WCF service arguments automatically URI decoded or do I manually do it?
EDITED TO ADD:
When I originally put forward this answer I seem to remember having verified it in a test. Upon #JohnSaunders comment below I revisited with a new project. And found that a console app submitting the string above returned the string exactly as it had been submitted, without having URI encoded it. It may be that I did something unexpected previously. Anyway, #JohnSaunders is correct.
Original Incorrect Answer:
No, they are not, and yes you have to URI encode them if you're expecting special characters.
If you were to submit this to a WCF service in the stream of data passed to it:
"http://localhost/users/email/bill#microsoft.com"
It would show up on the server like this:
"bill"
The "#microsoft" gets removed
The passed data must be URI encoded to get it all to pass through.
I have an ASP.NET MVC application with a route that allows searching for stuff via /search/<searchterm>.
When I supply "search/abc" it works well, but when I supply "/search/a+b+c" (correctly url encoded) then IIS7 rejects the request with HTTP Error 404.11 (The request filtering module is configured to deny a request that contains a double escape sequence). FIrst of all, why does it do this? It only seems to throw the error if it is part of the URL, but not as part of a query string ( /transmit?q=a+b+c works fine).
Now I could enable double escape requests in the security section of my web.config but I'm hesitant to do so as I don't understand the implications, and neither why the server would reject the request "a+b+c" as part of the URL but accept as part of a query string.
Can someone explain and give some advice what to do?
Edit: Added emphasis to relevant sections.
Basically: IIS is being excessively paranoid. You can safely disable this check if you're not doing anything particularly unwise with the uri decoded data (such as generating local filesystem URI's via string concatenation).
To disable the check do the following (from here): (see my comment below for what double escaping entails).
<system.webServer>
<security>
<requestFiltering allowDoubleEscaping="true"/>
</security>
</system.webServer>
If the plus symbol is a valid character in a search input, you will need to enable "allowDoubleEscaping" to permit IIS to process such input from the URI's path.
Finally, a very simple, if limited workaround is simply to avoid '+' and use '%20' instead. In any case, using the '+' symbol to encode a space is not valid url encoding, but specific to a limited set of protocols and probably widely supported for backwards-compatibility reasons. If only for canonicalization purposes, you're better off encoding spaces as '%20' anyhow; and this nicely sidesteps the IIS7 issue (which can still crop up for other sequences, such as %25ab.)
I would just like to add some information to Eamon Nerbonne's answer related to the "what to do" part of your question (not explaining the whys).
You can easily change a particular application's settings too with
opening the console with admin rights (Start - cmd - right click, Run as administrator)
typing in the following (taken from here: http://blogs.iis.net/thomad/archive/2007/12/17/iis7-rejecting-urls-containing.aspx):
%windir%\system32\inetsrv\appcmd set config "YOURSITENAME" -section:system.webServer/security/requestfiltering -allowDoubleEscaping:true
(you can e.g. substitute YOURSITENAME with Default Web Site for applying this rule to the default website)
Enter, ready.
An example:
firstly I had the same problem:
Typing in the text mentioned above:
Now it works as expected:
Have you thought about having the search URL like '/search/a/b/c'?
You'd need to setup a route like
search/{*path}
And then extract the search values from your path string in the action.
I ran into this under IIS 7.5 doing a Server.TransferRequest() in an application.
Encoding the filename caused the double-escape problem, but if I didn't encode it then I'd run into the "potentially dangerous Request.Path" error.
Putting an any protocol, even an empty one, on the URL I pass to Server.TranferRequest() fixed the problem.
Does not work:
context.Server.TransferRequest("/application_name/folder/bar%20bar.jpg");
Works:
context.Server.TransferRequest("://folder/bar%20bar.jpg");
My WebApp is part CMS, and when I serve up an HTML page to the user it typically contains relative paths in a.href and img.src attributes.
I currently have them accessed by urls like: ~/get-data.aspx/instance/user/page.html -- where instance indicates the particular instance for the report and "user/page.html" is a path created by an external application that generates the content.
This works pretty reliably with code in the application's BeginRequest method that translates the text after ".aspx" into a query string, then uses Context.RewritePath().
So far so good, but I've just tripped over something that took me by surprise: it appears that if any of the query string ("instance/user/page.html") happens to contain a plus sign ("+") the BeginRequest method is never called, and a 404 is immediately returned to the user.
So my question is two-fold:
Am I correct in my belief that a "+" would cause the 404, and if so are there other things that could cause similar problems? Is there a way around that problem (perhaps a different method than BeginRequest)?
Is there a better way to preserve relative URL paths for generated content than what I'm using? I'd rather not require site admins to install a 3rd party rewrite tool if I can help it.
Yes this is an IIS security setting. You can get round it by putting this in your web config.
<system.webServer>
<security>
<requestFiltering allowDoubleEscaping="True"/>
</security>
</system.webServer>
However youmight have to be carefull what you are doing. With double escaping enabled it will allow an escaped url to itself contain escape characters that could be used to exploit your url rewriting to pass escaped characters into your querystring to construct a different querystring than you intend.