Develop Reference

Decoding querystring values on the server side

I'm running a web service on my server using WCF and .Net 4. The contract type is WebGet. Here is my problem, at one point in time, someone was sending data through the querystring that was URL encoded. So I added HttpUtility.UrlDecode to decode the parameters. I think that fixed my issue at the time. Now, I've sent a URL encoded string to it and I see that the string is being URL decoded coming into the method (before even getting to the HttpUtility.UrlDecode).
So now I'm confused, if the .Net code is decoding it before it gets to my method, why would I need to call on decode explicitly? But for a time it wasn't, so is this a recent change to the underlying .Net framework?
My problem now is that my users are sending data (unencoded), where the data looks like this: "abc%1234" and I'm getting "abc34", the decoding is eating 3 characters. However, if I urlencode the % sign to be "abc%251234", the value coming into the method is "abc%1234" (what I expected) and then the call to HttpUtility.UrlDecode is changing it to "abc34" (which is not what I expected).
I'm not sure how to proceed here. Do I rip out the explicit call to URLDecode until it starts coming across encoded again or is there a better way to handle this?

It's a subtle thing in documentation, easily missed:
HttpRequest.QueryString Property
Property Value
NameValueCollection
The query string variables sent by the client. Keys and values are
URL-decoded.
So if you access the query string via HttpRequest.QueryString (or Params) collection they are already decoded.
You can get to the raw string in RawUrl, QueryString.ToString() (manually that is - re: manipulation, split, etc.).
End of day, %:
Because the percent ("%") character serves as the indicator for
percent-encoded octets, it must be percent-encoded as "%25" for that
octet to be used as data within a URI.
REF: RFC3986
Hth

C# HttpClient URI is not escaping properly

I'm trying to get my URL to escape but it's not working properly. Ironically, on my MacBook when I execute this part of code
Uri url = new Uri("http://www.example.com/?i=123%34", true);
// it returns http://www.example.com/?i=123%34 which is exactly what I want.
The problem is that my IDE says it's obsolete and it does not work on my Windows machine. It's the exact same project, and IDE. So I tried to find a solution, which someone suggested
Uri uri = new Uri(Uri.EscapeUriString("http://www.example.com/?i=123%34"));
// this returns http://www.example.com/?i=123%2534 which is what I DONT want.
So how do I approach this issue? I looked all over the web and I can't find any solutions. I need to know how to properly escape this URL. The second method posted above does not work like the first method above.
I verified the GET requests via Fiddler, so everything is indeed happening.
Update:
Again, I need the server to receive the URL exactly how the string is declared. I want the server to handle the conversion. I cannot substitute %25 for the % symbol. It MUST be received exactly how I the string is declared. Additionally, "http://www.example.com/?i=1234" is NOT what I want either.

The problem is with the configuration of your web server on Windows, that allows double escaping. Your original URL is http://www.example.com/?i=123%34, which when unescaped, becomes http://www.example.com/?i=1234.
Your web server on Windows, on the other hand, escapes the % character again instead of unescaping %34. Thus, it turns into http://www.example.com/?i=123%2534.
This is why you should not use characters like % in the URL before it gets escaped.
Edit -
I typed the following two URLs in Firefox to see how the parameters are received on the server.
The value of i in http://www.example.com/?i=123%34 is 1234.
The value of i in http://www.example.com/?i=123%2534 is 123%34
If the server must receive the % character, it must be escaped in order for it to be dispatched over HTTP. There's literally no other way to send it over the wire. If you don't escape the % character, it will be treated as an escape sequence along with 34 and automatically turn into 4 on the server.
If your network inspector shows you unescaped text in the request, it's because it's prettifying the URL before displaying it to you.

If you are okay with the string reading ht tp://www.example.com/?i=1234, you can try
Uri url = new Uri(Uri.UnescapeDataString("http://www.example.com/?i=123%34"));

ASP.NET MVC Override POST parse of fields

I wrote an MVC action to receive a post from a service. My problem is that the service is posting multipart data with wrong encoding.
Let me give an example:
The service will post the "á" for the form field "text".
I see (using Wireshark), that the byte written on the packet is 225, which is the right byte for "á" in ISO-8859-1 .
I do Request.Form["text"] and actually get a strange (different) char.
I believe this is cause by .NET attempting to convert the value 225 to a unicode char, when converting to string using the utf-8 encoding, but couldn't, as 225 isn't valid for utf-8.
So my question is: Is there a way to override the parsing of those bytes to string?

You could try to add a HttpModule and try to overwrite the ContentEncoding property of the Request object. Though I'm not sure this will work.
It's possible to set the default encoding in Web.Config's GlobalizationSection. The setting is called RequestEncoding and is taken to effect only if the HTTP request of your service does not contain a ContentType header. See http://msdn.microsoft.com/en-us/library/system.web.configuration.globalizationsection.requestencoding.aspx
You can further use inside Web.config to set the above setting only to a specific directory / MVC controller.

Which HttpUtility decode method to use?

this may be a silly question, but it trips me up every time.
HttpUtility has the methods HtmlDecode and UrlDecode. Do these two methods decode anything (Html/Http related) I might find? When do I have to use them, and which one am I supposed to use?
Just now I hit an error. This is my error log:
Payment receiver was not payment#mysite.com. (it was payment%40mysite.com).
But, I wrapped the email address here in HttpUtility.HtmlDecode before using it. It turns out I have to use .UrlDecode instead, but this email address didn't come from a URL so this wasn't obvious to me.
Can someone clarify this?

See What is meant by htmlencode and urlencode?
It's the reverse of your case, but essentially you need to use UrlEncode/Decode anytime you are using an address of sorts (urls and yes, email addresses). HtmlEncode/Decode is for code that typically a browser would render (html/xml tags).

This same encoding is also used in Form POST requests as well.
My guess is something read it 'naked' without decoding it.
Html Encoding/Decoding is only used to escape strings that contain characters that would otherwise be interpreted as html control characters. The process turns the characters into html entities and back again.
Url Encoding is to get around the fact that many characters are not allowed in Uris; or because they too could be misinterpreted. Thus the percent encoding is used.
Percent encoding is also used in the body of http requests.
In both cases, of course, it's also a way of expressing a specific character code in a request/response independent of character sets; but equally, interpreting what is meant by a particular code can also be dependent on knowing a particular character set. Generally you don't worry about that - but it can be important (especially in the HTML case).

URLEncode converts characters that aren't allowed in a URL into character equivalents which are parsable as a URL. In your example # became %40. URLDecode reverses this.
HTMLEncode is similar to URLEncode, but the target environment is text NESTED inside of HTML. This helps the browser from interpereting your content as HTML, but when rendered it should look like the decoded version. HTMLDecode reverses this.

When you see %xx this means percent encoding has occured - this is a URL encoding scheme, so you need to use UrlEncode / UrlDecode.
The HtmlEncode and HtmlDecode methods are for encoding and decoding elements for HTML display - so things like & get encoded to & and > to >.

C#, UTF-8 and encoding characters

This is a shot-in-the-dark, and I apologize in advance if this question sounds like the ramblings of a madman.
As part of an integration with a third party, I need to UTF8-encode some string info using C# so I can send it to the target server via multipart form. The problem is that they are rejecting some of my submissions, probably because I'm not encoding their contents correctly.
Right now, I'm trying to figure out how a dash or hyphen -- I can't tell which it is just by looking at it -- is received or interpreted by the target server as ?~#~S (yes, that's a 5-character string and is not your browser glitching out). And unfortunately I don't have a thorough enough understanding of Encoding.UTF8.GetBytes() to know how to use the byte array to begin identifying where the problem might lie.
If anybody can provide any tips or advice, I would greatly appreciate it. So far my only friend has been MSDN, and not much of one at that.
UPDATE 1: After some more digging around, I discovered that using System.Web.HttpUtility.UrlEncode()to encode an EM DASH character ("—") will hex-encode it into "%e2%80%94".
I'm currently sending this info in aHttpWebRequestpost, with a content type of "application/x-www-form-urlencoded" -- could this be what's causing the problem? And if so, what is the proper way to encode a series of name-value pairs whose values may contain Unicode characters, such that it will be understood by a server expecting a UTF-8 request?

byte[] test = System.Text.Encoding.UTF8.GetBytes("-");
Should give you
test[0] = 0x2D (45 as integer).
Verify that your sending 0x2D to the target server.

You may need to add a "charset=utf-8" parameter to your Content-Type header. You may also want to have a Content-Encoding header to set your encoding. The headers should contain the following:
Content-Type: multipart/form-data; charset=utf-8
Otherwise, the web server won't know your bytes are UTF-8 bytes, so it will misinterpret them.