EDIT: Please note that this is a WinForms application, not a web app.
I am using the WebClient.UploadValues(Uri, "POST", NameValueCollection) to send values to an instance of HttpListener. On the listener side, when the HttpListener.GetContext() method returns, I can access the sent data as a byte [].
I can convert this data to text using EncodingXXX.GetString(buffer) which returns the following:
Key1=Value1
Key2=Value2
...
Each item in the string is delimited by the ampersand sign &. Both the key and value are encoded using HttpUtility.HtmlEncode/HttpUtility.HtmlDecode so I can split the data based on ampersands fine. The equal = sign, however does not get encoded if the key or value contains it.
The equal sign in the data is to be expected and since HtmlEncode does not take care of it, are there other standard utility classes that can help out? I'd like to avoid manual string replacement if possible since it is error-prone.
It turns out HttpUtility.UrlEncode / HttpUtility.UrlDecode are better suited to this kind of data.
Related
I’m making a POST API call in C# using HttpWebRequest class. In the URL I do have password as query string. But the password has # in it which is getting truncated to vigne. Data after # are considered as Fragment which suppose not to happen, is there fix for it ?
Password example: vigne#ash#Test
URL = https://vigneashtesting.com/oauth/token?login_type=password&userid=vigneash&password=vigne#ash#Test;
You should never include passwords (or any other confidential) information in query strings because they are displayed in the browser.
If you want to include special characters in a query string then you need to use encodings. You can find the encodings here: https://www.w3schools.com/tags/ref_urlencode.asp.
You can also use Uri.EscapeDataString or System.Web.HttpUtility.UrlEncode to encode special characters. See the following answer for the differences between the two: https://stackoverflow.com/a/47877559/19214431.
I'm running a web service on my server using WCF and .Net 4. The contract type is WebGet. Here is my problem, at one point in time, someone was sending data through the querystring that was URL encoded. So I added HttpUtility.UrlDecode to decode the parameters. I think that fixed my issue at the time. Now, I've sent a URL encoded string to it and I see that the string is being URL decoded coming into the method (before even getting to the HttpUtility.UrlDecode).
So now I'm confused, if the .Net code is decoding it before it gets to my method, why would I need to call on decode explicitly? But for a time it wasn't, so is this a recent change to the underlying .Net framework?
My problem now is that my users are sending data (unencoded), where the data looks like this: "abc%1234" and I'm getting "abc34", the decoding is eating 3 characters. However, if I urlencode the % sign to be "abc%251234", the value coming into the method is "abc%1234" (what I expected) and then the call to HttpUtility.UrlDecode is changing it to "abc34" (which is not what I expected).
I'm not sure how to proceed here. Do I rip out the explicit call to URLDecode until it starts coming across encoded again or is there a better way to handle this?
It's a subtle thing in documentation, easily missed:
HttpRequest.QueryString Property
Property Value
NameValueCollection
The query string variables sent by the client. Keys and values are
URL-decoded.
So if you access the query string via HttpRequest.QueryString (or Params) collection they are already decoded.
You can get to the raw string in RawUrl, QueryString.ToString() (manually that is - re: manipulation, split, etc.).
End of day, %:
Because the percent ("%") character serves as the indicator for
percent-encoded octets, it must be percent-encoded as "%25" for that
octet to be used as data within a URI.
REF: RFC3986
Hth
I have a COM server app (App_A) that only supports native data types. I send the parameters over the COM server to a C# app (App_B) that then sends on the data as a web request.
My problem is that the String data read by App_A is Unicode, but App_A does not support non-UTF-8 encoding for its COM String values, so the data can be sent as a byte array or char array.
If I use the byte array, the generic App_B is now broken as I now have to handle this single data update differently to all the others (and I fear there will be more), so I would like to keep the App_B handling of values generic (obj.ToString).
If I hard code an App_B C# String as a literal, e.g. "\u5f90", the String contains a Unicode character and the HttpUtility.UrlEncode call in App_B works exactly as expected. If the String is passed in as a value (obj.ToString() = "\u5f90") the '\' is escaped and the UrlEncode does not UTF-8-encode a Unicode character as the '\u' escape sequence is lost.
I guess my question comes down to:
So far I have manipulated the byte array in App_A to replace the Unicode values (xxxx) with '\uxxxx': - is there any way I can use a String variable as a format string in the C# App_B?
Alternatively, if I'm going about this the wrong way, what would anyone suggest?
Please bear in mind that I have approx 300 data value updates that all use a generic o.ToString for part of the UrlEncode argument and I would like to keep this if possible.
Is it an option for you to support different encodings in your deserialization of the byte arrays in App_B? I'd suggest modifying App_A so that each sent string has an additional first byte which defines the encoding, which then has to be respected by App_B. That way it doesn't matter which encoding you use, as long as both apps support it.
I'd strongly suggest not modifying the strings as you've described by preceeding it with \u, that's just gonna be a mess of code later on which needs to be documented well and needs to be understood again if you come back to it later etc.
this may be a silly question, but it trips me up every time.
HttpUtility has the methods HtmlDecode and UrlDecode. Do these two methods decode anything (Html/Http related) I might find? When do I have to use them, and which one am I supposed to use?
Just now I hit an error. This is my error log:
Payment receiver was not payment#mysite.com. (it was payment%40mysite.com).
But, I wrapped the email address here in HttpUtility.HtmlDecode before using it. It turns out I have to use .UrlDecode instead, but this email address didn't come from a URL so this wasn't obvious to me.
Can someone clarify this?
See What is meant by htmlencode and urlencode?
It's the reverse of your case, but essentially you need to use UrlEncode/Decode anytime you are using an address of sorts (urls and yes, email addresses). HtmlEncode/Decode is for code that typically a browser would render (html/xml tags).
This same encoding is also used in Form POST requests as well.
My guess is something read it 'naked' without decoding it.
Html Encoding/Decoding is only used to escape strings that contain characters that would otherwise be interpreted as html control characters. The process turns the characters into html entities and back again.
Url Encoding is to get around the fact that many characters are not allowed in Uris; or because they too could be misinterpreted. Thus the percent encoding is used.
Percent encoding is also used in the body of http requests.
In both cases, of course, it's also a way of expressing a specific character code in a request/response independent of character sets; but equally, interpreting what is meant by a particular code can also be dependent on knowing a particular character set. Generally you don't worry about that - but it can be important (especially in the HTML case).
URLEncode converts characters that aren't allowed in a URL into character equivalents which are parsable as a URL. In your example # became %40. URLDecode reverses this.
HTMLEncode is similar to URLEncode, but the target environment is text NESTED inside of HTML. This helps the browser from interpereting your content as HTML, but when rendered it should look like the decoded version. HTMLDecode reverses this.
When you see %xx this means percent encoding has occured - this is a URL encoding scheme, so you need to use UrlEncode / UrlDecode.
The HtmlEncode and HtmlDecode methods are for encoding and decoding elements for HTML display - so things like & get encoded to & and > to >.
I have to encode a field so to make it secure of script injection.
I know I can use HttpUtility.HtmlEncode and Decode, but this method for HI-ASCII characters goes out of the range of the field size in database and I dont want to change the size of data field column.
Instead if I use HttpUtility.HtmlAttributeEncode, it works fine because it does not encode the HI-ASCII characters.
Is it safe what can be the disadvantages of it.
From HttpUtility..::.HtmlAttributeEncode Method (String):
The HtmlAttributeEncode method converts only quotation marks ("), ampersands (&), and left angle brackets (<) to equivalent character entities. It is considerably faster than the HtmlEncode method.
The string result from the HtmlAttributeEncode method should be used only for double-quoted attributes. Security issues might arise when using the HtmlAttributeEncode method with single-quoted attributes.
However it is not a usual practice to store the encoded input in the database. It is difficult to predict how much longer an encoded version will become.
Much better is directly store the input, and only encode it when needed (when you output it in HTML).
Storing data without encoding may cause SQL injection.