I wrote an MVC action to receive a post from a service. My problem is that the service is posting multipart data with wrong encoding.
Let me give an example:
The service will post the "á" for the form field "text".
I see (using Wireshark), that the byte written on the packet is 225, which is the right byte for "á" in ISO-8859-1 .
I do Request.Form["text"] and actually get a strange (different) char.
I believe this is cause by .NET attempting to convert the value 225 to a unicode char, when converting to string using the utf-8 encoding, but couldn't, as 225 isn't valid for utf-8.
So my question is: Is there a way to override the parsing of those bytes to string?
You could try to add a HttpModule and try to overwrite the ContentEncoding property of the Request object. Though I'm not sure this will work.
It's possible to set the default encoding in Web.Config's GlobalizationSection. The setting is called RequestEncoding and is taken to effect only if the HTTP request of your service does not contain a ContentType header. See http://msdn.microsoft.com/en-us/library/system.web.configuration.globalizationsection.requestencoding.aspx
You can further use inside Web.config to set the above setting only to a specific directory / MVC controller.
Related
When I use Mailkit to send emails, I noticed that it automatically decides to encode both the content as well as headers. Now, the content encoding is perfect however some email clients have difficulty decoding the headers which are like.
Is there a way to instruct the client to not encode certain headers?.
List-Unsubscribe:
=?us-ascii?q?=3Chttps=3A=2F=2Fbarlinkar=2Eus19=2Elist-manage=2Ecom=2Funsubscribe=3Fu=3D8c60690?=
=?us-ascii?q?5a7e637766f218816b&id=3D2e47bac84d&e=3D407e758886&c=3De27229afde=3E=2C?=
=?us-ascii?q?_=3Cmailto=3Aunsubscribe-mc=2Eus19=5F8c606905a7e637766f218816b=2Ee27229a?=
=?us-ascii?q?fde-407e758886=40mailin=2Emcsv=2Enet=3Fsubject=3Dunsubscribe=3E?=
X-Report-Abuse:
=?us-ascii?q?=3Chttps=3A=2F=2Fmailchimp=2Ecom=2Fcontact=2Fabuse=2F=3Fu=3D8c606905a7e637766f218?=
=?us-ascii?q?816b&id=3De27229afde&e=3D407e758886=3E?=
To: k****#****.***
EDIT: Jstedfast pointed out some errors and I fixed them but the overall result is the same.
I doubt the problem is that the header value is encoded. Your value is invalid to begin with.
Here's the raw value that you are using:
https://barlinkar.us19.list-manage.com/unsubscribe?u=8c606905a7e637766f218816b&id=2e47bac84d&e=407e758886&c=e27229afde>, <mailto:unsubscribe-mc.us19_8c606905a7e637766f218816b.e27229afde-407e758886#mailin.mcsv.net?subject=unsubscribe>List - Unsubscribe - Post: List - Unsubscribe = One - Click
Do you see anything wrong with that?
First, each URL should be enclosed in <>'s. Your first URL is missing the leading < character.
Secondly, you are including the List-Unsubscribe-Post header in the value of the List-Unsubscribe header. They need to be 2 distinct headers.
In other words, the receiving client is probably getting confused as to what the value is supposed to be because it is completely borked.
I'm running a web service on my server using WCF and .Net 4. The contract type is WebGet. Here is my problem, at one point in time, someone was sending data through the querystring that was URL encoded. So I added HttpUtility.UrlDecode to decode the parameters. I think that fixed my issue at the time. Now, I've sent a URL encoded string to it and I see that the string is being URL decoded coming into the method (before even getting to the HttpUtility.UrlDecode).
So now I'm confused, if the .Net code is decoding it before it gets to my method, why would I need to call on decode explicitly? But for a time it wasn't, so is this a recent change to the underlying .Net framework?
My problem now is that my users are sending data (unencoded), where the data looks like this: "abc%1234" and I'm getting "abc34", the decoding is eating 3 characters. However, if I urlencode the % sign to be "abc%251234", the value coming into the method is "abc%1234" (what I expected) and then the call to HttpUtility.UrlDecode is changing it to "abc34" (which is not what I expected).
I'm not sure how to proceed here. Do I rip out the explicit call to URLDecode until it starts coming across encoded again or is there a better way to handle this?
It's a subtle thing in documentation, easily missed:
HttpRequest.QueryString Property
Property Value
NameValueCollection
The query string variables sent by the client. Keys and values are
URL-decoded.
So if you access the query string via HttpRequest.QueryString (or Params) collection they are already decoded.
You can get to the raw string in RawUrl, QueryString.ToString() (manually that is - re: manipulation, split, etc.).
End of day, %:
Because the percent ("%") character serves as the indicator for
percent-encoded octets, it must be percent-encoded as "%25" for that
octet to be used as data within a URI.
REF: RFC3986
Hth
I need to read data from an MVC POST but want to make sure that I read the POST data in the correct encoding.
So far in my programming history (and in most code I've seen) people assume UTF8.
I'd like to prevent any encoding issues and ensure that my encoder matches what the client is POSTing.
Question
Is this necessary?
How do I get the correct encoder for an ASP.NET MVC form POST?
Have you tried the HttpRequest.ContentEncoding property?
http://msdn.microsoft.com/en-us/library/system.web.httprequest.contentencoding%28v=vs.110%29.aspx
How do I get the correct encoder for an ASP.NET MVC form POST?
Browsers don't actually send identification of what encoding the entity body is when submitting a form, so the ContentEncoding property is unlikely to tell you anything useful.
Usually it is enough to serve all pages as UTF-8. Then the browser will use UTF-8 to submit forms from those pages, so it's safe to assume UTF-8. Exceptions:
if you are worried about the user going to their browser's View->Encoding menu and changing it (this is very unlikely unless you have messed up the encoding already);
if you expect other people to be doing cross-site form submissions to your site, and you can't guarantee what encoding their pages will be using.
In principle you should be able to fix these cases by including the attribute accept-charset="utf-8" in your form tag. Unfortunately IE handles this wrong and only submits as UTF-8 if there are characters in the form that can't be represented in the current page's encoding. There are two ways to fix this; you can use one or both:
include a hidden input whose value contains a character that is not included in any non-UTF encoding (the Unicode snowman U+2603 is a traditional choice);
include a hidden input with name="_charset_", which the browser can fill in with the name of the encoding it is using when submitted. This is an IE hack that has been adopted by some other recent browsers.
When the browser is set to use a non utf-8 character set, we noticed MVC and jquery.unobtrusive-ajax mostly take care of encoding. But not when you are posting text values in a querystring.
This results in illegal unicode characters:
$.post("/Question/Save?Text=очень");
This works for non utf-8 charset:
var formValues = {};
formValues.Text = "очень";
$.ajax("/Question/Save", { type: "POST", data: formValues });
(Tested in Chrome and IE11)
I have a bit of a strange problem. I have a controller that is supposed to receive a URL as one of the parameters. I am encoding the URL, and trying to call the controller as such:
http://www.mysite.com/dowork/1/http%3a%2f%2fwww.otherurl.com
However, I am getting the error
A potentially dangerous Request.Path value was detected from the client (:).
What gives? Any way to handle this WITHOUT disabling validation?
If that is .NET 4.0, you can edit this configuration setting:
<system.web>
<httpRuntime requestPathInvalidCharacters="<,>,%,&,:,\,?" />
</system.web>
If you don't wish to edit this, you can additionally encode your incoming url - use custom encoding (like replacing dangerous characters) or use base64 encoding or omit protocol part if possible before sending to controller (drop http://, if https:// is possible also, you need to think how to pass that).
IF you pass it as a query string parameter (?path=....) you avoid the problem, the characters are only illegal as part of a path.
If you insist on passing the data as part of the path you will need to encode differently, like convert the string into a range of hexadecimal values or similar.
I am trying to throw together a screen scraper and keep getting invalid viewstate issues. It appears that during the System.Net.WebClient download of data or the System.Text.UTF8Encoding.Default.GetString call to convert the byte array returned by the WebClient DownloadData call to a string - that strings which match url character codes are being converted.
ie
Url encoded characters strings like %2B are being converted to their normal characters (+ for %2B).
Is this happening in the WebClient class? Is it the way I am converting the byte array to a string?
EDIT:
Based on suggestions I tried changing to the DownloadString call from the WebClient class and the resulting string has converted the character codes to the specific character so it appears WebClient is the culprit.
EDIT 2:
Solved. By making a call to System.Web.HttpUtility.UrlEncode I was able to convert the + back to %2B before sending the viewstate string back up to the server in subsequent requests. I am still at a loss as to where and why the problem was occurring but the server was expecting a viewstate string that contained ...%2B... and was getting ...+... and determining the viewstate to be invalid and throwing the exception. Kudos to Jon & Henk for forcing me to rethink my assumptions.
If you use System.Text.UTF8Encoding.Default then you're not using UTF-8 - you're using the default encoding for the system. It's equivalent to Encoding.Default, but in a more confusing form. Use Encoding.UTF8 to get a UTF-8 encoding... or use WebClient.DownloadString as Henk suggested.
On the other hand, it's not clear what you're trying to download. If you're trying to download geuinely binary data then you shouldn't be trying to convert it to a string at all.
It would help if you would clarify you question - try to provide a lot more context about what's making the requests, what's having problems, etc.
And what happened if you just use WebClient.DownloadString() instead of opening a binary stream?