I have some issue with Cloudfront Distribution. I have used a dropdownlist and a gridview in my page. When dropdownlist changed accordingly I wanted to update the gridview. Its working pretty fine on my local machine as well as on my server when I am trying with IP address.
I am using Amazon Cloudfront as CDN, Its not working behind Cloudfront.
I may suppose to add some behavior on cloudfront console to resolve this, but i am not sure about it.
Any help appreciated.
A shot in the dark here (as Michael - sqlbot says - you really need to provide more info).
Is the gridview an ASP.NET web control? If so, it might be that ASP.NET isn't recognising the CloudFront user-agent string: Amazon CloudFront (as opposed to something like Mozilla/5.0 (Windows; U; MSIE 9.0; WIndows NT 9.0; en-US which is an example of a user-agent string you'd typically see if making a direct request to your site) and therefore isn't rendering the appropriate Javascript (I've seen the __doPostBack javascript omitted in these circumstances)
From https://msdn.microsoft.com/en-us/library/x3k2ssx2.aspx
ASP.NET determines browser capabilities by reading the user-agent
information that is passed from the browser to the server during a
request. It compares the user-agent string that is received from the
browser to user agent strings that are stored in browser definition
files. These browser definition files contain information about the
capabilities of various user agents. When ASP.NET finds a match
between the current user-agent string and a user-agent string in a
browser definition file, it loads the corresponding browser
capabilities into the HttpBrowserCapabilities object. The properties
of the HttpBrowserCapabilities object can then be used to determine
whether the browser type that is represented by the user agent
supports scripting, styles, frames, and so on. Based on these
capabilities, the controls on the page render Web controls using
appropriate markup.
The page contains some details on how you can override this, but none of them seem ideal (i.e. explicitly targeting a specific browser / platform).
The other option is to configure CloudFront to whitelist the User-Agent header for cache behaviours that match the pages where you're using these controls (Edit Behavior > Forward Headers > Whitelist > Add Custom: User-Agent), but be aware this will effectively disable caching for those resources, as user-agent strings often vary per user.
Related
I am extracting user agent as:
string userAgent = HttpContext.Current.Request.UserAgent;
Is there any possibility that UserAgent would be null? Is it only possible for bot or for any legitimate user/device/client/browser too?
The User-Agent header is optional in RFC 2616. It SHOULD be used by clients, but it is not REQUIRED to be used. Also see RFC7231:
5.5.3 User-Agent
The "User-Agent" header field contains information about the user agent originating the request, which is often used by servers to help identify the scope of reported interoperability problems, to work around or tailor responses to avoid particular user agent limitations, and for analytics regarding browser or operating system use. A user agent SHOULD send a User-Agent field in each request unless specifically configured not to do so.
I think RFC7231 obsoletes 2616, but I have cited both for completeness.
So yes, you should check for a null or empty user agent.
I am using HtmlAgilityPack.
I am downloading articles and images from one web site. 80% images downloading without problem. But some images throwing error. I can see name of error in image_failed event.
I am downloading image like that:
Image = new BitmapImage(new Uri(img.Attributes["src"].Value));
I have searched google and found that this is really WTF problem.
There's a good chance the referrer header is screwing you up. You need to issue the calls yourself (instead of relying on BitmapImage to download the file).
There's a handy snippet/utility that 'extends' xaml and makes it easier to do.
http://blogs.msdn.com/b/swick/archive/2011/08/04/wp7-mango-image-download-with-custom-referer-header.aspx
Edit: Explanation
A lot of sites block requests for images not coming from their sites. That way, if you have http://mysite.com and you link to images in http://cnn.com, they can block images directly linked and redirect them or something.
Now, the reason it works is that the browser controls all calls made from the tag (or from any other mechanism such as AJAX) and it adds the REFERRER HTTP header saying where the request is coming from (http://mysite.com) - and then the cnn.com code can block it.
In .NET desktop, the Referrer header is not automatically added to the request - that means that the call would be blocked by some site that checks for an empty referrer and not for others that don't.
Switch to WP7/8 which is based on Silverlight. In Silverlight, the referrer is the site on which the Silverlight control is hosted. So if you have a SL control running on http://mysite.com and it makes [any] http request, the referrer header will be automatically set for you to http://mysite.com. There's no way to control that afaik (for security reasons). Windows Phone, however, while based on SL, does not need to be bound by the same security constraints. However, when they "ported" the code to Windows Phone, they put some value into referrer into it - the value is actually the package location inside the phone (you can see this by using fiddler). It's literally some path (/apps/storage/[guid]) or something like that - I don't recall the exact value. To fix that, you go and set the referrer to the site on the HTTP headers making the request.
Hope that makes it clear.
I am developing an iframe for use on a number of our partners websites.
Is there any way I can make sure it can only be used on those websites and not by anyone else
I was intending to add a compulsory querystring to the URL for the website.
Each partner would have a different value in the quesrystring dnd use that to look up an allowed domain
However, is there anyway to know the top level domain of the site hosting the iframe?
Presumably this is not sent in the http request for the iFrame? Or is it, I couldn’t see it?
Or do you need to send the domain from javascript?
Any advice?
However, is there anyway to know the top level domain of the site hosting the iframe?
Nothing reliable.
Presumably this is not sent in the http request for the iFrame? Or is it, I couldn’t see it?
It might be sent in the referer
Or do you need to send the domain from javascript?
If you want to fetch it from the framed page, you will be blocked by the same origin policy.
If you want to sent it from the framing page, you will be putting it in the query string and you can't trust it because it can be set to whatever the person writing the framing page likes.
There is also the X-Frame-Options header (but that has limited browser support).
The most reliable solution I can think of is:
Require the origin to be specified in the query string used to load the frame
Check the referer. If it doesn't match your white-list and is not blank, redirect to a page that is blank except for a link to your site with target="_top" and some JavaScript that top.location = "your site"
Check that the origin specified in the query string is on your whitelist, if it isn't act in the same way as a rejected step 2
Output an X-Frame-Options header that limits the framing to the specified origin
That is likely to catch enough browsers to discourage the framing site from framing your site.
You can try to check referrer which normal browser will send for IFrame requests on the page.
You also can use "x-frame-options" header covered in (How to Block Iframe call and MDN ) but not every browser will respect that (on other hand it is more reliable if browser supports it).
iframe's sanbox attribute might be helpful in controlling the various security aspects in an iframe including origins
http://www.html5rocks.com/en/tutorials/security/sandboxed-iframes/
The Architecture Journal of 2007 has a nice article about this: Secure Cross-Domain Communication in the Browser
Basically what the article suggests is:
If you have page A on domain 1 with an iframe with page B on domain 2 as its source , then having an iframe on page B to page C on domain 1, would allow you to pass information across domains
I haven't tested it, but this sounds like it could work.
Another possibility is create a file with a special filename (for instance a hash of the URL of page B on domain 2) and basic extension (like .htm) and place it in the root of domain 1. Checking whether the file exists on domain 1 cannot be done by javascript however, so it should be done with server side code.
Is it possible to make an exact identical POST with HttpWebRequest in C# as a browser would? Without a page being able to detect that it is actually no browser?
If so, were could i read up more on that?
Download and become familiar with a tool like Fiddler. It allows you to inspect web requests made from applications, like a normal browser, and see exactly what is being sent. You can then emulate the data being sent with a request created in C#, providing values for headers, cookies, etc.
I think this is doable.
Browser detection is done based on a header in the request. All you need to do is set that header. In HttpWebRequest we dont need to set the headers collection but rather the .UserAgent property.
Eg:
.UserAgent = "Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 6.0)";
There is quite a lot to user agents. Check this link for the complete list of User-Agents
Useful Links:
How to create a simple proxy in C#?
Is WebRequest The Right C# Tool For Interacting With Websites?
http://codehelp.smartdev.eu/2009/05/08/improve-webclient-by-adding-useragent-and-cookies-to-your-requests/
There is a reports website which content I want to parse in C#. I tried downloading the html with WebClient but then I don't get the complete source since most of it is generated via js when I visit the website.
I tried using WebBrowser but could't get it to work in a console app, even after using Application.Run() and SetApartmentState(ApartmentState.STA).
Is there another way to access this generated html? I also took a look into mshtml but couldn't figure it out.
Thanks
The Javascript is executed by the browser. If your console app gets the JS, then it is working as expected, and what you really need is for your console app to execute the JS code that was downloaded.
You can use a headless browser - XBrowser may server.
If not, try HtmlUnit as described in this blog post.
Just a comment here. There shouldn't be any difference between performing an HTTP request with some C# code and the request generated by a browser. If the target web page is getting confused and not generating the correct markup because it can't make heads or tails of from the type of browser it thinks it's serving then maybe all you have to do is set the user agent like so:
((HttpWebRequest)myWebClientRequest).UserAgent = "<a valid user agent>";
For example, my current user agent is:
Mozilla/5.0 (Windows NT 6.1; WOW64; rv:9.0.1) Gecko/20100101 Firefox/9.0.1
Maybe once you do that the page will work correctly. There may be other factors at work here, such as the referrer and so on, but I would try this first and see if it works.
Your best bet is to abandon the console app route and build a Windows Forms application. In that case the WebBrowser will work without any work needed.