C# HTTP GET returns 404 Not Found - c#

I'm trying to download .webp image, and it always returns 404, but if I hardcode URI or smply open it in browser, it returns 200 with image.
Details:
string uri1 = "https://some.site/static/.../.../0.webp";
string uri2 = parsedApiResponse;
var response1 = client.GetAsync(uri1).Result;
var response2 = client.GetAsync(uri2).Result;
Log($"{response1.StatusCode}\n", ConsoleColor.Magenta);
Log($"{response2.StatusCode}\n", ConsoleColor.Yellow);
StatusCode
parsedApiResponse contains string reply from API (it saves image on server and returns it's location) with full path leading to .webp image.
uri1 contains hardcoded parsedApiResponse with handly copied path to other image from exactly same earlier call.
response1 returns exactly what it should when response2, from absoletely same request with not hardcoded URI, returns... this (Fiddler screenshots:)
raw inspection of successful request
raw inspection of fail request
It's absolutely same! There's no custom headers, no content, but response is different. And if I'll log and copy path of uri2 and then will paste it on place of uri1, it will turn the same way.
Content of response2 is a full cloudflare 404 page, but main part says this:
<section id="text">
<div>
<h1>Error 404</h1>
<h3>This object could not be viewed</h3>
</div>
<div>
<p id="error-title">**You are not authorized to view this object**</p>
<p>
This object does not exist or is not publicly accessible at this
URL. Check the URL of the object that you're looking for or contact
the owner to enable Public access.
</p>
</div>
<div>
<p id="footer-title">Is this your bucket?</p>
<p>
Learn how to enable
<a
href="https://developers.cloudflare.com/r2/data-access/public-buckets/">Public Access
</a>
</p>
</div>
</section>
I've already tried to play with some HttpClientHandler params, tried to use other clients like RestClient and WebClient, tried to pass all possible headers to mimic my browser and many-many other stuff - no result. What can it be? Why does it require some authorize? How does hardcoded URI get's these credentials?..🤯

It was server side problem.
For some reason, server created image path and responded with it, before image itself was fully uploaded. That's the reason why it was Not found.
If you encounter similar problem, first of all try to make multiple requests with some delay in time:
public static async Task<byte[]?> DownloadImg(string url)
{
HttpClient client = new();
for (int i = 0; i < 10; i++)
{
try { return await client.GetByteArrayAsync(url); }
catch { Thread.Sleep(2000); }
}
return null;
}

Related

HttpRequest.Files is empty when sending through POSTMAN

I'm following this blog post to upload an image using C# Web API.
The article explains how to do it using ARC and it works fine.
But when I'm trying to do the same using POSTMAN it's failing.
Here is my request snapshot.
TL;DR: You are doing everything correct except that you are setting the Content-Type header explicitly in the tool. Get rid of the header and your issue will be resolved.
Detailed Explanation: To begin with, you attach the files using form-data option in the Body tab in the tool:
The moment you select the file(s), Postman auto-detects the Content-Type. Then behind the scenes, Postman inserts the automatically detected Content-Type into the POST request without us knowing it. So setting up Content-Type header explicitly on our own messes up the request.
Setting the value of Content-Type header to multipart/form-data involves a complex concept of setting up boundaries of multiple parts of the file as detailed here. It can be error-prone. So heavy lifting of setting up the boundaries is done automatically for us by the tool itself. This is the reason why it doesn't expect us to set the content-type header explicitly in this case. Please see how I've set only Authorization header while uploading the image file on my PC (Refer screenshot):
Authorization header is an optional header. It is required only if you've setup some sort of authentication on the web server. So if anonymous access is allowed on your website then Headers tab in your case should be empty i.e. no key value pairs at all.
Note: Just for information, the correct content type for image files in a POST request is multipart/form-data even though we don't need to set it explicitly in the tool. Screenshot in the question shows that Content-Type header is being set as application/x-www-form-urlencoded which is not right.
In the post you referrer to the data is being uploaded as "x-www-form-url-encoded"
Your Postman screen shot shows you are uploading it as "form-data"
Additionally, you adding a key "image01" where the ARC example doesn't appear to be sending a key.
If you want to upload the file using form-data you need a different approach:
// POST api/files
public async Task<HttpResponseMessage> Post()
{
// Check if the request contains multipart/form-data.
if (!Request.Content.IsMimeMultipartContent())
{
throw new HttpResponseException(HttpStatusCode.UnsupportedMediaType);
}
string root = HttpContext.Current.Server.MapPath("~/App_Data");
var provider = new MultipartFormDataStreamProvider(root);
string value;
try
{
// Read the form data and return an async data.
var result = await Request.Content.ReadAsMultipartAsync(provider);
// This illustrates how to get the form data.
foreach (var key in provider.FormData.AllKeys)
{
foreach (var val in provider.FormData.GetValues(key))
{
// return multiple value from FormData
if (key == "value")
value = val;
}
}
if (result.FileData.Any())
{
// This illustrates how to get the file names for uploaded files.
foreach (var file in result.FileData)
{
FileInfo fileInfo = new FileInfo(file.LocalFileName);
if (fileInfo.Exists)
{
//do somthing with file
}
}
}
HttpResponseMessage response = Request.CreateResponse(HttpStatusCode.Created, value);
response.Headers.Location = new Uri(Url.Link("DefaultApi", new { id = files.Id }));
return response;
}
catch (System.Exception e)
{
return Request.CreateErrorResponse(HttpStatusCode.InternalServerError, e);
}
}

Download file via GET and return the referrer url

I send a request via ajax and if response data success is true, I make a GET request in ajax success function:
success: function (data, status) {
if (!data["Success"]) {
alert("Error occurred: Cant read data properly," + data["Message"]);
return null;
}
window.location = '/Home/downloadanddelete?file=output.' + data["filetype"];
The problem is when Get request posted to controller the response is:
As you see the file request url is:"http://localhost:53091/Home/downloadanddelete?file=output.xml"
And I expect download this "output.xml" file and return the referrer page url.
here is download method in controller:
[HttpGet]
public ActionResult downloadanddelete(string file)
{
string fullName = Path.Combine(HttpRuntime.AppDomainAppPath, "App_Data", file);
if (System.IO.File.Exists(fullName))
{
return File(fullName, "application/xml");
}
return View("Index");
}
What is it wrong here?
You'll need to change two things. In the server code, you need to send a Content-Disposition header to indicate the content is an "attachment". Add these lines before sending the file:
var cd = new System.Net.Mime.ContentDisposition
{
FileName = "filename.xml",
Inline = false
};
Response.AppendHeader("Content-Disposition", cd.ToString());
Secondly, you ought to use window.location.assign(...); instead of setting window.location for a more seemless experience in the browser.
You can use the Header type "Content-Disposition" to tell the browser to preferably download and save the file, instead of displaying it. There is some information on it here: https://stackoverflow.com/a/20509354/1862405
For your specific case, you'd want to use the attachment disposition, which you can add to your controller action with AddHeader:
HttpContext.Response.AddHeader("Content-Disposition", "attachment; filename=\"download.xml\"");
Another thing is that Firefox might overrule this, but you can set up for file types how it should handle them.

show HTML formatted error message with HttpResponseMessage

I have a website that serves up media files. If a file is not found, I show a simple message on the page to the user with the code below:
var fileNotFoundResponse = new HttpResponseMessage(HttpStatusCode.NotFound);
if (!File.Exists(mediaFile.FilesystemLocation))
{
fileNotFoundResponse.Content = new StringContent("File not found! <br /> Please contact <a href='mailto:support#sbcdef.com'>support#sbcdef.com</a>");
return ResponseMessage(fileNotFoundResponse);
}
The problem is, is that it's just plain text on a white page that doesn't show HTML and I want it to be able to display the HTML.
Is there a way to do this?
Thanks!
You probably aren't sending the client the correct ContentType header, and likely need to set the HttpResponse.ContentType string:
fileNotFoundResponse.ContentType = "text/html";
fileNotFoundResponse.Clear();
fileNotFoundResponse.BufferOutput = true;
Hope this helps!
UPDATE: If you need UTF8 support, try this:
fileNotFoundResponse.ContentType = "text/html; charset=utf-8";

How do I get the destination URL of a shortened URL?

I have an API (https://www.readability.com/developers/api/parser#idm386426118064) to extract the contents of the webapges, but on passing a shortened url or an url that redirects to other, it gives error.
I am developing windows phone 8.1 (xaml) app. Is there any way to get the destination url in c# or any work around?
eg url - http://www.bing.com/r/2/BB7Q4J4?a=1&m=EN-IN
You could intercept the Location header value before the HttpClient follows it like this:
using (var handler = new HttpClientHandler())
{
handler.AllowAutoRedirect = false;
using (var client = new HttpClient(handler))
{
var response = await client.GetAsync("shortUrl");
var longUrl = response.Headers.Location.ToString();
}
}
This solution will always be the most efficient because it only issue one request.
It is possible however, that the short url will reference another short url and consequently cause this method to fail.
An alternative solution would be to allow the HttpClient to follow the Location header value and observe the destination:
using (var client = new HttpClient())
{
var response = client.GetAsync("shortUrl").Result;
var longUrl = response.RequestMessage.RequestUri;
}
This method is both terser and more reliable than the first.
The drawback is that this code will issue two requests instead of one.
You can get the ResponseUri from GetResponse():
string redirectedURL = WebRequest.Create("http://www.bing.com/r/2/BB7Q4J4?a=1&m=EN-IN")
.GetResponse()
.ResponseUri
.ToString();
Interesting article, by the way.
You need to inspect the headers returned from the URL.
If you get HTTP return codes 301 or 302, then you are being notified that the page is redirecting you to another URL.
See http://www.w3.org/Protocols/HTTP/HTRESP.html for more details about HTTP return codes.

Logging into a website using HttpWebRequest/Response in C#?

Now, first off, I want to understand whether or not its better to use HttpWebRequest and Response or whether its better to simply use a webbrowser control. Most people seem to prefer to use the web browser, however whenever I ask people about it, they tell me that HttpWebRequest and Response is better. So, if this question could be avoided by switching to a web browser (and there's a good reason as to why its better), please let me know!
Basically, I set up a test site, written in PHP, running on localhost. It consists of three files....
The first is index.php, which just contains a simple login form, all the session and everything is just me testing how sessions work, so its not very well written, like I said, its just for testing purposes:
<?php
session_start();
$_SESSION['id'] = 2233;
?>
<form method="post" action="login.php">
U: <input type="text" name="username" />
<br />
P: <input type="password" name="password" />
<br />
<input type="submit" value="Log In" />
</form>
Then, I have login.php (the action of the form), which looks like:
<?php
session_start();
$username = $_POST['username'];
$password = $_POST['password'];
if ($username == "username" && $password == "password" && $_SESSION['id'] == 2233)
{
header('Location: loggedin.php');
die();
}
else
{
die('Incorrect login details');
}
?>
And lastly, loggedin.php just displays "Success!" (using the element).
As you can see, a very simple test, and many of the things I have there are just for testing purposes.
So, then I go to my C# code. I created a method called "HttpPost". It looks like:
private static string HttpPost(string url)
{
request = HttpWebRequest.Create(url) as HttpWebRequest;
request.CookieContainer = cookies;
request.UserAgent = userAgent;
request.KeepAlive = keepAlive;
request.Method = "POST";
response = request.GetResponse() as HttpWebResponse;
if (response.StatusCode != HttpStatusCode.Found)
throw new Exception("Website not found");
StreamReader sr = new StreamReader(response.GetResponseStream());
return sr.ReadToEnd();
}
I built a Windows Form application, so in the button Click event, I want to add the code to call the HttpPost method with the appropriate URL. However, I'm not really sure what I'm supposed to put there to cause it to log in.
Can anyone help me out? I'd also appreciate some general pointers on programatically logging into websites!
Have you considered using WebClient?
It provides a set of abstract methods for use with web pages, including UploadValues, but I'm not sure if that would work for your purposes.
Also, it's probably better not to use WebBrowser as that's a full blown web browser that can execute scripts and such; HttpWebRequest and WebClient are much more light weight.
Edit : Login to website, via C#
Check this answer out, I think this is exactly what you're looking for.
Relevant code snippet from above link :
var client = new WebClient();
client.BaseAddress = #"https://www.site.com/any/base/url/";
var loginData = new NameValueCollection();
loginData.Add("login", "YourLogin");
loginData.Add("password", "YourPassword");
client.UploadValues("login.php", "POST", loginData);
You should use something like WCF Web Api HttpClient. It much easier to achieve.
Following code is writte off the top of my head. But it should give you the idea.
using (var client = new HttpClient())
{
var data = new Dictionary<string, string>(){{"username", "username_value"}, {"password", "the_password"}};
var content = new FormUrlEncodedContent(data);
var response = client.Post("yourdomain/login.php", content);
if (response.StatusCode == HttpStatusCode.OK)
{
//
}
}

Categories

Resources