I am using HttpWebRequest with the AddRange function like:
HttpWebRequest myHttpWebRequest = (HttpWebRequest)WebRequest.Create(URL);
myHttpWebRequest.AddRange(20, 30);
HttpWebResponse myHttpWebResponse = (HttpWebResponse)myHttpWebRequest.GetResponse();
Stream streamResponse = myHttpWebResponse.GetResponseStream();
SaveFileStream(name, streamResponse); //save file function
...but the entire file downloaded.
AddRange() in the above code expects the bytes between 20 and 30 to be downloaded (in other words, to download those 10 bytes from the file).
But my code is not working, since the download is not segmented. This link provides an example: http://stackoverflow.com/robots.txt That file was downloaded in its entirety. Why?
HTTP server are not required to support Range header requests. You can verify server's range support by issuing HEAD request and checking value of Accept-Ranges header in response (see HTTP range requests). But this still doesn't guarantee that server will not ignore Range header, in particular for very small ranges (it would be very inefficient for HTTP server to serve content in such small segments).
From RFC7233:
Because servers are free to ignore Range, many implementations will
simply respond with the entire selected representation in a 200 (OK)
response. That is partly because most clients are prepared to receive
a 200 (OK) to complete the task (albeit less efficiently) and partly
because clients might not stop making an invalid partial request until
they have received a complete representation. Thus, clients cannot
depend on receiving a 416 (Range Not Satisfiable) response even when
it is most appropriate.
To determine if server accepted or ignored Range header, you must check response status code. 200 (OK) indicates that server ignored Range header and returned whole response body, 206 (Partial Content) indicates that range specified in header was returned by server and 416 (Range Not Satisfiable) indicates that the set of ranges requested has been rejected due to invalid ranges or an excessive request of small or overlapping ranges.
In case of http://stackoverflow.com/robots.txt, server indicates support of Range header by returning Accept-Ranges: bytes header in response on HEAD request, but on GET request with AddRange(20, 30) specified, response is 200 (OK), so server just ignored such small range a returned whole reponse body. You have to cut requested range from response yourself, if you need to.
Related
I have an IP camera that I'm trying to receive events from. For example, if it detects motion in a specific part of the frame, I want to know about it and get the images captured with the event.
The problem is that it uses HTTP/1.0 POST to send this information. According to the spec (https://www.w3.org/Protocols/HTTP/1.0/draft-ietf-http-spec.html#BodyLength), the body length can either be communicated by the Content-Length header or by closing the connection when everything has been sent to the server. The IP camera does the latter of the two methods.
Kestrel's web server doesn't support this as seen here Microsoft.AspNetCore.Server.Kestrel.Core.Internal.Http.Http1MessageBody.For() contains the following (https://github.com/aspnet/KestrelHttpServer/blob/2191327b59f87f23f69fff2ac0dba9e58b67141b/src/Kestrel.Core/Internal/Http/Http1MessageBody.cs, Lines 308-318):
// Avoid slowing down most common case
if (!object.ReferenceEquals(context.Method, HttpMethods.Get))
{
// If we got here, request contains no Content-Length or Transfer-Encoding header.
// Reject with 411 Length Required.
if (context.Method == HttpMethod.Post || context.Method == HttpMethod.Put)
{
var requestRejectionReason = httpVersion == HttpVersion.Http11 ? RequestRejectionReason.LengthRequired : RequestRejectionReason.LengthRequiredHttp10;
BadHttpRequestException.Throw(requestRejectionReason, context.Method);
}
}
As a result, Kestrel throws a BadHttpRequestException every time.
Any ideas on how to work around this issue?
TCP Server that can run next to Kestrel?
A low-level hook to add the content-length?
I'm trying to download first bytes of a webpage.
I add Range to the HTTP request header. it's my code in C#:
HttpWebRequest request = (HttpWebRequest)WebRequest.Create("http://example.com");
request.AddRange(0,1000);
//request.Proxy = null;
using (HttpWebResponse response = (HttpWebResponse)request.GetResponse())
{
Stream st = response.GetResponseStream();
StreamReader sr = new StreamReader(st);
string str = sr.ReadToEnd();
sr.Close();
st.Close();
}
for some webpages it works fine, but some servers will ignore the HTTP range header, so the server send all the page in the response.
I changed my code to this:
string str = sr.Read(buffer,0,999);
but it doesn't work! because the problem is not in this line. actually the response will send to my program when I call request.GetResponse()! in this line all the bytes received by the program and write the all bytes to the RAM. I want to stop receiving data from the server when I received first 1000 bytes.
but there is no control on HttpWebRequest class and .GetResponse() method to stop receiving data after 1000 bytes received.
How can I do that?
I think there would be another HTTP Request custom class that allow us to stop receiving data when we want.
please tell me any Idea about this problem. I'm thinking to override the HttpWebRequest or write a MFC Library (C++ language) and import it into my project, but I don't know how to do this.
EDIT: I know it's optional for server to allow or ignore the Range header! but my question is how can I stop receiving data from the server! for example the server is sending 10,000 bytes to my computer, I want to stop receiving bytes after I see the 1000th byte! (I don't want the server to send just first 1000 bytes, I want to close connection and stop receiving the bytes after first 1000 bytes! even if the server send all 10,000 bytes)
I'm not sure what response code to use when the request is valid but there is no result for the given parameter
The JSON returns a success of true/false and a message
When I do a GET and there is some data I use:
200 HttpStatusCode.OK
However where there is no data should I still use OK and return the JSON success / message or should I use 400 HttpStatusCode.BadRequest to indicate something in the request is bad.
It depends on what does it mean for the client to not have results.
Typically lack of data would still be http 200 ok.
e.g. /employees etc.
However, for some scenarios you could return
HTTP 404 Not Found.
Especially when the client expects a particular resource to be present.
e.g. employees/update/32
Normally any other response code (204 etc.) though technically valid and fitting, might confuse the client.
Also, 400 Bad Request should also be not used, if there is nothing wrong with the request.
If the operation was successful but there is really no response data, use the status 204 NO CONTENT. If an expected entity was missing, return 404 NOT FOUND. If there was some sort of internal error, return 500 SERVER ERROR.
According to the HTTP/1.1 spec,
The server has fulfilled the request but does not need to return an
entity-body, and might want to return updated metainformation. The
response MAY include new or updated metainformation in the form of
entity-headers, which if present SHOULD be associated with the
requested variant.
If the client is a user agent, it SHOULD NOT change its document view
from that which caused the request to be sent. This response is
primarily intended to allow input for actions to take place without
causing a change to the user agent's active document view, although
any new or updated metainformation SHOULD be applied to the document
currently in the user agent's active view.
The 204 response MUST NOT include a message-body, and thus is always
terminated by the first empty line after the header fields.
If ther was nothing wrong with the request but you just don't have any data (i.e. search returns 0 rows) than I would not use BadRequest:
BadRequest is sent when no other error is applicable, or if the exact
error is unknown or does not have its own error code.
http://msdn.microsoft.com/en-us/library/system.net.httpstatuscode%28v=vs.110%29.aspx
I know how to make a request that says "fetch the file only if it has been modified since a specific date".
HttpWebRequest request = (HttpWebRequest) WebRequest.create(url);
request.IfModifiedSince = lastWriteTime; // lastWriteTime is the time when
// the local file was last written.
try
{
HttpWebResponse response = (HttpWebResponse) request.GetResponse();
...
But, suppose this scenario occurs -
the local file is not modified since the last write time, but when it was downloaded from the server the last time, the file was incomplete. In other words, the content length in the local file is not the same as the content length in the file on the server.
So I would like to specify that the file should be fetched from the server, if the content length on the server is not the same as the content length on the local machine.
How can I do this?
For Your scenario and with HTTP only I would check file length header:
string length = response.Headers[HttpResponseHeader.ContentLength];
So You will know right from the start what length this file should have. If it's too short then it's not complete and You need to download again.
If You are more concerned about content of file You can check also MD5:
string md5 = response.Headers[HttpResponseHeader.ContentMd5];
If You insist on request condition You can try HTTP header If-Range. It is described by W3C as:
...
Informally, its meaning is `if the entity is unchanged, send me the part(s) that I am missing; otherwise, send me the entire new entity'.
...
Check Header Field Definitions and HTTP header fields to see what Your options are.
I am creating a Http request parser. I am getting data in chunks (byte array) and parsing it simultaneously. I want to know the condition of detecting end of http request. The request may or may not contain message body.
Thanks
Three different ways:
content-length header (number of bytes following the headers)
chunked encoding (content length unknown at start of request, chunked encoding will indicate when the end is reached)
connection closed by server (http "0.9")