Develop Reference

DataContractJsonSerializer: serialize object or array (can be both)

I'm accessing the tomtom json api, and the api either returns me an array of objects, or a single object, when an error has happen.
Example:
[{"driverno": "...
Error Example:
{"errorCode": "8011","errorMsg": "request quota reached, error code: 8011"}
The data is accessed WebRequest, WebResponse and they return a stream, which can then be passed to a DataContractJsonSerializer. However, I can't create a serialization class, which accepts both forms of JSON, and the stream can't be passed twice, because the seek function is not supported.
Is there a way, to create a serialization class which supports both types of JSON input?

I found a workaround, where I copy the Stream to a MemoryStream, which enables seeking. I'm not completly settisfied with th solution, becuase it does a Stream copying and the DataContractJsonSerializer twice.
Sample:
string text = File.ReadAllText(PAHT);
text = Regex.Replace(text, "\\{[\\n\\r ]*\"__type", "{\"__type");
// copy to MemoryStream
using (MemoryStream dataStream = new MemoryStream(Encoding.UTF8.GetBytes(text)))
{
DataContractJsonSerializer errorDeserializer = new DataContractJsonSerializer(typeof(RequestError));
RequestError errorSerilaized = (RequestError)errorDeserializer.ReadObject(dataStream);
// check if an error happened
if (errorSerilaized.errorCode == null)
{
// seek the stream to position 0
dataStream.Position = 0;
DataContractJsonSerializer _deserializer = new DataContractJsonSerializer(typeof(NoneErrorSerializationClass));
NoneErrorSerializationClass tripReportsSerialized = (NoneErrorSerializationClass)_deserializer.ReadObject(dataStream);
// ...
}
else
{
MessageBox.Show(errorSerilaized.errorMsg);
}
}

Is it safe to return a FileStream in a method in C#?

In ASP.NET WebForms 4.5, I have WebAPI Controller with a GET method for getting a PDF.
Then down in the business layer of the application, I have an API class with a method that contains the logic for actually finding and returning the PDF to the controller.
So MyController class basically has:
public HttpResponseMessage GetStatement(string acctNumber, string stmtDate) {
MyApi myApi = new MyApi();
HttpResponseMessage response = new HttpResponseMessage(HttpStatusCode.OK);
FileStream stream = myApi.GetStatement(acctNumber, stmtDate);
...set the response.Content = stream...
... set the mime type..
... close the stream...
return response;
}
And MyApi class has:
public FileStream GetStatement(string acctNumber, string stmtDate) {
... makes an HttpWebRequest to get a PDF from another system ...
HttpWebRequest req = WebRequest.Create(......)....
FileStream stream = new FileStream(accountNumber +"_" + stmtDate + ".pdf", FileMode.Create);
response.GetResponseStream().CopyTo(stream);
return stream;
}
The API class is not in the web layer of the application because it's used by other (non-web) parts of the software.
I guess my concern is there's no explicit closing of the FileStream in the API method. I could do it in the Controller method, but I'd be relying on others to do the same when they're calling it from other areas.
Is there a better way to return the PDF file from the API method? Possibly just as a byte array or something like that? Preferably as little overhead as possible.
Thanks-

You should not be returning a file stream but instead an array of bytes. This way you can correctly dispose of the object correctly and not worry about other calling methods in the stack.
byte[] currentFile = ....
You can then deliver your file as follows this, a byte array is easy to convert to anything. Below example is for MVC4.
return new FileContentResult(currentFile, "application/pdf");

It's not uncommon for methods to return FileStreams, hoping that the caller will remember to put the method call in a using statement. But it's understandable to not want to make that assumption. One alternative is to use an interesting form of Inversion of Control, where you require the caller to give you the callback function that knows what to do with the FileStream, and then you wrap a call to that handler inside a using statement.
public T GetStatement<T>(string acctNumber, string stmtDate,
Func<FileStream, T> callback) {
... makes an HttpWebRequest to get a PDF from another system ...
HttpWebRequest req = WebRequest.Create(......)....
using(FileStream stream = new FileStream(accountNumber +"_" + stmtDate + ".pdf", FileMode.Create))
{
response.GetResponseStream().CopyTo(stream);
return callback(stream);
}
}
However, this is going to require a little extra hackery in your use case because you're returning a response whose content is feeding through the stream, so you'd need to find a way to cause your message to push out the entire response before the callback returns.
It may be best in this case to just throw comments on your method to document the fact that you're expecting the caller to ensure the stream gets closed. That's what we do in our application, and it's worked well so far.

Generally, you should put your FileStream inside a using block as already described by other answers or have some faith that it will be disposed of by other parts of your code. However, this is tricky when you are returning a FileStream from your controller. For example:
public ActionResult GetImage()
{
Stream stream = //get the stream
return base.File( stream, "image/png" );
}
Happily, the stream is disposed of by the framework once written, so you don't need to worry about the disposing of it. See here for details.

This is not an answer to your exact requirement, but thought of sharing it so that it can give you an idea of other ways of doing this.
public async Task<HttpResponseMessage> Get(string acctNumber, string stmtDate)
{
HttpResponseMessage response2 = new HttpResponseMessage();
HttpClient client= new HttpClient();
string url = "http://localhost:9090/BusinessLayer?acctNumber=" + acctNumber + "&stmtDate=" + stmtDate;
// NOTE 1: here we are only reading the response1's headers and not the body. The body of response1 would be later be
// read by response2.
HttpResponseMessage response1 = await client.GetAsync(url, HttpCompletionOption.ResponseHeadersRead);
// NOTE 2: here we need not close response1's stream as Web API would close it when disposing
// response2's stream content.
response2.Content = new StreamContent(await response1.Content.ReadAsStreamAsync());
response2.Content.Headers.ContentLength = response1.Content.Headers.ContentLength;
response2.Content.Headers.ContentType = response1.Content.Headers.ContentType;
return response2;
}

Store Stream in cache

I'd like to cache the response I get from a HttpWebRequest. I need both the ResponseStream and the headers. Using HttpRequestCachePolicy(HttpCacheAgeControl.MaxAge, TimeSpan.FromDays(1)) when creating the request doesn't seem to work (IsFromCache is never true for some reason), and I'm a bit scared of manually caching the entire HttpWebResponse since it's containing a stream of gzipped data, and I have a bad feeling about storing streams in the ASP.NET cache.
The response is mapped to an object like this (simplified):
public readonly Stream Response;
public readonly string Etag;
private MyObject(Stream response, string etag)
{
this.Response = response;
this.Etag = etag;
}
Since the object also contains the response stream I face the same issue here.
How do I cache this?

A Stream is a pipe, not a bucket. If you need to store it, you must first get the actual contents. For example, you could read it all into a byte[]. For example:
using(var ms = new MemoryStream()) {
response.CopyTo(ms);
byte[] payload = ms.ToArray();
}
You can store the byte[] trivially. Then if you want a Stream later, you can use new MemoryStream(payload).

Can you create a custom class that contains a byte array for storing the data and another field with HttpWebRequest.Headers for the headers? Then cache that.

Getting a Stream from an absolute path?

I have this method:
public RasImage Load(Stream stream);
if I want to load a url like:
string _url = "http://localhost/Application1/Images/Icons/hand.jpg";
How can I make this url in to a stream and pass it into my load method?

Here's one way. I don't really know if it's the best way or not, but it works.
// requires System.Net namespace
WebRequest request = WebRequest.Create(_url);
using (var response = request.GetRespone())
using (var stream = response.GetResponseStream())
{
RasImage image = Load(stream);
}
UPDATE: It looks like in Silverlight, the WebRequest class has no GetResponse method; you've no choice but to do this asynchronously.
Below is some sample code illustrating how you might go about this. (I warn you: I wrote this just now, without putting much thought into how sensible it is. How you choose to implement this functionality would likely be quite different. Anyway, this should at least give you a general idea of what you need to do.)
WebRequest request = WebRequest.Create(_url);
IAsyncResult getResponseResult = request.BeginGetResponse(
result =>
{
using (var response = request.EndGetResponse(result))
using (var stream = response.GetResponseStream())
{
RasImage image = Load(stream);
// Do something with image.
}
},
null
);
Console.WriteLine("Waiting for response from '{0}'...", _url);
getResponseResult.AsyncWaitHandle.WaitOne();
Console.WriteLine("The stream has been loaded. Press Enter to quit.");
Console.ReadLine();

Dan's answer is a good one, though you're requesting from localhost. Is this a file you can access from the filesystem? If so, I think you should be able to just pass in a FileStream:
FileStream stream = new FileStream(#"\path\to\file", FileMode.Open);

Reading "chunked" response with HttpWebResponse

I'm having trouble reading a "chunked" response when using a StreamReader to read the stream returned by GetResponseStream() of a HttpWebResponse:
// response is an HttpWebResponse
StreamReader reader = new StreamReader(response.GetResponseStream());
string output = reader.ReadToEnd(); // throws exception...
When the reader.ReadToEnd() method is called I'm getting the following System.IO.IOException: Unable to read data from the transport connection: The connection was closed.
The above code works just fine when server returns a "non-chunked" response.
The only way I've been able to get it to work is to use HTTP/1.0 for the initial request (instead of HTTP/1.1, the default) but this seems like a lame work-around.
Any ideas?
#Chuck
Your solution works pretty good. It still throws the same IOExeception on the last Read(). But after inspecting the contents of the StringBuilder it looks like all the data has been received. So perhaps I just need to wrap the Read() in a try-catch and swallow the "error".

Haven't tried it this with a "chunked" response but would something like this work?
StringBuilder sb = new StringBuilder();
Byte[] buf = new byte[8192];
Stream resStream = response.GetResponseStream();
string tmpString = null;
int count = 0;
do
{
count = resStream.Read(buf, 0, buf.Length);
if(count != 0)
{
tmpString = Encoding.ASCII.GetString(buf, 0, count);
sb.Append(tmpString);
}
}while (count > 0);

I am working on a similar problem. The .net HttpWebRequest and HttpWebRequest handle cookies and redirects automatically but they do not handle chunked content on the response body automatically.
This is perhaps because chunked content may contain more than simple data (i.e.: chunk names, trailing headers).
Simply reading the stream and ignoring the EOF exception will not work as the stream contains more than the desired content. The stream will contain chunks and each chunk begins by declaring its size. If the stream is simply read from beginning to end the final data will contain the chunk meta-data (and in case where it is gziped content it will fail the CRC check when decompressing).
To solve the problem it is necessary to manually parse the stream, removing the chunk size from each chunk (as well as the CR LF delimitors), detecting the final chunk and keeping only the chunk data. There likely is a library out there somewhere that does this, I have not found it yet.
Usefull resources :
http://en.wikipedia.org/wiki/Chunked_transfer_encoding
https://www.rfc-editor.org/rfc/rfc2616#section-3.6.1

I've had the same problem (which is how I ended up here :-). Eventually tracked it down to the fact that the chunked stream wasn't valid - the final zero length chunk was missing. I came up with the following code which handles both valid and invalid chunked streams.
using (StreamReader sr = new StreamReader(response.GetResponseStream(), Encoding.UTF8))
{
StringBuilder sb = new StringBuilder();
try
{
while (!sr.EndOfStream)
{
sb.Append((char)sr.Read());
}
}
catch (System.IO.IOException)
{ }
string content = sb.ToString();
}

After trying a lot of snippets from StackOverflow and Google, ultimately I found this to work the best (assuming you know the data a UTF8 string, if not, you can just keep the byte array and process appropriately):
byte[] data;
var responseStream = response.GetResponseStream();
var reader = new StreamReader(responseStream, Encoding.UTF8);
data = Encoding.UTF8.GetBytes(reader.ReadToEnd());
return Encoding.Default.GetString(data.ToArray());
I found other variations work most of the time, but occasionally truncate the data. I got this snippet from:
https://social.msdn.microsoft.com/Forums/en-US/4f28d99d-9794-434b-8b78-7f9245c099c4/problems-with-httpwebrequest-and-transferencoding-chunked?forum=ncl

It is funny. During playing with the request header and removing "Accept-Encoding: gzip,deflate" the server in my usecase did answer in a plain ascii manner and no longer with chunked, encoded snippets. Maybe you should give it a try and keep "Accept-Encoding: gzip,deflate" away. The idea came while reading the upper mentioned wiki in topic about using compression.