I am attempting to get a file from Ebays API
https://developer.ebay.com/api-docs/sell/feed/resources/task/methods/getResultFile#h2-samples
I am getting data back but it is not decompressing properly or encoding properly. It should be an gziped xml file. The documentation is not very clear on this actually. I am using RestSharp for my http calls (106.15.0).
Exception:
Found invalid data while decoding
My Code:
const string url = "sell/feed/v1/task/task-16-SOMENUMBER/download_result_file";
var restClient = new RestClient(_restApiUrl)
{
Authenticator = new OAuth2AuthorizationRequestHeaderAuthenticator(authentication.AuthorizationToken, "Bearer")
};
var httpRequest = new RestRequest(url, Method.GET);
httpRequest.AddHeader("Accept-Encoding", "application/gzip");
httpRequest.AddHeader("Accept", "*/*");
byte[] myfile = restClient.DownloadData(httpRequest);
var decodedString = Encoding.UTF8.GetString(myfile);
using (var stream = new MemoryStream(myfile))
{
string res;
using (GZipStream zipStream = new GZipStream(stream, CompressionMode.Decompress))
{
using (var sr = new StreamReader(zipStream))
{
res = sr.ReadToEnd(); //ERROR HERE: Found invalid data while decoding
}
}
var result = res;
}
First 30 of returned string (Encoding.Default.GetString(myfile))
PK\u0003\u0004\u0014\0\b\b\b\0\f£†T\0\0\0\0\0\0\0\0\0\0\0\0>\0\0\0ActiveInventoryReport
Hex
50-4B-03-04-14-00-08-08-08-00-0C-A3-86-54-00-00-00-00-00-00-00-00-00-00-00-00-3E-00-00-00-41-63-74-69-76-65-49-6E-76-65-6E-74-6F-72-79-52-65-70-6F-72-74-2D-41-70-72-2D-30-36-2D-32-30-32-32-2D-32-30-3A-32-34-3A-32-30-2D-30-37-30-30-2D-31-33-33-34-39-39-38-35-32-34-2E-78-6D-6C-BD-9D-5B-53-9B-47-B6-86-AF-77-7E-45-CA-F7-32-7D-3E-4C-79-3C-25-09-03-1E-C0-D6-48-38-60-DF-B1-8D-F6-98-0A-01-17-86-4C-3C-BF-7E-F7-E1-
I am creating a console application that
Connects to a vendor API to pull voucher numbers for submitted expenses between two dates and
Downloads a PDF copy of receipts submitted with the expense
The first part, I have working fine. I am able to connect to the Vendor API and parse out the returned XML to create an array of voucher numbers (needed to get the PDF images) using the following code:
static async Task RunAsyncCR()
{
using (var client = new HttpClient())
{
var values = new Dictionary<string, string>
{
{"un","SomeUser"},
{"pw","SomePassword"},
{"method","getVoucherInvoices"},
{"fromDate","05/30/2016"},
{"toDate", "06/13/2016"}
};
var content = new FormUrlEncodedContent(values);
Console.WriteLine("Connecting...");
var response = await client.PostAsync("https://www.chromeriver.com/receipts/doit", content);
Console.WriteLine("Connected...");
var responseString = await response.Content.ReadAsStringAsync();
char[] DelimiterChars = {'<'};
String[] xmlReturn = responseString.Split(DelimiterChars);
string[] VoucherNumber = new string[500];
int i = 0;
foreach (string s in xmlReturn)
{
if (s.Contains("voucherInvoice>") && s != "/voucherInvoice>\n ")
{
VoucherNumber[i] = s.Substring(15, 16);
i++;
}
}
Array.Resize(ref VoucherNumber, i);
Yes, there is likely a better way of doing this, but it works and returns the values I am expecting.
Now, what I am having trouble with, is when I connect back to the API to retrieve the file, I cannot seem to be able to download the file to a specified file path.
I can connect back to the API using
i = 0;
foreach (string x in VoucherNumber)
{
Console.WriteLine("Get receipt: " + x);
var NewValues = new Dictionary<string, string>
{
{"un","SomeUser"},
{"pw","SomePassword"},
{"method","getReceiptsWithCoverPage"},
{"voucherInvoiceForPdf", VoucherNumber[i]}
};
var NewContent = new FormUrlEncodedContent(NewValues);
var NewResponse = await client.PostAsync("https://www.chromeriver.com/receipts/doit", NewContent);
string NewResponseString = await NewResponse.Content.ReadAsStringAsync();
But I cannot seem to write the response to a valid file (PDF)
Here is a screen shot of my Autos window as I step through the code, where I would need to download the file:
My question is, from this point, how do I go about saving the file to my system?
I have tried to take the encoded response I get from doing Console.WriteLine(NewResponseString); and write it to a file using the System.IO.File.WriteAllLines() method, using a specified filepath/name, but this results in a blank file. I have also spent some time researching the depths of Google/Stackoverflow, but do not understand how to implement the results I find.
Any and all help would be greatly appreciated.
So I think you need help with Streams. The returned HttpContent is actually a System.Net.Http.StreamContent instance which shows that you are getting content back. Its just a matter of getting the Stream (content) from that instance and saving that to a file.
var NewResponse = await client.PostAsync("https://www.chromeriver.com/receipts/doit", NewContent);
System.Net.Http.HttpContent content = NewResponse.Content; // actually a System.Net.Http.StreamContent instance but you do not need to cast as the actual type does not matter in this case
using(var file = System.IO.File.Create("somePathHere.pdf")){ // create a new file to write to
var contentStream = await content.ReadAsStreamAsync(); // get the actual content stream
await contentStream.CopyToAsync(file); // copy that stream to the file stream
}
I respectfully recommend that you do a little reading on how Streams work. This is a common construct in many languages that you will probably have to deal with again in the near future.
First of all, are you sure there is a file to begin with? May I suggest using the open source library PdfSharp. I personally use it myself and it works great. As far as downloading the file, maybe this may help you...
Download Synchronously
using System.Net;
WebClient webClient = new WebClient();
webClient.DownloadFile("http://example.com/myfile.txt", #"c:\\myfile.txt");
http://www.csharp-examples.net/download-files/
At first Create StreamReader from NewResponse
Stream receiveStream = NewResponse.GetResponseStream ();
StreamReader readStream = new StreamReader (receiveStream, Encoding.UTF8);
Then Define a StremaWriter to write into a file.
using (var writer = new StreamWriter(#"C:\MyNewFile.pdf", append: false))
{
writer.Write(readStream.ReadToEnd());
}
Alternative Approach is
var httpContent = NewResponse.Content;
using(var newFile = System.IO.File.Create(#"C:\MyNewFile.pdf"))
{
var stream = await httpContent.ReadAsStreamAsync();
await stream.CopyToAsync(newFile);
}
Here is what I did, did not find any other solution that satisfied my situation:
using (var client = new System.Net.Http.HttpClient())
{
client.DefaultRequestHeaders.Add("Authorization", "someapikey");
client.BaseAddress = new Uri("https://someurl.com");
byte[] bytes = client.GetByteArrayAsync(client.BaseAddress).ConfigureAwait(false).GetAwaiter().GetResult();
string pdfFilePath = #"c:\somepath"
System.IO.File.WriteAllBytes(pdfFilePath, bytes);
//Note that below is only to open PDF in standard viewer, not necessary
var process = new System.Diagnostics.Process();
var startInfo = new System.Diagnostics.ProcessStartInfo()
{
FileName=pdfFilePath,
WorkingDirectory = System.IO.Path.GetDirectoryName(pdfFilePath),
UseShellExecute = true
}
process.StartInfo = startInfo;
process.Start();
}
Use this code for download a pdf from the API. It will convert the string data to bytes and provide you the necessary solution.
HttpWebRequest request = (HttpWebRequest) WebRequest.Create(URL);
request.ContentType = "application/pdf;charset=UTF-8";
request.Method = "GET";
using(HttpWebResponse response = (HttpWebResponse) request.GetResponse()) {
BinaryReader bin = new BinaryReader(response.GetResponseStream());
byte[] buffer = bin.ReadBytes((Int32) response.ContentLength);
Response.Buffer = true;
Response.Charset = "";
Response.AppendHeader("Content-Disposition", "attachment; filename=+ filename);
Response.Cache.SetCacheability(HttpCacheability.NoCache);
Response.ContentType = "application/pdf";
Response.BinaryWrite(buffer);
Response.Flush();
Response.End();
}
I download a web page as follows. I want to save it as UTF-8 text. But how?
HttpWebRequest req = (HttpWebRequest)WebRequest.Create(url);
using (HttpWebResponse resp = (HttpWebResponse)req.GetResponse())
{
Encoding enc = Encoding.GetEncoding(resp.CharacterSet);
Encoding utf8 = Encoding.UTF8;
using (StreamWriter w = new StreamWriter(new FileStream(pathname, FileMode.Create), utf8))
{
using (StreamReader r = new StreamReader(resp.GetResponseStream()))
{
// This works, but it's bad because you read the whole response into memory:
string s = r.ReadToEnd();
w.Write(s);
// This doesn't work :(
char[] buffer = new char[1024];
int n;
while (!r.EndOfStream)
{
n = r.ReadBlock(buffer, 0, 1024);
w.Write(utf8.GetChars(Encoding.Convert(enc, utf8, enc.GetBytes(buffer))));
}
// This means that r.ReadToEnd() is doing the transcoding to UTF-8 differently.
// But how?!
}
}
return resp.StatusCode;
}
Don't read this paragraph. It's just here to make the warning message about having too much code go away.
You could simply use the WebClient Class. It supports encoding and easier use:
WebClient webClient = new WebClient();
webClient.Encoding = System.Text.Encoding.UTF8;
webClient.DownloadFile(url, "file.txt");
this one is baking my head, i need an XML feed, but the feed is inside a gzip file.
now i can download, unpack, save file etc.
but what i want to do is get the gzip in memory
put the contents into memory
read the contents into memory
and then pass that to the serializer
problem here is that results contain unreadable gibberish and reader1 and reader both contain none. I am missing another line here but cant seem to find what it is.
var request = (HttpWebRequest)HttpWebRequest.Create("urltofeed.com");
request.Credentials = CredentialCache.DefaultCredentials;
// auto unpack gzip
request.AutomaticDecompression = DecompressionMethods.GZip;
HttpWebResponse response = (HttpWebResponse)request.GetResponse();
StreamReader data = new StreamReader(response.GetResponseStream());
var result = data.ReadToEnd();
XmlReader reader1 = XmlReader.Create(response.GetResponseStream());
var reader = new XmlTextReader(response.GetResponseStream());
Direclty from MSDN:
//Create the decompressed file.
using (FileStream outFile = File.Create(origName))
{
using (GZipStream Decompress = new GZipStream(inFile,
CompressionMode.Decompress))
{
// Copy the decompression stream
// into the output file.
Decompress.CopyTo(outFile);
Console.WriteLine("Decompressed: {0}", fi.Name);
}
}
So in your case something like
using (var GZipStream decompress = new GZipStream(response.GetResponseStream(), CompressionMode.Decompress))
{
// XmlReader reader1 = XmlReader.Create(decompress);
// or ? var reader = new XmlTextReader(decompress);
}
You have to decompress the gzip-file, that step is missing. Try this: GZipStream and decompression
Behold the code:
using (var client = new WebClient())
{
using (var stream = client.OpenWrite("http://localhost/", "POST"))
{
stream.Write(post, 0, post.Length);
}
}
Now, how do I read the HTTP output?
It looks like you have a byte[] of data to post; in which case I expect you'll find it easier to use:
byte[] response = client.UploadData(address, post);
And if the response is text, something like:
string s = client.Encoding.GetString(response);
(or your choice of Encoding - perhaps Encoding.UTF8)
If you want to keep streams everywhere and avoid allocating huge arrays of bytes, which is good practise (for example, if you plan to post big files), you still can do it with a derived version of WebClient. Here is a sample code that does it.
using (var client = new WebClientWithResponse())
{
using (var stream = client.OpenWrite(myUrl))
{
// open a huge local file and send it
using (var file = new FileStream(filePath, FileMode.Open, FileAccess.Read, FileShare.ReadWrite))
{
file.CopyTo(stream);
}
}
// get response as an array of bytes. You'll need some encoding to convert to string, etc.
var bytes = client.Response;
}
And here is the customized WebClient:
public class WebClientWithResponse : WebClient
{
// we will store the response here. We could store it elsewhere if needed.
// This presumes the response is not a huge array...
public byte[] Response { get; private set; }
protected override WebResponse GetWebResponse(WebRequest request)
{
var response = base.GetWebResponse(request);
var httpResponse = response as HttpWebResponse;
if (httpResponse != null)
{
using (var stream = httpResponse.GetResponseStream())
{
using (var ms = new MemoryStream())
{
stream.CopyTo(ms);
Response = ms.ToArray();
}
}
}
return response;
}
}