Develop Reference

How to properly encode binary data to send over REST api PUT call

I'm trying to simply upload an image file to Azure blob store and the file is making it there but the file size is bigger than it should be and when I download the file via the browser it won't open as an image.
I manually uploaded the same file via the Azure interface and the file works and is 22k, uploading via my code is 29k and doesn't work.
NOTE: One thing to note is that all the examples used with this code only send text files. Maybe the Encoding.UTF8.GetBytes(requestBody); line is doing it?
This is where I'm Base64 encoding my data
HttpPostedFileBase hpf = Request.Files[file] as HttpPostedFileBase
byte[] binData;
using (BinaryReader b = new BinaryReader(hpf.InputStream))
binData = b.ReadBytes(hpf.ContentLength);
var result = System.Convert.ToBase64String(binData);
new BlobHelper("STORENAMEHERE", "SECRETKEY").PutBlob("images", "test.jpg", result);
This is what I'm using to PUT the data, it's from https://azurestoragesamples.codeplex.com/
public bool PutBlob(string container, string blob, string content)
{
return Retry<bool>(delegate()
{
HttpWebResponse response;
try
{
SortedList<string, string> headers = new SortedList<string, string>();
headers.Add("x-ms-blob-type", "BlockBlob");
response = CreateRESTRequest("PUT", container + "/" + blob, content, headers).GetResponse() as HttpWebResponse;
response.Close();
return true;
}
catch (WebException ex)
{
if (ex.Status == WebExceptionStatus.ProtocolError &&
ex.Response != null &&
(int)(ex.Response as HttpWebResponse).StatusCode == 409)
return false;
throw;
}
});
}
public HttpWebRequest CreateRESTRequest(string method, string resource, string requestBody = null, SortedList<string, string> headers = null,
string ifMatch = "", string md5 = "")
{
byte[] byteArray = null;
DateTime now = DateTime.UtcNow;
string uri = Endpoint + resource;
HttpWebRequest request = HttpWebRequest.Create(uri) as HttpWebRequest;
request.Method = method;
request.ContentLength = 0;
request.Headers.Add("x-ms-date", now.ToString("R", System.Globalization.CultureInfo.InvariantCulture));
request.Headers.Add("x-ms-version", "2009-09-19");
if (headers != null)
{
foreach (KeyValuePair<string, string> header in headers)
request.Headers.Add(header.Key, header.Value);
}
if (!String.IsNullOrEmpty(requestBody))
{
request.Headers.Add("Accept-Charset", "UTF-8");
byteArray = Encoding.UTF8.GetBytes(requestBody);
request.ContentLength = byteArray.Length;
}
request.Headers.Add("Authorization", AuthorizationHeader(method, now, request, ifMatch, md5));
if (!String.IsNullOrEmpty(requestBody))
request.GetRequestStream().Write(byteArray, 0, byteArray.Length);
return request;
}

I believe the problem is with how you're converting the binary data to string and then converting it back to byte array.
While converting binary data to string, you're using System.Convert.ToBase64String however while getting the byte array from string you're using Encoding.UTF8.GetBytes. Most likely this is causing the mismatch.
Please change the following line of code:
byteArray = Encoding.UTF8.GetBytes(requestBody);
to
byteArray = System.Convert.FromBase64String(requestBody);
and that should fix the problem.

no http-connection possible after occurrence of server error 503

I build a windows-mobile 6.5 application (based on cf 2.0) and have a problem with a special test case of one method. So I hope someone can give me an advice or has a helpful idea what the reason for this behaviour is...
The method is called continuous every 30 seconds from inside a thread, looks for files to be transferred via a HTTP request to a web server (jboss) and brings them on their way. The server url itself is under my control.
Everything works fine ... until I stop the web server and force an 503 server error. So far so good. But after restarting the web server, I would expect, that the next call of the transfer method will end in success - but it does not. Every further try ends in a timeout exception and I have to restart the application to make it work again.
So my question is: where is the problem, when I want to connect to an uri after an earlier try has failed with error 503? It seems, that there is something cached, but what the hell should it be?
Many thanks for every hint you have.
Juergen
public static Boolean HttpUploadFile2(string url, string file)
{
HttpWebRequest requestToServer = null;
WebResponse response = null;
try
{
Logger.writeToLogFileCom(string.Format("Uploading {0} to {1}", file, url));
requestToServer = (HttpWebRequest)WebRequest.Create(url);
requestToServer. Timeout = 40000;
string boundaryString = "----SSLBlaBla";
requestToServer.AllowWriteStreamBuffering = false;
requestToServer.Method = "POST";
requestToServer.ContentType = "multipart/form-data;
boundary=" + boundaryString;
requestToServer.KeepAlive = false;
ASCIIEncoding ascii = new ASCIIEncoding();
string boundaryStringLine = "\r\n--" + boundaryString + "\r\n";
byte[] boundaryStringLineBytes = ascii.GetBytes(boundaryStringLine);
string lastBoundaryStringLine = "\r\n--" + boundaryString + "--\r\n";
byte[] lastBoundaryStringLineBytes = ascii.GetBytes(lastBoundaryStringLine);
// Get the byte array of the myFileDescription content disposition
string myFileDescriptionContentDisposition = String.Format(
"Content-Disposition: form-data; name=\"{0}\"\r\n\r\n{1}",
"myFileDescription",
"A sample file description");
byte[] myFileDescriptionContentDispositionBytes
= ascii.GetBytes(myFileDescriptionContentDisposition);
string fileUrl = file;
// Get the byte array of the string part of the myFile content
// disposition
string myFileContentDisposition = String.Format(
"Content-Disposition: form-data;name=\"{0}\"; "
+ "filename=\"{1}\"\r\nContent-Type: {2}\r\n\r\n",
"myFile", Path.GetFileName(fileUrl), Path.GetExtension(fileUrl));
byte[] myFileContentDispositionBytes =
ascii.GetBytes(myFileContentDisposition);
FileInfo fileInfo = new FileInfo(fileUrl);
// Calculate the total size of the HTTP request
long totalRequestBodySize = boundaryStringLineBytes.Length * 2
+ lastBoundaryStringLineBytes.Length
+ myFileDescriptionContentDispositionBytes.Length
+ myFileContentDispositionBytes.Length
+ fileInfo.Length;
// And indicate the value as the HTTP request content length
requestToServer.ContentLength = totalRequestBodySize;
// Write the http request body directly to the server
using (Stream s = requestToServer.GetRequestStream())
{
//TIMEOUT OCCURED WHEN CALLING GetRequestStream
// Send the file description content disposition over to the server
s.Write(boundaryStringLineBytes, 0, boundaryStringLineBytes.Length);
s.Write(myFileDescriptionContentDispositionBytes, 0,
myFileDescriptionContentDispositionBytes.Length);
// Send the file content disposition over to the server
s.Write(boundaryStringLineBytes, 0, boundaryStringLineBytes.Length);
s.Write(myFileContentDispositionBytes, 0,
myFileContentDispositionBytes.Length);
// Send the file binaries over to the server, in 1024 bytes chunk
FileStream fileStream = new FileStream(fileUrl, FileMode.Open,
FileAccess.Read);
byte[] buffer = new byte[1024];
int bytesRead = 0;
Logger.writeToLogFileCom("writing data...");
while ((bytesRead = fileStream.Read(buffer, 0, buffer.Length)) != 0)
{
s.Write(buffer, 0, bytesRead);
} // end while
fileStream.Close();
Logger.writeToLogFileCom("... finished, File closed");
// Send the last part of the HTTP request body
s.Write(lastBoundaryStringLineBytes, 0, lastBoundaryStringLineBytes.Length);
Logger.writeToLogFileCom("... finished, File closed");
} // end using
// Grab the response from the server. WebException will be thrown
// when a HTTP OK status is not returned
Logger.writeToLogFileCom("lese Response");
response = requestToServer.GetResponse();
StreamReader responseReader = new StreamReader(response.GetResponseStream());
string replyFromServer = responseReader.ReadToEnd();
response.Close();
if (Regex.Split(Regex.Split(replyFromServer, "content\\:RESPONSE\"\\>")[1], "\\</span\\>")[0].Equals("OK"))
{
return true;
}
else
{
return false;
}
}
catch (Exception ex)
{
Logger.writeToLogFileCom("Fehler im HTML Sender");
Logger.writeToLogFileCom(ex.Message);
Logger.writeToLogFileCom(ex.StackTrace);
}
finally
{
try
{
if (response != null)
{
response.Close();
}
}
catch (Exception ex) { }
}
return false;
}

I solved the problem.
I added an additional try / catch block inside the finally clause to call getResponse in every situation.
finally
{
try { response = requestToServer.GetResponse(); }
catch (Exception ex) { }
[...]

Google Translate V2 cannot hanlde large text translations from C#

I've implemented C# code using the Google Translation V2 api with the GET Method.
It successfully translates small texts but when increasing the text length and it takes 1,800 characters long ( including URI parameters ) I'm getting the "URI too large" error.
Ok, I burned down all the paths and investigated the issue across multiple pages posted on Internet. All of them clearly says the GET method should be overriden to simulate a POST method ( which is meant to provide support to 5,000 character URIs ) but there is no way to find out a code example to of it.
Does anyone has any example or can provide some information?
[EDIT] Here is the code I'm using:
String apiUrl = "https://www.googleapis.com/language/translate/v2?key={0}&source={1}&target={2}&q={3}";
String url = String.Format(apiUrl, Constants.apiKey, sourceLanguage, targetLanguage, text);
Stream outputStream = null;
byte[] bytes = Encoding.ASCII.GetBytes(url);
// create the http web request
HttpWebRequest webRequest = (HttpWebRequest)WebRequest.Create(url);
webRequest.KeepAlive = true;
webRequest.Method = "POST";
// Overrride the GET method as documented on Google's docu.
webRequest.Headers.Add("X-HTTP-Method-Override: GET");
webRequest.ContentType = "application/x-www-form-urlencoded";
// send POST
try
{
webRequest.ContentLength = bytes.Length;
outputStream = webRequest.GetRequestStream();
outputStream.Write(bytes, 0, bytes.Length);
outputStream.Close();
}
catch (HttpException e)
{
/*...*/
}
try
{
// get the response
HttpWebResponse webResponse = (HttpWebResponse)webRequest.GetResponse();
if (webResponse.StatusCode == HttpStatusCode.OK && webRequest != null)
{
// read response stream
using (StreamReader sr = new StreamReader(webResponse.GetResponseStream(), Encoding.UTF8))
{
string lista = sr.ReadToEnd();
DataContractJsonSerializer serializer = new DataContractJsonSerializer(typeof(TranslationRootObject));
MemoryStream stream = new MemoryStream(Encoding.UTF8.GetBytes(lista));
TranslationRootObject tRootObject = (TranslationRootObject)serializer.ReadObject(stream);
string previousTranslation = string.Empty;
//deserialize
for (int i = 0; i < tRootObject.Data.Detections.Count; i++)
{
string translatedText = tRootObject.Data.Detections[i].TranslatedText.ToString();
if (i == 0)
{
text = translatedText;
}
else
{
if (!text.Contains(translatedText))
{
text = text + " " + translatedText;
}
}
}
return text;
}
}
}
catch (HttpException e)
{
/*...*/
}
return text;
}

Apparently using WebClient won't work as you cannot alter the headers as needed, per the documentation:
Note: You can also use POST to invoke the API if you want to send more data in a single request. The q parameter in the POST body must be less than 5K characters. To use POST, you must use the X-HTTP-Method-Override header to tell the Translate API to treat the request as a GET (use X-HTTP-Method-Override: GET).
You can use WebRequest, but you'll need to add the X-HTTP-Method-Override header:
var request = WebRequest.Create (uri);
request.Method = "POST";
request.ContentType = "application/x-www-form-urlencoded";
request.Headers.Add("X-HTTP-Method-Override", "GET");
var body = new StringBuilder();
body.Append("key=SECRET");
body.AppendFormat("&source={0}", HttpUtility.UrlEncode(source));
body.AppendFormat("&target={0}", HttpUtility.UrlEncode(target));
//--
body.AppendFormat("&q={0}", HttpUtility.UrlEncode(text));
var bytes = Encoding.ASCII.GetBytes(body.ToString());
if (bytes.Length > 5120) throw new ArgumentOutOfRangeException("text");
request.ContentLength = bytes.Length;
using (var output = request.GetRequestStream())
{
output.Write(bytes, 0, bytes.Length);
}

The accepted answer appears to be out of date. You can now use the WebClient (.net 4.5) successfully to POST to the google translate API making sure to set the X-HTTP-Method-Override header.
Here is some code to show you how.
using (var webClient = new WebClient())
{
webClient.Headers.Add("X-HTTP-Method-Override", "GET");
var data = new NameValueCollection()
{
{ "key", GoogleTranslateApiKey },
{ "source", "en" },
{ "target", "fr"},
{ "q", "<p>Hello World</p>" }
};
try
{
var responseBytes = webClient.UploadValues(GoogleTranslateApiUrl, "POST", data);
var json = Encoding.UTF8.GetString(responseBytes);
var result = JsonConvert.DeserializeObject<dynamic>(json);
var translation = result.data.translations[0].translatedText;
}
catch (Exception ex)
{
loggingService.Error(ex.Message);
}
}

? What? it is trivial to post using C# - it is right there in the documentation.
HttpWebRequest request = (HttpWebRequest)WebRequest.Create(uri);
{
// Set type to POST
request.Method = "POST";
From there on you bascially put the data into fom fields into the content stream.
This is not "simulate a post meethod", it is fully doing a post request as per specifications.
Btw. hwhere does json enter here? You say "in C#". There is no need to use json?

Uploading to imgur.com

Imgur is a image uploading website who offers an API to upload
My code looks exactly like the PHP code they provide as an example. however, in their php code they are http_build_query($pvars);
It seems like they are URLEncoding their query before posting it. edit: Note that I have changed to full .NET 3.5 rather then the client profile. This gave me access to system.web so I used httputliity.urlencode(). This made the api return a "fail" with a "no image was sent". If I don't encode then the API returns an "okay" with a link to the picture, however no picture is uploaded (like a blank file).
How can I fix my code to work properly against their API?
Image image = Image.FromFile("C:\\Users\\Affan\\Pictures\\1509310.jpg");
MemoryStream ms = new MemoryStream();
// Convert Image to byte[]
image.Save(ms, System.Drawing.Imaging.ImageFormat.Jpeg);
byte[] imageBytes = ms.ToArray();
WebRequest wb = WebRequest.Create(new Uri("http://imgur.com/api/upload.xml"));
wb.ContentType = "application/x-www-form-urlencoded";
wb.Method = "POST";
wb.Timeout = 10000;
Console.WriteLine(imageBytes.Length);
string parameters = "key=433a1bf4743dd8d7845629b95b5ca1b4&image=" + Convert.ToBase64String(imageBytes);
Console.WriteLine("parameters: " + parameters.Length);
System.Text.UTF8Encoding encoding = new System.Text.UTF8Encoding();
byte[] bytes = encoding.GetBytes(parameters);
// byte[] bytes = Convert.FromBase64String(parameters);
System.IO.Stream os = null;
try { // send the Post
wb.ContentLength = bytes.Length; //Count bytes to send
os = wb.GetRequestStream();
os.Write(bytes, 0, bytes.Length); //Send it
} catch (WebException ex) {
MessageBox.Show(ex.Message, "HttpPost: Request error");
Console.WriteLine(ex.Message);
} finally {
if (os != null) {
// os.Close();
}
}
try { // get the response
WebResponse webResponse = wb.GetResponse();
StreamReader sr = new StreamReader(webResponse.GetResponseStream());
//MessageBox.Show(sr.ReadToEnd().Trim());
Console.WriteLine(sr.ReadToEnd().Trim());
} catch (WebException ex) {
MessageBox.Show(ex.Message, "HttpPost: Response error");
}

I've just uploaded this image
using this code:
using (var w = new WebClient())
{
var values = new NameValueCollection
{
{ "key", "433a1bf4743dd8d7845629b95b5ca1b4" },
{ "image", Convert.ToBase64String(File.ReadAllBytes(#"hello.png")) }
};
byte[] response = w.UploadValues("http://imgur.com/api/upload.xml", values);
Console.WriteLine(XDocument.Load(new MemoryStream(response)));
}
You might want to change your API key now :-)
The output was:
<rsp stat="ok">
<image_hash>IWg2O</image_hash>
<delete_hash>fQAXiR2Fdq</delete_hash>
<original_image>http://i.imgur.com/IWg2O.png</original_image>
<large_thumbnail>http://i.imgur.com/IWg2Ol.jpg</large_thumbnail>
<small_thumbnail>http://i.imgur.com/IWg2Os.jpg</small_thumbnail>
<imgur_page>http://imgur.com/IWg2O</imgur_page>
<delete_page>http://imgur.com/delete/fQAXiR2Fdq</delete_page>
</rsp>

Here's an updated version of dtb's answer for the v3 API using anonymous uploading (you need to register your app at http://api.imgur.com/ to get your client ID):
using (var w = new WebClient())
{
string clientID = "<<INSERT YOUR ID HERE>>";
w.Headers.Add("Authorization", "Client-ID " + clientID);
var values = new NameValueCollection
{
{ "image", Convert.ToBase64String(File.ReadAllBytes(#"hello.png")) }
};
byte[] response = w.UploadValues("https://api.imgur.com/3/upload.xml", values);
Console.WriteLine(XDocument.Load(new MemoryStream(response)));
}
And the response is now like this (see http://api.imgur.com/models/image):
<data success="1" status="200">
<id>SbBGk</id>
<title/>
<description/>
<datetime>1341533193</datetime>
<type>image/jpeg</type>
<animated>false</animated>
<width>2559</width>
<height>1439</height>
<size>521916</size>
<views>1</views>
<bandwidth>521916</bandwidth>
<deletehash>eYZd3NNJHsbreD1</deletehash>
<section/>
<link>http://i.imgur.com/SbBGk.jpg</link>
</data>

I Guess that the dtb solution is deprecated
using (var w = new WebClient())
{
var values = new NameValueCollection
{
{"image", Convert.ToBase64String(imageData)},
{"type", "base64"}
};
w.Headers.Add("Authorization", "Client-ID xxxxxxxxx");
var response = w.UploadValues("https://api.imgur.com/3/image", values);
}
another way to do:
HttpWebRequest request = (HttpWebRequest)WebRequest.Create("https://api.imgur.com/3/image");
request.Headers.Add("Authorization", "Client-ID xxxxxxx");
request.Method = "POST";
ASCIIEncoding enc = new ASCIIEncoding();
string postData = Convert.ToBase64String(imageData);
byte[] bytes = enc.GetBytes(postData);
request.ContentType = "application/x-www-form-urlencoded";
request.ContentLength = bytes.Length;
Stream writer = request.GetRequestStream();
writer.Write(bytes, 0, bytes.Length);
HttpWebResponse response = (HttpWebResponse)request.GetResponse();

How do I programatically download a file from a sharepoint site?

I have a sharepoint site that has an excel spreadsheet that I need to download on a schedulad basis
Is this possible?

Yes it is possible to download the file from sharepoint.
Once you have the url for the document, it can be downloaded using HttpWebRequest and HttpWebResponse.
attaching a sample code
DownLoadDocument(string strURL, string strFileName)
{
HttpWebRequest request;
HttpWebResponse response = null;
request = (HttpWebRequest)WebRequest.Create(strURL);
request.Credentials = System.Net.CredentialCache.DefaultCredentials;
request.Timeout = 10000;
request.AllowWriteStreamBuffering = false;
response = (HttpWebResponse)request.GetResponse();
Stream s = response.GetResponseStream();
// Write to disk
if (!Directory.Exists(myDownLoads))
{
Directory.CreateDirectory(myDownLoads);
}
string aFilePath = myDownLoads + "\\" + strFileName;
FileStream fs = new FileStream(aFilePath, FileMode.Create);
byte[] read = new byte[256];
int count = s.Read(read, 0, read.Length);
while (count > 0)
{
fs.Write(read, 0, count);
count = s.Read(read, 0, read.Length);
}
// Close everything
fs.Close();
s.Close();
response.Close();
}
You can also use the GetItem API of Copy service to download a file.
string aFileUrl = mySiteUrl + strFileName;
Copy aCopyService = new Copy();
aCopyService.UseDefaultCredentials = true;
byte[] aFileContents = null;
FieldInformation[] aFieldInfo;
aCopyService.GetItem(aFileUrl, out aFieldInfo, out aFileContents);
The file can be retrieved as a byte array.

You can also do this:
try
{
using (WebClient client = new WebClient())
{
client.Credentials = new NetworkCredential("username", "password", "DOMAIN");
client.DownloadFile(http_path, path);
}
}
catch (Exception ex)
{
MessageBox.Show("Error: " + ex.Message);
}

The link the to document in Sharepoint should be a static URL. Use that URL in whatever solution you have to grab the file on your schedule.

Why not just use wget.exe <url>. You can put that line in a batch file and run that through windows scheduler.