Develop Reference

How to print all results of a JSON API call

I'm making a web-app that needs data through an API. I am trying to re-purpose some of the developer's example code of a different function than the original code was written for. Here's the important part:
WebRequest request = WebRequest.Create(urlPrefix + "getthingjson/" + Key);
WebResponse response = request.GetResponse();
Stream dataStream = response.GetResponseStream();
StreamReader reader = new StreamReader(dataStream);
string responseFromServer = reader.ReadToEnd();
reader.Close();
response.Close();
// Parse returned JSON into data
//
using (var web = new WebClient())
{
web.Encoding = System.Text.Encoding.UTF8;
var jsonString = responseFromServer;
var jss = new JavaScriptSerializer();
var ThingsList = jss.Deserialize<List<Things>>(jsonString);
string ThingsListStr = "";
foreach (Things x in ThingsList)
ThingsListStr = ThingsListStr + ", " + x.Name;
MessageBox.Show(ThingsListStr);
}
I know that I need to change 'Name' in order to get a different piece of info on the 'Things' call. The thing is, I need to call results on a different function, instead of 'Things' say 'Details'. I don't know what to look for in place of 'Name' since when I search that it returns nothing. How could I just deserialize all of what JSON returns? Sorry if my terminology was off or I made a simple mistake, I'm new to JSON and C#. Thanks!

To get Details instead of Name property you should:
Check if you receive this data in json.
Adjust your c# class with Details property if it doesn't exist. (Go to Things class and add new Details property)
Change
ThingsListStr = ThingsListStr + ", " + x.Name;
to
ThingsListStr = ThingsListStr + ", " + x.Details;

Post content to a webpage and return what it gets?

I'm not very good with any networking type of things (as far as C# goes) but I need to post data (not get!) to a web url and then return what it outputs.
So far I'm using this for post and determining if it's logged in or not.
//credentials
string username = textBox1.Text;
string password = textBox2.Text;
using (WebClient client = new WebClient())
{
byte[] data = Encoding.Default.GetBytes("username="+username+"&password="+password);
client.Headers["Content-Type"] = "application/Json";
try
{
var response2 = client.UploadData(base_url + "/users/authenticate", "POST", data);
MessageBox.Show(Encoding.Default.GetString(response2));
}
catch { }
}
Note: Error I'm getting is 400 bad request. Any ideas?

Looking at the vine code, I would say you need to send it JSON data instead of URL data. With something simple like that you can just write the string.
String jsonData = "{\"username\": \"" + username + "\", \"password\": \"" + password + "\"}";
Updated to add encoding and be more strict with JSON data
PHP's JSON decoder wants UTF-8 encoded data, so also change the encoding line to
byte[] data = Encoding.UTF8.GetBytes(jsonData);

Upload to S3 from HTTPWebResponse.GetResponseStream() in c#

I am trying to upload from an HTTP stream directly to S3, without storing in memory or as a file first. I am already doing this with Rackspace Cloud Files as HTTP to HTTP, however the AWS authentication is beyond me so am trying to use the SDK.
The problem is the upload stream is failing with this exception:
"This stream does not support seek operations."
I've tried with PutObject and TransferUtility.Upload, both fail with the same thing.
Is there any way to stream into S3 as the stream comes in, rather than buffering the whole thing to a MemoryStream or FileStream?
or is there any good examples of doing the authentication into S3 request using HTTPWebRequest, so I can duplicate what I do with Cloud Files?
Edit: or is there a helper function in the AWSSDK for generating the authorization header?
CODE:
This is the failing S3 part (both methods included for completeness):
string uri = RSConnection.StorageUrl + "/" + container + "/" + file.SelectSingleNode("name").InnerText;
var req = (HttpWebRequest)WebRequest.Create(uri);
req.Headers.Add("X-Auth-Token", RSConnection.AuthToken);
req.Method = "GET";
using (var resp = req.GetResponse() as HttpWebResponse)
{
using (Stream stream = resp.GetResponseStream())
{
Amazon.S3.Transfer.TransferUtility trans = new Amazon.S3.Transfer.TransferUtility(S3Client);
trans.Upload(stream, config.Element("root").Element("S3BackupBucket").Value, container + file.SelectSingleNode("name").InnerText);
//Use EITHER the above OR the below
PutObjectRequest putReq = new PutObjectRequest();
putReq.WithBucketName(config.Element("root").Element("S3BackupBucket").Value);
putReq.WithKey(container + file.SelectSingleNode("name").InnerText);
putReq.WithInputStream(Amazon.S3.Util.AmazonS3Util.MakeStreamSeekable(stream));
putReq.WithMetaData("content-length", file.SelectSingleNode("bytes").InnerText);
using (S3Response putResp = S3Client.PutObject(putReq))
{
}
}
}
And this is how I do it successfully from S3 to Cloud Files:
using (GetObjectResponse getResponse = S3Client.GetObject(new GetObjectRequest().WithBucketName(bucket.BucketName).WithKey(file.Key)))
{
using (Stream s = getResponse.ResponseStream)
{
//We can stream right from s3 to CF, no need to store in memory or filesystem.
var req = (HttpWebRequest)WebRequest.Create(uri);
req.Headers.Add("X-Auth-Token", RSConnection.AuthToken);
req.Method = "PUT";
req.AllowWriteStreamBuffering = false;
if (req.ContentLength == -1L)
req.SendChunked = true;
using (Stream stream = req.GetRequestStream())
{
byte[] data = new byte[32768];
int bytesRead = 0;
while ((bytesRead = s.Read(data, 0, data.Length)) > 0)
{
stream.Write(data, 0, bytesRead);
}
stream.Flush();
stream.Close();
}
req.GetResponse().Close();
}
}

As no-one answering seems to have done it, I spent the time working it out based on guidance from Steve's answer:
In answer to this question "is there any good examples of doing the authentication into S3 request using HTTPWebRequest, so I can duplicate what I do with Cloud Files?", here is how to generate the auth header manually:
string today = String.Format("{0:ddd,' 'dd' 'MMM' 'yyyy' 'HH':'mm':'ss' 'zz00}", DateTime.Now);
string stringToSign = "PUT\n" +
"\n" +
file.SelectSingleNode("content_type").InnerText + "\n" +
"\n" +
"x-amz-date:" + today + "\n" +
"/" + strBucketName + "/" + strKey;
Encoding ae = new UTF8Encoding();
HMACSHA1 signature = new HMACSHA1(ae.GetBytes(AWSSecret));
string encodedCanonical = Convert.ToBase64String(signature.ComputeHash(ae.GetBytes(stringToSign)));
string authHeader = "AWS " + AWSKey + ":" + encodedCanonical;
string uriS3 = "https://" + strBucketName + ".s3.amazonaws.com/" + strKey;
var reqS3 = (HttpWebRequest)WebRequest.Create(uriS3);
reqS3.Headers.Add("Authorization", authHeader);
reqS3.Headers.Add("x-amz-date", today);
reqS3.ContentType = file.SelectSingleNode("content_type").InnerText;
reqS3.ContentLength = Convert.ToInt32(file.SelectSingleNode("bytes").InnerText);
reqS3.Method = "PUT";
Note the added x-amz-date header as HTTPWebRequest sends the date in a different format to what AWS is expecting.
From there it was just a case of repeating what I was already doing.

Take a look at Amazon S3 Authentication Tool for Curl. From that web page:
Curl is a popular command-line tool for interacting with HTTP
services. This Perl script calculates the proper signature, then calls
Curl with the appropriate arguments.
You could probably adapt it or its output for your use.

I think the problem is that according to the AWS Documentation Content-Length is required and you don't know what the length is until the stream has finished.
(I would guess the Amazon.S3.Util.AmazonS3Util.MakeStreamSeekable routine is reading the whole stream into memory to get around this problem which makes it unsuitable for your scenario.)
What you can do is read the file in chunks and upload them using MultiPart upload.
PS, I assume you know the C# source for the AWSSDK for dotnet is on Github.

This is a true hack (which would probably break with a new implementation of the AWSSDK), and it requires knowledge of the length of the file being requested, but if you wrap the response stream as shown with this class (a gist) as shown below:
long length = fileLength;
you can get file length in several ways. I am uploading from a dropbox link, so they give me the
length along with the url. Alternatively, you can perform a HEAD request and get the Content-Length.
string uri = RSConnection.StorageUrl + "/" + container + "/" + file.SelectSingleNode("name").InnerText;
var req = (HttpWebRequest)WebRequest.Create(uri);
req.Headers.Add("X-Auth-Token", RSConnection.AuthToken);
req.Method = "GET";
using (var resp = req.GetResponse() as HttpWebResponse)
{
using (Stream stream = resp.GetResponseStream())
{
//I haven't tested this path
Amazon.S3.Transfer.TransferUtility trans = new Amazon.S3.Transfer.TransferUtility(S3Client);
trans.Upload(new HttpResponseStream(stream, length), config.Element("root").Element("S3BackupBucket").Value, container + file.SelectSingleNode("name").InnerText);
//Use EITHER the above OR the below
//I have tested this with dropbox data
PutObjectRequest putReq = new PutObjectRequest();
putReq.WithBucketName(config.Element("root").Element("S3BackupBucket").Value);
putReq.WithKey(container + file.SelectSingleNode("name").InnerText);
putReq.WithInputStream(new HttpResponseStream(stream, length)));
//These are necessary for really large files to work
putReq.WithTimeout(System.Threading.Timeout.Infinite);
putReq.WithReadWriteTimeout(System.Thread.Timeout.Infinite);
using (S3Response putResp = S3Client.PutObject(putReq))
{
}
}
}
The hack is overriding the Position and Length properties, and returning 0 for Position{get}, noop'ing Position{set}, and returning the known length for Length.
I recognize that this might not work if you don't have the length or if the server providing the source does not support HEAD requests and Content-Length headers. I also realize it might not work if the reported Content-Length or the supplied length doesn't match the actual length of the file.
In my test, I also supply the Content-Type to the PutObjectRequest, but I don't that that is necessary.

As sgmoore said, the problem is that your content length is not seekable from the HTTP response. However HttpWebResponse does have a content length property available. So you can actually form your Http post request to S3 yourself instead of using the Amazon library.
Here's another Stackoverflow question that managed to do that with what looks like full code to me.

Convert JSON curl to C#

I have the following curl code:
curl 'localhost:8983/solr/sessions/update?commit=true' -H 'Content-type:application/json' -d '[{"Session_SessionId":"da7007e9-fe7a-4bdf-b9e4-1a55034cf08f","Session_HasComments":{"set":true}}]'
I am trying to convert to C#, but I am getting an error every time so unsure if my code is correct...
Here's what I have so far:
string path = "http://localhost:8983/solr/sessions/update?commit=true";
var httpWebRequest = (HttpWebRequest)WebRequest.Create(path);
httpWebRequest.ContentType = "application/json";
httpWebRequest.Method = "POST";
using (var streamWriter = new StreamWriter(httpWebRequest.GetRequestStream()))
{
string json = "{\"Session_SessionId\":\"" + sessionId + "\"," +
"\"" + fieldName + "\":{\"set\":\"" + fieldValue + "\"}}";
streamWriter.Write(json);
streamWriter.Flush();
streamWriter.Close();
var httpResponse = (HttpWebResponse)httpWebRequest.GetResponse();
using (var streamReader = new StreamReader(httpResponse.GetResponseStream()))
{
var result = streamReader.ReadToEnd();
}
}
It always seems to error () on this line:
var httpResponse = (HttpWebResponse)httpWebRequest.GetResponse();
The error i get is:
The remote server returned an error: (400) Bad Request.","StackTrace":" at System.Net.HttpWebRequest.GetResponse()
Any ideas? Thanks in advance!
Dave

Maybe it's that you have removed square brackets in your JSON content that you are streaming into request? Try adding the [ ] back to the start/end of data. Although the "BadRequest" is usually quite strict error that tells you that your HTTP request is malformed, your server may actually return that code also for other cases - like missing session id - which probably occurred here.
note the diff:
-d '[{"Session_SessionId":"da70.....
^ bracket
and
string json = "{\"Session_SessionId\":\"" + sessionId + "\"," + ....
^ no bracket
and the same at the end of data.
But, of course, that's just a guess. :)

Your request cannot be understood by the server. Did you check the output of json variable. I believe the JSON string is not generating properly.
Why don't you use JavaScriptSerializer.Serialize to create JSON string.

Your use the Update Handler with a JSON object will need to follow the JSON format outlined in Update JSON - Update Commands

As another user has suggested your JSON output does not match the CURL. Try the following rather than typing the text.
var data = new[]
{
new
{
Session_SessionId = "da7007e9-fe7a-4bdf-b9e4-1a55034cf08f",
Session_HasComments = new {set = true}
}
};
var json = new System.Web.Script.Serialization.JavaScriptSerializer().Serialize(data);
Also, you are using the using keyword to write the data through, and attempt to handle response while inside the block - it probably does make a difference but it might be worth moving this outside this block.
Lastly, you may need to encode the data as a byte array.
Here is the code implementing the above suggestions.
string path = "http://localhost:8983/solr/sessions/update?commit=true";
var httpWebRequest = (HttpWebRequest)WebRequest.Create(path);
httpWebRequest.ContentType = "application/json";
httpWebRequest.Method = "POST";
var data = new[]
{
new
{
Session_SessionId = "da7007e9-fe7a-4bdf-b9e4-1a55034cf08f",
Session_HasComments = new {set = true}
}
};
string json = new System.Web.Script.Serialization.JavaScriptSerializer().Serialize(data);
byte[] byteData = new System.Text.ASCIIEncoding().GetBytes(json);
httpWebRequest.ContentLength = byteData.Length;
using (Stream stream = httpWebRequest.GetRequestStream())
{
stream.Write(byteData,0,byteData.Length);
}
HttpWebResponse resp = (HttpWebResponse)httpWebRequest.GetResponse();
string respStr = new StreamReader(resp.GetResponseStream()).ReadToEnd();
Console.WriteLine("Response : " + respStr);

Just use SolrNet. If you are doing this with Solr 4+, you need to download latest code and built it yourself, but that's very easy.

Dave, it worked for me.
You were missing square brackets. Just replace respective line with below:
string json = "[{\"Session_SessionId\":\"" + sessionId + "\"," +
"\"" + fieldName + "\":{\"set\":\"" + fieldValue + "\"}}]";

Send object through HttpWebRequest (REST Service)

I have a REST Service that accepts a id in the URL for PUT requests. So far the PUT request looks like this:
string url = "http://localhost:3596/WidgetManager.svc/Widgets/" + TextBox3.Text;
WebRequest req = WebRequest.Create(url);
req.Method = "PUT";
using (HttpWebResponse resp = req.GetResponse() as HttpWebResponse)
{
StreamReader reader = new StreamReader(resp.GetResponseStream());
Label4.Text = reader.ReadToEnd();
}
But I also need to send a Widget object in my request.
Widget w = new Widget();
w.Name = "worked!!!";
w.CogCount = 1000;
w.SprocketSize = 2000;
I saw a lot of examples on how to send strings. But what about objects like this?

You could serialise it using XML or JSON.
If it is such a small object, you could write your own small method like
.toJSON() {
return '{"Name":"' + this.name + '", "CogCount":' + this.CogCount + ', "SprocketSize":' + this.SprocketSize + '}';
}
//Output: '{"Name":"worked!!!", "CogCount":1000, "SprocketSize":2000}'
On the other hand: C# provides powerful (XML) serialisation tools!
This here: http://www.codeproject.com/Articles/1789/Object-Serialization-using-C is only one of many examples.
But if you use PHP or similar, JSON might even be more interesting.