Downloading JSON through WebClient - c#

I'm trying to download a JSON from Twitter with all the authorization and stuff.
WebClient wc = new WebClient();
wc.DownloadStringCompleted += (b, a) => {
if (a.Cancelled)
MessageBox.Show("Download Canceled!");
else if (a.Error != null)
MessageBox.Show("Download Error!");
else
string g = a.Result;
};
wc.DownloadStringAsync(new Uri("TWITTER_JSON"));
(TWITTER_JSON is a long address with many authorization headers that gives the JSON)
When I run this the 2nd message ("Download Error!") shows up. Why? And how do I fix this?

a.Error is actually an Exception object. Have you tried examining it to see what the exception details contain?
MessageBox.Show( a.Error.ToString() );
That will give you more information about what actually went wrong.
You may also find it helpful to read Eric Lippert's recent blog post on how to debug your code.

Related

Vimeo: Get download link from API

I've Vimeo PRO and I'm trying to get the download link so the end user can download the video source. However, the lack of documentation makes it really hard to figure that out.
I'm trying VimeoDotNet but I cannot authenticate, I'm doing the following:
var client = new VimeoClientFactory().GetVimeoClient(key, secret)
var downloadLink = client.GetVideo(video_id).download;
However, the call to GetVideo throws an error saying I have to authenticate first, but I don't see how!
I've also tried with another VimeoClient, but it doesn't seem to implement the download link part.
Can anyone help? Or better yet, share a working example. Thanks.
After 2 days I was finally able to do it, I'll share what I did in case someone needs it. First, download this library:
https://github.com/saeedafshari/VimeoDotNet3
Open in Visual Studio and compile it. It's pretty simple so it compiled right away.
Then reference that compiled DLL from your project and do the following:
var VimeoClient3 = Vimeo.VimeoClient.ReAuthorize(_vimeoAccessToken,
_vimeoAppConsumerKey, _vimeoAppClientSecret);
// videoId is the ID of the video as in the public URL (eg, 123874983)
var result = VimeoClient3.Request("/videos/" + videoId, null, "GET");
if (result == null)
{
throw new Exception("Video not found.");
}
if (result["download"] == null)
{
throw new Exception("Download link not available.");
}
foreach (var item in (ArrayList)result["download"])
{
var downloadLinkInfo = item as Dictionary<string, object>;
if (downloadLinkInfo == null) continue;
// For example, get the link for SD quality.
// As of today, Vimeo was returning an HD quality and a 'mobile' one
// that is for streaming.
if (string.Equals((downloadLinkInfo["quality"] as string), "sd", StringComparison.InvariantCultureIgnoreCase))
{
return downloadLinkInfo["link"] as string;
}
}

"TargetInvocationException" in windows phone 8 sdk

An exception of type System.Reflection.TargetInvocationException
occurred in System.ni.dll but was not handled in user code.
var client = new WebClient();
client.DownloadStringCompleted += (sender, e) =>
{
XDocument doc = XDocument.Load(e.Result);
};
client.DownloadStringAsync(new Uri("http://mylocation.com/myfile.php?userid=xyz"));
Maybe you could add a bit of error checking to narrow things down a bit
WebClient client = new WebClient();
client.DownloadStringCompleted += (sender, e) =>
{
if (e.Error != null && !string.IsNullOrEmpty(e.Result))
{
XDocument doc = XDocument.Load(e.Result);
}
};
client.DownloadStringAsync(new Uri("http://mylocation.com/myfile.php?userid=xyz"));
Maybe the data passed between app and Service exceed certain size. For example, when you pass a string larger than 8K chars back to Service, you will get a 404 error. need to change the default configuration to handle large data.
Mostly e.Result or one of its children is null and hence this exception is being thrown.
Go to Tools -> Options -> Debugging and select Enable Just My Code in Visual Studio. That should help you figure out what the problem is.

How to properly extract content from a blog article?

I am trying to extract content from a blog article like this:
static void GetBlogData (string blogPostUrl)
{
string blogPostContent = null;
WebClient client = new WebClient ();
//client.Headers.Add (HttpRequestHeader.Referer, "http://www.stackoverflow.com");
TextWriter writer = new StreamWriter ("/home/nanda/projects/mono/common/article");
try
{
blogPostContent = client.DownloadString (blogPostUrl);
}
catch (Exception ex)
{
Term.PrintLn ("Unable to download\n{0}", ex.Message);
}
if (blogPostContent != null)
{
writer.WriteLine (blogPostContent);
}
else
{
Term.PrintLn ("No content found");
}
}
I am aware that this is too simple of an approach, but I want to know why I am unable to extract content from some URLs like they have a block or something. How can I detect if a website/blog is blocking me from downloading its content?
A website cannot block you from downloading its content without blocking the site's consultation from a browser.
If your download fails, it means either:
a) your url is wrong
b) the website needs some form of identification and your request lacks something (probably a cookie)

Uploading Any File To Google Docs Using GData

I'm trying to use the Google Docs GData API (.NET) to upload a file to my docs, but I keep getting errors thrown. I can't find any example that uses this method, so I'm not even sure that I am usign it correctly.
DocumentsService docService = new DocumentsService("MyDocsTest");
docService.setUserCredentials("w****", "*****");
DocumentsListQuery docQuery = new DocumentsListQuery();
DocumentsFeed docFeed = docService.Query(docQuery);
foreach (DocumentEntry entry in docFeed.Entries)
{
Console.WriteLine(entry.Title.Text);
}
Console.ReadKey();
Console.WriteLine();
if (File.Exists(#"testDoc.txt") == false)
{
File.WriteAllText(#"testDoc.txt", "test");
}
docService.UploadDocument(#"testDoc.txt", null); // Works Fine
docService.UploadFile(#"testDoc.txt", null, #"text/plain", false); // Throws Error
The above code will throw a GDataRequestException:
Execution of request failed: https://docs.google.com/feeds/default/private/full?convert=false
This is kind of aggrivating, seeing as this API could be so insanely helpful. Does anyone know what I am doing wrong?
After a lot of experimentation and research, I got it to work. Gonna leave this here for others in my predicament. I will leave in the using shorthands for reference.
// Start the service and set credentials
Docs.DocumentsService service = new Docs.DocumentsService("GoogleApiTest");
service.setUserCredentials("username", "password");
// Initialize the DocumentEntry
Docs.DocumentEntry newEntry = new Docs.DocumentEntry();
newEntry.Title = new Client.AtomTextConstruct(Client.AtomTextConstructElementType.Title, "Test Upload"); // Set the title
newEntry.Summary = new Client.AtomTextConstruct(Client.AtomTextConstructElementType.Summary ,"A summary goes here."); // Set the summary
newEntry.Authors.Add(new Client.AtomPerson(Client.AtomPersonType.Author, "A Person")); // Add a main author
newEntry.Contributors.Add(new Client.AtomPerson(Client.AtomPersonType.Contributor, "Another Person")); // Add a contributor
newEntry.MediaSource = new Client.MediaFileSource("testDoc.txt", "text/plain"); // The actual file to be uploading
// Create an authenticator
Client.ClientLoginAuthenticator authenticator = new Client.ClientLoginAuthenticator("GoogleApiTest", Client.ServiceNames.Documents, service.Credentials);
// Setup the uploader
Client.ResumableUpload.ResumableUploader uploader = new Client.ResumableUpload.ResumableUploader(512);
uploader.AsyncOperationProgress += (object sender, Client.AsyncOperationProgressEventArgs e) =>
{
Console.WriteLine(e.ProgressPercentage + "%"); // Progress updates
};
uploader.AsyncOperationCompleted += (object sender, Client.AsyncOperationCompletedEventArgs e) =>
{
Console.WriteLine("Upload Complete!"); // Progress Completion Notification
};
Uri uploadUri = new Uri("https://docs.google.com/feeds/upload/create-session/default/private/full?convert=false"); // "?convert=false" makes the doc be just a file
Client.AtomLink link = new Client.AtomLink(uploadUri.AbsoluteUri);
link.Rel = Client.ResumableUpload.ResumableUploader.CreateMediaRelation;
newEntry.Links.Add(link);
uploader.InsertAsync(authenticator, newEntry, new object()); // Finally upload the bloody thing
Can you check the ResponseString property of the GDataRequestException that is being thrown in order to get a detailed error message?
Capturing your requests with a tool like Fiddler will also help you a lot when trying to debug this kind of issues.

How to ignore a certificate error with c# 2.0 WebClient - without the certificate

Using Visual Studio 2005 - C# 2.0, System.Net.WebClient.UploadData(Uri address, byte[] data) Windows Server 2003
So here's a stripped down version of the code:
static string SO_method(String fullRequestString)
{
string theUriStringToUse = #"https://10.10.10.10:443"; // populated with real endpoint IP:port
string proxyAddressAndPort = #"http://10.10.10.10:80/"; // populated with a real proxy IP:port
Byte[] utf8EncodedResponse; // for the return data in utf8
string responseString; // for the return data in utf16
WebClient myWebClient = new WebClient(); // instantiate a web client
WebProxy proxyObject = new WebProxy(proxyAddressAndPort, true);// instantiate & popuylate a web proxy
myWebClient.Proxy = proxyObject; // add the proxy to the client
myWebClient.Headers.Add("Content-Type", "application/x-www-form-urlencoded"); // stick some stuff in the header
UTF8Encoding utf8Encoding = new UTF8Encoding(false);// create a utf8 encoding
Byte[] utf8EncodedRequest = HttpUtility.UrlEncodeToBytes(fullRequestString, utf8Encoding); // convert the request data to a utf8 byte array
try
{
utf8EncodedResponse = myWebClient.UploadData(theUriStringToUse, "POST", utf8EncodedRequest); // pass the utf8-encoded byte array
responseString = utf8Encoding.GetString(utf8EncodedResponse); // get a useable string out of the response
}
catch (Exception e)
{
// some other error handling
responseString = "<CommError><![CDATA[" + e.ToString() + "]]></CommError>";// show the basics of the problem
}
return responseString;// return whatever ya got
}
This is the error I get:
The underlying connection was closed: Could not establish trust relationship for the SSL/TLS secure channel.
I don't have much control to see what's happening when the request goes out. I'm told that it's reaching the correct destination and there's a "certificate error". This is supposedly because there's a literal mismatch between the IP address in my request and the URL it resolves to. I have more than one IP I'm supposed to round-robin to so specifying the URL won't work. I'm not attaching a certificate - nor am I supposed to according to the endpoint owners. Per "them" the certificate error is 'normal and I am supposed to ignore it.
The cert in question is supposedly one of the many verisign certs that is "just there" on our server. The examples I've seen for ignoring cert errors all seem to imply that the requestor is attaching a specific x509 certificate (which I'm not).
I looked over .net WebService, bypass ssl validation! which kinda-sorta describes my problem - except it also kinda-sorta doesn't because I don't know which certificate (if any) I should reference.
Is there a way for me to ignore the error without actually knowing/caring what certificate is causing the problem?
and please - kid gloves, small words, and "for dummies" code as I'm not exactly a heavy hitter.
This traffic is over a private line - so my understanding is that ignoring the cert error is not as big a deal as if it were open internet traffic.
The SSL certificate is for a machine to establish a trust relationship. If you type in one IP address, and end up talking to another, that sounds the same as a DNS hijack security fault, the kind of thing SSL is intending to help you avoid - and perhaps something you don't want to put up with from "them".
If you may end up talking to more than machine (ideally they would make it appear as one for you), you will need a certificate for each of the possible machines to initiate trust.
To ignore trust (I've only ever had to do this temporarily in development scenarios) the following snippet may work for you, but I strongly recommend you consider the impact of ignoring trust before using it:
public static void InitiateSSLTrust()
{
try
{
//Change SSL checks so that all checks pass
ServicePointManager.ServerCertificateValidationCallback =
new RemoteCertificateValidationCallback(
delegate
{ return true; }
);
}
catch (Exception ex)
{
ActivityLog.InsertSyncActivity(ex);
}
}
I realize this is an old post, but I just wanted to show that there is a more short-hand way of doing this (with .NET 3.5+ and later).
Maybe it's just my OCD, but I wanted to minimize this code as much as possible. This seems to be the shortest way to do it, but I've also listed some longer equivalents below:
// 79 Characters (72 without spaces)
ServicePointManager.ServerCertificateValidationCallback = (a, b, c, d) => true;
Shortest way in .NET 2.0 (which is what the question was specifically asking about)
// 84 Characters
ServicePointManager.ServerCertificateValidationCallback = delegate { return true; };
It's unfortunate that the lambda way requires you to define the parameters, otherwise it could be even shorter.
And in case you need a much longer way, here are some additional alternatives:
ServicePointManager.ServerCertificateValidationCallback = (sender, cert, chain, errors) => true;
ServicePointManager.ServerCertificateValidationCallback = delegate(object sender, X509Certificate certificate, X509Chain chain, SslPolicyErrors errors) { return true; };
// 255 characters - lots of code!
ServicePointManager.ServerCertificateValidationCallback =
new RemoteCertificateValidationCallback(
delegate(object sender, X509Certificate certificate, X509Chain chain, SslPolicyErrors errors)
{
return true;
});
This is somewhat the code we're using (not polished yet - I don't think I have the error-handling setup correctly but it should be close) based on thomas's suggestion (this is .NET 4.0 code, though):
var sslFailureCallback = new RemoteCertificateValidationCallback(delegate { return true; });
try
{
if (ignoreSslErrors)
{
ServicePointManager.ServerCertificateValidationCallback += sslFailureCallback;
}
response = webClient.UploadData(Options.Address, "POST", Encoding.ASCII.GetBytes(Options.PostData));
}
catch (Exception err)
{
PageSource = "POST Failed:\r\n\r\n" + err;
return PageSource;
}
finally
{
if (ignoreSslErrors)
{
ServicePointManager.ServerCertificateValidationCallback -= sslFailureCallback;
}
}
This code is much broader than you might expect. It is process-wide. The process might be the exe, IIS on this machine, or even DLLHost.exe. After calling it, have a finally block that restores things to normal by removing the delegate that always returns true.
I wanted to disable SSL verification for a specific domain without globally deactivating it because there might be other requests running which should not be affected, so I came up with this solution (please note that uri is a variable inside a class:
private byte[] UploadValues(string method, NameValueCollection data)
{
var client = new WebClient();
try
{
ServicePointManager.ServerCertificateValidationCallback +=
ServerCertificateValidation;
returnrclient.UploadValues(uri, method, parameters);
}
finally
{
ServicePointManager.ServerCertificateValidationCallback -=
ServerCertificateValidation;
}
}
private bool ServerCertificateValidation(object sender,
X509Certificate certificate, X509Chain chain,
SslPolicyErrors sslPolicyErrors)
{
var request = sender as HttpWebRequest;
if (request != null && request.Address.Host.Equals(
this.uri.Host, StringComparison.OrdinalIgnoreCase))
return true;
return false;
}
Here is the VB.net code to make WebClient ignore the SSL cert.
Net.ServicePointManager.ServerCertificateValidationCallback = New Net.Security.RemoteCertificateValidationCallback(Function() True)

Categories

Resources