I wanted to download the data from https://eauction.ccmc.gov.in/frm_scduled_items.aspx using the date listed in the dropdown.
private async Task Cbetest()
{
using (var client = new HttpClient())
{
client.BaseAddress = new Uri("https://eauction.ccmc.gov.in");
var content = new FormUrlEncodedContent(new[]
{
new KeyValuePair<string, string>("ctl00$ContentPlaceHolder1$gridedit$ctl14$ctl02","17/02/2016")
});
var result = await client.PostAsync("/frm_scduled_items.aspx", content);
string resultContent = await result.Content.ReadAsStringAsync();
Console.WriteLine(resultContent);
}
}
I wanted to download the data shown in the above image
You need to do a little extra work to simulate a post to begin scraping against a ASP.NET WebForms application. Mostly, you're going to need to pass along valid ViewState and EventValidation parameters, which you can retrieve from an initial GET request.
I'm using the HTML Agility Pack to ease with parsing the initial response, I recommend you look into it if you're planning to scrape HTML.
The following seems to get the results you're looking for, though I haven't looked too deeply in the response HTML.
using(var client = new HttpClient())
{
client.BaseAddress = new Uri("https://eauction.ccmc.gov.in");
var initial = await client.GetAsync("/frm_scduled_items.aspx");
var initialContent = await initial.Content.ReadAsStringAsync();
var htmlDoc = new HtmlDocument();
htmlDoc.LoadHtml(initialContent);
var viewState = htmlDoc.DocumentNode.SelectSingleNode("//input[#id='__VIEWSTATE']").GetAttributeValue("value", string.Empty);
var eventValidation = htmlDoc.DocumentNode.SelectSingleNode("//input[#id='__EVENTVALIDATION']").GetAttributeValue("value", string.Empty);
var content = new FormUrlEncodedContent(new Dictionary<string, string>{
{"__VIEWSTATE", viewState},
{"__EVENTVALIDATION", eventValidation},
{"ctl00$ContentPlaceHolder1$drp_auction_date", "17/02/2016"}
});
var res = await client.PostAsync("/frm_scduled_items.aspx", content);
var resContent = await res.Content.ReadAsStringAsync();
Console.WriteLine(resContent);
}
From there you'll want to parse the resulting table to get useful information. If you want to crawl through the DataGrid's pages, you're going to need to get updated EventValidation and ViewState values and simulate additional posts for each page.
Related
I am a beginner to GET and POST. I apologize if the question sounds stupid. But I need your help in guiding me to do it the correct way.
I have to periodically do GET from server A indefinitely until user terminate using Ctrl + C
To check at interval I use the timer below
Timer timer = new Timer(RunTask, null, 3000, 3000);
Where RunTask is the callback method consists of GET method to retrieve data below
using (HttpClient client = new HttpClient())
{
HttpResponseMessage response = await client.GetAsync(urlA);
response.EnsureSuccessStatusCode();
string responseBody = await response.Content.ReadAsStringAsync();
var data = JsonConvert.DeserializeObject<RootObject>(responseBody);
}
Then register the data to server B through POST method
using (HttpClient client = new HttpClient())
{
var json = JsonConvert.SerializeObject(rootIssue);
var output = new StringContent(json, Encoding.UTF8, "application/json");
var result = await client.PostAsync(urlB, output);
string response = result.Content.ReadAsStringAsync().Result;
}
Once registered, server B will return with updated info which I have to POST back to server A
using (HttpClient client = new HttpClient())
{
var json = JsonConvert.SerializeObject(rootIssue);
var output = new StringContent(json, Encoding.UTF8, "application/json");
var result = await client.PostAsync(urlA, output);
string response = result.Content.ReadAsStringAsync().Result;
}
My question here is, what is the best approach for me to do all these queries in my console app? Should I separate each GET and POST to these
GetFromA()
PostToB()
PostToA()
I am concerned if I do it this way, is it possible that PostToB() will POST with incomplete response from GetFromA()?
Or should I combine GET and POST from A to B in one method to something like
GetFromAPostToB()
I hope that someone can point me in the correct direction.
Thanks.
When creating an anonymous sharing link for a notebook section group using the OneNote API, the shared link opens the OneDrive directory listing instead of opening the OneNote web page for the shared section group.
The URL returned from the call to GetOrCreateAnonymousSharingLink looks like this:
https://company-my.sharepoint.com/:o:/g/personal/aaa_company_onmicrosoft_com/Eas72kadUjRBs9_EmWWWVncBaNLxzujnkFlGRobX6IOBHw
The directory listing displayed has a .one link and a .onetoc2 link. Clicking either link will open the OneNote site and display the section and its pages.
Shared links that are manually created are in the same format as above, however, they contain a query parameter e.g. e=1zreF2
Should this parameter be included in the URL returned by GetOrCreateAnonymousSharingLink? Or, have I done something to my Notebooks that might be causing the directory listing to display instead of the actual notebook.
Here's the code:
internal static async Task<string> GenerateSharingLink(string sectionGroupId, SDKHelper.NotebookSecurityRole securityRole) {
string result;
string accessLevel = "{\"accessLevel\":\"Edit\"}";
var accessToken = await SDKHelper.GetAccessToken(SDKHelper.ApiResourceType.OneNoteApi);
string requestUri = $"https://www.onenote.com/api/v1.0/users/aaa#company.onmicrosoft.com/notes/sectiongroups/{sectionGroupId}/Microsoft.OneNote.Api.GetOrCreateAnonymousSharingLink";
var accessLevelJson = new StringContent(JsonConvert.SerializeObject(accessLevel), Encoding.UTF8, "application/json");
using(var client = new HttpClient()) {
client.BaseAddress = new Uri("https://onenote.com");
client.DefaultRequestHeaders.Authorization = new AuthenticationHeaderValue("Bearer", accessToken);
client.DefaultRequestHeaders
.Accept
.Add(new MediaTypeWithQualityHeaderValue("application/json"));
using(var request = new HttpRequestMessage(HttpMethod.Post, requestUri)) {
request.Content = new StringContent(accessLevel, Encoding.UTF8, "application/json");
using(var response = await client.SendAsync(request)) {
if (!response.IsSuccessStatusCode) {
throw new Microsoft.Graph.ServiceException(
new Error {
Code = response.StatusCode.ToString(),
Message = await response.Content.ReadAsStringAsync()
});
}
result = await response.Content.ReadAsStringAsync();
}
}
}
return result;
}
I've been trying to get a grasp on C#/.NET and UWP applications. At the moment I'm trying to follow this guide to complete CRUD operations against my database. So far so good, but when I come to the part where I want to create I run into som troubles. The code below is where I create a new object and try to post it to the database. Piece in this context is a book.
var newPiece = new Piece
{
PieceTitle = Title.Text,
PieceAuthor = Author.Text,
PieceIsbn = Isbn.Text,
PieceDescription = Description.Text
};
using (var client = new HttpClient())
{
var content = JsonConvert.SerializeObject(newPiece);
Task task = Task.Run(async () =>
{
var data = new HttpFormUrlEncodedContent(
new Dictionary<string, string>
{
["value"] = content
});
await client.PostAsync(App.BaseUri, data);
});
task.Wait();
}
The problem here is that because I am using both HttpClient() and HttpFormUrlEncodedContent() VS is telling me that HttpClient() is "an ambiguous reference" and that I need to specify if it belongs to System.Net.Http; or Windows.Web.Http;. If I choose System.Net.Http then it tells me that Argument 1 in PostAsync() "cannot be converted from 'System.Uri' to 'string'", and that Argument 2 "cannot convert from 'Windows.Web.Http.HttpFormUrlEncodedContent' to 'System.Net.Http.HttpContent'"
If I try letting HttpClient() use Windows.Web.Http; then it doesn't give me any errors while compiling, but after the object has been created and posted it displays as null in my tables.
System.Net.Http name space has FormUrlEncodedContent. You can use that.
var newPiece = new Piece
{
PieceTitle = Title.Text,
PieceAuthor = Author.Text,
PieceIsbn = Isbn.Text,
PieceDescription = Description.Text
};
using (var client = new System.Net.Http.HttpClient()) {
var json = JsonConvert.SerializeObject(newPiece);
var content = new System.Net.Http.FormUrlEncodedContent(
new Dictionary<string, string> {
["value"] = json
});
await client.PostAsync(App.BaseUri, content);
}
Should also try to avoid mixing async and blocking calls like .Wait() or .Result as they can lead to deadlocks.
It was also indicated that the data when received is null. This could be an indication that the format being used does not match the expected format.
Try using a different format. For example the following sends JSON content.
using (var client = new System.Net.Http.HttpClient()) {
var json = JsonConvert.SerializeObject(newPiece);
var content = new System.Net.Http.StringContent(json, Encoding.UTF8, "application/json");
await client.PostAsync(App.BaseUri, content);
}
So currently I get a list of images with ID's. When the user then clicks on it, it downloads and displays the image. Everything works, I just need a progress bar to show where it is currently at.
Current code:
var values = new Dictionary<string, string>
{
{ "myPost", "Data" },
};
var client = new System.Net.Http.HttpClient();
var content = new FormUrlEncodedContent(values);
client.DefaultRequestHeaders.Add("test", "header");
var response = await client.PostAsync("myScript.php", content);
var responseString = await response.Content.ReadAsStringAsync();
Since you are using UWP. I would suggest you use the HttpClient provided in Windows.Web.Http. This has post/put and get methods which also provides progress.
See HttpClient for more details specifically HttpProgress in HttpClient.
Hope this helps!
I want to, in code behind, call a Web Api I've built and save some data that I have stored in the following JObject:
var json = new JObject(
new JProperty("firstName", txtFirstName.Text),
new JProperty("lastName", txtLastName.Text),
new JProperty("companyName", txtCompanyName.Text),
new JProperty("email", txtEmail.Text),
new JProperty("phone", txtPhone.Text)
);
Problem is, I'm not sure the best way to go about this. I've found a plethora of examples that are close to the answer, but not quite what I'm looking for. So, how would someone construct an http call to a web api(in C#) and have the aforementioned JObject be posted in the body of the message? I wouldn't necessarily need anything returned, except maybe a generic success or failure message. I appreciate any help given.
Here's an example using System.Net.HttpClient
string jsonText = json.ToString();
using (var client = new HttpClient())
{
var httpContent = new StringContent(jsonString);
httpContent.Headers.ContentType = new System.Net.Http.Headers.MediaTypeHeaderValue("application/json");
HttpResponseMessage message = await client.PostAsync("http://myWebUrl/send", httpContent);
}
I ended up changing the json into a .net object and got my problem solved.
using System.Net.Http;
using System.Net.Http.Headers;
var userData = new User()
{
firstName = txtFirstName.Text,
lastName = txtLastName.Text,
companyName = txtCompanyName.Text,
email = txtEmail.Text,
phone = txtPhone.Text
};
client.BaseAddress = new Uri("http://yourBaseUrlHere");
client.DefaultRequestHeaders.Accept.Clear();
client.DefaultRequestHeaders.Accept.Add(new MediaTypeWithQualityHeaderValue("application/json"));
HttpResponseMessage response = await client.PostAsJsonAsync("requestUrlHere", user);
if (response.IsSuccessStatusCode)
{
//Success code
}