Develop Reference

How to retrieve user names of commenters on an Instagram post

I have set a session cookie and can make a request to:
https://www.instagram.com/{username}/?__a=1 to get a list of the media ids (shortcodes).
I can then navigate to https://www.instagram.com/p/{shortcode}/ to view the media. However, I can't view the page source as it is populated by some JavaScript execution.
Is there a way to either view the page source after JavaScript execution, or another way to obtain the usernames of people who have commented?
Thank you

You can use the tool Swagger-Codegen to generate APIs for a bunch of websites.
Haven't tried with IG but a quick search on google gave me this:
https://app.swaggerhub.com/apis/saketthakare/instagram/1.0.0
You can use Export -> Client SDK -> your programming language to generate a client API.
EDIT: This might also interest you: https://developers.facebook.com/docs/instagram-basic-display-api

Handling Authentication for a File Display using a Web Service

This is my first time developing this kind of system, so many of these concepts are very new to me. Any and all help would be appreciated. I'll try to sum up what I'm doing as efficiently as possible.
Background: I have a web application running AngularJS with Bootstrap. The app communicates with the server and DB through a web service programmed using C#. On the site, users can upload files and reference them later using direct links. There's no restriction to file type (yet), so just about anything is allowed.
My Goal: Having direct links creates a big security problem for me, since the documents/images are supposed to be private data. What I would prefer to do is validate a user's credentials when the link is clicked, then load the file in the browser using a more generic url path.
--Example--
"mysite.com/attachments/1" ---> (Image)
--instead of--
"mysite.com/data/files/importantImg.jpg"
Where I'm At: Not very far. My first thought was to add a page that sends the server request and receives a file byte stream along with mime type that I can reassemble and present to the user. However, I have no idea if this is possible using a web service that sends JSON requests, nor do I have a clue about how the reassembling process would work client-side.
Like I said, I'll take any and all advice. I'd love to learn more about this subject for future projects as well, but for now I just need to be pointed in the right direction.

Your first thought is correct, for it, you need to use the Response object, and more specifically the AddHeader and Write functions. Of course this will be a different page that will only handle file downloads, so it will be perfectly fine in your JSON web service.

I don't think you want to do this with a web service. Just use a regular IHttpHandler to perform the validation and return the data. So you would have the URL "attachments/1" get rewritten to "attachments/download.ashx?id=1". When you've verified access, write the data to the response stream. You can use the Content Disposition header to set the file name.

accessing websites using C#

I have a problem here. Assume there's a basic calculator implemented in javascript hosted on a website ( I have googled it and to find an example and found this one: http://www.unitsconverter.net/calculator/ ). What I want to do is make a program that opens this website, enters some value and gets the return value. So, in our website calculator, the program:
- open the website
- enters an operand
- enters an operation
- enters an operand
- retrieve the result
Note: things should be done without the need to show anything to the user ( the browser for example ).
I did some search and found about HttpWebRequest and HttpWebRespond. But I think those can be used to post data to the server, which means, The file I'm sending data to must be php, aspx or jsp. But Javascript is client side. So, I think they are kind of useless to me in this case.
Any help?
Update:
I have managed to develop the web bot using WebBrowser Control tool ( found in System.Windows.Forms )
Here's a sample of the code:
webBrowser1.Navigate("LinkOfTheSiteYouWant"); // this will load the page specified in the string. You can add webBrowser1.ScriptErrorsSuppressed = true; to disable the script in a page
webBrowser1.Document.GetElementById("ElementId").SetAttribute("HTMLattrbute", "valueToBeSet");
Those are the main methods I have used to do what I wanted to.
I have found this video useful: http://www.youtube.com/watch?v=5P2KvFN_aLY

I guess you could use something like WatiN to pipe the user's input/output from your app to the website and return the results, but as another commenter pointed out, the value of this sort of thing when you could just write your own calculator fairly escapes me.

You'll need a JavaScript interpreter (engine) to parse all the JavaScript code on the page.
https://www.google.com/search?q=c%23+javascript+engine

What you're looking for is something more akin to a web service. The page you provided doesn't seem like it accepts any data in an HTTP POST and doesn't have any meaningful information in the source that you could scrape. If for example you wanted to programmatically make searches for eBay auctions, you could figure out how to correctly post data to it eg:
http://www.ebay.com/sch/i.html?_nkw=http+for+dummies&_sacat=267&_odkw=http+for+dummies&_osacat=0
and then look through the http response for the information you're looking for. You'd probably need to create a regular expression to match the markup you're looking for like if you wanted to know how many results, you'd search the http response for this bit of markup:
<div class="alt w"><div class="cnt">Your search returned <b>0 items.</b></div></div>
As far as clientside/javascript stuff, you just plain aren't going to be able to do anything like what you're going for.

It is a matter of API: "Does the remote website expose any API for the required functionality?".
Well web resources that expose interactive API are called web service. There are tons of examples (Google Maps for istance).
You can access the API -depending on the Terms & Conditions of the service- through a client. The nature of the client depends on the kind of web service you are accessing.
A SOAP based service is based on SOAP protocol.
A REST based service is based on REST principles.
So, if there is an accessible web service called "Calculator", then you can access the service and, for istance, invoke the sum method.
In your example, the calculator is a Javascript implementation, so it is not a web service and it cannot be accessed via HTTP requests. Though, its implementation is still accessible: it is the javascript file where the calculator is implemented. You can always include the file in your website and access its functions via javascript (always mind terms and conditions!!).
A very common example is the jQuery library stored in Google Libraries.

Redirecting url to index.aspx page using standard asp.net3.5 and web.config

I have a subdomain that is http://trade.businessbazaar.in . I am dynamically creating urls from database something in this manner http://trade.businessbazaar.in/mycompany. To display details, I have an index.aspx file there,thinking that on every request the index.aspx page will load and display data accodingly. Also, There is a masterpage on the index.aspx page from where i am capturing the text mycompany and query it in database to fetch result. But nothing seems to work.
A genuine link is http://trade.businessbazaar.in/Symparlife. But its unable to load index.aspx. I need a clean approach without any third party dll or rewriters. Directly to push some lines in config and start working. That is url will be the same but index page will get loaded...
In short, i want to say
I need the StackOverflow type clean url mechanism to fetch pages
Thanks in Advance

You can handle the Begin_Request event in Global.asax and add custom code to redirect to index.aspx and convert the parts of the URL into query string arguments. You should use Server.Transfer to keep the URL in the browser.
I'd recommend upgrading to 4.0 and using the Routing enine though. You should check if the standard routing is available as a download for ASP.NET 3.5. I am sure your code will get messy very soon. Been there, done that.
As #Mike Miller mentions in the comments the Routing engine ships with ASP.NET 3.5. You can check the documentation here - http://msdn.microsoft.com/en-us/library/system.web.routing(v=vs.90).aspx
Here is a tutorial on how to use it with Web Forms - http://weblogs.asp.net/scottgu/archive/2009/10/13/url-routing-with-asp-net-4-web-forms-vs-2010-and-net-4-0-series.aspx
For your case the code would be something like:
routes.MapPageRoute("company-index", "/{company}", "~/index.aspx")
And in index.aspx you can access the route value for company like this:
string company = (string)Page.RouteData.Values["company"];
Keep in mind that you'd better add something in the URL before your actual argument (the company name). If you don't you will have problems later on when because you may want to add a URL like "/Login" but then you will have to validate that users can't create a company named "Login". Not how Stack Overflow has "/questions/" before the actual question info in the URL.

Getting data from a webpage

I have an idea for an App that would really help me out in work but I'm not sure if it's possible.
I want to run a C# desktop application that will ask for a value. When a value is supplied, the application will open a browswer, go to a webpage and add the value into a form on an online website. The form is then submitted and a new page is loaded that contains a table of results. I then want to extract the table of results from the page source and write code to parse the result values.
It is not important that the user see's this happen in an actual browser. In other words if there's a way to do it by reading HTTP requests then thats great.
The biggest problem I have is getting the values into the form and then retrieving the page source after the form is submitted and the next page loads.
Any help really appreciated.
Thanks

Provided that you're only using this in a legal context:
Usually, web forms are sent via POST request to the web server, specifically some script that handles it. You can look at the HTML code for the form's page and find out the destination for the form (form's action).
You can then use a HttpWebRequest in C# to "pretend you are the form", sending a POST request with all the required parameters (adding them to the HTTP header).
As a result you will get the source code of the destination page as it would be sent to the browser. You can parse this.

This is definitely possible and you don't need to use an actual web browser for this. You can simply use a System.Net.WebClient to send your HTTP request and get an HTTP response.
I suggest to use wireshark (or you can use Firefox + Firebug) it allows you to see HTTP requests and responses. By looking at the HTTP traffic you can see exactly how you should pass your HTTP request and which parameters you should be setting.

You don't need to involve the browser with this. WebClient should do all that you require. You'll need to see what's actually being posted when you submit the form with the browser, and then you should be able to make a POST request using the WebClient and retrieve the resulting page as a string.
The docs for the WebClient constructor have a nice example.

See e.g. this question for some pointers on at least the data retrieval side. You're going to know a lot more about the http protocol before you're done with this...

Why would you do this through web pages if you don't even want the user to do anything?
Web pages are purely for interaction with users, if you simply want data transfer, use WCF.
#Brian using Wireshark will result in a very angry network manager, make sure you are actually allowed to use it.