I am a beginner in programming and have recently learnt and used http uri to get data and then parse json string from the stream using C# in SSIS and was able to load the data into sql server database.
Below is the sample code I used..
System.Uri uri = new Uri("API URL");
WebRequest webRequest = HttpWebRequest.Create(uri);
HttpWebRequest httpWebRequest = (HttpWebRequest)webRequest;
NetworkCredential networkCredential = new NetworkCredential("LOGIN","Password");
credentialCache.Add(uri,"Basic",networkCredential);
WebResponse webResponse = webRequest.GetResponse();
...
However I am trying to setup the same type of connection for another api which uses POST method.
The query looks something like.. URL + JSON API query which is similar to..
{"JSON" : {
"name": "Dataset",
"ColumnSelect": ["Name","Age","Email"],
"sort":["Name"],
"filterList": [
{
"Attribute": "Age",
"Operator": ">",
"Value": ["25"]
}],"returnObject"
I am not sure how this can be used to query the data and load data to sql like the http request. Can someone please advise me on the right direction to achieve this. Thank you for all your help.
I'm not %100 sure what your scenario is, but as far as I understand you are trying to get data from some server, and store the data in your database.
You have achieved it using HTTP GET, but this time you need to do it with POST.
The basic difference with GET and POST is, GET is used to query the existing data, but in POST, you are delivering something to the server. (of course there are more differences, check this link: https://www.w3schools.com/tags/ref_httpmethods.asp)
This can be done easily:
Prepare your POST request, you need to have a content for your parameters.
Create classes for your incoming response string, from your JSON posted:
public class JSON
{
public string name { get; set; }
public string[] ColumnSelect { get; set; }
public string[] sort { get; set; }
public filterList filterList { get; set; }
}
public class filterList
{
public string Attribute { get; set; }
public string Operator { get; set; }
public string[] Value { get; set; }
}
When you have the response string (stringified JSON), deserialize it via JSON.NET.
Now you have them as .NET objects. You can use Entity Framework to commit them to the database.
I can edit my answer if you need advice on any steps.
Related
I am retrieving data from an odata service. The response contains a link for loading more data i.e. odata.nextLink which I need to retrieve to load more data(odata.nextLink). How do I do this using C# code? The server's response is something like this:
{
"odata.metadata":"http://localhost:60497/odata/$metadata#tables","value":[
{"id":001,"name":"abc" }
.
.
],
"odata.nextLink":"http://localhost:60497/odata/tables?$skip=10"
}
The presence of the odata.nextLink element is used to indicate that there are more results available.
If you simply want to fetch all results then you would keep fetching additional results from each successive nextLink URI until you receive a response that does not contain a nextLink element.
In the most simple case you could simply have a while loop, however in most situations if a query is returning a large set of results you would want to retrieve some and then offer the user the means to request more rows.
In C# you might be using the OData Client library from Microsoft which allows you to perform LINQ queries on OData services.
Microsoft's documentation has a comprehensive example at https://learn.microsoft.com/en-gb/odata/client/pagination
Create a class for the Json Object which is returned, Like this
public class Value
{
public int id { get; set; }
public string name { get; set; }
}
public class Root
{
[JsonProperty("odata.metadata")]
public string OdataMetadata { get; set; }
public List<Value> value { get; set; }
[JsonProperty("odata.nextLink")]
public string OdataNextLink { get; set; }
}
Try to deserialize the Json object returned, here data is going to be the Json Object in String Format.
var RootObj = JsonConvert.DeserializeObject<Root>(data);
From this RootObj, you will be able to get the OdataNextLink value and other values you need, which you can store in your local variable.
Let me know if you have more questions on this.
I've managed to build a console application that successfully requests API data and converts it into a JSON object. However, I don't know how to put these DTO files (recently stored API objects) into my SQL server.
So far I have this for the console application.
class Program
{
static void Main(string[] args)
{
getRequestData();
}
public static void getRequestData()
{
var client = new RestClient(URL);
var request = new RestRequest();
var response = client.Execute(request);
if (response.StatusCode == System.Net.HttpStatusCode.OK)
{
string rawResponse = response.Content;
AllRequests.Rootobject result = JsonConvert.DeserializeObject<AllRequests.Rootobject>(rawResponse);
}
}
}
As you can see, the console application program.cs file successfully obtains a get request then converts the API data into an object (A Json object I think). Here is the DTO file (AllRequests) that the data is being saved into.
class AllRequests
{
public class Rootobject
{
public Operation operation { get; set; }
}
public class Operation
{
public Result result { get; set; }
public Detail[] details { get; set; }
}
public class Result
{
public string message { get; set; }
public string status { get; set; }
}
public class Detail
{
public string requester { get; set; }
public string workorderid { get; set; }
public string accountname { get; set; }
}
}
When testing the application in debug mode, the objects are stored correctly and everything is fine. However, I don't know how to save these objects into an SQL database. I do already have an SQL server, however, I'm not sure how to push the already converted data and save it into a retrospective table. So for instance, saving this data into a Request table in my SQL database.
At the moment I only know how to make API calls in this console application. I can't find any data that will assist me with completing the second part of this task, which is storing the data into an SQL database.
Any help would be greatly appreciated, apologise if there is too much unnecessary information.
There are a few methods to make your application interact with a Database. You can setup your application to use Entity Framework (EF) to manage your DB. It is a powerfull and very popular ORM; there are a lot of resources in the web for help.
Personaly, I like to work with Dapper, that is very simple to use in comparison with EF. You just have to provide your connection-string to create your SqlConnection object.
You can find a simple tutorial in this link.
I avoid EF and other tools frankly. Just more hills to learn.
In .Net 6, I use RestSharp to do the API calls and conversion to JSON and Microsoft.Data.SqlClient to save the data into SQL Server.
Get your API call to respond
var client = new RestClient("your API url");;
Apply the API call
var req = new RestRequest();
req.AddHeader("x-api-key", "your API key");
Get a response object in JSON by default
RestResponse resp = client.Execute(req);
Make sure it worked
if (resp.IsSuccessful) {}
Deserialise your response, where RootObject is my name for my SQL Server table schema as a Class
RootObject ro = new RootObject();
ro = JsonConvert.DeserializeObject<RootObject>(resp.Content);
Build a List<>
//iterate through each device data and add it into a DataRow
List<DeviceClass> device = new List<DeviceClass>();
DataTable dt = GenericList2Datatable<DeviceClass>(device);
//only do anything if the API call worked!
if (!dt.HasErrors)
{
foreach (Device d in ro.Devices)
{
here do your mapping from "ro" object over to data objects like DataRow and DataTable and put them across to SQL server further down. Plenty on the web on how to do that.
The GenericList2Datatable() method I found on the web. Again google it.
public static DataTable GenericList2Datatable<T>(List<T> list)
{}
I might not have worded this question in the correct format as it's difficult to explain what the problem is. From the research I've gathered I think database hierarchy is something that I'm trying to utilize when using Entity Framework to create a database, I'll try to explain my problem in the best way possible.
Firstly, I'm running a Visual Studio 2019 project with a console application. This is my Program.cs file which is currently using a get request from an API URL with RESTSHARP then performing deserialization.
static void Main(string[] args)
{
getAPIData()
}
public static void getAPIData()
{
var client = new RestClient(URL);
var request = new RestRequest();
var response = client.Execute(request);
if (response.StatusCode == System.Net.HttpStatusCode.OK)
{
string rawResponse = response.Content;
Requests.Rootobject result = JsonConvert.DeserializeObject<Requests.Rootobject>(rawResponse);
}
}
As you can see above, the raw data that's gathered from the URL (my API) is converted to a JSON format. And when I debug the code, it stores the code perfectly into the Requests DTO, as I'll show below.
public class Requests
{
public int Id { get; set; }
public class Rootobject
{
public Operation operation { get; set; }
}
public class Operation
{
public Result result { get; set; }
public Detail[] details { get; set; }
}
public class Result
{
public string message { get; set; }
public string status { get; set; }
}
public class Detail
{
public string requester { get; set; }
public string workorderid { get; set; }
public string accountname { get; set; }
}
}
The recently converted API data is now stored in this class in this correct format. Which is what's necessary for the API itself since it comes in numerous classes. This object now holds all the relevant data I need, now I can utilize the entity framework to generate an appropriate table that will hold this into an SQL server.
Here is the Entity Framework class.
public class TransitionContext : DbContext
{
private const string connectionString = #"(connection string here)";
protected override void OnConfiguring(DbContextOptionsBuilder optionsBuilder)
{
optionsBuilder.UseSqlServer(connectionString);
}
public DbSet<Requests> Requesting { get; set; }
}
Currently, when I run the package manager console commands to build a migration and setup a connection to the server, it works fine (Obviously I've not included my real connection string for privacy reasons). However, when I use the "Update-Database" command from the DbSet which is my DTO for the API data. The only key that is displayed is an ID, nothing else.
SQL requests table
All that's generated here is an Id key from the Requests Dto. Now, on the API itself, there is no ID key, I added it in there because Entity Framework said I had to have an ID key and was displaying errors until I made one. But now that's all it's generating when I "Update-Database".
The reason why I showed as much data as I have done, is because the process is simply exporting the converted JSON data (a recently gathered API) into an SQL table. Or tables so that it supports a hierarchy, anything that will map correctly from the DTO to the database. But as of right now, my only issue with the project is that only an ID is being generated.
So I'm not sure where I'm going wrong here or how to get this working as I intend it to work. Apologies for the really long explanation, I tried to condense it as much as I could, any help would be appreciated and any questions I'm free to answer, thank you!
I am trying to get json string from a url by using following line of code. When I enter the url in Google Chrome I get whole string and data. but when I use my code it returns only this line of string {"expand":"projects","projects":[]} it is exact what I get when I enter the url in IE 10. How can I get same data that I get when I enter the url in Chrome? here is my code to get the json data. var jsonStr = new WebClient().DownloadString("https_my_url");
You will need to authenticate the request via WebClient.
See this answer for how to do that if the site uses Forms Authentication.
WebClient accessing page with credentials
You need to use a JSON parser to turn it in to something useful. ServiceStack.Text (available via NuGet or download) has an excellent one that can turn json strings in to first-class POCOs.
using ServiceStack.Text;
public sealed class SomeObject
{
public string expand { get; set; }
public List<string> projects {get; set; }
}
And convert thusly:
SomeObject object = jsonString.FromJson<SomeObject>();
Note, I would make my POCOs a little more c# friendly and alias away the lower-case:
using ServiceStack.Text;
using ServiceStack.DataAnnotations;
public sealed class SomeObject
{
[Alias("expand")]
public string Expand { get; set; }
[Alias("projects")]
public List<string> Projects {get; set; }
}
I am looking for new wcf rest web service in asp.net 4.0 using vs2010.
here is my code for passing url:
"homewizard/Service1.svc/monthlytips/country_state=US_IL_N/tip_code=NoInformation/feature=ACC,FAHD,WHG,FP,WA,DY,DWSH,GD,REF,STV,OVN,MW,CPTR,ATT,ROOF,RG,BSMT,FDN,SPX,GAR,EGF,PLB,DOOR,WIND,WS,LWN,DKG,PF,BBQ,WSD,OWF,DWY,OLIT,HL,SPTC,CF,WF,CPTS,DVB,FURW,FURL,FURU,MAT,BATH,KITC,CLST,LITE,SD,COD,FE,EMS,PC,SS,MED,EAUD,ENR,GARR,INR,MGR,TAXR,TELR,CGD,DOOR,WIND,WS/dwelling_type=1/tip_priority=1/month=3/tip_knowledge_level=1/tipbr_ensav=0/tipbr_safe=0/tipbr_avoid=1/tipbr_comfort=1/tipbr_value=1/tipbr_appear=1/tipbr_green=0/tipbr_money=0/tipbr_space=1/tipbr_allergy=2/tipbr_elderly=2/tipbr_children=2/tip_location_temp=0/tip_location_humidity=0"
output:Bad Request - Invalid URL HTTP Error 400. The request URL is invalid.
my web config is: httpRuntime maxUrlLength="1024"
but it's working my local host not in server pc.
thanks in advance.
To be honest your issue is that that's not a restfull URL, the url should just be the location of the resource you are dealing with, the body of the request should contain the details of the request (as xml or json for example). It will both solve your problem and result in less code to manage the parameters via deserialization into a request object server side, not to mention clear up that url.
UPDATE
Are you sure thats a GET and not a PUT, what is the call meant to do?
If it is a GET I'd do something like the following...
Service Interface
[WebGet(UriTemplate = "monthlytips")]
AppropriateReturnType MonthlyTips(MonthlyTip monthlytip);
DTO Object
public class MonthlyTip
{
public string CountryState { get; set; }
public string TipCode { get; set; }
public List<string> Feature { get; set; }
public int DwellingType { get; set; }
public int TipPriority { get; set; }
...
...
}
Thats off the top of my head so it might need a little refining and you'll need to implement the interface, finish the DTO and so on but that's the approach you should take.