I want to take existing data and put it into RavenDB.
My existing data was in an XML format, so I converted it to JSON.
What should my next step be?
Can I store it in RavenDB as is?
Do I need to create new objects to store it?
Thanks in advance!
It is not mandatory to submit content to RavenDB using the RavenDB Client, nor is it necessary to populate a domain model first. This is unnecessary effort and only complicates the process of data submission/insertion/migration/import.
You can submit JSON formatted documents directly to RavenDB using the HTTP API, specifically you may wish to review the "Single Document Operations" topic for simple examples which (currently) show examples using 'curl'.
Consider the following .NET code:
var url = string.Format("http://ravendb-server:8080/databases/{0}/docs/{1}", databaseName, docId);
var webRequest = System.Net.HttpWebRequest.CreateHttp(url);
webRequest.Method = "PUT";
webRequest.ContentType = "application/json";
webRequest.Headers["Raven-Entity-Name"] = entityName;
var stream = webRequest.GetRequestStream();
using (var writer = new System.IO.StreamWriter(webRequest.GetRequestStream()))
{
writer.Write(json);
}
var webResponse = webRequest.GetResponse();
webResponse.Close();
The above snippet allows you to submit a valid JSON document into a specific database and a specific document collection with the specified ID. Database selection and ID designation is performed through URL paths, and the Document Collection is specified with the metadata header Raven-Entity-Name.
There are additional metadata headers you may want to send up, such as Raven-Clr-Type or Last-Modified but they are not required.
I suppose that your json-data represents the data of your applications domain, and you want to have classes with properties to work with that data in your application, right?
If that is the case, you need to write a simple import-application, that populates your domain model once and then stores all your objects as regular RavenDB documents, just the way you would store any other object with RavenDB.
Does that make sense?
Related
I have trouble in retrieve data from Json Object from Restful.
Currently, I am using RestSharp to get the response from the api.
Based on what I have studied in Use C# to get JSON Data
I do not quite understand on the data retrieving. If I want to retrieve just a single data from the API,
Can I declared on specific data that I want to retrieve from the api in Model class or I need to declare every parameters from the API?
How do I retrieve only specific data from the API?
If the Api consists of an Object in outter part, how to do get the data in the nested object?
Please enlighten me on this matter. Thank you in advanced.
This is the sample code that I created,
var client = new RestClient(<myAPIkey>);
var request = new RestRequest(String.Format("post", Method.GET));
client.ExecuteAsync(request, response =>
{
Console.WriteLine(response.Content);
});
EDITED
I have edited my project as I am testing to get data from hardcoded Json data as such,
[{"id":"518523721","name":"ftyft"},
{"id":"527032438","name":"ftyftyf"},
{"id":"527572047","name":"ftgft"},
{"id":"531141884","name":"ftftft"}]
With Model class of,
public class TestInfo
{
public string id { get; set; }
public string name { get; set; }
}
and the code for deserialize,
TestInfo curTest = new TestInfo();
curTest = JsonConvert.DeserializeObject<TestInfo>(json1);
Console.WriteLine(curTest.id);
I still failed to get the id and name from the json data as it returns empty in Console.WriteLine. Can you please guide me through on how to read the json data?
Can I declared on specific data that I want to retrieve from the api
in Model class or I need to declare every parameters from the API?
It depends on the way which the api was designed. When you say parameters, are you trying to say routes or end-points? Your question is a little bit confused.
How do I retrieve only specific data from the API?
Can you be more specific? Are you talking about the route which you are trying to retrieve your data?
If the Api consists of an Object in other part, how to do get the
data in the nested object?
You can use Newtonsoft.Json(which is the most popular package in nuget) to deserialize your json object. Also, it will only be possible if you create an C# class which will be a mirror from your json model.
You can use an online tool like https://jsonutils.com/ to create this class automatically.
edit:
Since you are trying to deserialize a list of several objetcs, you should try the following approach:
var json1 =
"[{\"id\":\"518523721\",\"name\":\"ftyft\"},\r\n{\"id\":\"527032438\",\"name\":\"ftyftyf\"},\r\n{\"id\":\"527572047\",\"name\":\"ftgft\"}, \r\n{\"id\":\"531141884\",\"name\":\"ftftft\"}]";
var curTest = JsonConvert.DeserializeObject<List<TestInfo>>(json1);
Console.WriteLine(curTest[0].id);
It will works now.
Use the returned structure and put it on something like http://json2csharp.com/ to get a mapping object. And then you can deserialize it ref:
RootObject obj = new JavaScriptSerializer().Deserialize<RootObject>(response.Content);
//obj.YourDesiredProperty;
I was little late but it works for me.
var request = HttpWebRequest.Create(#"http://peerff.azurewebsites.net/api/team");
request.ContentType = "application/json";
request.Method = "GET";
using (HttpWebResponse response = request.GetResponse() as HttpWebResponse)
{
}
I’ve got 2 projects that run on different servers but both have a reference to a common dll that contains a complex type called LeadVM. LeadVM contains sub-objects, like Employer which contains properties. Ex:
LeadVM.FirstName
LeadVM.LastName
LeadVM.Employer.Name
LeadVM.Employer.Phone
So, project 1 can create a LeadVM type of object, and populate it. I need to then, via an HTTP call, POST in the data to a controller/action in the 2nd project. The 2nd project knows what a LeadVM object is.
How can I...serialize(?) LeadVM and pass it to the accepting Action in the 2nd project?
EDIT: Thanks to #Unicorno Marley, I ended up using the Newtonsoft JSON stuff.
Now I just create my object in project 1, then do the following code (I'm sure it's clunky but it works).
LeadVM NewCustomer = new LeadVM();
NewCustomer.set stuff here....
var js = Newtonsoft.Json.JsonSerializer.Create();
var tx = new StringWriter();
js.Serialize(tx, NewCustomer);
string leadJSON = tx.ToString();
And then I can use HttpWebRequest to send a web request to my project 2.
HttpWebRequest request = (HttpWebRequest)WebRequest.Create("http://localhost:1234/Lead/Create");
request.Method = "POST";
StreamWriter streamOut = new StreamWriter(request.GetRequestStream(), System.Text.Encoding.ASCII);
streamOut.Write(System.Web.HttpUtility.UrlEncode(leadJSON));
streamOut.Close();
HttpWebResponse resp = null;
resp = (HttpWebResponse)request.GetResponse();
StreamReader responseReader = new StreamReader(resp.GetResponseStream(), Encoding.UTF8);
sResponse = responseReader.ReadToEnd();
resp.Close();
In project 2, I can catch the json sent like this and my NewCustomer object in project 2 is already populated, ready for use.
var buffer = new byte[Request.InputStream.Length];
Request.InputStream.Read(buffer, 0, buffer.Length);
string json = System.Text.Encoding.Default.GetString(buffer);
json = System.Web.HttpUtility.UrlDecode(json);
LeadVM NewCustomer = Newtonsoft.Json.JsonConvert.DeserializeObject<PhoenixLead.LeadVM>(json);
I'm sure I'm doing things very awkwardly. I'll clean it up, but wanted to post the answer that I was led to.
Json is probably the most common option, there is a good json library for c# that can be found here:
http://james.newtonking.com/json
Alternatively, since youre just doing an HTTP post and you only have one object, the fastest option would just be to write out the data line by line and have the recipient machine parse it. Since they both know what a LeadVM is, and both presumably have the same definition, it would be trivial to read a text string into the proper variables. This quickly becomes the slower option if you decide to add more object types to this process though.
I am trying to get specific information from a website. Right Now I have this html string as you can see my code, the html source code of the website is placed in "responseText". I know I can do this with If's statement but it would be really tedious. I'm a newbie so I have no idea what I'm doing with this. I'm sure there must be another easier way to retrieve information from a website... This is c# for windows store so I can't use webclient. This codes get the string but isn't there is a way I can remove the html code and only leave the variables or something? I just want to do this for a webpage and I know the variables I want because I looked at the html code of the webpage. Isn't it a way to request a list of variables with its information from the website? I'm just kind of lost here. So basically I just want to get specific information from a website in c#, I'm making an app for windows store.
StringBuilder sb = new StringBuilder();
// used on each read operation
byte[] buf = new byte[8192];
// prepare the web page we will be asking for
HttpClient searchClient;
searchClient = new HttpClient();
searchClient.MaxResponseContentBufferSize = 256000;
HttpResponseMessage response = await searchClient.GetAsync(url);
response.EnsureSuccessStatusCode();
responseText = await response.Content.ReadAsStringAsync();
This codes get the string but isn't there is a way I can remove the html code and only leave the variables or something?
What "variables"? You get the HTML - that's the response from the web server. If you want to strip that HTML, that's up to you. You might want to use HTML Tidy to make it more pleasant to work with, but the business of extracting relevant information from HTML is up to you. HTML isn't designed to be machine-readable as a raw information source - it's meant to be mark-up to present to humans.
You should investigate whether the information is available in a more machine-friendly source, with no presentation information etc. For example, there may be some way of getting the data as JSON or XML.
I would like to make a console application that can query a website for data, process it, and then display it. ie- Access http://data.mtgox.com/ and then display the rate for currency X to currency Y.
I am able to get a large string of text via WebClient and StreamReader (though I don't really understand them), and I imagine that I could trim down the string to what I want and then loop the query with a delay to enable updating of the data without running the program again. However I'm only speculating and it seems like there would be a more efficient way of accessing data than this. Am I missing something?
EDIT - The general consensus seems to be to use JSON to do this; which is exactly was I was looking for! Thanks guys!
It looks like that site provides an API serving up JSON data. If you don't know what JSON is you need to look into that, but basically you could create object models representing this JSON. If you have the latest version of VS2012 you can copy the JSON and right click, hit paste special, then paste as class. This will automatically generate models for you. You then contact the API, retrieve the JSON, deserialize it into your models, and do whatever you want from there.
A better way:
string url = "http://data.mtgox.com/";
HttpWebRequest myWebRequest = (HttpWebRequest)HttpWebRequest.Create(url);
myWebRequest.Method = "GET"; // This can also be "POST"
HttpWebResponse myWebResponse = (HttpWebResponse)myWebRequest.GetResponse();
StreamReader myWebSource = new StreamReader(myWebResponse.GetResponseStream());
string myPageSource = string.Empty;
myPageSource= myWebSource.ReadToEnd();
myWebResponse.Close();
// Do something with the data in myPageSource
Once you have the data, look at JSON.NET to parse it
I have a system that basically copies data from other databases to my own SQL Server so I can format the data to suit my needs. That data is used to populate dropdowns to end up with a variable can return the rows I want to show to the user.
All of the databases I've encountered up to now had a logical way the data of the dropdowns was stored. But now I have one that is different and not logical.
What I want to do is use the existing website that provides a somewhat logical dropdown itself to get the data I need. That dropdown fills a multiple selection dropdown after a selection.
I want to select each option in the first dropdown and then read the data it puts in the second multiple selection dropdown. Is there a way to do this?
What you essentially want to do is web scraping. In a similar project, this is what I did:
Use the available objects in .NET that can send http requests and can receive http responses. In my case, I used WebClient:
byte[] response;
using (var client = new WebClient())
{
try
{
response = client.UploadValues(URL, "POST", postData);
}
catch (Exception ex)
{
// Log error here
}
}
See the postData? This is a NameValueCollection that contains the data that is sent by the page when you do a POST. Use Fiddler or a similar tool to find out what data needs to be included. If the page uses .NET web forms, you may have to do something special for the __VIEWSTATE, like do a sample request and get the __VIEWSTATE value from there. Here's a sample for postData:
var postData = new NameValueCollection();
postData.Add("__EVENTTARGET", "");
postData.Add("__EVENTARGUMENT", "");
postData.Add("__VIEWSTATE", "");
Use HtmlAgilityPack to easily "parse" the response that you get when you post. Using Xpath, you will be able to get the new items in the second dropdown.
var doc = new HtmlDocument();
doc.Load(new MemoryStream(requestBytes));
return doc;