MongoDB C# Driver, httprequest.json to BSON, then insertMany - c#

I want to replace a stock price data AccessDB with a MongoDB using MongoDB c# driver.
The download data using xmlhttprequest as json string is shown below (2 dates only):
[{"date":"2016-01-04T00:00:00.000Z","close":2.77,"high":2.82,"low":2.63,"open":2.77,"volume":32516771},{"date":"2016-01-05T00:00:00.000Z","close":2.75,"high":2.8,"low":2.64,"open":2.77,"volume":12972260},....]
I have studied the c# driver documentation and conducted web searches, but can't find anything that I can get to work. I could parse the each document (i.e, price for a symbol and a date) in the json string and insertOne as in the BsonDocument snippet below.
I would think that json or BSON methods exist that make to appropriate conversions to accomplish a BSON insertMany() DB load.
Please help me get up the curve by finding appropriate info and learn how to do this.
Thank you.
The below works with manual conversion from above JSON data for 1 date
BsonDocument stkDoc = new BsonDocument
{
{"symbol","AMD"},
{"date","2016-01-04T00:00:00.000Z"},
{"close",2.77},
{"high",2.82},
{"low",2.63},
{"open",2.77},
{"volume",32516771}
};
_client = new MongoClient();
_db = _client.GetDatabase("stkprices");
var collection = _db.GetCollection("stkdata1");
collection.InsertOne(stkDoc);

Related

Get xml data from database to mvc application

I have a variable named "value" that are connected to a remote database and has the value of a xml string. Im wondering how I can take the values from that xml string and display them in my mvc index.cshtml. Do you need to do a model class that repressents the values in the xml string? I need help knowing what to search for, any help will do alot
thx in advance and I hope you understand my concern.

You're question is a little vague but I will try my best to answer it. You can use this to load an XElement from and XML file and then convert that to JSON and then convert to an object.
XElement xml = XElement.Load(path);
var json = JsonConvert.SerializeXNode(xml, Newtonsoft.Json.Formatting.Indented, omitRootObject: true);
Console.WriteLine(json);
File.Delete(path);
var wbexport = JsonConvert.DeserializeObject<WbExport<Column>>(json);
If you are trying to load a string value from a field in a database then you should look at an ORM solution like Dapper or Entity Framework
Dapper:
using(var sql = new SqlConnection(_configuration.GetConnectionString("ConnectionString")))
{
await sql.OpenAsync();
var val = await sql.QueryAsync("<query to get your xml value>");
}

AWS Athena query on parquet data to return JSON output

I have follwoing setup.
Application sends serialized JSON data into Firehose.
Firehose is configured with Data conversion to praquet using a glue table definition for efficient query execution.
I am able to run query in Athena and see the results.
Now what I need is to create another application which can query Athena using AWSSDK (C#) and read the data back in JSON format.
Is it possible to somehow use the table's input/output format and serde to read the data back in JSON format using Athena SDK? Or I need to implement custom logic to convert the data back to JSON?

Question is old, but might help someone who trying to export results from Athena table into different output format. AWS CTAS can be used to export data into different formats (ORC, PARQUET, AVRO, JSON, or TEXTFILE) using simple Athena CTAS (https://docs.aws.amazon.com/athena/latest/ug/create-table-as.html) statement. You can also specify compression format for saving output data.
Below example would export data into JSON format on s3.
output_location='s3://s3_bucket/output.json'
CREATE TABLE output
WITH (
format = 'JSON',
external_location = output_location) AS SELECT * FROM target_table
Downside of this approach : Output is always compressed.

Athena bases on Presto version 0.172, so you will find the answer in Presto documentation.For instance, this chapter can be useful to you:
https://prestodb.io/docs/current/functions/json.html

So I get something like this from Athena:
SELECT output FROM "test"."prod" limit 10;
{score=41, avg_time_solving={double=17439.333333333332, long=null}
If I use cast for this field I get:
SELECT CAST(output AS JSON) FROM "test"."prod" limit 10;
[51, [8205.57142857143,null], [null,5159], 7, [0.8571428571428571,null], null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null]

how can search data on multiple field on mongoDB with C#?

I am using MongoDB with C# Driver
I try to fetch data based on search as an like from collection using following code:
col.Find(new BsonDocument(new BsonElement("Name", new BsonRegularExpression(search)))).ToList();
it's work for single field data search but how can do with this multiple field.

Handle time change (summer- and default time) in MongoDB with C# driver

I have a centralized logging service written in C#. This service receives log entries (in JSON string) and add them in a MongoDB collection. I use the official C# MongoDB driver and MongoDB Version 3.0.
Let's say I have a document with this basic structure (and example):
{
"_id" : ObjectId("562c2785a075b738c484ad99"),
"Service" : "WebServer A",
"Description" : "User has logged in",
"SourceTime" : ISODate("2015-10-25T00:41:15.469Z")
}
As you can see in this document I have a date field called "SourceTime". This UTC format doesn't tell me whether the document is created in summer- (+02:00) or in default time (+01:00).
I would like to ask you what's the best way to store a date field with an ISODate object without having problems with summer- and default time?
Is it better to store an ISODate with following format in C#?
{
"SourcTime" : ISODate("2015-10-25T01:41:15.469+01:00")
}
If yes, does anyone has an example how to achieve that with C#?
At the moment, I add a document as follows:
// parse String _document to a BsonDocument
BsonDocument document = BsonDocument.Parse(_document);
// Change SourceTime field to a DateTime object
var TimeElem = document.GetElement("SourceTime").Value.ToString();
DateTime newTime = DateTime.Parse(TimeElem);
// Update document
document.Set("SourceTime", newTime.ToUniversalTime()); // result ISODate("2015-10-25T00:41:15.469Z")
// Add to MongoDB
var collection = _database.GetCollection<BsonDocument>(_collectionName);
collection.InsertOneAsync(document);
Thank you very much for your help. Very appreciate it.
Regards

MongoDB uses a Bson Date which does not store the timezone within the date -
The official BSON specification refers to the BSON Date type as the UTC datetime.
https://docs.mongodb.org/manual/reference/bson-types/#date
The ISODate("2015-10-25T01:41:15.469+01:00") is just a javascript helper method to allow you to convert between an ISODate and a BsonDate so the offset within the string just change converted back to a UTC time.
If you require the timezone or the offset you will need to manually store this within the document, and convert it back after deserialization.

problem with conversion of JSOn string to Mongo document

I am trying to convert a JSOn string into a Mongo document and there is not much help availabel online.
the only helpful thing that I found was:
1:
2:
MongoDB.Bson.BsonDocument doc4=MongoDB.Bson.Serialization
.BsonSerializer.Deserialize(genericjson);
Toggle HighlightingOpen in New WindowSelect All
as described in the post #
Convert string into MongoDB BsonDocument
It creates the document in the databse but it's not the simulation of what it is here in C#. When I click on the nodes they don't show me the data inside. this is what Mongo does when the conversion has corrupted the file.
is there any other way to solve it?

Try the BsonDocument.Parse() method, e.g.
var bsonDoc = BsonDocument.Parse(jsonString);

Categories

Resources