I want to add an option to save data locally in my application using the mongo databse tools, I want to configure all the server information from within my application.
I have 2 questions.
the following code is working only after manual setup of mongodb localhost database in this way:
but on A computer that didn't configure the database setting, the code will not work.
my code is :
public void createDB()
{
MongoClient client = new MongoClient();
var db = client.GetDatabase("TVDB");
var coll = db.GetCollection<Media>("Movies");
Media video = new Media("", "");
video.Name = "split";
coll.InsertOne(video);
}
this code works only after manual set the database like the picture above.
without it I get in the last line A timeout exception.
how can I configure it from my application to make it work (define Server) ?
Is the user will be must install MongoDB software on his PC, or the API Package is enough in order to use the database?
Many Thanks!
By using that command you're not "configuring the database", you're running it.
If you don't want to manually run it, but want it to be always running, you should install it as a Windows Service as explained in How to run MongoDB as Windows service?.
You need to install and/or run a MongoDB server in order to use it. Using the API alone is not enough, it's not like SQLite.
The Code you are using will search for local mongodb.
Related
Can someone help me to understand why I don't see the graph from JanusGraph.Net client?
I'm running the latest Docker image for JanusGraph from Docker Hub. After connecting to the JanusGraph using the built-in console I created a sample Graph of the Gods and was able to query it using the following commands:
graph = JanusGraphFactory.open('conf/janusgraph-berkeleyje-lucene.properties')
GraphOfTheGodsFactory.load(graph)
g = graph.traversal()
g.V().count()
Because the graph remained across container restarts (was able to query it again without GraphOfTheGodsFactory.load(graph) command) and some files were created inside the /opt/janusgraph/db/berkeley/ folder I assume that everything works.
Then I updated the graphs.graph property of the /opt/janusgraph/conf/gremlin-server/gremlin-server.yaml (path taken from docker-entrypoint.sh) to this value:
graphs: {
graph: conf/janusgraph-berkeleyje-lucene.properties
}
and restarted container.
After that I created a simple .NET console application using JanusGraph.Net from with the following code:
static void Main(string[] args)
{
var client = JanusGraphClientBuilder
.BuildClientForServer(new GremlinServer("localhost", 8182))
.Create();
var g = AnonymousTraversalSource
.Traversal()
.WithRemote(new DriverRemoteConnection(client));
var count = g.V().Count().Next();
}
and the count variable is always zero. It looks like that my .NET application connected to some another (probably in-memory) empty graph on this server.
What else should I change or update? Please help to figure this out.
OK, it was not very obvious, but JanusGraph Docker image is pre-configured to use BerkeleyDB by default (BTW according to the documentation it should be Cassandra). I found it taking a look at the Gremlin Server logs. Gremlin Server is configured to use /etc/opt/janusgraph/janusgraph.properties on startup. This file contains a completely different configuration for BerkeleyDB than, for example, conf/janusgraph-berkeleyje.properties - different folders, etc. That is why my .NET application hasn't seen any data - it was connected (through Gremlin Server) to the different BerkeleyDB database.
I also wasn't able to load this file to the Gremlin console via graph = JanusGraphFactory.open('/etc/opt/janusgraph/janusgraph.properties') - got access issues. After I copied this file to the conf directory (and changed the access rights) - I got another error: Could not instantiate implementation: org.janusgraph.diskstorage.berkeleyje.BerkeleyJEStoreManager probably because the BerkeleyDB already exists.
So the only way I've figured out how to connect to the existing database via the Gremlin Console is to use the :remote connect command. I was able to load a sample "Graph of the Gods" database and later access it from my .NET application using the following commands:
:remote connect tinkerpop.server conf/remote.yaml
:> GraphOfTheGodsFactory.load(graph)
:> g.V().count()
==>12
I'm looking for a solution or at least some advice on how to build a function app in c# allowing to backup a tabular database hosted on an SSAS Azure Instance using the REST API. Idea is to backup it on an azure storage.
I've seen several code for SQL but I don't find any relevant documentation or sample.
Thanks for your help,
Miguel
At this moment I've start with this but I've got errors
string ConnectionString = "Provider = MSOLAP; Data Source = asazure://northeurope.asazure.wxxxxxxxxxxxxxxxx";
//
// The using syntax ensures the correct use of the
// Microsoft.AnalysisServices.Tabular.Server object.
//
using (Server server = new Server())
{
server.Connect(ConnectionString); }
I am using Azure cosmos dB Emulator to do CRUD operations on MongoDB using MongoDB C# Drivers.
I am able to create DB and collection using C# in emulator. This is my sample code to create DB and Collection..
IMongoDatabase db = dbClient.GetDatabase("<My DB name>");
db.CreateCollection("<Collection Name>");
These queries are working fine but when I am trying to insert sample data into this collection its throwing below error
Command insert failed: Unknown server error occurred when processing this request..
My sample code to insert sample data is
IMongoCollection<UserProfile> collection = db.GetCollection<UserProfile("<Collection Name>");
UserProfile c = new UserProfile();
c.ID = 21;
c.UserName = "<Some Name> ";
c.Email = "<Email ID>";
collection.InsertOne(c);
How to use MongoDB C# Drivers to do CRUD operations in Azure cosmos dB Emulator And how to run mongo queries in Emulator instead of SQL queries?
Thanks in Advance
The UI for MongoDB API in Emulator is not yet implemented (it's coming though), but everything else should work. There are two tutorials you need to combine for your use case:
https://learn.microsoft.com/en-us/azure/cosmos-db/local-emulator
(look for MongoDB section there)
https://learn.microsoft.com/en-us/azure/cosmos-db/create-mongodb-dotnet
- build, run and make sure it works new connection string for emulator and then just inject your code, it will work.
As per title, I would like to request a calculation to a Spark cluster (local/HDInsight in Azure) and get the results back from a C# application.
I acknowledged the existence of Livy which I understand is a REST API application sitting on top of Spark to query it, and I have not found a standard C# API package. Is this the right tool for the job? Is it just missing a well known C# API?
The Spark cluster needs to access Azure Cosmos DB, therefore I need to be able to submit a job including the connector jar library (or its path on the cluster driver) in order for Spark to read data from Cosmos.
As a .NET Spark connector to query data did not seem to exist I wrote one
https://github.com/UnoSD/SparkSharp
It is just a quick implementation, but it does have also a way of querying Cosmos DB using Spark SQL
It's just a C# client for Livy but it should be more than enough.
using (var client = new HdInsightClient("clusterName", "admin", "password"))
using (var session = await client.CreateSessionAsync(config))
{
var sum = await session.ExecuteStatementAsync<int>("val res = 1 + 1\nprintln(res)");
const string sql = "SELECT id, SUM(json.total) AS total FROM cosmos GROUP BY id";
var cosmos = await session.ExecuteCosmosDbSparkSqlQueryAsync<IEnumerable<Result>>
(
"cosmosName",
"cosmosKey",
"cosmosDatabase",
"cosmosCollection",
"cosmosPreferredRegions",
sql
);
}
If your just looking for a way to query your spark cluster using SparkSql then this is a way to do it from C#:
https://github.com/Azure-Samples/hdinsight-dotnet-odbc-spark-sql/blob/master/Program.cs
The console app requires an ODBC driver installed. You can find that here:
https://www.microsoft.com/en-us/download/details.aspx?id=49883
Also the console app has a bug: add this line to the code after the part where the connection string is generated.
Immediately after this line:
connectionString = GetDefaultConnectionString();
Add this line
connectionString = connectionString + "DSN=Sample Microsoft Spark DSN";
If you change the name of the DSN when you install the spark ODBC Driver you will need to change the name in the above line then.
Since you need to access data from Cosmos DB, you could open a Jupyter Notebook on your cluster and ingest data into spark (create a permanent table of your data there) and then use this console app/your c# app to query that data.
If you have a spark job written in scala/python and need to submit it from a C# app then I guess LIVY is the best way to go. I am unsure if Mobius supports that.
Microsoft just released a dataframe based .NET support for Apache Spark via the .NET Foundation OSS. See http://dot.net/spark and http://github.com/dotnet/spark for more details. It is now available in HDInsight per default if you select the correct HDP/Spark version (currently 3.6 and 2.3, soon others as well).
UPDATE:
Long ago I said a clear no to this question.
However times has changed and Microsoft made an effort.
Pleas check out https://dotnet.microsoft.com/apps/data/spark
https://github.com/dotnet/spark
// Create a Spark session
var spark = SparkSession
.Builder()
.AppName("word_count_sample")
.GetOrCreate();
Writing spark applications in C# now is that easy!
OUTDATED:
No, C# is not the tool you should choose if you would like to work with Spark! However if you really want to do the job with it try as mentioned above Mobius
https://github.com/Microsoft/Mobius
Spark has 4 main languages and API-s for them: Scala, Java, Python, R.
If you are looking for a language in production I would not suggest the R API. The Other 3 work well.
For Cosmo DB connection I would suggest: https://github.com/Azure/azure-cosmosdb-spark
I currently run MongoDB pointing to the appropriate data directory using the command line below:
mongod --dbpath "somePath/data"
But currently this is a manual step that I run before running a particular suite of tests. Is there a way I can set the path within the code (without calling a script or batch file) using the Mongo C# driver to use a specific data directory?
Update:
To clarify, the reason I'm looking to do this isn't for use in production code, but to isolate test databases for different test suites and to point at a disposable and isolated data directory so that each server instance is clean at the time of running tests and is only populated with the data it requires for the same server settings as production.
You probably won't find any way to do that. The Mongo C# Driver is for programming a MongoClient, not a server. The documentation for C# Driver for MongoDB says - MongoClient class serves as the root object for working with a MongoDB server. When you are programming a client, you automatically would assume that the server is up and running. Whether you do it manually or you write another code for it, that is a different story.
Very rarely would you allow people to connect to a machine and let them start a server AND A CLIENT on it. And why is it rare? You may try to start a server on another machine and screw up with that machine (which may be providing some other completely different service too!). There are some ways (and there are times when it is needed) to start a server remotely, but that is not what you can do using the MongoDB C# Driver.
Now, in order to get your task done, you can try this:
Start one mongod per database on your server, and make each mongod listen to a different port. Then in your code, you can connect your MongoClient to mongod running on the concerned database's port. You can achieve this by using a simple if condition (or a switch case) and checking what database the MongoClient wants to connect to and thus finding the right port value to put in the connection string. Each mongod can serve only one database or more or whatever you want.
So if you are running three mongod's on port1, port2 and port3 and all those three are connected to their respective db paths, the code can be somewhat like this:
var DBNAME = name_of_the_db;
string connectionString;
switch (DBNAME)
{
case name_of_first_DB:
connectionString = "mongodb://[user:pass#]hostname[:port1][/[DBNAME][?options]]";
break;
case name_of_second_DB:
connectionString = "mongodb://[user:pass#]hostname[:port2][/[DBNAME][?options]]";
break;
case name_of_third_DB:
connectionString = "mongodb://[user:pass#]hostname[:port3][/[DBNAME][?options]]";
break;
default:
Console.WriteLine("Invalid DB Name");
}
Answering the updated part of the question:
You can start mongod's on different partitions of the server. Even start the daemons from different drives altogether and make them listen to different ports. Goes without saying that the dbpaths should not be pointing to the same drive for any two databases to at least pretty closely mimic what you wanted.
Just to complete this answer I am adding what #Schaliasos has mentioned in comments.. Consider installing mongo as a window service.