Develop Reference

Azure Cosmos DB error in creating database: The remote name could not be resolved

I'm trying to make a functional basic Azure Cosmos Database workflow.
From my Azure account, I'm getting the endpoint address from the HOST param in the "Connection String" in "Settings", and the key from the "PRIMARY PASSWORD" param.
string EndpointUri = "https://<my-database-name>.mongo.cosmos.azure.com";
string PrimaryKey ="<my-key>";
then I try to create the client (following these instructions):
CosmosClient cosmosClient = new(
uri,
key,
new CosmosClientOptions(){
ApplicationRegion = Regions.WestUS
}
);
database = cosmosClient.CreateDatabaseIfNotExistsAsync(id: databaseId).Result;
But I keep getting and error
The remote name could not be resolved
Am I using the Endpoint and Key wrong...?

I guess you are using the Mongo API and using the SQL .Net SDK against it,
Here is the example to connect with Mongo API using the driver.
var client = new MongoClient(Environment.GetEnvironmentVariable("MONGO_CONNECTION"));

You've mixed-n-matched protocols. Every instance of Cosmos DB has an associated protocol (SQL API, MongoDB API, etc). Your example shows you chose the MongoDB API (which has a MongoDB endpoint) with the Cosmos DB API's SDK (and that won't work). Note: When I mentioned instance of Cosmos DB, I mean a complete namespace (which you created), and that namespace encapsulates all of your databases and related containers, permissions, etc.
When using the MongoDB API, you need to use standard MongoDB SDKs (provided by MongoDB or by the community), and then code against it just as you would code against native MongoDB databases.
The only protocol that uses the Cosmos DB SDK is the native SQL API (which is the SDK referenced in the code in your question).
If you really wanted to use different protocols (e.g. some Cosmos DB native containers, some MongoDB containers, some Cassandra keyspaces...), you'd need to create individual instances of Cosmos DB (three instances, in this case).

Am I using the Endpoint and Key wrong...?
Yes. You are using the Endpoint incorrectly.
The endpoint should be https://<account-name>.documents.azure.com.

MongoDB Connecting to Atlas Cloud Database C#

I'm having some trouble getting my C# app connected to my MongoDB on Atlas. The documentation on the website wants me to do this:
using MongoDB.Bson;
using MongoDB.Driver;
// Replace the uri string with your MongoDB deployment's connection string.
var client = new MongoClient(
"mongodb+srv://<username>:<password>#<cluster-address>/test?w=majority"
);
var database = client.GetDatabase("test");
But the drivers I downloaded (version 2.13) dont seem to have the GetDatabase function in the MongoClient namespace. Doing some digging I found a code snippet that seems to indicate that GetDatabase was depreciated in lieu of just using MongoClient. I just wanted to know if that was correct or if I had some bad drivers. Thanks in advance!

How to connect C# and Cassandra with

I am new to .NET C# and Cassandra and I am not able to connect them with each other. I have searched a lot and haven't found a clear explaining on how it works.
I have downloaded Cassandra installed it with Python 2.7, I can run the server and I can run cqlsh. Then I open Visual studio, create a new .NET Core project and install package of Cassandra C# driver.
That's all, I don't know how to create a table, key-spaces from C# to Cassandra.
Can anyone give a simple explanation on how can I create a simple key-space, tables with columns to output the code, so that I can see how it works?

It's five years old, but I wrote-up an article on how to use Cassandra as a backend for a ASP.NET MVC project: http://www.aaronstechcenter.com/aspnet_mvc_cassandra.php
My Git repo for the article is still out there, too: https://github.com/aploetz/ShipCrew
The meat of it will be in the CassandraDAO.cs:
private Cluster Connect() {
string user = getAppSetting("cassandraUser");
string pwd = getAppSetting("cassandraPassword");
string[] nodes = getAppSetting("cassandraNodes").Split(',');
QueryOptions queryOptions = new QueryOptions().SetConsistencyLevel(ConsistencyLevel.One);
Cluster cluster = Cluster.Builder()
.AddContactPoints(nodes)
.WithCredentials(user, pwd)
.WithQueryOptions(queryOptions)
.Build();
return cluster;
}
I'm sure the driver versions are way out-of-date, but it should be enough to get you started.

Submit a Spark job from C# and get results

As per title, I would like to request a calculation to a Spark cluster (local/HDInsight in Azure) and get the results back from a C# application.
I acknowledged the existence of Livy which I understand is a REST API application sitting on top of Spark to query it, and I have not found a standard C# API package. Is this the right tool for the job? Is it just missing a well known C# API?
The Spark cluster needs to access Azure Cosmos DB, therefore I need to be able to submit a job including the connector jar library (or its path on the cluster driver) in order for Spark to read data from Cosmos.

As a .NET Spark connector to query data did not seem to exist I wrote one
https://github.com/UnoSD/SparkSharp
It is just a quick implementation, but it does have also a way of querying Cosmos DB using Spark SQL
It's just a C# client for Livy but it should be more than enough.
using (var client = new HdInsightClient("clusterName", "admin", "password"))
using (var session = await client.CreateSessionAsync(config))
{
var sum = await session.ExecuteStatementAsync<int>("val res = 1 + 1\nprintln(res)");
const string sql = "SELECT id, SUM(json.total) AS total FROM cosmos GROUP BY id";
var cosmos = await session.ExecuteCosmosDbSparkSqlQueryAsync<IEnumerable<Result>>
(
"cosmosName",
"cosmosKey",
"cosmosDatabase",
"cosmosCollection",
"cosmosPreferredRegions",
sql
);
}

If your just looking for a way to query your spark cluster using SparkSql then this is a way to do it from C#:
https://github.com/Azure-Samples/hdinsight-dotnet-odbc-spark-sql/blob/master/Program.cs
The console app requires an ODBC driver installed. You can find that here:
https://www.microsoft.com/en-us/download/details.aspx?id=49883
Also the console app has a bug: add this line to the code after the part where the connection string is generated.
Immediately after this line:
connectionString = GetDefaultConnectionString();
Add this line
connectionString = connectionString + "DSN=Sample Microsoft Spark DSN";
If you change the name of the DSN when you install the spark ODBC Driver you will need to change the name in the above line then.
Since you need to access data from Cosmos DB, you could open a Jupyter Notebook on your cluster and ingest data into spark (create a permanent table of your data there) and then use this console app/your c# app to query that data.
If you have a spark job written in scala/python and need to submit it from a C# app then I guess LIVY is the best way to go. I am unsure if Mobius supports that.

Microsoft just released a dataframe based .NET support for Apache Spark via the .NET Foundation OSS. See http://dot.net/spark and http://github.com/dotnet/spark for more details. It is now available in HDInsight per default if you select the correct HDP/Spark version (currently 3.6 and 2.3, soon others as well).

UPDATE:
Long ago I said a clear no to this question.
However times has changed and Microsoft made an effort.
Pleas check out https://dotnet.microsoft.com/apps/data/spark
https://github.com/dotnet/spark
// Create a Spark session
var spark = SparkSession
.Builder()
.AppName("word_count_sample")
.GetOrCreate();
Writing spark applications in C# now is that easy!
OUTDATED:
No, C# is not the tool you should choose if you would like to work with Spark! However if you really want to do the job with it try as mentioned above Mobius
https://github.com/Microsoft/Mobius
Spark has 4 main languages and API-s for them: Scala, Java, Python, R.
If you are looking for a language in production I would not suggest the R API. The Other 3 work well.
For Cosmo DB connection I would suggest: https://github.com/Azure/azure-cosmosdb-spark

Get number of MongoDB connections with C# driver

How to get the number of (open) MongoDB connections with the C# driver (1.9.0 NuGet package)?
MongoDB documentations describes that db.serverStatus() should give iformation about the count of open and available connections but I can't find any function in the C# driver which represents this information.
(Documentation of db.serverStatus(): http://docs.mongodb.org/manual/reference/command/serverStatus/ )

I searched my fork of the driver for serverStatus, there's no results, so currently, I don't think this command is supported.
There's no associated jira (with serverStatus, at least) that i can find
Adding this functionality would be fairly trivial I would think
Edit
I wrote on the mongodb driver google group, and got this reply from Craig at MongoDB inc.
You can simply run:
MongoDatabase adminDb = ...;
adminDb.RunCommand("serverStatus");
I believe this needs to be run on the admin database.