I am new to .NET C# and Cassandra and I am not able to connect them with each other. I have searched a lot and haven't found a clear explaining on how it works.
I have downloaded Cassandra installed it with Python 2.7, I can run the server and I can run cqlsh. Then I open Visual studio, create a new .NET Core project and install package of Cassandra C# driver.
That's all, I don't know how to create a table, key-spaces from C# to Cassandra.
Can anyone give a simple explanation on how can I create a simple key-space, tables with columns to output the code, so that I can see how it works?
It's five years old, but I wrote-up an article on how to use Cassandra as a backend for a ASP.NET MVC project: http://www.aaronstechcenter.com/aspnet_mvc_cassandra.php
My Git repo for the article is still out there, too: https://github.com/aploetz/ShipCrew
The meat of it will be in the CassandraDAO.cs:
private Cluster Connect() {
string user = getAppSetting("cassandraUser");
string pwd = getAppSetting("cassandraPassword");
string[] nodes = getAppSetting("cassandraNodes").Split(',');
QueryOptions queryOptions = new QueryOptions().SetConsistencyLevel(ConsistencyLevel.One);
Cluster cluster = Cluster.Builder()
.AddContactPoints(nodes)
.WithCredentials(user, pwd)
.WithQueryOptions(queryOptions)
.Build();
return cluster;
}
I'm sure the driver versions are way out-of-date, but it should be enough to get you started.
Related
Can someone help me to understand why I don't see the graph from JanusGraph.Net client?
I'm running the latest Docker image for JanusGraph from Docker Hub. After connecting to the JanusGraph using the built-in console I created a sample Graph of the Gods and was able to query it using the following commands:
graph = JanusGraphFactory.open('conf/janusgraph-berkeleyje-lucene.properties')
GraphOfTheGodsFactory.load(graph)
g = graph.traversal()
g.V().count()
Because the graph remained across container restarts (was able to query it again without GraphOfTheGodsFactory.load(graph) command) and some files were created inside the /opt/janusgraph/db/berkeley/ folder I assume that everything works.
Then I updated the graphs.graph property of the /opt/janusgraph/conf/gremlin-server/gremlin-server.yaml (path taken from docker-entrypoint.sh) to this value:
graphs: {
graph: conf/janusgraph-berkeleyje-lucene.properties
}
and restarted container.
After that I created a simple .NET console application using JanusGraph.Net from with the following code:
static void Main(string[] args)
{
var client = JanusGraphClientBuilder
.BuildClientForServer(new GremlinServer("localhost", 8182))
.Create();
var g = AnonymousTraversalSource
.Traversal()
.WithRemote(new DriverRemoteConnection(client));
var count = g.V().Count().Next();
}
and the count variable is always zero. It looks like that my .NET application connected to some another (probably in-memory) empty graph on this server.
What else should I change or update? Please help to figure this out.
OK, it was not very obvious, but JanusGraph Docker image is pre-configured to use BerkeleyDB by default (BTW according to the documentation it should be Cassandra). I found it taking a look at the Gremlin Server logs. Gremlin Server is configured to use /etc/opt/janusgraph/janusgraph.properties on startup. This file contains a completely different configuration for BerkeleyDB than, for example, conf/janusgraph-berkeleyje.properties - different folders, etc. That is why my .NET application hasn't seen any data - it was connected (through Gremlin Server) to the different BerkeleyDB database.
I also wasn't able to load this file to the Gremlin console via graph = JanusGraphFactory.open('/etc/opt/janusgraph/janusgraph.properties') - got access issues. After I copied this file to the conf directory (and changed the access rights) - I got another error: Could not instantiate implementation: org.janusgraph.diskstorage.berkeleyje.BerkeleyJEStoreManager probably because the BerkeleyDB already exists.
So the only way I've figured out how to connect to the existing database via the Gremlin Console is to use the :remote connect command. I was able to load a sample "Graph of the Gods" database and later access it from my .NET application using the following commands:
:remote connect tinkerpop.server conf/remote.yaml
:> GraphOfTheGodsFactory.load(graph)
:> g.V().count()
==>12
I want to add an option to save data locally in my application using the mongo databse tools, I want to configure all the server information from within my application.
I have 2 questions.
the following code is working only after manual setup of mongodb localhost database in this way:
but on A computer that didn't configure the database setting, the code will not work.
my code is :
public void createDB()
{
MongoClient client = new MongoClient();
var db = client.GetDatabase("TVDB");
var coll = db.GetCollection<Media>("Movies");
Media video = new Media("", "");
video.Name = "split";
coll.InsertOne(video);
}
this code works only after manual set the database like the picture above.
without it I get in the last line A timeout exception.
how can I configure it from my application to make it work (define Server) ?
Is the user will be must install MongoDB software on his PC, or the API Package is enough in order to use the database?
Many Thanks!
By using that command you're not "configuring the database", you're running it.
If you don't want to manually run it, but want it to be always running, you should install it as a Windows Service as explained in How to run MongoDB as Windows service?.
You need to install and/or run a MongoDB server in order to use it. Using the API alone is not enough, it's not like SQLite.
The Code you are using will search for local mongodb.
I am a newbie when it comes to C# connector and MySQL, so please bear with me.
If C/C++ application uses .mylogin.cnf file to connect to MySQL server, can we use the same file in C#, and how?
I did search if it is possible to use the same file to connect to server when using C#, but could not find the way.
I would really appreciate if somebody could let me know if this is possible and how (short example or a link to a site with an example would be great). If it is not possible then what would be the best way to accomplish functionally similar way of connecting? Custom plugin? We use Community Server.
Thanks.
It is not possible to use .mylogin.cnf from the .NET Connector library. In .NET, the connection options are always specified in a "connection string" passed to MySqlConnection.
If the settings in .mylogin.cnf were created with:
mysql_config_editor set --host=HostName --port=1234 \
--user=UserName --password=Passw0rd
then the equivalent C# code is:
new MySqlConnection("Server=HostName;Port=1234;UserID=UserName;Password=Passw0rd")
or:
var csb = new MySqlConnectionStringBuilder
{
Server = "HostName",
Port = 1234u,
UserID = "UserName",
Password = "Passw0rd",
};
var connection = new MySqlConnection(csb.ConnectionString);
As per title, I would like to request a calculation to a Spark cluster (local/HDInsight in Azure) and get the results back from a C# application.
I acknowledged the existence of Livy which I understand is a REST API application sitting on top of Spark to query it, and I have not found a standard C# API package. Is this the right tool for the job? Is it just missing a well known C# API?
The Spark cluster needs to access Azure Cosmos DB, therefore I need to be able to submit a job including the connector jar library (or its path on the cluster driver) in order for Spark to read data from Cosmos.
As a .NET Spark connector to query data did not seem to exist I wrote one
https://github.com/UnoSD/SparkSharp
It is just a quick implementation, but it does have also a way of querying Cosmos DB using Spark SQL
It's just a C# client for Livy but it should be more than enough.
using (var client = new HdInsightClient("clusterName", "admin", "password"))
using (var session = await client.CreateSessionAsync(config))
{
var sum = await session.ExecuteStatementAsync<int>("val res = 1 + 1\nprintln(res)");
const string sql = "SELECT id, SUM(json.total) AS total FROM cosmos GROUP BY id";
var cosmos = await session.ExecuteCosmosDbSparkSqlQueryAsync<IEnumerable<Result>>
(
"cosmosName",
"cosmosKey",
"cosmosDatabase",
"cosmosCollection",
"cosmosPreferredRegions",
sql
);
}
If your just looking for a way to query your spark cluster using SparkSql then this is a way to do it from C#:
https://github.com/Azure-Samples/hdinsight-dotnet-odbc-spark-sql/blob/master/Program.cs
The console app requires an ODBC driver installed. You can find that here:
https://www.microsoft.com/en-us/download/details.aspx?id=49883
Also the console app has a bug: add this line to the code after the part where the connection string is generated.
Immediately after this line:
connectionString = GetDefaultConnectionString();
Add this line
connectionString = connectionString + "DSN=Sample Microsoft Spark DSN";
If you change the name of the DSN when you install the spark ODBC Driver you will need to change the name in the above line then.
Since you need to access data from Cosmos DB, you could open a Jupyter Notebook on your cluster and ingest data into spark (create a permanent table of your data there) and then use this console app/your c# app to query that data.
If you have a spark job written in scala/python and need to submit it from a C# app then I guess LIVY is the best way to go. I am unsure if Mobius supports that.
Microsoft just released a dataframe based .NET support for Apache Spark via the .NET Foundation OSS. See http://dot.net/spark and http://github.com/dotnet/spark for more details. It is now available in HDInsight per default if you select the correct HDP/Spark version (currently 3.6 and 2.3, soon others as well).
UPDATE:
Long ago I said a clear no to this question.
However times has changed and Microsoft made an effort.
Pleas check out https://dotnet.microsoft.com/apps/data/spark
https://github.com/dotnet/spark
// Create a Spark session
var spark = SparkSession
.Builder()
.AppName("word_count_sample")
.GetOrCreate();
Writing spark applications in C# now is that easy!
OUTDATED:
No, C# is not the tool you should choose if you would like to work with Spark! However if you really want to do the job with it try as mentioned above Mobius
https://github.com/Microsoft/Mobius
Spark has 4 main languages and API-s for them: Scala, Java, Python, R.
If you are looking for a language in production I would not suggest the R API. The Other 3 work well.
For Cosmo DB connection I would suggest: https://github.com/Azure/azure-cosmosdb-spark
Similar questions have been asked before but after a day of going through the answers I'm still very confused.
I'm using Microsoft's Sync Framework with SQL2008 on the server and SQL CE on Windows Mobile devices. I would have thought this was a VERY common requirement. I don't want to replicate large tables onto the mobile device. I only want the records that are needed. For example, each user will need their "jobs" out of the jobs table. They don't need any other user's jobs. So I need something like "where jobId = 3" for one device and "where jobId=4" for another etc.
This looked promising: http://jtabadero.spaces.live.com/blog/cns!BF49A449953D0591!1203.entry
but unfortunately it doesn't work with my code. This code from the sample seems to be trying to get the code that contains the SQL:
var remoteProvider = (LocalDataCache1ServerSyncProvider)syncAgent.RemoteProvider;
var selectIncrementalInsertsCommand = remoteProvider.SalesLT_CustomerSyncAdapter.SelectIncrementalInsertsCommand;
BUT the code containing the SQL (generated by VS) is on the server-side and only a proxy is available in the client-side code. This is how the proxy is added:
// The WCF Service
var webSvcProxy = new MicronetCacheSyncService();
// The Remote Server Provider Proxy
var serverProvider = new ServerSyncProviderProxy(webSvcProxy);
// The Sync Agent
var syncAgent = new MicronetCacheSyncAgent();
syncAgent.RemoteProvider = serverProvider;
So how can I get to the server-side code that contains the sql from the client-side? Sorry I'm not explaining this very well but I guess it's unlikely anyone will have an answer. The short version is does anyone know a SIMPLE way to limit the records that are synced to a mobile device is this type of app? I think the example was meant for desktop apps.
It looks to me like this sync framework is another one of Microsoft's half-baked releases that is really just a beta. It's starting to remind me of some previous horrible experiences with Entity Framework 1.0 :(
The tutorial at http://msdn.microsoft.com/en-us/library/dd918848%28SQL.105%29.aspx contains everything you need to provision filtering for a scope.
FYI, that tutorial is for Sync Framework 2.0, whereas from your code above it appears you're using Sync Framework 1.0 -- a legacy product.