Mobius is the c# binding for apache spark. I wanted to know if MLlib and graphx can also be used via C#. As in can machine learning and graph processing jobs be made for apache spark via C#.
If your spark driver is working over a Linux plateform, i suggest that you better better use LIVY. In fact, LIVY is Apache REST API, that gathers all the functionalities offered by all Spark api (Pyspark, scala, java ..). you can use LIVY to submit job on spark using pyspark snippets. I this seems to be a good solution for you, at least you don't have to ask each time which is working on mobius or not.
Related
As title states I'm trying to use my Keras (tf backend) pretrained model for predicitions in c#. What's the best approach?
I've tried IronPython but it gave me errors, after search I found it isn't supported. Simply calling python script won't work since target Windows devices won't have python interpreters installed.
This question is somewhat vague, but I'll try to answer it the best I can:
Tensorflow core is written in c++ and exposes an interface, which can be consumed by a client written in whichever language (although yes, the Python one is the most developed so far).
What could you do:
You could use a python script to train the keras model and save its computation graph in a file. Then use TensorFlowSharp --- which is the project that contracts with the tensorflow API using c# -- to reload the computation graph and generate predictions with the trained network.
If your devices are phones, tablets and such, Tensorflow for mobile could help you too. It would work similarly to the option above. I'm not sure if they offer support to windows phones, though.
You mentioned:
[...] devices won't have python interpreters installed.
Well, okay. You could, however, define an boot process that would cascade the installation of the necessary dependencies (cpython included). Some applications do that to set databases, for example. In this case, you could create a microservice that boots tensorflow and answers to prediction requests through a file, pipe or socket. Your c# application would then connect to your microservice and request predictions.
Try Keras.NET. That uses C# numpy and calls the installed Keras and backend.
https://github.com/SciSharp/Keras.NET
I have used it succesfully to train and predict with Keras directly from C#. Not ideal but it works.
it does not work like that, since you wont even be able to install tensorflow in a C# project. Abandon C# stack and learn framework in python stack instead, ex. if if you need to consume the prediction result in a web app, learn Flask or Django
I was exploring different storage solutions for my Unity app:
C# code communicates with PHP script on server and this php script communicates with SQL database on server. It was good, but writing Universal PHP script was hard and we ended up with many different php scripts for different queries. It wasn't a flexible solution.
AWS - we decided to try DynamoDB, it was ok - Amazon provided nice UnityPackage with elegant wrapper for integrating our code with their back end; however their solution is expensive and it doesn't provide much more then a storage space.
IBM Bluemix- now we're investigating Bluemix solution. It provides many interesting add-ons like Watson, and we think we would like to stay with it. However it seems like no one have ever bothered to create some integration points with Unity, and it seems like all communications needs to go through REST api.
Does anybody have some experience with IBM cloud solution, and could provide some tips for integrating an existing project / redesigning it to work best with IBM?
Thanks !
I have to consume some C# dll's under a linux host through java. Rewriting the code is currently not an option, and we can't expose the dll as a service because this linux server will exist in isolation.
Is it possible to generate my "java to .net" proxy JARS on a windows host with JNBridge, and then consume them under my linux box (presumably with mono).
This seems feasible in theory, but seeing as I have no experience with JNBridge or even mono I don't know if this is even possible, or what the degree of difficulty would be. Or, is there is a better route to go to solve my problem.
Thanks.
The JNBridge web site says that Windows is required. How about using a Windows VM for the .NET code (it shouldn't matter if your Linux server is in isolation)?
Update: JNBridgePro now supports Mono and can be run under Linux. Please see the JNBridge website for more information.
Shredder -- For some reason I can't do this as a comment on your comment, so I have to provide a reply.
JNBridgePro does not support Mono directly. jdigital's suggestion, to run the .NET inside a Windows VM, is the one we would recommend, although we would recommend that you use JNBridgePro to bridge between the Java code and the .NET code running in the VM. In this case, you would use the JNBridgePro tcp/binary channel to do the bridging.
You could use a Web service if it would work for you, but since you were considering JNBridgePro, I had figured that Web services weren't appropriate for your project. In general, JNBridgePro provides a number of capabilities not offered by Web services, including faster performance, cross-platform access to a finer-grained object-oriented API, and use of cross-platform pass-by-value and remote references.
Wayne Citrin -- JNBridge
We currently using OpenFire as our XMPP server and using SoapBox C# library to implement XMPP client. But now we want to move to secure XMPP over HTTP (BOSH) so users can instant chat on different network. I've search for days but could not find any C# examples to implement XMPP over HTTP. Our application is in C# and so I need to find a C# or .NET solution, ideally keep SoapBox if possible. Anyone has any examples or pointers to share?
Just use an HTTPS URL for polling from any library that supports BOSH. Although why you'd use BOSH from a real programming language other than to cut through a particularly virulent firewall, I don't know.
SoapBox mentions that "SoapBox Server,BOSH implementation works well with JSJaC, the most popular XMPP BOSH JavaScript library" on their official website. Even though, they didn't mention anything about BOSH implementation using SoapBox library.
But, MatriX C# library supports http-bind(bosh) implementation. MatriX is a SDK/library for XMPP, written in managed C# dedicated to .NET and Silverlight technologies. But, MatriX is a commercial one.
I'm also looking the sample source for http-bind(bosh) implementation using SoapBox C# library. If anyone found please, share the link in here.
Has anyone worked with mongodb from within Silverlight? What driver did you use? Silverlight supports only asynchronous sockets; but it looks like available C# drivers are synchronous.
Believe it or not, my first project with MongoDB was in Silverlight. IronPython, some C#. This was in 2009 and we went live with something like 0.8 or 0.9 of MongoDB ... so it has been awhile.
This was I think 2.x Silverlight so it was harder to accomplish things like direct database connections. That said, I would recommend still going the route that I went at the time.
Use a REST+JSON API on your server end and have your client communicate through that. We did ours in Python, but there are these days a few awesome prebuilt Mongo REST Interfaces like Sleepy Mongoose if you don't want to roll your own.
The biggest advantage of this is the question of security: with a direct database connection from client to server even with authentication you run the risk of the user getting into your database... at which point they can do anything they want as MongoDB Authentication restricts access at a DB level, but not what people can see or delete. This doesn't even require disassembling code in some cases as there are, as I recall, development browser plugins that let you script a running Silverlight app from IronPython and IronRuby.
There are other reasons including the synchronous question, but overall I think having a serverside interface to Mongo with a clean, secured facade that talks to Silverlight would serve you better.
It'd probably be best to use the official 10gen MongoDB C# driver on the server with a lightweight web API sitting over it that can be consumed by your Silverlight app with WCF, SOAP, REST, etc.
This not seems to be a good practice to access the mongoDB from silverlight.
I suggest you to wrap all your commands in a clean WCF service (maybe a data service), and completely abstract the mongo plumbing.
this will allow you to control exactly what kind of operation can be done, who can execute the command, and with an evolutive protocol.