When I am working on a .NET Spark (C#) Notebook in Azure Synapse I always get the following error message: Fail to start interpreter. detail: org.apache.spark.api.dotnet.DotnetBackend. When changing the language from .NET Spark (C#) to Python or Scala the notebook is working as expected.
What is causing this error?
I have tried:
Working in a completely new resource group
Working with a larger Spark Node
Running the notebook Hitchhiker's Guide to .NET for Apache Spark available via the Gallery
Search online for a solution
Short answer: Use spark 3.2 runtime.
I asked the same question on the microsoft Q&A where I got a working solution.
https://learn.microsoft.com/en-us/answers/questions/1179148/azure-synapse-net-c-sparkpool-fail-to-start-interp
Related
I'm trying to start developing RFID-reader apps. I've read that in the past you had to do it in C or C++, but nowadays there's a way to do it in .net (C# for ex.). I downloaded ETK, Documents and tools and latest SDK but I didn't find information about developing embedded apps in C# nor compiling .net solutions for arm processor (cause it's supposed to be executed by the reader).
According to Impinj Support Portal they have GCC based tools for on reader application. They have detailed installation instructions.
The do support C# for the PC side as part of the Octane SDK
I work for a team that currently specializes in creating windows desktop applications in C#/.NET that interwork with local user instances of Microsoft Excel via Office COM Interop. I am currently charged with specifying a new product but I’ve been told that the project will not go ahead unless the app can run on Mac as well as Windows – that is, we have to be able to produce a Mac version of the app that can install natively on OSX and interact with the object model of a user’s instance of Microsoft Excel for Mac.
Running the app with Parallels, Mono or Wine (see COM Interop Through Wine in OSX, which was never answered) is not a solution because the spec for the app requires that the user machine is not modified in any way. We just have to assume the user has a licenced local copy of Excel running, and work with that… which is what we have always done on Windows, where it works great.
I think .NET Core is the answer but I can’t find anywhere where it says definitely that we can (or will be able to) access the Excel Object Library from C# when running .NET Core on Mac OSX with a local copy of Excel for Mac installed. Can anybody point me please to where this has been discussed already; but if not, then tell me how I can go about finding out from Microsoft if this is in their roadmap for .NET Core?
I have received some helpful feedback from a similarly worded question on MSDN. I have also posted an issue on the .NET Core Github, but I think it is already clear what the answer is to this, which is as follows:
The .NET Core team may perhaps be persuadable to add access to the Office.Interop.Excel namespace in the .NET Core Class library, but only for Windows
This cannot be done for .NET Core on OSX because the environment is
not suitable for interchanging COM objects.
This leaves only two options for manipulating objects in Excel for
Mac, both of which already exist: Embedded VBA or Javascript (Office
Add-ins).
Office Addins is a great new solution for web-driven data-oriented
objects in Excel - and its multiplatform nature is awesome - so is enjoying the main focus of
development from Microsoft. However it is not intended to be any
match for the performance of COM for managing complex spreadsheets,
since that's not its focus.
Which means there is no solution for managing objects within Excel
for Mac using C#, in the way that there is on Windows, and there
doesn't appear to be any prospect of one.
The bottom line: The very welcome .NET Core initiative looked like it was a fix, but sadly isn't, because of architecture limitations on OSX.
I have submitted this as an answer because I think I've come to the end of the line of enquiry on it, but if anyone thinks any of the above is incorrect, please shout!
Thanks.
I am basically a .net programmer and I am tasked with analyzing data with SPARK and Cassandra. I was looking for a C# API to work with SPARK and I found out Mobius (as I don’t know JAVA). I started downloading the Mobius project, from GitHub and as per the build for windows I followed the steps mentioned and not able to get it work. I have the following questions regarding it.
1) I have a DataStax enterprise in an Ubuntu machine where my Cassandra and SPARK is available (Standalone). Now I would like to connect from my .NET project to the SPARK and then process the data in Cassandra. Is it possible for me to do that? I want to do that in Debug mode? I will be working using SPARK-SQL only as i am comfortable in SQL.
2) Is it a MUST to install SOLR and SPARK in my windows machine in order for Mobius to work? Will I be able to connect to CASSANDRA (Ubuntu machine) from the Windows SPARK and Mobius?
3) When I run the the command “sparkclr-submit.cmd debug” to get the value for CSharpBackendPortNumber, I am getting an error that “load-spark-env.cmd” is missing. Where can I find this file and how will I get the value for CSharpBackendPortNumber? Is it necessary to have SPARK in my windows machine?
Using Windows client to connect to YARN-based Spark cluster in Linux is a supported/validated scenario for Mobius. I have never tried using a Windows client for Mobius with Standalone Linux-based Spark cluster. I recommend using a Linux machine as Mobius client to verify a basic functionality in Mobius first.
Mobius does not need Solr. You should be able to use Mobius to connect to Cassandra deployed in any OS.
load-spark-env.cmd is a part of the Spark release. You need to set SPARK_HOME environment variable before running sparkclr-submit.cmd
I am trying configuring the latest stable release version of the Mondrian 3.7 using below applications:
JDK 1.8;
Apache Tomcat 7.0;
MySQL 5.6;
MySQL JDBC driver 5.1.21.
But, unable to get lib/mondrian.war from mondrian-3.7.0.0-752.zip.
File missing as mentioned in the document of Mondrian website.
Also there is the task in JIRA for this issue in there website.
Please provide if any other document to deploy this new version.
Thanks in advance.
The mondrian project does not ship the web application anymore. There are plenty of UIs out there to connect to Mondrian and the one we used to ship (jPivot) was terribly outdated.
I'm creating an application that uses amazon s3 for storage in C#. It may be seen to be a stupid question. Where is the documentation on C# aws library?
I'm really like some documentation on the functions calls I've been using.
Thanks.
There is the AWS SDK for .NET site, at the bottom of which you will find the Reference Documentation. Note that the reference documentation also gets installed to your local Visual Studio Help.