I am basically a .net programmer and I am tasked with analyzing data with SPARK and Cassandra. I was looking for a C# API to work with SPARK and I found out Mobius (as I don’t know JAVA). I started downloading the Mobius project, from GitHub and as per the build for windows I followed the steps mentioned and not able to get it work. I have the following questions regarding it.
1) I have a DataStax enterprise in an Ubuntu machine where my Cassandra and SPARK is available (Standalone). Now I would like to connect from my .NET project to the SPARK and then process the data in Cassandra. Is it possible for me to do that? I want to do that in Debug mode? I will be working using SPARK-SQL only as i am comfortable in SQL.
2) Is it a MUST to install SOLR and SPARK in my windows machine in order for Mobius to work? Will I be able to connect to CASSANDRA (Ubuntu machine) from the Windows SPARK and Mobius?
3) When I run the the command “sparkclr-submit.cmd debug” to get the value for CSharpBackendPortNumber, I am getting an error that “load-spark-env.cmd” is missing. Where can I find this file and how will I get the value for CSharpBackendPortNumber? Is it necessary to have SPARK in my windows machine?
Using Windows client to connect to YARN-based Spark cluster in Linux is a supported/validated scenario for Mobius. I have never tried using a Windows client for Mobius with Standalone Linux-based Spark cluster. I recommend using a Linux machine as Mobius client to verify a basic functionality in Mobius first.
Mobius does not need Solr. You should be able to use Mobius to connect to Cassandra deployed in any OS.
load-spark-env.cmd is a part of the Spark release. You need to set SPARK_HOME environment variable before running sparkclr-submit.cmd
Related
I'm using Visual Studio Community 2022, I've developed a web application with MySQL database and it works fine in Windows hosting and IIS, the problem appeared when I tried to deploy it in GoDaddy Linux host not Windows host. I did all my best with all required steps to deploy it on Linux hosting, when I contacted with godaddy help desk they tolled me that there is no any solution for my problem.
Are there any one published ASP.NET Core MVC on Linux server especially on GoDaddy?
Well, first up, it seems to me a VERY and VERY BEYOND idea to develop on one platform, and worse yet a different OS then what you are deploying to.
Next up, you have to think of a web server JUST like your desktop computer. If you say use some PDF library, then that PDF viewer or library HAS to be installed on your computer for this to work.
Same goes for a web hosting server. If you going to run the .net runtimes on Linux, then you have to setup and install the .net runtimes and system on that Linux server.
You can no more build a program for windows desktop, and THEN decide that program is to now run on a Mac computer, or run on a Android phone.
So, in addition to placing your web site and your .net code that drives the web site on that Linux computer?
You also have to get onto that Linux computer, and install the .net framework, and systems you require.
For example, if you use MySQL as the database? Then that system ALSO has to be installed on that web server.
And if you using SQL server, as the database? Then you have to pick and choose a hosting system that has SQL server installed. Or better yet, pick a hosting plan, and then YOU must develop to the systems and software that is existing on that server.
So, if you decide to develop using PHP and say MySQL? That is VERY unlikely to run on a web site that runs the windows OS.
And if you decide to use PHP with SQL server? Then you have to ensure that the web hosting server (and plan) has support for PHP, and ALSO has SQL server installed and supported.
Unless you can say install SQL server on that server? And unless you can install the .net runtimes and systems on that server? Then you can't use that hosting plan, or else you go and find a hosting plan that:
Lets you install these systems.
Or
You choose a hosting plan that supports the set of tools you used to develop that web site and code with.
You can't just out of the blue install some windows software on a Mac computer.
And you can't out of the blue develop some software for Linux, and then expect that software to run on windows computers.
So, unless that hosting plan has all the librries, and bits and parts you used to devleop your software with? Then your software not even close to be able to work.
So, check your hosting plan. Does it support SQL server, or MySql? Maybe it only has Postgres SQL support.
Mabe the server does not support PHP - so you can't use that either.
Given that a windows hosting plan on go-daddy is what -- about an extra $2 or $3 dollars per month?
Then why not use and choose and adopt a plan that is windows based in place of Linux based.
The simple matter?
You either buy a hosting plan, check tools and software systems it supports. And then YOU MUST then choose those same developer tools for development.
Or you pick a set of development tools to use, and THEN go find a hosting plan that supports the software and systems you used for development.
However, be it desktop or web development?
You can no more install Android software on windows or Mac. And you can no more install Mac software on a windows desktop.
So, you have to ensure the given computer in question (your web server) has all the software systems and bits and parts installed on that server for YOUR software to run.
After you say been developing for a few years in .net, then I would suggest you get a local Linux computer up and running, and then say spend a month or two, and learning how to run a .net web site on that Linux computer. It certainly can be done, but there is a huge number of bits and parts you have to install, setup, learn to use, and quite a bit more to get this all working.
But you MUST drop this silly idea that any old web server will run any old web site you create.
in fact, EVEN WHEN you say develop using Linux tools, unless that target web server has the same database system, and all of the required systems you used for development? Then you can't even deploy software developed for and on a Linux computer to that Linux web server (unless it supports and has all of the systems and frameworks installed that YOU used to develop your site with).
Until you spend say a 6 months or maybe year or 2 at this?
I would fork out the extra $2 for a windows hosting plan on go-daddy, and go with that. But, even then, you have to determine if that web server in question supports say .net core and you used .net core. Or if not, then what .net framework version did you use for development? If the web server does not have that framework, then you can't use that hosting plan. For example, the latest .net framework (non core) is version 4.8, but some web hosting plans only recent adopted and support .net 4.8. As a result, you have to use .net 4.5, or even maybe .net 4.0 for your development platform.
So, you MUST ALWAYS develop your software based on what software and systems EXIST on the target web server.
No such thing as "just publish" to any old web server EVER existed in our industry. How this works for desktop computers, or a web server computer box is not really different.
I should also point out that asp.net REQUIRES the IIS web server and NOT say Apache web server. So, then how can and does a .net core publish work on a Linux box running Apache web server?
Well, during your publish, you INCLUDE a IIS compatible web server. Then you configure Apache to FORWARD all requests to IIS (that low foot print web server is called Kestrel if you wondering).
So, not only will the .net core runtimes be required on that web server, but ALSO a working copy of IIS web server. It turns out that BOTH the .net core runtimes and the included IIS web server CAN be included as part of a .net core publish to a Linux web server. (and to be 100% clear, your server has to be running CentOS compatible version of Linux).
ONCE you do this publish, then you STILL have to configure Apache to FORWARD all requests to the IIS compatible web server.
I have create a Windows application in WPF using Visual Studio 2013, and I'm using MySQL as database, now I want to include MySQL as a distributable package in my setup file.
I am new to development and have very little idea about deploying the software, and i have been searching over the internet but hardly finds any good article which explains the steps to perform the task.
A MySQL database is supposed to be setup on a (remote) database server that the client applications connect to. You don't (or at least you shouldn't) set up a server on each workstation where your client application is installed on.
What you probably want is a self-contained database that can be shipped with your application and run "as-is" without any remote servers or specific setup.
SQLite is a popular example of such a database.
Getting started with SQLite in C#: http://blog.tigrangasparian.com/2012/02/09/getting-started-with-sqlite-in-c-part-one/
Use this:
https://marketplace.visualstudio.com/items?itemName=UnniRavindranathan-MSFT.MicrosoftVisualStudio2013InstallerProjects
And then follow this:
http://geekswithblogs.net/TarunArora/archive/2014/04/24/visual-studio-2013-installer-projects-ndash-hello-world-installer.aspx
There will be settings which you will be setting when creating installer with the wizard, one will be like pre-requisites for the program and will ask you for a MSI location for that program. Dump the location of your LocalDB or whatever MySQL software that you are using (Popular ones are SQLite/SQL Server Express). When the user starts to install the program it will make them install your SQL prerequisite software before moving onto installing your program.
I am trying to learn how to use the ASP.NET framework; I'm already familiar with C# having already used it for desktop utilities a couple of times.
I have a hosted VPS that I want to host a project on, which I then want to work on from my local machine. I have tried using VSCode on my MacBook to work on the files before re-uploading them to the web server with SFTP, and then running the project server-side with dotnet run. This doesn't work, however, because it doesn't open port 5000 externally so I'm unable to view the project.
This probably just comes down to how much of a novice I am when it comes to ASP.NET. How do you or your organisation go about working on a project based on a server? Do I need to install the core on my local machine and run it locally before uploading changes back to the server with SFTP, or is there something I'm missing?
My company recently using tivoli workload scheduler (TWS) to remote trigger jobs from an z/OS to window server. we have tested the tivoli can trigger the mssql services successfully.
The problem now is we have a .NET CL program that we used daily to extract some data in AS400 to mssql server, it was worked perfectly before when using windows scheduler and trigger daily in specific time frame. now we trying to centralize the scheduler so the TWS remotely trigger a prepared batch file (it will trigger the CL program).
but the execution of the CL program show following error while trying to connect to AS400 DB.
The .Net Framework Data Providers require Microsoft Data Access Components(MDAC). Please install Microsoft Data Access Components(MDAC) version 2.6 or later.
EDITED:
in normal scenario, we were assumed the program can trigger successfully, it should be just like using windows scheduler, set a schedule and execute it. the only differences is the scheduler is not windows scheduler for now, but switch to TWS and trigger the CL program remotely. but the execute show the above error during executing the CL program. we have no idea why this error comes up.
we tried to rerun the CL program and schedule it in windows scheduler, works fine. but schedule on TWS remotely, error.
For the testing and observation we have done so far:
Our server is Windows server 2008 SP2 x64, I have made some researches the MDAC used in old version windows while server 2008 should deliver with a newer version of MDAC (WDAC 6.0) and it cannot be reinstall so i assume the MDAC/WDAC must be install correctly.
the CL program was compiled with .NET 2.0/ 3.0 / 3.5, and tested all of them produce the same error.
they error logs were able to produce to sql server DB, so I assume the connection driver using in CL program have no problem. but it might be caused by IBMDA400 driver.
TWS use the admin account in our server to trigger the batch files, a TWS client (listener) is installed in our server for trigger programs in our server, but we dont know how they connect to our server (SSH? telnet?) and they seems donot actually login to our server for remote trigger(trigger our job in silent mode).
we are desperate in seeking any solutions, if anyone could provide any clues or thoughts, it would be very helpful and provide a big help to the people with the same problem in the future.
Thank you very much!.
For those searching, I recently got this error in a web app despite MDAC 2.8 SP1 being already installed on the 2008 box. We re-installed MDAC and it did not seem to fix. Stopping and starting the app pool for the affected web app fixed the problem. It's not 100% clear if the re-install was necessary, or if windows updates or something required an iisreset that didn't happen.
The reason for this error is that the application pool is trying to read a registry key from the HKey_Current_User hive which isn't always loaded.
The solution is as follow:
Open IIS management console
Click on "Application Pools"
Right-click the pool for your web site and select "Advanced Settings"
Change the setting "Load User Profile" to True
I can only give you some avenues to investigate.
You may want to try using the IBM DB2 iSeries ADO.NET Data provider instead of using the IBMDA400 OLE DB provider. My team had a similar experience when we went through a re-platforming project to a newer Windows Server that was x64. For some reason we had very strange results trying to use the older OLE DB providers on our .NET Windows Services. We later found out it was due to our server being 64bit.
I am suspecting you are having an issue with the IBMDA400 is a 32bit driver. Check if you are compiling your .NET CL program as 64bit. You could try to compile your program as a 32 bit application and enabling Wow64 on your server.
Hope one of these leads you to a solution!
I am building an application which needs to connect to a central MySQL database. The application is written in C#. I have installed the .NET connector from MySQL for the VS integration.
I was wondering if there is anything special I need to do to be able to have my application be portable and not require an install of the connector on every machine it is used on?
There is a similar question I found here on SO, but it is for a server instead instead of client which is very different.
Edit:
Connector I used: .NET connector 6.2.3 (
http://www.mysql.com/downloads/connector/net/)
When you make an installer, you can include the dll for the mysql connecter such that it is copied along with the other side. This way, whenever the application is installed properly, you wont have the problem.