I am working with HDInsight .NET SDK to use C# sdk with pig. I am getting error when specifying the c# application path.
here's how am defining the C# app in pig script
DEFINE pigudf `PigUDF.exe` SHIP('wasb://book#storage.blob.core.windows.net/PIG/app/PigUDF.exe');
am getting error "invalid ship specification" 'wasb://bookstore#storage160203122480.blob.core.windows.net/PIG/app/PigUDF.exe' doesn't exists, however PigUDF.exe does exists at the given path.
If I run the same query from HDInsight cluster console with both pig script file and c# app stored locally on cluster, it runs successfully.. i.e the below works on hdinsight cluster console
DEFINE pigudf `PigUDF.exe` SHIP('C:/PigUDF.exe');
where pigudf.exe is locally stored on cluster.
I even tried running it through HDInsight tools for visual studio, but I get same error.
Any help here will be appreciated.
thanks,
Saleem
Try using http:// instead of wasb://. The wasb protocol is used to access Windows Azure blob storage.
DEFINE pigudf `PigUDF.exe` SHIP('http://book#storage.blob.core.windows.net/PIG/app/PigUDF.exe');
You can copy your udf to local, update its permissions, ship it, and eventually remove it
fs -copyToLocal wasb://<container>#account/udf.exe
sh cacls udf.exe /E /G <group>:F
define myUdf `udf.exe`ship('udf.exe')
-- run your computation...
sh del udf.exe
Related
I use CyberArk 12.1 to get database password by SDK. Application was written in NetCore3.1, we migrate it to NET6. Sadly CA 12.1 isn't compatible with NET6, support appears in 12.6, but my organisation at this moment doesn't plan upgrade
I try to execute PowerShell script from my application (NET6) to connect with CyberArk agent
Script returns from CA error
APPAP308E - Invalid process path. Full path is required
PowerShell
C:\Program Files (x86)\...\CLIPasswordSDK.exe GetPassword /p AppId=xxx /p Query="Safe=xxx;Folder=Root;Object=xxx" /p Reaseon="test"
I don't understand what means path in this context?
How can I resolve problem, what could be wrong in my script?
Script is executed in application by creating new process
Process.Start("script.bat"); // Error about path from CyberArk
There has to be entered absolute path
Process.Start("c:\\app\\script.bat"); // works
I have installed Java version 1.8.0_73 or later.
please suggest the later steps to configure it
Just follow official guide how to install Elasticsearch on Windows - https://www.elastic.co/guide/en/elasticsearch/reference/current/windows.html
Another way, maybe easier, to install Docker and then run Elasticsearch just with one command, more details here - https://hub.docker.com/_/elasticsearch/
Steps are as follows :
Step1 : Download the latest version of the elastic Search from elastic.co website
Step2 : Install the latest JDK in your machine. must be 1.8 or later.
Step3 : Set the bin path of jre as JAVA_HOME in environment variable.
Step4 : open the command prompt and move to the elasticsearch bin folder then type elasticsearch.
OR
Step4 : goto the elasticsearch bin folder run the service.bat file and elasticsearch.bat file.
Step5 : goto browser and check this url http://localhost:9200/ you must get JSON in return if so, then you have successfully configure the elasticsearch at your machine.
One easy way to install elasticsearch is to use homebrew:
Linux
OS X
Then you can just type:
brew install elasticsearch
brew services start elasticsearch
I am trying to use this azure commandlets form C# application that is console job hosted in my azure Virtual machine. this job runs more then once in a day automatically but my azure commandlets are fails to execute. yesterday that works fine but after a day it is not working please help me. ☺
Note : here i am doing some administration task and i am using organizational account. but it fails to add account other commandlets goes fail to work.
Install the Azure PowerShell tools. The download link is here: https://azure.microsoft.com/en-us/documentation/articles/powershell-install-configure/#how-to-install-azure-powershell
It's renamed to Add-AzureAccount.
I'm assuming that the original question was "The term 'Login-AzureRmAccount' is not recognized as the name of a cmdlet, function, script file. I'm seeing the same thing, despite "Install-Module AzureRM" having been done (from an elevated powershell prompt) and "Get-Module AzureRM" returning a version of 4.3.1 (i.e. powershell tools being installed).
The first part of the solution for me was to also do "Install-Module Azure" (from an elevated powershell prompt) as well after which "Add-AzureAccount" worked. However a command such as "New-AzureRmResourceGroup" still failed with "Run Login-AzureRmAccount to login".
The second part of the solution was simply to reboot, after which:
Add-AzureRmAccount
New-AzureRmResourceGroup -Name wibble -Location UKSouth
worked.
I just registered at AWS services (micro, free) and trying to deploy asp.net mvc 4 application.
Everything is working fine, web application deployed and running (the web part, database, etc), but when i am trying to write to filesystem (for example, when i`m saving a file), i'm gettig an exception
System.UnauthorizedAccessException: Access to the path
'C:\inetpub\wwwroot.logs\xxx.log
The folder "C:\inetpub\wwwroot\.logs" exists.
I tried to grant a permission to folder to DefaultAppPool using commands.
I have following web project structure:
WebHost
.ebextensions
aws.config
aws.cofig has following content:
container_commands:
01-logscreate:
command: "mkdir C:\inetpub\wwwroot\.logs > create-logs.log"
cwd: "C:/inetpub/wwwroot/.ebextensions"
01-logspermission:
command: "icacls \"C:/inetpub/wwwroot/.logs\" /grant DefaultAppPool:(OI) (CI) > p-logs.log"
cwd: "C:/inetpub/wwwroot/.ebextensions"
When i accessing the file, i resolving the full path using
HostingEnvironment.MapPath("~/.logs/xxx.log")
But when i trying to write to this file, a get an exception.
Relative path of '.ebextensions' in deployment package is
Content\D_C.git\udsmonitoring.app\WebHost\obj\Debug\Package\PackageTmp\.ebextensions
Thanx in advance
The app pool that gets deployed runs with limited permissions. It doesn't have access to write to that directory.
Further, you really shouldn't need to be writing logs to the instance file system itself. These instances are transient. If you need a logging solution, consider using CloudWatch or LogEntries. Both probably fit into your existing code base with only a config file change required. In general, if what you're doing ends up with you needing to remote into the EC2 instance to do or see something, consider taking an alternate approach.
Finally, if you really do have a need to write to that directory from within the context of your app, you can add a command in your ebextensions that uses appcmd to set the app pool identity to one with greater privileges...or have a command that sets the directory permissions explicitly.
I'm getting the error below while starting the storage emulator. By mistake I deleted the mdf and log files in users/admin folder. It's holding reference someplace and not allowing me to create the DB again. Please help me.
Added reservation for 127.0.0.1:10000/ in user account RaviBorra-PC\Ravi Borra.
Added reservation for 127.0.0.1:10001/ in user account RaviBorra-PC\Ravi Borra.
Added reservation for 127.0.0.1:10002/ in user account RaviBorra-PC\Ravi Borra.
Found SQL Instance (localdb)\v11.0.
Creating database DevelopmentStorageDb201206 on SQL instance '(localdb)\v11.0'.
Cannot create database 'DevelopmentStorageDb201206' : Database 'DevelopmentStorageDb201206' already exists. Choose a different database name..
One or more initialization actions have failed. Resolve these errors before attempting to run the storage emulator again. Please refer to http://go.microsoft.com/fwlink/?LinkID=248088 for more details.
Thank you,
Ravi Krishna B.
Do these steps:
Open cmd.exe (Command Prompt)
Write this:
SqlLocalDb stop "v11.0"
SqlLocalDb delete "v11.0"
and press Enter
Note that "v11.0" is MyInstance.
For more details, visit this link.
This problem may occur if you delete the data file C:\Users\<Username>\DevelopmentStorageDb201206.mdf but do not delete the corresponding database on the LocalDB server instance. One possible solution is:
Connect to the (localdb)\v11.0 instance from SQL Server Management Studio.
Delete the DevelopmentStorageDb201206 database. There will be the error message that the database deletion ended with an error. Ignore this message and refresh the database list manually.
Now you can start the Azure Storage Emulator.
This is not a problem related with Azure components rather LocalDB instance is reporting back to Azure request that DB still exist. Check LocalDB instances in your machine using "sqllocaldb i" first and the delete the instance using "sqllocaldb d 'instance_name'". I could not test it in my machine but this may unblock you if localdb instance is blocking you.
In Windows Azure SDK 1.7, Windows Azure Storage Emulator uses LocalDB instance specific configuration at following location:
%LocalAppData%\DevelopmentStorage\DevelopmentStorage.201206.config
And based on that you will see that v11.0 instance is Windows Azure Storage Emulator specific so verify if that is running and deleting it will let DSInit to create the DB again in LocalDB.
If you have Sql Express installed on your local box then (localdb)\v11.0 is most likely not your local Sql instance. It is more likely {MyComputerName}\SQLEXPRESS.
Going to:
C:\Users\<yourloginname>\AppData\Local\DevelopmentStorage\DevelopmentStorage.201206.config
And changing:
<SQLInstance>(localdb)\v11.0</SQLInstance>
To:
<SQLInstance>{MyComputerName}\SQLEXPRESS</SQLInstance>
solved this error for me.
I hope this helps..
If you already tried and failed perform:
SQLLocalDB stop v11.0
SQLLocalDB delete v11.0
Delete all the files in C:\Users\<accontname>\WAStorageEmulatorDb* (usually one mdf and one ldf)
Create a new account (I called it Azure, with administrative rights)
Run again the installation
Installation completed!
For me this works... I had tried to format the PC before use this way. I supposte my problem is related to my username that contains "invalid" characters like Name 'NickName' Surname while the "Azure" account seems to be OK for him.
I figure it out 'cause somewhere in the log I have a property destination path set to "C:\Users\Name$_" that is far away for the path of my user account folder.
After that you can go on CMD.exe and prompt:
control userpasswords2
From the control you can delete the Azure login without delete the file folders!
Hope it helps!
I simply deleted all the mdf's and ldf's in C:\Users\accountname\ that started with DevelopmentStorage... or WAStorageEmulator.
Re-ran the install and it worked.
This can also be caused by the folder not existing for localDB. It looks like Visual Studio / Azure will not automatically create containing folders. I just copied the path from the error code into explorer and created the directory up to the .MDF file.
In visual studio go to Tools -> Options -> Database Tools-> Data connections and change a localdb instance name from v11.0 to some other name.
I did have SQL Express installed, as Brian Ogden suggested in his answer, but I wanted to try & point Azure Storage Emulator to my SQL Express instance without modifying configuration files.
I did this by running the initialization process for Storage Emulator & passing my server\instance details for SQL Express using the available switches. Below example of the command I executed (from an elevated command prompt):
AzureStorageEmulator.exe init -server MY-PC-NAME -instance MYSQLINST
This created the database successfully (in my SQL Express instance) & got my Storage Emulator working.
I am using a named instance, but I'm thinking this same approach can be used for default instances by omitting the -instance switch.