I want to connect hadoop in c# using HDInsight. I have created a cluster in AZURE and it was created successfully. and also I enabled remote desktop connection in it.
When I entered the credentials in C# and execute the job then I get the connection error. I am confused in providing the parameters. Kindly assist me.
var hadoop = Hadoop.Connect(new Uri("https://clustername.azurehdinsight.net"), "admin", "");
//I have set remote desktop password
var config = new HadoopJobConfiguration();
config.InputPath = "input/CodeFiles";
config.OutputFolder = "output/CodeFiles";
var result = hadoop.MapReduceJob.ExecuteJob<NamespaceCounterJob>();
Last line gives the exception.
Exception message is:
One or more errors occured
And this is the inner exception:
Unable to connect to the remote server
The Uri should be your Cluster Name, not Username, like:
var hadoop = Hadoop.Connect(new Uri("https://clustername.azurehdinsight.net"), "username", "password");
var config = new HadoopJobConfiguration();
config.InputPath = "input/CodeFiles";
config.OutputFolder = "output/CodeFiles";
var result = hadoop.MapReduceJob.ExecuteJob<NamespaceCounterJob>();
The cluster name is shown at the top of the detail page in the Azure portal for your cluster. Also, you don't need to enable RDP to use this method, it's actually the username set for "Hadoop Services" in the configuration tab of the cluster. Launching a job in this manner makes use of the WebHCat/Templeton endpoint, so RDP is not required.
Related
I have this code
// EXTERNAL MACHINE SCANNING START
ConnectionOptions connectionOptions = new();
connectionOptions.SecurePassword = dataContext.Cred.SecurePassword;
connectionOptions.Username = dataContext.Cred.UserName;
ManagementObjectCollection allUsers = null;
ManagementScope scope = new ManagementScope(#"\\" + dataContext.Comp.Ipaddress + #"\root\cimv2", connectionOptions);
// create connection and run query.
try
{
scope.Connect();
ObjectQuery objectQuery = new ObjectQuery(WQL);
ManagementObjectSearcher searcher = new ManagementObjectSearcher(scope, objectQuery);
allUsers = searcher.Get();
}
But i get this error code.
System.Runtime.InteropServices.COMException: 'The stub received bad data. (0x800706F7)'
Ive also tried to use CimSession with this code
string nameSpc = #"root\cimv2";
string query = "SELECT * FROM Win32_Group WHERE LocalAccount = true AND (SID='S-1-5-32-544' OR SID='S-1-5-32-555')";
CimCredential credentials = new CimCredential(PasswordAuthenticationMechanism.Basic, dataContext.Cred.Domain, dataContext.Cred.UserName, dataContext.Cred.SecurePassword);
WSManSessionOptions sessionOptions = new WSManSessionOptions()
{
DestinationPort = (uint)dataContext.Comp.Port
};
sessionOptions.AddDestinationCredentials(credentials);
sessionOptions.NoEncryption = true;
CimSession session = CimSession.Create(dataContext.Comp.Ipaddress, sessionOptions);
IEnumerable<CimInstance> queryInstance = session.QueryInstances(nameSpc, "WQL", query);
But this code creates this error:
"The WinRM client cannot process the request. If the authentication scheme is different from Kerberos, or if the client computer is not joined to a domain, then HTTPS transport must be used or the destination machine must be added to the TrustedHosts configuration setting. Use winrm.cmd to configure TrustedHosts. Note that computers in the TrustedHosts list might not be authenticated. You can get more information about that by running the following command: winrm help config."
The machine this code is running on is NOT a member of the domain, and never will be. This is a way to collect the useraccounts needed from thousands of computers, for later processing by the company.
This functionality was handled by powershell before, but the company want it rewritten into c#. In powershell it is easy, just a couple of lines of code and thats it.
Ive setup a testplatform in a vm with a domaincontroller and a machine which logs on to the domain controller.
The machine where the code is running has had its group policy for WinRM set to allow basic authentication and unencrypted traffic.
Ive never delved into this area of coding before, so in using those frameworks and the general lingo of that area im quite a newb.
Does anyone have any idea what i might be doing wrong.
I have a connection string for LDAP protocol
ldap://ldap.example.com:636/DC=users,DC=buyers
which works fine.
But I need to use a LDAPS connection :
ldaps://ldap.example.com/DC=users,DC=buyers
which does show up in ldp.exe windows form when I test the connection.
Unfotunately it does not work in the Asp.Net application. I get "Unknown error (0x80005000)".
I am not sure whether LDAPS string is even possible with Asp.Net. I downloaded the source code into LDAPConnection.cs class and was unable to find any valuable information.
The method you found that works is indeed using LDAPS:
ldap://ldap.example.com:636/DC=users,DC=buyers
That's the only way to do it. I do that in one of my existing projects. It doesn't understand "LDAPS://".
If you don't believe me :) fire up Wireshark as you debug. When it connects, you'll see the SSL handshake to your domain controller.
Port 636 is only for LDAPS. Port 389 is the non-SSL port.
If you have more than one domain, you can use port 3269 for the global catalog via SSL.
Below code worked for me to connect to AD using LDAPS
ldapConnection = new LdapConnection(new LdapDirectoryIdentifier("your.LDAPSserver.com", 636));
var networkCredential = new NetworkCredential("UsernameWithoutDomain", "yourPassword", "AD.yourDOMAIN.com");
ldapConnection.SessionOptions.SecureSocketLayer = true;
ldapConnection.SessionOptions.ProtocolVersion = 3;
ldapConnection.SessionOptions.VerifyServerCertificate = new VerifyServerCertificateCallback(ServerCallback);
ldapConnection.AuthType = AuthType.Negotiate;
ldapConnection.Bind(networkCredential);
SearchRequest Srchrequest = new SearchRequest("CN=Users,DC=AD,DC=YOURCOMPANY,DC=COM", "mail=useremail#company.com", System.DirectoryServices.Protocols.SearchScope.Subtree);
SearchResponse SrchResponse = (SearchResponse)ldapConnection.SendRequest(Srchrequest);
// ServerCallback
private static bool ServerCallback(LdapConnection connection, X509Certificate certificate)
{
return true;
}
Surprisingly it is also working when I am not using networkCredential and just using ldapConnection.Bind(); Seems it is using my local credentials as default on my local machine.
I have a problem: I need to connect from a remote server to Active Directory, but the code has to be using the LdapConnection class. I need this because that way I can only test change notifiers when some event happen (such as user is deactivated or he changed group, data etc). OS on the remote server is Windows Server 2012.
I managed to do this from local using DirectoryServices with the following code:
String ldapPath = "LDAP://XRMSERVER02.a24xrmdomain.info";
directoryEntry = new DirectoryEntry(ldapPath, #"A24XRMDOMAIN\username", "pass");
//// Search AD to see if the user already exists.
DirectorySearcher search = new DirectorySearcher(directoryEntry);
search.Filter = "(&(objectClass=user))";
SearchResult result = search.FindOne();
This is okay and connection works but now I need to connect using the LdapConnection class.
I tried something like this on many ways but none of that helped me:
LdapConnection connection = new LdapConnection(XRMSERVER02.a24xrmdomain.info);
var credentials = new NetworkCredential(#"A24XRMDOMAIN\username", "pass");
connection.Credential = credentials;
connection.Bind();
It says that credentials are invalid but that is not true.
Explanations:
XRMSERVER02 - Domain controller
a24xrmdomain.info - Domain
A24XRMDOMAIN - Domain used for logging
Thanks for your help.
Even though I solved my problem I want to share with other developers what I achieved so far. Problem that I encountered was that I had remote server with OS Windows server 2012 and Active directory on it. I needed to connect on him via my local machine(Windows 10).
As I stated in my question it is possible to do that via DirectoryServices with the following code:
String ldapPath = "LDAP://(DomainController).a24xrmdomain.info";
directoryEntry = new DirectoryEntry(ldapPath, #"DOMAIN\username","pass");
//// Test search on AD to see if connection works.
DirectorySearcher search = new DirectorySearcher(directoryEntry);
search.Filter = "(&(objectClass=user))";
SearchResult result = search.FindOne();
This is one of the solutions, but since my task was to get notification and to identify when ever some object has changed in Active Directory, I needed connection to Active Directory on Remote server via LDAP class. Code for getting notifiers is taken from:
- Registering change notification with Active Directory using C#
I succeeded to connect with LDAP class via next code:
String ldapPath2 = "(DomainController).a24xrmdomain.info";
LdapConnection connection = new LdapConnection(ldapPath2);
var credentials = new NetworkCredential(#"username", "pass");
connection.Credential = credentials;
connection.Bind();
Want to mention that no IP address of remote server is needed, just Domain Controller that is used on him, and that Domain used for logging is unnecessary.
Happy coding
Try using NetworkCredential constructor with 3 parameters: username, password and domain. Specify domain separately from user name
Ok, first of all here is the error message from WMI-Activity / Operational in EventLog (Windows 8):
Id = ; ClientMachine = machinename; User = machinename\Dev; ClientProcessId = 1440; Component = WMIService; Operation = connect to namespace : root\CIMV2\Security\MicrosoftVolumeEncryption; ResultCode = 0x80041003; PossibleCause = the user was not granted appropriate permission on the namespace.
the thing is User name "Dev" is not the username that I fed to the ConnectionOptions of ManagementScope() when I attempted connection to remote computer.
The machinename(I hide) was correct.
Trying to make a remote connection is a service and most suspicious scenario is service running as local machine somehow overwrites the username..
I need a confirmation with somebody who might know and able to explain what I am doing wrong here..
I see Username and Password are correct until fed into properties of ConnectionOptions and made Scope.Connect() call.
I have UI where I have the user enter the Url where I can find the Exchange Web Service(EWS). My application uses EWS to get Free/Busy information but my configuration tool just needs to set it for the user.
For now I am just asking for the host name and building up the Url from there. For example they enter example.org as the host and I build https://example.org/EWS/Exchange.asmx from that.
I would like to add a test button to ensure the host they entered is reachable by the machine they are configuring. But I'm not sure how simple or complex I need to be to test the service.
Is there any noop or bind I can do to make sure I can establish communication with EWS?
Something like:
var serviceUri = new Uri(_textBoxEwsUrl.Text));
var exchangeService = new ExchangeService();
exchangeService.Url = serviceUri;
// what can I call here to test that I can talk to the exchangeService?
exchangeService.????
Bind to the inbox folder of the users mail address. Of course, you would need his credentials to this.
Any kind of operation that results in communication with server can be used as test. Simple binding to any folder or item will throw ServiceRequestException or some different type of exception if your url or credentials are incorrect.
Sample code:
try
{
var inbox = Folder.Bind(service, WellKnownFolderName.Inbox);
}
catch(ServiceRequestException e)
{
//handle exception in some way
}