I'm trying to re write a search from System.DirectoryServices to System.DirectoryServices.Protocol
In S.DS I get all the requested attributes back, but in S.DS.P, I don't get the GUID, or the HomePhone...
The rest of it works for one user.
Any Ideas?
public static List<AllAdStudentsCV> GetUsersDistinguishedName( string domain, string distinguishedName )
{
try
{
NetworkCredential credentials = new NetworkCredential( ConfigurationManager.AppSettings[ "AD_User" ], ConfigurationManager.AppSettings[ "AD_Pass" ] );
LdapDirectoryIdentifier directoryIdentifier = new LdapDirectoryIdentifier( domain+":389" );
using ( LdapConnection connection = new LdapConnection( directoryIdentifier, credentials ) )
{
SearchRequest searchRequest = new SearchRequest( );
searchRequest.DistinguishedName = distinguishedName;
searchRequest.Filter = "(&(objectCategory=person)(objectClass=user)(sn=Afcan))";//"(&(objectClass=user))";
searchRequest.Scope = SearchScope.Subtree;
searchRequest.Attributes.Add("name");
searchRequest.Attributes.Add("sAMAccountName");
searchRequest.Attributes.Add("uid");
searchRequest.Attributes.Add("telexNumber"); // studId
searchRequest.Attributes.Add("HomePhone"); //ctrId
searchRequest.SizeLimit = Int32.MaxValue;
searchRequest.TimeLimit = new TimeSpan(0, 0, 45, 0);// 45 min - EWB
SearchResponse searchResponse = connection.SendRequest(searchRequest) as SearchResponse;
if (searchResponse == null) return null;
List<AllAdStudentsCV> users = new List<AllAdStudentsCV>();
foreach (SearchResultEntry entry in searchResponse.Entries)
{
AllAdStudentsCV user = new AllAdStudentsCV();
user.Active = "Y";
user.CenterName = "";
user.StudId = GetstringAttributeValue(entry.Attributes, "telexNumber");
user.CtrId = GetstringAttributeValue(entry.Attributes, "HomePhone");
user.Guid = GetstringAttributeValue(entry.Attributes, "uid");
user.Username = GetstringAttributeValue(entry.Attributes, "sAMAccountName");
users.Add(user);
}
return users;
}
}
catch (Exception ex)
{
throw;
}
}
Also, if I want to fetch EVERY user in AD, so I can synch data with my SQL DB, how do I do that, I Kept getting max size exceeded, errors. I set the size to maxInt32... is there an "ignore size" option?
Thanks,
Eric-
I think that the standard way is to use System.DirectoryServices, not System.DirectoryServices.Protocol. Why do you want to user the later ?
Concerning your second question about the error message "max sized exceeded", it may be because you try to fetch too many entries at once.
Active Directory limits the number of objects returned by query, in order to not overload the directory (the limit is something like 1000 objects). The standard way to fetch all the users is using paging searchs.
The algorithm is like this:
You construct the query that will fetch all the users
You specify a specific control (Paged Result Control) in this query indicating that this is
a paged search, with 500 users per page
You launch the query, fetch the first page and parse the first 500 entries in
that page
You ask AD for the next page, parse the next 500 entries
Repeat until there are no pages left
Related
I can't see the name of the tables already created. I'm working on a project in which I have access to the DynamoDB database through an IAM client, I create the AmazonClient using the credentials and configs that were made available to me, but I can't see the tables already created in the database.
I have already created the client and connected it to the database, I am trying to see the number of tables as follows, but the result is always 0
new code
List<string> currentTables = client.ListTablesAsync().Result.TableNames;
MessageBox.Show(currentTables.Count.ToString());
Try awaiting the API call:
List<string> currentTables = await client.ListTablesAsync().Result.TableNames;
MessageBox.Show(currentTables.Count.ToString());
Try this sync code instead:
AmazonDynamoDBClient client = new AmazonDynamoDBClient();
// Initial value for the first page of table names.
string lastEvaluatedTableName = null;
do
{
// Create a request object to specify optional parameters.
var request = new ListTablesRequest
{
Limit = 10, // Page size.
ExclusiveStartTableName = lastEvaluatedTableName
};
var response = client.ListTables(request);
ListTablesResult result = response.ListTablesResult;
foreach (string name in result.TableNames)
Console.WriteLine(name);
lastEvaluatedTableName = result.LastEvaluatedTableName;
} while (lastEvaluatedTableName != null);
I have the JSON documents in the document DB (~30k documents) where each document has a unique ID something like AA123, AA124. There is a tool we use to pull those documents from the document DB where it has a restriction of 500 documents per GET request call. So this has to go through 60 times GET requests to fetch the result which takes sometime. I am looking to get this optimized to run this in quick time(run threads parallely), so that I can get the data quickly. Below is the sample code on how I am pulling the data from the DB as of now.
private int maxItemsPerCall = 500;
public override async Task<IEnumerable<docClass>> Getdocuments()
{
string accessToken = "token";
SearchResponse<docClass> docs = await db.SearchDocuments<docClass>(initialload, accessToken); //Gets top 500
List<docClass> routeRules = new List<docClass>();
routeRules.AddRange(docs.Documents);
var remainingCalls = (docs.TotalDocuments / maxItemsPerCall);
while (remainingCalls > 0 && docs.TotalDocuments > maxItemsPerSearch)
{
docs = await db.SearchDocuments<docClass>(GetFollowUp(docs.Documents.LastOrDefault().Id.Id), requestOptions);
routeRules.AddRange(docs.Documents);
remainingCalls--;
}
return routeRules;
}
private static SearchRequest initialload = new SearchRequest()
{
Filter = new SearchFilterGroup(
new[]
{
new SearchFilter(Field.Type, FilterOperation.Equal, "documentRule")
},
GroupOperator.And),
OrderBy = Field.Id,
Top = maxItemsPerCall,
Descending = false
};
private static SearchRequest GetFollowUp(string lastId)
{
SearchRequest followUpRequest = new SearchRequest()
{
Filter = new SearchFilterGroup(
new[] {
new SearchFilter(Field.Type, FilterOperation.Equal, "documentRule"),
new SearchFilter(Field.Id, FilterOperation.GreaterThan, lastId)
},
GroupOperator.And),
OrderBy = Field.Id,
Top = maxItemsPerCall,
};
return followUpRequest;
}
Help needed: Since I am using the each GET request(500 documents based on IDs depending on the ID of the previous run), how can I use to run this parallely (atleast 5 parallel threads at a time) fetching 500 records per thread (i.e. 2500 parallely in total for 5 threads at a time). I am not familiar with threading, so it would be helpful if someone can suggest how to do this.
You are able to create a csv load job to load data from a csv file in Google Cloud Storage by using the BigQueryClient in Google.Cloud.BigQuery.V2 which has a CreateLoadJob method.
How can you guarantee idempotency with this API to ensure that say the network dropped before getting a response and you kicked off a retry you would not end up with the same data being loaded into BigQuery multiple times?
Example API usage
private void LoadCsv(string sourceUri, string tableId, string timePartitionField)
{
var tableReference = new TableReference()
{
DatasetId = _dataSetId,
ProjectId = _projectId,
TableId = tableId
};
var options = new CreateLoadJobOptions
{
WriteDisposition = WriteDisposition.WriteAppend,
CreateDisposition = CreateDisposition.CreateNever,
SkipLeadingRows = 1,
SourceFormat = FileFormat.Csv,
TimePartitioning = new TimePartitioning
{
Type = _partitionByDayType,
Field = timePartitionField
}
};
BigQueryJob loadJob = _bigQueryClient.CreateLoadJob(sourceUri: sourceUri,
destination: tableReference,
schema: null,
options: options);
loadJob.PollUntilCompletedAsync().Wait();
if (loadJob.Status.Errors == null || !loadJob.Status.Errors.Any())
{
//Log success
return;
}
//Log error
}
You can achieve idempotency by generating your own jobid based on e.g. file location you loaded and target table.
job_id = 'my_load_job_{}'.format(hashlib.md5(sourceUri+_projectId+_datasetId+tableId).hexdigest())
var options = new CreateLoadJobOptions
{
WriteDisposition = WriteDisposition.WriteAppend,
CreateDisposition = CreateDisposition.CreateNever,
SkipLeadingRows = 1,
JobId = job_id, #add this
SourceFormat = FileFormat.Csv,
TimePartitioning = new TimePartitioning
{
Type = _partitionByDayType,
Field = timePartitionField
}
};
In this case if you try reinsert the same job_id you got error.
You can also easily generate this job_id for check in case if pooling failed.
There are two places you could end up losing the response:
When creating the job to start with
When polling for completion
The first one is relatively tricky to recover from without a job ID; you could list all the jobs in the project and try to find one that looks like the one you'd otherwise create.
However, the C# client library generates a job ID so that it can retry, or you can specify your own job ID via CreateLoadJobOptions.
The second failure time is much simpler: keep the returned BigQueryJob so you can retry the polling if that fails. (You could store the job name so that you can recover even if your process dies while waiting for it to complete, for example.)
I have some code (below) that runs every 15 minutes. Sometimes it will fail to query AD with the following error:
System.DirectoryServices.Protocols.DirectoryOperationException: The server does not support the control. The control is critical.
at System.DirectoryServices.Protocols.LdapConnection.SendRequest(DirectoryRequest request, TimeSpan requestTimeout)
When it runs successfully, the entire process takes about one minute to run with the AD query taking about 30 seconds with 32 pages.
When it does fail, it's always on the first page.
It doesn't seem to fail in a pattern (always different times of the day) as far as I can tell.
After googling that error, I found two SO questions (one, two) that point to using AuthType.Ntlm to fix the issue. This hasn't solved it for me though. Another says to check if the server supports paging (it does).
Any ideas as to why this might be happening?
var attributesToReturn = new[] {
"givenName",
"sn",
"middleName",
"extensionAttribute8",
"department",
"sAMAccountName",
"userAccountControl"
};
var filter = "(&(objectclass=user)(!(objectclass=computer))(sn=*)(givenName=*)(extensionAttribute8=*)(|(sn=a*)(sn=b*)(sn=c*)(sn=d*)(sn=e*)(sn=f*)(sn=g*)(sn=h*)(sn=i*)(sn=j*)(sn=k*)(sn=l*)(sn=m*)(sn=n*)(sn=o*)(sn=p*)(sn=q*)(sn=r*)(sn=s*)(sn=t*)(sn=u*)(sn=v*)(sn=w*)(sn=x*)(sn=y*)(sn=z*)))";
var currentBatch = 1;
var searchRequest = new SearchRequest("DC=foo,DC=bar,DC=baz", filter, SearchScope.Subtree, attributesToReturn);
var pageRequestControl = new PageResultRequestControl(500);
searchRequest.Controls.Add(pageRequestControl);
using (var ldapConnection = new LdapConnection("server.foo.bar.baz"))
{
ldapConnection.Credential = new NetworkCredential("user", "pass", "domain");
ldapConnection.Timeout = new TimeSpan(0, 4, 0);
ldapConnection.AuthType = AuthType.Ntlm; // https://stackoverflow.com/a/14255413
while (true)
{
log.Debug("Fetching batch {0} from AD", currentBatch);
var searchResponse = (SearchResponse)ldapConnection.SendRequest(searchRequest);
var pageResultResponse = (PageResultResponseControl)searchResponse.Controls[0];
log.Debug("Parsing AD response for batch {0}", currentBatch);
ParseResponse(_return, searchResponse, includeDisabled);
if (pageResultResponse.Cookie.Length == 0)
break;
pageRequestControl.Cookie = pageResultResponse.Cookie;
currentBatch++;
}
}
This may not be the issue since it only fails for you sometimes, but I had this error every time and had to set
ldapConnection.SessionOptions.ProtocolVersion=3
for it to work at all.
I'm using the Business Objects Web Services SDK to access our Business Objects data. I've successfully got a list of reports, and from that found the LastSuccessfulInstance of a report that has been previously run. However, I can't seem to get the LastRunTime to be populated. When I do a query with no attributes specified it comes back as not set, and I get the same result when I ask for that attribute in particular. I've looked at the report itself and the instance and they both don't have this information. Does anyone know where I can get it from?
Here's my code (hacked from one of SAP's demos):
var sessConnUrl = serviceUrl + "/session";
var boConnection = new BusinessObjects.DSWS.Connection(sessConnUrl);
var boSession = new Session(boConnection);
// Setup the Enterprise Credentials used to login to the Enterprise System
var boEnterpriseCredential = new EnterpriseCredential
{
Domain = cmsname,
Login = username,
Password = password,
AuthType = authType
};
// Login to the Enterprise System and retrieve the SessionInfo
boSession.Login(boEnterpriseCredential);
/************************** DISPLAY INBOX OBJECTS *************************/
// Retrieve the BIPlatform Service so it can be used to add the USER
var biPlatformUrl = boSession.GetAssociatedServicesURL("BIPlatform");
var boBiPlatform = BIPlatform.GetInstance(boSession, biPlatformUrl[0]);
// Specify the query used to retrieve the inbox objects
// NOTE: Adding a "/" at the end of the query indicates that we want to
// retrieve the all the objects located directly under the inbox.
// Without the "/" Path operator, the inbox itself would be returned.
const string query = "path://InfoObjects/Root Folder/Reports/";
// Execute the query and retrieve the reports objects
var boResponseHolder = boBiPlatform.Get(query, null);
var boInfoObjects = boResponseHolder.InfoObjects.InfoObject;
// If the reports contains a list of objects, loop through and display them
if (boInfoObjects != null)
{
// Go through and display the list of documents
foreach (var boInfoObject in boInfoObjects)
{
var report = boInfoObject as Webi;
if (report == null)
continue;
if (!string.IsNullOrEmpty(report.LastSuccessfulInstanceCUID))
{
var instanceQuery = "cuid://<" + report.LastSuccessfulInstanceCUID + ">";
var instanceResponseHolder = boBiPlatform.Get(instanceQuery, null);
var instance = instanceResponseHolder.InfoObjects.InfoObject[0];
}
}
}
Both report.LastRunTimeSpecified and instance.LastRunTimeSpecified are false and both LastRunTime are 01\01\0001, but I can see a last run time in the Web Intelligence UI.
With a little help from Ted Ueda at SAP support I figured it out. Not all the properties are populated by default you need to append #* to the query string to get everything, i.e. change the line:
const string query = "path://InfoObjects/Root Folder/Reports/";
to:
const string query = "path://InfoObjects/Root Folder/Reports/#*";