Develop Reference

Get LDAP TokenGroups using System.DirectoryServices.Protocols in .NET5

I'm porting some code from System.DirectoryServices to System.DirectoryServices.Protocols due to the need to run on linux.
The majority of the code is fine but this little piece that retrieves tokenGroups isn't coming across.
The original code reads:
using (var ldapObject = new DirectoryEntry(objectPath, username, password, AuthenticationTypes.Secure))
{
ldapObject.Options.Referral = ReferralChasingOption.All;
ldapObject.RefreshCache(new string[] { "tokenGroups" });
foreach (byte[] sid in ldapObject.Properties["tokenGroups"])
{
// Won't run on linux
var groupSID = new System.Security.Principal.SecurityIdentifier(sid, 0).ToString();
try
{
var group = this.GetSecurityGroupBySID(groupSID);
}
catch (Exception ex)
{
log.Warn($"Failed to get group with SID {groupSID}", ex);
}
}
}
Where objectPath is of the form $"LDAP://{domainName}/{distinguishedName}"
My attempt looks like this:
public void GetExpandedGroups(string objectMail, LdapConnection ldapConnection)
{
var usersPath = "OU=active,OU=users,DC=path,DC=etc"; //Not real values
var filter = $"(&(objectClass = user)(!userAccountControl:1.2.840.113556.1.4.803:= 2)(mail={objectMail}))";
var request = new SearchRequest(usersPath, filter, SearchScope.Subtree, new string[] { "tokenGroups" });
request.Controls.Add(new SecurityDescriptorFlagControl(SecurityMasks.Dacl | SecurityMasks.Group | SecurityMasks.Owner));
var ldapObject = (SearchResponse)ldapConnection.SendRequest(request); // Error thrown here
foreach (byte[] sid in ldapObject.Entries[0].Attributes["tokenGroups"])
{
// Do stuff...
}
}
Whenever this runs I get an error, thrown. The stack trace and error response are below:
at System.DirectoryServices.Protocols.LdapConnection.<ConstructResponseAsync>d__57.MoveNext()
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at System.Threading.Tasks.ValueTask`1.get_Result()
at System.Runtime.CompilerServices.ValueTaskAwaiter`1.GetResult()
at System.DirectoryServices.Protocols.LdapConnection.SendRequest(DirectoryRequest request, TimeSpan requestTimeout)
at System.DirectoryServices.Protocols.LdapConnection.SendRequest(DirectoryRequest request)
at LDAPTesting.Connectors.Protocols.GetExpandedGroups(String objectMail, LdapConnection ldapConnection) in C:\code\LDAPTesting\LDAPTesting\Connectors\Protocols.cs:line 62
at LDAPTesting.Connectors.Protocols.GetGroups() in C:\code\LDAPTesting\LDAPTesting\Connectors\Protocols.cs:line 103
at LDAPTesting.Program.Main(String[] args) in C:\code\LDAPTesting\LDAPTesting\Program.cs:line 17
00002120: SvcErr: DSID-031404CC, problem 5012 (DIR_ERROR), data 0
My thoughts are that I'm querying the wrong path but it looks to match the path that the original code requests on so I'm not sure what else I'm missing.

For anyone that comes along this, the solution is incredibly simple.
The SearchRequest object needs to be of the scope Base. Really obvious when reading what the error message. So the line:
var request = new SearchRequest(usersPath, filter, SearchScope.Subtree, new string[] { "tokenGroups" });
Should instead read:
var request = new SearchRequest(usersPath, filter, SearchScope.Base, new string[] { "tokenGroups" });
I had a feeling that there would be an obvious and simple answer due to overlooking something and indeed there was!

I think I found the solution - it's just the first value that is not correct - if that is ignored, everything works fine!

NServiceBus with AzureStorageQueues not removing poison messages from input queue due to changing message properties

I am experimenting with a new NServiceBus project utilizing Azure Storage Queues for message transport and JSON serialization using custom message unwrapping logic seen here:
var jsonSerializer = new Newtonsoft.Json.JsonSerializer();
transportExtensions.UnwrapMessagesWith(cloudQueueMessage =>
{
using (var stream = new MemoryStream(cloudQueueMessage.AsBytes))
using (var streamReader = new StreamReader(stream))
using (var textReader = new JsonTextReader(streamReader))
{
try
{
var jObject = JObject.Load(textReader);
using (var jsonReader = jObject.CreateReader())
{
// Try deserialize to a NServiceBus envelope first
var wrapper = jsonSerializer.Deserialize<MessageWrapper>(jsonReader);
if (wrapper.MessageIntent != default)
{
// This was a envelope message
return wrapper;
}
}
// Otherwise this was an EventGrid event
using (var jsonReader = jObject.CreateReader())
{
var #event = jsonSerializer.Deserialize<EventGridEvent>(jsonReader);
var wrapper = new MessageWrapper
{
Id = #event.Id,
Headers = new Dictionary<string, string>
{
{ "NServiceBus.EnclosedMessageTypes", #event.EventType },
{ "NServiceBus.MessageIntent", "Publish" },
{ "EventGrid.topic", #event.Topic },
{ "EventGrid.subject", #event.Subject },
{ "EventGrid.eventTime", #event.EventTime.ToString("u") },
{ "EventGrid.dataVersion", #event.DataVersion },
{ "EventGrid.metadataVersion", #event.MetadataVersion },
},
Body = Encoding.UTF8.GetBytes(JsonConvert.SerializeObject(#event.Data)),
MessageIntent = MessageIntentEnum.Publish
};
return wrapper;
}
}
catch
{
logger.Error("Message deserialization failed, sending message to error queue");
throw;
}
}
});
The custom message unwrapping logic works correctly for properly formatted JSON messages and when an improperly formatted JSON message is put into the input queue the custom message unwrapping logic will error out on the first line inside the usings where I create the jObject which is the expected behavior. However, when the custom message unwrapping logic fails the error will get caught by the logic in the MessageRetrieved class which is part of the NServiceBus.Azure.Transports.WindowsAzureStorageQueues NuGet package (v8.2.0) seen below:
public async Task<MessageWrapper> Unwrap()
{
try
{
Logger.DebugFormat("Unwrapping message with native ID: '{0}'", rawMessage.Id);
return unwrapper.Unwrap(rawMessage);
}
catch (Exception ex)
{
await errorQueue.AddMessageAsync(rawMessage).ConfigureAwait(false);
await inputQueue.DeleteMessageAsync(rawMessage).ConfigureAwait(false);
throw new SerializationException($"Failed to deserialize message envelope for message with id {rawMessage.Id}. Make sure the configured serializer is used across all endpoints or configure the message wrapper serializer for this endpoint using the `SerializeMessageWrapperWith` extension on the transport configuration. Please refer to the Azure Storage Queue Transport configuration documentation for more details.", ex);
}
}
The first line of the try catch runs correctly adding the message to the configured error queue, however, when it does that, it appears to be changing the message ID and popreceipt of the raw message as seen here:
Initial Message Values
Updated Message Values
Then when the next line runs attempting to remove the original message from the input queue it is unable to find it as according to this article https://learn.microsoft.com/en-us/rest/api/storageservices/delete-message2#remarks it requires the original message ID and pop reciept which have now changed leading to the following error being thrown:
2020-04-20 14:17:58,603 WARN : Azure Storage Queue transport failed pushing a message through pipeline
Type: Microsoft.WindowsAzure.Storage.StorageException
Message: The remote server returned an error: (404) Not Found.
Source: Microsoft.WindowsAzure.Storage
StackTrace:
at Microsoft.WindowsAzure.Storage.Core.Executor.Executor.EndExecuteAsync[T](IAsyncResult result) in c:\Program Files (x86)\Jenkins\workspace\release_dotnet_master\Lib\ClassLibraryCommon\Core\Executor\Executor.cs:line 50
at Microsoft.WindowsAzure.Storage.Core.Util.AsyncExtensions.<>c__DisplayClass7.<CreateCallbackVoid>b__5(IAsyncResult ar) in c:\Program Files (x86)\Jenkins\workspace\release_dotnet_master\Lib\ClassLibraryCommon\Core\Util\AsyncExtensions.cs:line 121
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at NServiceBus.Transport.AzureStorageQueues.MessageRetrieved.<Unwrap>d__3.MoveNext() in C:\BuildAgent\work\3c19e2a032c05076\src\Transport\MessageRetrieved.cs:line 40
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at NServiceBus.Transport.AzureStorageQueues.MessagePump.<InnerReceive>d__7.MoveNext() in C:\BuildAgent\work\3c19e2a032c05076\src\Transport\MessagePump.cs:line 153
TargetSite: T EndExecuteAsync[T](System.IAsyncResult)
Is this an issue with the NServiceBus package logic, or is something in my custom message unwrapping logic causing these values to change?

This is a bug. When unwrapping is failing, the message is not yet going through the processing pipeline. As a result of that, the normal recoverability is not applicable. The CloudQueueMessage needs to be "cloned" and the clone to be sent to the error queue while the original message used to remove it from the input queue. I've raised a bug issue in GitHub and you can track the process there.

WebJob occasionally throwing System.Threading.Tasks.TaskCanceledException

I have a continuous WebJob which executes a function which is queue triggered and it occasionally throws System.Threading.Tasks.TaskCanceledException.
WebJobs SDK version : 2.1.0
StackTrace:
Microsoft.Azure.WebJobs.Host.FunctionInvocationException:
Microsoft.Azure.WebJobs.Host.FunctionInvocationException : Exception while executing function: Functions.ProcessCreationQueueMessage --->
System.AggregateException : One or more errors occurred. ---> A task was canceled. at
System.Threading.Tasks.Task.ThrowIfExceptional(Boolean includeTaskCanceledExceptions) at
System.Threading.Tasks.Task`1.GetResultCore(Boolean waitCompletionNotification) at System.Threading.Tasks.Task`1.get_Result() at
Normandy.Portal.Reporting.Creation.WebJobs.Functions.ProcessCreationQueueMessage(String message,String& outputQueueMessage,TextWriter log) at
D:\a\1\s\Portal\src\Normandy.Portal.Reporting.Creation.WebJobs\Functions.cs : 105 at lambda_method(Closure ,Functions ,Object[] ) at Microsoft.Azure.WebJobs.Host.Executors.VoidMethodInvoker`2.InvokeAsync(TReflected instance,Object[] arguments) at
async Microsoft.Azure.WebJobs.Host.Executors.FunctionInvoker`2.InvokeAsync[TReflected,TReturnValue](Object instance,Object[] arguments) at
async Microsoft.Azure.WebJobs.Host.Executors.FunctionExecutor.InvokeAsync(IFunctionInvoker invoker,ParameterHelper parameterHelper,CancellationTokenSource timeoutTokenSource,CancellationTokenSource functionCancellationTokenSource,Boolean throwOnTimeout,TimeSpan timerInterval,IFunctionInstance instance) at
async Microsoft.Azure.WebJobs.Host.Executors.FunctionExecutor.ExecuteWithWatchersAsync(IFunctionInstance instance,ParameterHelper parameterHelper,TraceWriter traceWriter,CancellationTokenSource functionCancellationTokenSource)
at async Microsoft.Azure.WebJobs.Host.Executors.FunctionExecutor.ExecuteWithLoggingAsync(??)
at async Microsoft.Azure.WebJobs.Host.Executors.FunctionExecutor.ExecuteWithLoggingAsync(??) ---> (Inner Exception #0) System.Threading.Tasks.TaskCanceledException : A task was canceled.<--- End of inner exception
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw() at async Microsoft.Azure.WebJobs.Host.Executors.FunctionExecutor.ExecuteWithLoggingAsync(??) at
async Microsoft.Azure.WebJobs.Host.Executors.FunctionExecutor.TryExecuteAsync(IFunctionInstance functionInstance,CancellationToken cancellationToken)
WebJob:
public void ProcessCreationQueueMessage([QueueTrigger("reportcreation")] string message, [Queue("reportretrieval")] out string outputQueueMessage, TextWriter log)
{
JsonSerializerSettings settings = new JsonSerializerSettings { TypeNameHandling = TypeNameHandling.All };
log.WriteLine(message);
...
try
{
var jobId = _reportingService.RequestReport(req.ReportName, searchString).Result;
var output = new RetrieveReportDataRequest()
{
CreationRequest = new ReportCreationRequest()
{
UserId = req.UserId,
UID = req.UID,
CreateDate = req.CreateDate,
IsMultiSchoolUser = req.IsMultiSchoolUser,
CentreCode = req.CentreCode,
ReportName = req.ReportName,
InputSearchParameters = req.InputSearchParameters,
},
SplunkSearchParameters = searchString,
SplunkJobId = jobId,
Created = DateTime.UtcNow,
};
outputQueueMessage = JsonConvert.SerializeObject(output, settings);
}
catch (Exception ex)
{
log.WriteLine(ex.Message);
throw;
}
}
There is no any time taking tasks apparently. This is failing once for certain messages. What am I missing?

It can be related to host-settings - look at Azure Portal -> AppService configuration, make sure that Always On is set to On:

Azure Service Bus: AMQP object is closing. Operation 'attach' cannot be performed

I'm using Azure Service Bus Topics with the AMQP protocol in the West Europe datacenter.
This is a schematic way of how the solution implented works:
private Microsoft.Azure.ServiceBus.SubscriptionClient CreateClient() {
string serviceBusConnectionString;
strin serviceBusTopicName;
string subscriptionName;
var subscriptionClient = new Microsoft.Azure.ServiceBus.SubscriptionClient(serviceBusConnectionString, serviceBusTopicName, subscriptionName) {
PrefetchCount = 0
};
return subscriptionClient;
}
public async Task<ISubscriptionClient> CreateSubscriptionClientAsync() {
//Some logic on the subscriptionClient, caching, creating a new one if it doesn't exists, etc.
return CreateClient()
}
private async Task CallbackAsync(Message msg, CancellationToken cancellationToken) {
//Do stuff with the Message
//when you're done Completethe message
}
public async Task<string> OpenAsync(CancellationToken cancellationToken) {
subscriptionClient = await CreateSubscriptionClientAsync().ConfigureAwait(false);
var messageHandlerOptions = new MessageHandlerOptions(ExceptionReceivedHandlerAsync) {
AutoComplete = false,
MaxAutoRenewDuration = TimeSpan.FromHours(8)
};
subscriptionClient.RegisterMessageHandler(CallbackAsync, messageHandlerOptions);
return string.Empty;
}
But last night I had couple of thousand exception like this one:
Exception: "System.InvalidOperationException: The AMQP object g2b-sessionXXXXXXXX is closing. Operation 'attach' cannot be performed.
at Microsoft.Azure.ServiceBus.Core.MessageReceiver.<OnReceiveAsync>d__86.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at Microsoft.Azure.ServiceBus.Core.MessageReceiver.<>c__DisplayClass64_0.<<ReceiveAsync>b__0>d.MoveNext()
I'm using the Microsoft.Azure.ServiceBus v3.1.0.0,
On line I didn't find anything useful about this "Operation 'attach' cannot be performed".
On the message bus exception list page I didn't find any information about this specific problem. And in the status history site there is no reference about any outage involving Service Bus.
Did anyone experience this problem before?
What is causing the exception?
Do I need to implement any retry logic? How?
Any help is apreciated, thanks.

HttpClient SendAsync Cannot access a disposed object ResponseHeadersRead

I am looking into an issue with some legacy code which is relaying a Http Request to another URL. It is an API Controller that reads the request and forwards it to another address asynchronously.
Very rarely, it throws a "Cannot access a disposed object" exception - full stack trace shown later in this question. It seems to be on the line where the HttpClient is calling the SendAsync method. I think it might be the ResponseHeadersRead option - I suspect it is happening when a large packet is being sent and it gets closed because it has just read the header and quit. Just thought I'd sanity check this with you all, for your thoughts. I'll change the option to be ResponseContentsRead option and see how that goes (but it can take a long time for the error to surface).
Here is the code:
using (var client = new HttpClient())
{
var request = BuildRelayHttpRequest(Request);
await client.SendAsync(request, HttpCompletionOption.ResponseHeadersRead);
}
private static HttpRequestMessage BuildRelayHttpRequest(HttpRequestMessage incomingRequest)
{
var forwardToUrl = new Uri(ConfigurationManager.AppSettings["ForwardFeedURL"]);
var relayRequest = new HttpRequestMessage(incomingRequest.Method, forwardToUrl);
if (incomingRequest.Method != HttpMethod.Get && incomingRequest.Content != null)
{
relayRequest.Content = incomingRequest.Content;
}
//Copies contents
relayRequest.Content = incomingRequest.Content;
return relayRequest;
}
And here is the exception:
System.ObjectDisposedException: Cannot access a disposed object.
Object name: 'System.Web.Http.WebHost.HttpControllerHandler+LazyStreamContent'.
at System.Net.Http.HttpContent.CheckDisposed()
at System.Net.Http.HttpContent.CopyToAsync(Stream stream, TransportContext context)
at System.Net.Http.HttpClientHandler.GetRequestStreamCallback(IAsyncResult ar)
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at System.Runtime.CompilerServices.TaskAwaiter`1.GetResult()
at CHO.Web.Services.ETrains.Controllers.ETrainsApiController.<CopyandForwardFeedAsyn>d__18.MoveNext() in \Controllers\MyAPIController.cs:line 289
Note, line 289 is the "await client.SendAsync" line of code

Odds are there is an error code being set by the server.
Follow your code with a response.EnsureSuccessStatusCode(); that is wrapped in a Try Catch block:
try
{
var request = BuildRelayHttpRequest(Request);
await client.SendAsync(request, HttpCompletionOption.ResponseHeadersRead);
response.EnsureSuccessStatusCode();
// Handle success
}
catch (HttpRequestException)
{
// Handle failure
}