For my Azure functions application, a fan-out architecture with (sub-)orchestrators and a lot of activities running in parallel is required. Additionally, all these activities have to interact with an Azure blob storage account. When dealing with this high degree of parallelization, I often run into runtime errors like these:
[2023-02-07T14:11:36.859Z] 4c6a215f040141aaaa995f9cfbf1a234:10: An error occurred while processing message [TaskScheduled#46]: DurableTask.AzureStorage.Storage.DurableTaskStorageException: The specified pop receipt did not match the pop receipt for a dequeued message.[2023-02-07T14:11:36.860Z] ---> Microsoft.WindowsAzure.Storage.StorageException: The specified pop receipt did not match the pop receipt for a dequeued message.
[2023-02-07T14:11:36.861Z] at Microsoft.WindowsAzure.Storage.Core.Executor.Executor.ExecuteAsyncInternal[T](RESTCommand`1 cmd, IRetryPolicy policy, OperationContext operationContext, CancellationToken token)[2023-02-07T14:11:36.874Z] at DurableTask.AzureStorage.Storage.AzureStorageClient.WrapFunctionWithReturnType(Func`3 storageRequest, OperationContext context, CancellationToken cancellationToken) in /_/src/DurableTask.AzureStorage/Storage/AzureStorageClient.cs:line 156[2023-02-07T14:11:36.875Z] at DurableTask.AzureStorage.TimeoutHandler.ExecuteWithTimeout[T](String operationName, String account, AzureStorageOrchestrationServiceSettings settings, Func`3 operation, AzureStorageOrchestrationServiceStats stats, String clientRequestId)[2023-02-07T14:11:36.876Z] at DurableTask.AzureStorage.Storage.AzureStorageClient.MakeStorageRequest[T](Func`3 storageRequest, String accountName, String operationName, String clientRequestId, Boolean force)
[2023-02-07T14:11:36.877Z] Request Information
[2023-02-07T14:11:36.879Z] RequestID:7db00ba3-1e70-42fc-84f5-f6dfb9ec630a
[2023-02-07T14:11:36.880Z] RequestDate:Tue, 07 Feb 2023 15:11:36 GMT
[2023-02-07T14:11:36.896Z] StatusMessage:The specified pop receipt did not match the pop receipt for a dequeued message.
[2023-02-07T14:11:36.897Z] ErrorCode:PopReceiptMismatch
[2023-02-07T14:11:36.898Z] ErrorMessage:The specified pop receipt did not match the pop receipt for a dequeued message.
RequestId:7db00ba3-1e70-42fc-84f5-f6dfb9ec630a
Time:2023-02-07T14:11:36.137Z
[2023-02-07T14:11:36.901Z] --- End of inner exception stack trace ---
These neither trigger try-catch breakpoints nor stop code execution, but are really annoying and cause the orchestrators to restart which leads to data corruption down the road. Is there a proper way of fixing this?
When function projects get created in Visual Studio, the AzureWebJobsStorage value in the local.settings.json file is set to true by default. Configuring Azure functions to use a real storage account for orchestrators can eliminate concurrency issues and faulty state management. Simply replace the value with the connection string of your own storage:
{
...
"Values": {
"AzureWebJobsStorage": "[YOUR CONNECTION STRING HERE]",
...
}
}
Related
I'm using FlaUI library to automate desktop app.
I took an error on try to run code to take a window of launched program. Error: Could not find process with id: ***'
Details of error.
System.Exception
HResult=0x80131500
Message=Could not find process with id: 13536
Source=FlaUI.Core
StackTrace:
at FlaUI.Core.Application.FindProcess(Int32 processId)
at FlaUI.Core.Application.<WaitWhileMainHandleIsMissing>b__33_0()
at FlaUI.Core.Tools.Retry.<>c__DisplayClass12_0.<WhileTrue>b__0()
at FlaUI.Core.Tools.Retry.While[T](Func`1 retryMethod, Func`2 checkMethod, Nullable`1 timeout, Nullable`1 interval, Boolean throwOnTimeout, Boolean ignoreException, String timeoutMessage, Boolean lastValueOnTimeout, T defaultOnTimeout)
at FlaUI.Core.Application.WaitWhileMainHandleIsMissing(Nullable`1 waitTimeout)
at FlaUI.Core.Application.GetMainWindow(AutomationBase automation, Nullable`1 waitTimeout)
at Program.<Main>$(String[] args) in C:\Users\jekug\source\repos\FlaUI test\FlaUI test\Program.cs:line 10
This exception was originally thrown at this call stack:
[External Code]
ArgumentException: Process with an Id of 13536 is not running.
What can be wrong and are there other ways to take
a window in FlaUI?
Thank you
calc.exe on any newer windows is actually just a proxy executable to in the end run the "WindowsStoreApp" calculator which behaves quite differently. For those apps, you need to use LaunchStoreApp instead of Launch in order to get the correct process.
Event Viewer on my workstation have the following error log:
ERROR NServiceBus.Transports.Msmq.MsmqDequeueStrategy [(null)] - Error in receiving messages.
System.Transactions.TransactionAbortedException: The transaction has aborted. ---> System.Transactions.TransactionManagerCommunicationException: Communication with the underlying transaction manager has failed. ---> System.Runtime.InteropServices.COMException: The Transaction Manager is not available. (Exception from HRESULT: 0x8004D01B)
at System.Transactions.Oletx.IDtcProxyShimFactory.ConnectToProxy(String nodeName, Guid resourceManagerIdentifier, IntPtr managedIdentifier, Boolean& nodeNameMatches, UInt32& whereaboutsSize, CoTaskMemHandle& whereaboutsBuffer, IResourceManagerShim& resourceManagerShim)
at System.Transactions.Oletx.DtcTransactionManager.Initialize()
--- End of inner exception stack trace ---
at System.Transactions.Oletx.OletxTransactionManager.ProxyException(COMException comException)
at System.Transactions.Oletx.DtcTransactionManager.Initialize()
at System.Transactions.Oletx.DtcTransactionManager.get_ProxyShimFactory()
at System.Transactions.Oletx.OletxTransactionManager.CreateTransaction(TransactionOptions properties)
at System.Transactions.TransactionStatePromoted.EnterState(InternalTransaction tx)
--- End of inner exception stack trace ---
at System.Transactions.TransactionStateAborted.CheckForFinishedTransaction(InternalTransaction tx)
at System.Transactions.Transaction.Promote()
at System.Transactions.TransactionInterop.ConvertToOletxTransaction(Transaction transaction)
at System.Transactions.TransactionInterop.GetDtcTransaction(Transaction transaction)
at System.Messaging.MessageQueue.StaleSafeReceiveMessage(UInt32 timeout, Int32 action, MQPROPS properties, NativeOverlapped* overlapped, ReceiveCallback receiveCallback, CursorHandle cursorHandle, IntPtr transaction)
at System.Messaging.MessageQueue.ReceiveCurrent(TimeSpan timeout, Int32 action, CursorHandle cursor, MessagePropertyFilter filter, MessageQueueTransaction internalTransaction, MessageQueueTransactionType transactionType)
at System.Messaging.MessageQueue.Receive(TimeSpan timeout, MessageQueueTransactionType transactionType)
at NServiceBus.Transports.Msmq.MsmqDequeueStrategy.TryReceiveMessage(Func`1 receive, Message& message) in C:\BuildAgent\work\3206e2123f54fce4\src\NServiceBus.Core\Transports\Msmq\MsmqDequeueStrategy.cs:line 332
Facts:
Distributed Transaction Coordinator (DTC) Component Service ==> DTC Enabled
Distributed Transaction Coordinator (DTC) Service ==> Running (referenced to Answered Stack Oveflow Question)
Realtek Audio Universal Service ==> Disabled (as per reference in Fix for 0x8004d01b)
Can anyone recommend a direction on how to check/troubleshoot the cause of the issue?
After long ardous hours, the issue I encountered boils down to permission issue. The services involved in DTC which uses credentials of LocalSystem do not have enough permission and blocked by company-imposed policy implemented on the server.
Unfortunately, I only managed to find an alternative or workaround by using account that have enough permissions to operate and manage the services involved (including custom services).
#gnud - Thanks a lot for your inputs giving me a direction for investigation and that help me continue move forward on my investigation. I appreciate the help. Thanks.
I have an aplication which writes a message using Publish/Subscriber pattern with Rebus implementation.
At some point, without any code changes, Rebus started writing the MSMQ Extension binary content with some weird characters after JSON content, here follows an example of one taken message with the issue:
{"rbs2-intent":"pub","rbs2-msg-id":"ac543d60-e28c-49bb-8783-b5c6574a90ea","rbs2-return-address":"myqueuename#SERVERNAME","rbs2-senttime":"2018-04-26T00:20:48.0453055-03:00","rbs2-corr-id":"ac543d60-e28c-49bb-8783-b5c6574a90ea","rbs2-corr-seq":"0","rbs2-msg-type":"ClassNamespace.BusinessClassName,
ClassNamespace","rbs2-content-type":"application/json;charset=utf-8","rbs2-content-encoding":"gzip"}?k?h?3??
? /??w
Here is the full error message I receive on my lof4net log file, written by Rebus when doing the message receive from MSMQ.
2018-05-03 10:33:49,516 WARN
Rebus.Workers.ThreadPoolBased.ThreadPoolWorker - An error occurred
when attempting to receive the next message:
System.Runtime.Serialization.SerializationException: Could not
deserialize JSON text
'{"rbs2-intent":"pub","rbs2-msg-id":"ac543d60-e28c-49bb-8783-b5c6574a90ea","rbs2-return-address":"myqueuename#SERVERNAME","rbs2-senttime":"2018-04-26T00:20:48.0453055-03:00","rbs2-corr-id":"ac543d60-e28c-49bb-8783-b5c6574a90ea","rbs2-corr-seq":"0","rbs2-msg-type":"ClassNamespace.BusinessClassName,
ClassNamespace","rbs2-content-type":"application/json;charset=utf-8","rbs2-content-encoding":"gzip"}?k?h?3??
? /??w R? ? Newtonsoft.Json.JsonReaderException:
Additional text encountered after finished reading JSON content: .
Path '', line 1, position 432. at
Newtonsoft.Json.JsonTextReader.Read() at
Newtonsoft.Json.Serialization.JsonSerializerInternalReader.Deserialize(JsonReader
reader, Type objectType, Boolean checkAdditionalContent) at
Newtonsoft.Json.JsonSerializer.DeserializeInternal(JsonReader reader,
Type objectType) at
Newtonsoft.Json.JsonConvert.DeserializeObject(String value, Type type,
JsonSerializerSettings settings) at
Newtonsoft.Json.JsonConvert.DeserializeObject[T](String value,
JsonSerializerSettings settings) at
Rebus.Serialization.HeaderSerializer.DeserializeFromString(String str)
--- End of inner exception stack trace --- at Rebus.Serialization.HeaderSerializer.DeserializeFromString(String str)
at Rebus.Msmq.MsmqTransport.d__15.MoveNext()
--- End of stack trace from previous location where exception was thrown --- at
System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw() at
System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task
task) at
Rebus.Workers.ThreadPoolBased.ThreadPoolWorker.d__17.MoveNext()
After some time digging into this problem, I managed do diassembly Rebus implemetation for MSMQ "MsmqTransport" class and I could see how Rebus is communication with MSMQ, also how is the Extension property being filled with Encoding.UTF8.GetBytes binary data.
What is not clear to me is how come a Encoding.UTF8.GetBytes(JsonConvert.Serialize(Dictionary<string,string>)) could result in that messed up array of bytes.
I'm considering that this implementation of Rebus MsmqTransport for MSMQ cannot be affected by my code, either by the object I'm sending to MSMQ, in which is going to be placed at Message Body, not Extension binary or by the way I'm configuring it.
As a workaround, in order to keep all solution running, I've implemented a service in which peeks the message from MSMQ, does a Encoding.UTF8.GetString from Exteions property bytes, analyse it, fix it in case of wierd characters and pull it back (the healthy JSON) at Extension property, then dequeue it from MSMQ and re enqueue it back to MSMQ, so the real process using Rebus (the Subscriber) will be able to receive it properly.
Is there a way I can figure it out how Rebus is messing up with the binary data written at Extension property of MSMQ message!?
Here is how I'm configuring Rebus at the Publisher implementation:
Bus = Configure.With(activator)
.Logging(l => l.Trace())
.Options(o =>
{
o.SimpleRetryStrategy(Config.ErrorQueue, Config.MaxDeliveryAttempts == 0 ? 3 : Config.MaxDeliveryAttempts);
if (Config.EnableCompression)
o.EnableCompression();
})
.Transport(t => t.UseMsmq(Config.PublisherQueue))
.Subscriptions(s => s.Register(c => SubscriptionStorage))
.Start();
Many thanks.
That sounds really weird!
As you have correctly figured out, the way Rebus populates the Extension property of the MSMQ Message does not leave that much room for weird bugs and funny behavior like what you're experiencing.
I once used Rebus at a customer's site though, and stuff was working fine for a long time – but suddenly, also without any code changes, weird stuff started to turn up in the Extension part of our MSMQ messages too, very similar to what you're seeing.
After much pondering and head-shaking, we found out that out customer had hired another company to help them with general performance monitoring and stuff, and one of their initiatives involved something that would use the Extension property of MSMQ messages, messing with Rebus' headers, thus breaking everything.
I don't know if it's "standard MSMQ practice", or why they would just start messing with our messages like that without even asking us if it would be a problem, but maybe it could be something similar in your case?
If not, I don't really have any clues.
I have a worker role which consumes messages from an Azure Queue does some processing in the background.
When I check my log there appears to be no exception logged while processing the message but still in my logs I get the following Exception(pasting some of the relevant texts from the long error log only):
System.Net.WebException
Microsoft.WindowsAzure.Storage.StorageException Exception messages:
The remote server returned an error: (404) Not Found. The remote
server returned an error: (404) Not Found. Stack Traces: at
System.Net.HttpWebRequest.GetResponse() at
Microsoft.WindowsAzure.Storage.Core.Executor.Executor.ExecuteSync[T](RESTCommand1
cmd, IRetryPolicy policy, OperationContext operationContext) --- Next
Call Stack: at
Microsoft.WindowsAzure.Storage.Core.Executor.Executor.ExecuteSync[T](RESTCommand1
cmd, IRetryPolicy policy, OperationContext operationContext) at
Microsoft.WindowsAzure.Storage.Queue.CloudQueue.DeleteMessage(String
messageId, String popReceipt, QueueRequestOptions options,
OperationContext operationContext) at
InnovativeExams.Azure.CloudStorage.AzureQueue`1.DeleteMessage(T
message) The specified message does not exist.
ErrorCode:MessageNotFound Prod-WorkerError Context
Here is the code I have in the worker role:
private void ProcessQueueMessage(object queueMessageToProcess)
{
var queueMessage = queueMessageToProcess as EventCompletedQueueMessage;
try
{
if (_eventCompletedProcessor.Process(queueMessage))
_azureQueue.DeleteMessage(queueMessage);
}
catch (Exception ex)
{
_logger.LogError(string.Format("Event Completed message <{0}> was not processed due to an exception", queueMessage.Id), ex, LogSources.WorkerRole_EventCompletedDispatcher);
}
}
The above exception is caught in the above catch block and logged.
I think there is some problem when the worker role tries to Delete the message in the Queue and the message is not found.
Need help from someone to help me resolve this error and to know the background what's going wrong here.
Few Questions asked in comment:
1) Are you running multiple instances of your worker role?
A: This is an existing application and I figured out we are using ThreadPool to pre-instantiate, threads which stand ready to be given work.
2) How are you "getting" the messages in your worker role? Are you using some kind of leader election pattern to decide which instance gets the messages?
A: Yes there is a framework which determines the appropriate dispatcher for a QueueMessage to be processed.
3) When you get messages, what's the visibility timeout for those messages?
A: Its set to 120.
4) How long does it take for you to process these messages i.e. how much time between getting the message and deleting the messages?
A: I am not sure on this.
Let me explain under what situation you would get the error you're encountering.
When you dequeue a message (i.e. GET Messages in Azure terminology), Azure Queue Service returns something called popreceipt which must be used to delete or update a message. This popreceipt is an opaque value (i.e. you should not build any business logic around it) which remains valid till the time that same message is dequeued again. When a message is dequeued again, you get a new value for popreceipt and you should use this new value to delete or update the message.
If you try to use an old popreceipt value to delete a message while the message was dequeued again (by some other process), you will get the error you're getting.
My guess is that this is what is happening in your application. Please check if this is indeed the case:
One of the worker role instances dequeues the message and starts working on that message. Based on what you told above, you hide the message for 120 seconds when you dequeue a message. I am assuming that the actual time taken for processing the message is more than 120 seconds and thus the message reappears in the queue. There's another process which now dequeues this message (and thus you get a new popreceipt). However soon after the 2nd process dequeued the message, 1st process finished working on the message and now it wants to delete the message using the popreceipt it has. Because this popreceipt is no longer valid, any attempt to perform delete operation on that message using this popreceipt will cause a message not found error.
I'm connecting to a JMS queue. The queue is hosted by WebSphere Application Server (version 8.0.0.5), using SIBus.
I have a simple program to reproduce some problematic behavior, it is written in C# and uses XMS (IBM's .NET API) to connect to the queue. Note: I obtained the IBM.XMS dlls from here: MQC71: WebSphere MQ V7.1 Clients
The scenario is this:
run the program (use an empty queue, so it sits waiting on a blocking Receive call)
disconnect network (unplug ethernet)
after a few minutes, the Receive call throws an exception
the program attempts to connect anew
this fails due to the network being down
...let it loop a bit to verify this error keeps occurring
restore the network (plug in ethernet)
the same error continues to occur (verified with WireShark that no network traffic occurs during the connectionFactory.CreateConnection() call)
So, the question is: why does the CreateConnection() call do nothing (does not send any packets) but fail?
using IBM.XMS;
using System;
namespace SimpleTest
{
class Program
{
static void Main(string[] args)
{
while (true)
{
try
{
string queueURI = "queue://your.details.GoHereDearReader";
string providerEndpoint = "1.2.3.4:1234";
string targetTransportChain = "InboundBasicMessaging";
string busName = "some_bus_name";
XMSFactoryFactory factoryFactory = XMSFactoryFactory.GetInstance(XMSC.CT_WPM);
IConnectionFactory connectionFactory = factoryFactory.CreateConnectionFactory();
connectionFactory.SetStringProperty(XMSC.WPM_PROVIDER_ENDPOINTS, providerEndpoint);
connectionFactory.SetStringProperty(XMSC.WPM_TARGET_TRANSPORT_CHAIN, targetTransportChain);
connectionFactory.SetStringProperty(XMSC.WPM_BUS_NAME, busName);
Log("Connecting...");
using (var connection = connectionFactory.CreateConnection())
{
using (var session = connection.CreateSession(false, AcknowledgeMode.AutoAcknowledge))
{
using (var destination = session.CreateQueue(queueURI))
{
destination.SetIntProperty(XMSC.DELIVERY_MODE, XMSC.DELIVERY_NOT_PERSISTENT);
connection.Start();
using (IMessageConsumer consumer = session.CreateConsumer(destination))
{
Log("Receiving...");
IMessage recvMsg = consumer.Receive();
Log("recvMsg:" + recvMsg);
}
}
}
}
}
catch (Exception e)
{
Log(e.ToString());
}
Log("Sleeping some before consuming more...");
System.Threading.Thread.Sleep(10 * 1000);
}//while
}//Main
static void Log(string text)
{
Console.WriteLine(DateTime.UtcNow.ToString("HH:mm:ss.fff") + "-------------------------- " + text);
}
}
}
And some output:
16:38:19.483-------------------------- Connecting...
16:38:19.936-------------------------- Receiving...
16:43:31.952-------------------------- IBM.XMS.XMSException: EXCEPTION_RECEIVED_CWSIA0022
EXCEPTION_RECEIVED_CWSIA0022.explanation
EXCEPTION_RECEIVED_CWSIA0022.useraction
at IBM.XMS.Impl.Connection.Dispose(Boolean disposing)
at IBM.XMS.Impl.Connection.Dispose()
at SimpleTest.Program.Main(String[] args) in c:\Users\Administrator\Documents\Visual Studio Projects\TestJMSDequeue\SimpleTest\Program.cs:line 43
Linked Exception : IBM.XMS.SIB.JFAP.JFapConversationClosedException: Exception of type 'IBM.XMS.SIB.JFAP.JFapConversationClosedException' was thrown.
at IBM.XMS.SIB.JFAP.ConversationImpl.Send(IByteBuffer[] data, JFAPSegmentType segmentType, UInt16 requestNumber, IoPriority priority, Boolean pooledBufferHint, IReceiveListener recvListener, ISendListener sendListener, Object state)
at IBM.XMS.SIB.JFAP.ConversationImpl.RequestReplyExchange(IByteBuffer[] data, JFAPSegmentType segmentType, UInt16 requestNumber, IoPriority priority, Boolean pooledBufferHint)
at IBM.XMS.SIB.JFAP.ConversationImpl.RequestReplyExchange(IByteBuffer data, JFAPSegmentType segmentType, UInt16 requestNumber, IoPriority priority, Boolean pooledBufferHint)
at IBM.XMS.SIB.Comms.Client.JFAPCommunicator.RequestReplyExchange(IByteBuffer data, JFAPSegmentType sendSegType, MessagePriority priority, Boolean canPoolOnReceive)
at IBM.XMS.SIB.Comms.Client.ConnectionProxy.Close()
at IBM.XMS.Impl.Connection.Dispose(Boolean disposing)
16:43:31.952-------------------------- Sleeping some before consuming more...
16:43:41.967-------------------------- Connecting...
Exception : System.InvalidOperationException: UniqueLinkObject no seen on handshake.
at IBM.XMS.Formats.MFP.SIB.SchemaManager.SendSchemas(ICommsConnection connection, JMFSchema[] schemas)
at IBM.XMS.Formats.MFP.SIB.JsMsgObject.Encode(Object connection)
FFDC to xmsffdc8416_2013.03.08T10.43.41.967444.txt
Exception : IBM.XMS.Formats.MFP.MessageEncodeFailedException: Exception of type 'IBM.XMS.Formats.MFP.MessageEncodeFailedException' was thrown. ---> System.InvalidOperationException: UniqueLinkObject no seen on handshake.
at IBM.XMS.Formats.MFP.SIB.SchemaManager.SendSchemas(ICommsConnection connection, JMFSchema[] schemas)
at IBM.XMS.Formats.MFP.SIB.JsMsgObject.Encode(Object connection)
--- End of inner exception stack trace ---
at IBM.XMS.Formats.MFP.SIB.JsMsgObject.Encode(Object connection)
at IBM.XMS.Formats.MFP.SIB.TrmFirstContactMessageImpl.Encode(Object connection)
at IBM.XMS.SIB.Trm.ClientBootstrapHandler.Connect(IClientConnection clientConnection)
FFDC to xmsffdc8416_2013.03.08T10.43.41.983044.txt
16:43:41.983-------------------------- IBM.XMS.XMSException: EXCEPTION_RECEIVED_CWSIA0241
EXCEPTION_RECEIVED_CWSIA0241.explanation
EXCEPTION_RECEIVED_CWSIA0241.useraction
at IBM.XMS.Impl.ConnectionFactory.CreateConnection(String userName, String password)
at IBM.XMS.Impl.ConnectionFactory.CreateConnection()
at SimpleTest.Program.Main(String[] args) in c:\Users\Administrator\Documents\Visual Studio Projects\TestJMSDequeue\SimpleTest\Program.cs:line 26
Linked Exception : IBM.XMS.Core.SIResourceException: CWSIT0006E: It is not possible to connect to bus your_bus_name_here because the following bootstrap servers could not be contacted 1.2.3.4:1234:BootstrapBasicMessaging and the following bootstrap servers returned an error condition . See previous messages for the reason for each bootstrap server failure.
The client cannot connect to the bus. This situation may be due to configuration problems, network problems or it may be that none of the required bootstrap servers or messaging engines are currently available.
Ensure that the network is working correctly and that the required bootstrap servers and messaging engines are available. ---> IBM.XMS.Core.SIConnectionLostException: Exception of type 'IBM.XMS.Core.SIConnectionLostException' was thrown.
at IBM.XMS.SIB.Comms.Client.ClientSideConnection.Connect(ConnectionProperties cp, IClientComponentHandshake cch, SICoreConnectionProperties siProps)
at IBM.XMS.SIB.Trm.TrmSICoreConnectionFactoryImpl.Bootstrap(Credentials credentials, SICoreConnectionProperties connectionProperties, ClientAttachProperties cap)
--- End of inner exception stack trace ---
at IBM.XMS.SIB.Trm.TrmSICoreConnectionFactoryImpl.Bootstrap(Credentials credentials, SICoreConnectionProperties connectionProperties, ClientAttachProperties cap)
at IBM.XMS.SIB.Trm.TrmSICoreConnectionFactoryImpl.CreateConnection(Credentials credentials, SICoreConnectionProperties connectionProperties)
at IBM.XMS.SIB.Trm.TrmSICoreConnectionFactory.CreateConnection(String username, String password, SICoreConnectionProperties connectionProperties)
at IBM.XMS.Impl.ConnectionFactory.CreateConnection(String userName, String password)
16:43:41.983-------------------------- Sleeping some before consuming more...
16:43:51.998-------------------------- Connecting...
Exception : System.InvalidOperationException: UniqueLinkObject no seen on handshake.
at IBM.XMS.Formats.MFP.SIB.SchemaManager.SendSchemas(ICommsConnection connection, JMFSchema[] schemas)
at IBM.XMS.Formats.MFP.SIB.JsMsgObject.Encode(Object connection)
FFDC to xmsffdc8416_2013.03.08T10.43.51.998262.txt
Exception : IBM.XMS.Formats.MFP.MessageEncodeFailedException: Exception of type 'IBM.XMS.Formats.MFP.MessageEncodeFailedException' was thrown. ---> System.InvalidOperationException: UniqueLinkObject no seen on handshake.
at IBM.XMS.Formats.MFP.SIB.SchemaManager.SendSchemas(ICommsConnection connection, JMFSchema[] schemas)
at IBM.XMS.Formats.MFP.SIB.JsMsgObject.Encode(Object connection)
--- End of inner exception stack trace ---
at IBM.XMS.Formats.MFP.SIB.JsMsgObject.Encode(Object connection)
at IBM.XMS.Formats.MFP.SIB.TrmFirstContactMessageImpl.Encode(Object connection)
at IBM.XMS.SIB.Trm.ClientBootstrapHandler.Connect(IClientConnection clientConnection)
FFDC to xmsffdc8416_2013.03.08T10.43.51.998262.txt
16:43:51.998-------------------------- IBM.XMS.XMSException: EXCEPTION_RECEIVED_CWSIA0241
EXCEPTION_RECEIVED_CWSIA0241.explanation
EXCEPTION_RECEIVED_CWSIA0241.useraction
at IBM.XMS.Impl.ConnectionFactory.CreateConnection(String userName, String password)
at IBM.XMS.Impl.ConnectionFactory.CreateConnection()
at SimpleTest.Program.Main(String[] args) in c:\Users\Administrator\Documents\Visual Studio Projects\TestJMSDequeue\SimpleTest\Program.cs:line 26
Linked Exception : IBM.XMS.Core.SIResourceException: CWSIT0006E: It is not possible to connect to bus your_bus_name_here because the following bootstrap servers could not be contacted 1.2.3.4:1234:BootstrapBasicMessaging and the following bootstrap servers returned an error condition . See previous messages for the reason for each bootstrap server failure.
The client cannot connect to the bus. This situation may be due to configuration problems, network problems or it may be that none of the required bootstrap servers or messaging engines are currently available.
Ensure that the network is working correctly and that the required bootstrap servers and messaging engines are available. ---> IBM.XMS.Core.SIConnectionLostException: Exception of type 'IBM.XMS.Core.SIConnectionLostException' was thrown.
at IBM.XMS.SIB.Comms.Client.ClientSideConnection.Connect(ConnectionProperties cp, IClientComponentHandshake cch, SICoreConnectionProperties siProps)
at IBM.XMS.SIB.Trm.TrmSICoreConnectionFactoryImpl.Bootstrap(Credentials credentials, SICoreConnectionProperties connectionProperties, ClientAttachProperties cap)
--- End of inner exception stack trace ---
at IBM.XMS.SIB.Trm.TrmSICoreConnectionFactoryImpl.Bootstrap(Credentials credentials, SICoreConnectionProperties connectionProperties, ClientAttachProperties cap)
at IBM.XMS.SIB.Trm.TrmSICoreConnectionFactoryImpl.CreateConnection(Credentials credentials, SICoreConnectionProperties connectionProperties)
at IBM.XMS.SIB.Trm.TrmSICoreConnectionFactory.CreateConnection(String username, String password, SICoreConnectionProperties connectionProperties)
at IBM.XMS.Impl.ConnectionFactory.CreateConnection(String userName, String password)
16:43:51.998-------------------------- Sleeping some before consuming more...
16:44:02.013-------------------------- Connecting...
Exception : System.InvalidOperationException: UniqueLinkObject no seen on handshake.
at IBM.XMS.Formats.MFP.SIB.SchemaManager.SendSchemas(ICommsConnection connection, JMFSchema[] schemas)
at IBM.XMS.Formats.MFP.SIB.JsMsgObject.Encode(Object connection)
FFDC to xmsffdc8416_2013.03.08T10.44.02.013479.txt
Exception : IBM.XMS.Formats.MFP.MessageEncodeFailedException: Exception of type 'IBM.XMS.Formats.MFP.MessageEncodeFailedException' was thrown. ---> System.InvalidOperationException: UniqueLinkObject no seen on handshake.
at IBM.XMS.Formats.MFP.SIB.SchemaManager.SendSchemas(ICommsConnection connection, JMFSchema[] schemas)
at IBM.XMS.Formats.MFP.SIB.JsMsgObject.Encode(Object connection)
--- End of inner exception stack trace ---
at IBM.XMS.Formats.MFP.SIB.JsMsgObject.Encode(Object connection)
at IBM.XMS.Formats.MFP.SIB.TrmFirstContactMessageImpl.Encode(Object connection)
at IBM.XMS.SIB.Trm.ClientBootstrapHandler.Connect(IClientConnection clientConnection)
FFDC to xmsffdc8416_2013.03.08T10.44.02.013479.txt
16:44:02.013-------------------------- IBM.XMS.XMSException: EXCEPTION_RECEIVED_CWSIA0241
EXCEPTION_RECEIVED_CWSIA0241.explanation
EXCEPTION_RECEIVED_CWSIA0241.useraction
at IBM.XMS.Impl.ConnectionFactory.CreateConnection(String userName, String password)
at IBM.XMS.Impl.ConnectionFactory.CreateConnection()
at SimpleTest.Program.Main(String[] args) in c:\Users\Administrator\Documents\Visual Studio Projects\TestJMSDequeue\SimpleTest\Program.cs:line 26
Linked Exception : IBM.XMS.Core.SIResourceException: CWSIT0006E: It is not possible to connect to bus your_bus_name_here because the following bootstrap servers could not be contacted 1.2.3.4:1234:BootstrapBasicMessaging and the following bootstrap servers returned an error condition . See previous messages for the reason for each bootstrap server failure.
The client cannot connect to the bus. This situation may be due to configuration problems, network problems or it may be that none of the required bootstrap servers or messaging engines are currently available.
Ensure that the network is working correctly and that the required bootstrap servers and messaging engines are available. ---> IBM.XMS.Core.SIConnectionLostException: Exception of type 'IBM.XMS.Core.SIConnectionLostException' was thrown.
at IBM.XMS.SIB.Comms.Client.ClientSideConnection.Connect(ConnectionProperties cp, IClientComponentHandshake cch, SICoreConnectionProperties siProps)
at IBM.XMS.SIB.Trm.TrmSICoreConnectionFactoryImpl.Bootstrap(Credentials credentials, SICoreConnectionProperties connectionProperties, ClientAttachProperties cap)
--- End of inner exception stack trace ---
at IBM.XMS.SIB.Trm.TrmSICoreConnectionFactoryImpl.Bootstrap(Credentials credentials, SICoreConnectionProperties connectionProperties, ClientAttachProperties cap)
at IBM.XMS.SIB.Trm.TrmSICoreConnectionFactoryImpl.CreateConnection(Credentials credentials, SICoreConnectionProperties connectionProperties)
at IBM.XMS.SIB.Trm.TrmSICoreConnectionFactory.CreateConnection(String username, String password, SICoreConnectionProperties connectionProperties)
at IBM.XMS.Impl.ConnectionFactory.CreateConnection(String userName, String password)
16:44:02.013-------------------------- Sleeping some before consuming more...
Note that in the above output I unplugged the network shortly after "Receiving..." and plugged it in again at 16:43:52, but the errors continued after that until I killed the program.
It reads like the ConnectionFactory does not reset it's state properly when the network becomes available again.
I have scoured the reference manual for any hint of a reset function on the factory - but havn't found one.
I would try this however: Move the GetInstance(...) and createConnectionFactory() out of the loop, as well as the SetStringProperty() lines.
If that doesn't work I cannot see why your client would not work.
Have you considered if this is a problem with the server? Which ethernet cable did you unplug? the one connected to the client or the server? If it was the server, does the same problem occur when you unplug the client cable?
If you restart the program, will it work then?
I know this is an old post, but maybe it will be still helpful.
You receive this error after you created 65.535 connections. It doesn't matter if you close the connection / dispose the object.
I see the following solutions:
1.) You can either create a connection once and reuse the connection for all your operations on the queue, until the connection breaks. Then you have to do a reconnect. (yes that's bad and against the common patterns. I wouldn't advice to do this)
2.) There is a bugfix for the version 8.0.0.6 available. EDIT: APAR #(IJ01244)
I don't know yet when it will included into future releases.
I guess you can contact the IBM support and refer to this PMR ID to receive an updated version of the IBM.XMS.SIB.dll
Hope this helps.