None of the specified endpoints were reachable - c#

There is intermittent issue in rabbitmq while publishing message from .net application. It is creating lots of noise in the system.
I have tried googling but could not found root cause of an issue.
Here is the error I can see in application log,
None of the specified endpoints were reachable
Endpoints attempted:
------------------------------------------------
endpoint=amqp-0-9://localhost:5672, attempts=1
System.TimeoutException: Connection to amqp-0-9://localhost:5672 timed out
at RabbitMQ.Client.Impl.SocketFrameHandler_0_9.Connect(TcpClient socket, AmqpTcpEndpoint endpoint, Int32 timeout)
at RabbitMQ.Client.Impl.SocketFrameHandler_0_9..ctor(AmqpTcpEndpoint endpoint, ObtainSocket socketFactory, Int32 timeout)
at RabbitMQ.Client.Framing.Impl.v0_9_1.ProtocolBase.CreateFrameHandler(AmqpTcpEndpoint endpoint, ObtainSocket socketFactory, Int32 timeout)
at RabbitMQ.Client.ConnectionFactory.FollowRedirectChain(Int32 maxRedirects, IDictionary`2 connectionAttempts, IDictionary`2 connectionErrors, AmqpTcpEndpoint[]& mostRecentKnownHosts, AmqpTcpEndpoint endpoint)
================================================
Stack trace:
at RabbitMQ.Client.ConnectionFactory.CreateConnection(Int32 maxRedirects)
at RabbitMQ.Client.ConnectionFactory.CreateConnection()

I have resolved this issue by making RabbitMQ Connection object singleton to make sure reuse connection instead of making new connection everytime.

Related

Re-registration of WCF endpoint thows "already exists" even after an abort

I'm facing a strange problem with WCF endpoint manager highly available windows service using Azure Service fabric, which provides callbacks when the service becomes active.
Everytime the service becomes active, we register the WCF endpoints using:
A)
serviceHost = CreateServiceHost(IWcfContractInstance)
serviceHost.AddServiceEndpoint(...)
serviceHost.Open()
Whenever the service is deactivated (note: process does not go down in this case)
B)
serviceHost.Close()
They both are under lock and there is no synchronization issue here. The problem is that A fails , saying that the endpoint is already registered.
StackTrace:
System.InvalidOperationException: The ChannelDispatcher at 'net.tcp://localhost:[PORT]/IWcfContract '"ContractName"' is unable to open its IChannelListener. ---> System.InvalidOperationException:
A registration already exists for URI 'net.tcp://localhost:[PORT]/IWcfContract'.
at System.ServiceModel.Channels.UriPrefixTable`1.RegisterUri(Uri uri; HostNameComparisonMode hostNameComparisonMode; TItem item)
at System.ServiceModel.Channels.ConnectionOrientedTransportManager`1.Register(TransportChannelListener channelListener)
at System.ServiceModel.Channels.TransportManager.Open(TransportChannelListener channelListener)
at System.ServiceModel.Channels.TransportManagerContainer.Open(SelectTransportManagersCallback selectTransportManagerCallback)
at System.ServiceModel.Channels.TransportChannelListener.OnOpen(TimeSpan timeout)
at System.ServiceModel.Channels.ConnectionOrientedTransportChannelListener.OnOpen(TimeSpan timeout)
at System.ServiceModel.Channels.TcpChannelListener`2.OnOpen(TimeSpan timeout)
at System.ServiceModel.Channels.CommunicationObject.Open(TimeSpan timeout)
at System.ServiceModel.Channels.ReliableChannelListenerBase`1.OnOpen(TimeSpan timeout)
at System.ServiceModel.Channels.CommunicationObject.Open(TimeSpan timeout)
at System.ServiceModel.Dispatcher.ChannelDispatcher.OnOpen(TimeSpan timeout)
at System.ServiceModel.Dispatcher.ChannelDispatcher.OnOpen(TimeSpan timeout)
at System.ServiceModel.Channels.CommunicationObject.Open(TimeSpan timeout)
at System.ServiceModel.ServiceHostBase.OnOpen(TimeSpan timeout)
at System.ServiceModel.Channels.CommunicationObject.Open(TimeSpan timeout)
The strange thing is that B) also fails and throws CommunicationObjectFaultedException. So the question is, calling a Close on a faulted session should dispose off the resoureces, so that when we go ahead and register this endpoint again, it should just work.
Am I missing something here ?

Azure cache exception when storing data

currently I'm using Azure cache in my WCF application, and I got this exception when putting data into cache:
Microsoft.ApplicationServer.Caching.DataCacheException: ErrorCode< ERRCA0017 >:SubStatus< ES0006 >:There is a temporary failure. Please retry later. (One or more specified cache servers are unavailable, which could be caused by busy network or servers. For on-premises cache clusters, also verify the following conditions. Ensure that security permission has been granted for this client account, and check that the AppFabric Caching Service is allowed through the firewall on all cache hosts. Also the MaxBufferSize on the server must be greater than or equal to the serialized object size sent from the client.)
And here's the inner exception:
System.ServiceModel.CommunicationException: The socket connection was aborted. This could be caused by an error processing your message or a receive timeout being exceeded by the remote host, or an underlying network resource issue. Local socket timeout was '00:00:30'. ---> System.Net.Sockets.SocketException: An existing connection was forcibly closed by the remote host
at System.Net.Sockets.Socket.Send(Byte[] buffer, Int32 offset, Int32 size, SocketFlags socketFlags)
at System.ServiceModel.Channels.SocketConnection.Write(Byte[] buffer, Int32 offset, Int32 size, Boolean immediate, TimeSpan timeout)
--- End of inner exception stack trace ---
at System.ServiceModel.Channels.SocketConnection.Write(Byte[] buffer, Int32 offset, Int32 size, Boolean immediate, TimeSpan timeout)
at System.ServiceModel.Channels.SocketConnection.Write(Byte[] buffer, Int32 offset, Int32 size, Boolean immediate, TimeSpan timeout, BufferManager bufferManager)
at System.ServiceModel.Channels.BufferedConnection.WriteNow(Byte[] buffer, Int32 offset, Int32 size, TimeSpan timeout, BufferManager bufferManager)
at System.ServiceModel.Channels.BufferedConnection.Write(Byte[] buffer, Int32 offset, Int32 size, Boolean immediate, TimeSpan timeout, BufferManager bufferManager)
at System.ServiceModel.Channels.FramingDuplexSessionChannel.OnSend(Message message, TimeSpan timeout)
at System.ServiceModel.Channels.OutputChannel.Send(Message message, TimeSpan timeout)
at Microsoft.ApplicationServer.Caching.CacheResolverChannel.Send(Message message, TimeSpan timeout)
at Microsoft.ApplicationServer.Caching.WcfClientChannel.SendOnChannel(EndpointID endpoint, TimeSpan& timeout, WaitCallback callback, Object state, Boolean async, IDuplexSessionChannel channel, Message message)
--- End of inner exception stack trace ---
at Microsoft.ApplicationServer.Caching.DataCache.ThrowException(ResponseBody respBody)
at Microsoft.ApplicationServer.Caching.DataCache.ExecuteAPI(RequestBody reqMsg, IMonitoringListener listener)
at Microsoft.ApplicationServer.Caching.DataCache.InternalPut(String key, Object value, DataCacheItemVersion oldVersion, TimeSpan timeout, DataCacheTag[] tags, String region, IMonitoringListener listener)
at Microsoft.ApplicationServer.Caching.DataCache.<>c_DisplayClass19.b_18()
at Microsoft.ApplicationServer.Caching.MonitoringListenerFactory.EmptyListener.Microsoft.ApplicationServer.Caching.IMonitoringListener.Listen[TResult](Func`1 innerDelegate)
at Microsoft.ApplicationServer.Caching.DataCache.Put(String key, Object value)
The problem is this message is very general, doesn't give much details
I already tried to increase requestTimeout in dataCacheClient to 2 minutes. This doesn't solve the problem, so it should not because of connection to cache timeout
I know it's not because of amount of data too big also, because the same amount of data on production server doesn't have problem (this happens on testing server). I connect from local computer to testing database and also don't have problem
Anyone has any ideas how to solve this problem? Thanks
Below is my current configuration for Azure cache
<dataCacheClient name="default" requestTimeout="30000" isCompressionEnabled="true" >
<localCache isEnabled="true" sync="TimeoutBased" objectCount="4194304" ttlValue="80000" />
</dataCacheClient>
Please set ReceiveTimeout Property between 40-50 seconds in your code to avoid socket timeouts.
Have a retry logic in your code to retry the operation if the error suggests so.
And if this issue persists over longer period of time then take a look at the cache instance performance counters to see if you are nearing 100% limit or maxing out capacity in some sense in which case you should reassess your capacity, refer Capacity Planning Considerations for In-Role Cache (Windows Azure Cache)

WCF Error while receiving the HTTP response to my service

Hi guys I created a service using WCF but when I test my services some completed successfully why some is giving me the error below. Any Idea on how to solve these ? Thanks
An error occurred while receiving the HTTP response to http://localhost:8733/PortOperation/Operator_Service/ws. This could be due to the service endpoint binding not using the HTTP protocol. This could also be due to an HTTP request context being aborted by the server (possibly due to the service shutting down). See server logs for more details.
Server stack trace:
at System.ServiceModel.Channels.HttpChannelUtilities.ProcessGetResponseWebException(WebException webException, HttpWebRequest request, HttpAbortReason abortReason)
at System.ServiceModel.Channels.HttpChannelFactory`1.HttpRequestChannel.HttpChannelRequest.WaitForReply(TimeSpan timeout)
at System.ServiceModel.Channels.RequestChannel.Request(Message message, TimeSpan timeout)
at System.ServiceModel.Channels.ClientReliableChannelBinder`1.RequestClientReliableChannelBinder`1.OnRequest(TRequestChannel channel, Message message, TimeSpan timeout, MaskingMode maskingMode)
at System.ServiceModel.Channels.ClientReliableChannelBinder`1.Request(Message message, TimeSpan timeout, MaskingMode maskingMode)
at System.ServiceModel.Channels.ClientReliableChannelBinder`1.Request(Message message, TimeSpan timeout)
at System.ServiceModel.Security.SecuritySessionClientSettings`1.SecurityRequestSessionChannel.Request(Message message, TimeSpan timeout)
at System.ServiceModel.Dispatcher.RequestChannelBinder.Request(Message message, TimeSpan timeout)
at System.ServiceModel.Channels.ServiceChannel.Call(String action, Boolean oneway, ProxyOperationRuntime operation, Object[] ins, Object[] outs, TimeSpan timeout)
at System.ServiceModel.Channels.ServiceChannelProxy.InvokeService(IMethodCallMessage methodCall, ProxyOperationRuntime operation)
at System.ServiceModel.Channels.ServiceChannelProxy.Invoke(IMessage message)
Exception rethrown at [0]:
at System.Runtime.Remoting.Proxies.RealProxy.HandleReturnMessage(IMessage reqMsg, IMessage retMsg)
at System.Runtime.Remoting.Proxies.RealProxy.PrivateInvoke(MessageData& msgData, Int32 type)
at IOperator_Service.GetAllUser(String ConnectionString)
at Operator_ServiceClient.GetAllUser(String ConnectionString)
Inner Exception:
The underlying connection was closed: An unexpected error occurred on a receive.
at System.Net.HttpWebRequest.GetResponse()
at System.ServiceModel.Channels.HttpChannelFactory`1.HttpRequestChannel.HttpChannelRequest.WaitForReply(TimeSpan timeout)
Inner Exception:
Unable to read data from the transport connection: An existing connection was forcibly closed by the remote host.
at System.Net.Sockets.NetworkStream.Read(Byte[] buffer, Int32 offset, Int32 size)
at System.Net.PooledStream.Read(Byte[] buffer, Int32 offset, Int32 size)
at System.Net.Connection.SyncRead(HttpWebRequest request, Boolean userRetrievedStream, Boolean probeRead)
Inner Exception:
An existing connection was forcibly closed by the remote host
at System.Net.Sockets.Socket.Receive(Byte[] buffer, Int32 offset, Int32 size, SocketFlags socketFlags)
at System.Net.Sockets.NetworkStream.Read(Byte[] buffer, Int32 offset, Int32 size)
There are many scenarios when this type of error occurs.
First
Check that have you provided proper [DataContract] and [DataMember]? if this is not provided then this type of error occurs. You have to write [DataContract] above the class which you pass in response, and write [DataMember] above the class member which goes into the client response.
eg.
[DataContract]
class Program
{
[DataMember]
public string Example{get;set}
}
Second
Check in the response if some minvalue of Data type is passed. That means some time data member is not initialized at that time it takes the minvalue.
e.g. minValue of int is '-2147483648' so at some time it cannot be serialized and throws an error.
If you want to trace this type of error then add the following code in your server-side web.config
<system.diagnostics>
<sources>
<source name="System.ServiceModel" switchValue="Information, ActivityTracing" propagateActivity="true">
<listeners>
<add name="traceListener" type="System.Diagnostics.XmlWriterTraceListener" initializeData= "D:\Traces.svclog"/>
</listeners>
</source>
</sources>
</system.diagnostics>
I think it will help you.
I was able to solve the problem.
If you are using EF "code first"-approach and you want access to your data using WCF you should consider not enabling lazy loading using the "virtual" keyword in your poco-class because WCF serialization cannot serialize dynamic proxy data.

WCF Service TargetInvocationException - Possible Cyclic Reference

I need a little help with an Exception being thrown at my WCF service. The point is: I need to develop a WPF application and I've heard that to treat data WCF is the best choice so I used it only with basic knowledge and now I have this problem.
The point is, I have a Customer class, a Supplier class and an Order class. I have then two services: the first one has all methods to add, read and edit customers and suppliers from database. This service works perfectly fine. The second service has methods to add new orders, edit orders and read them from database, and this service has a problem.
The class Customer has a list of Suppliers (the suppliers from which the customer buys), the class Supplier has a list of Customers (the customers that buy from that supplier). The class order has a Customer, a Supplier and a list of Itens (and the itens are added to the database together with the order).
The method for adding an order works fine, however, once an order has been added, the method to list the orders throws the TargetInvocationException. I searched a lot on the internet and the best answer I've found is that it's possibly a "Ciclyc Reference".
The method to list the orders is inside the repository class and is simply:
public IQueryable<Order> ListOrders()
{
return Context.Set<Order>().Include(x => x.Customer).Include(x => x.Supplier);
}
Where Context is the entity framework context. I tried to remove the includes and load the customer and supplier separately, but the result was the same exception.
Can someone please help me ? If I need to post more information just tell me. And sorry if this matter is trivial, I'm new to WPF and WCF so I'm still a little confused.
Thanks a lot in advance.
EDIT Exception below
An error occurred while receiving the HTTP response to
http://SomeService.com/OrdersService.svc. This could be due to the
service endpoint binding not using the HTTP protocol. This could also
be due to an HTTP request context being
aborted by the server (possibly due to the service shutting down). See
server logs for more details.
Server stack trace: at
System.ServiceModel.Channels.HttpChannelUtilities.ProcessGetResponseWebException(WebException
webException, HttpWebRequest request, HttpAbortReason abortReason)
at
System.ServiceModel.Channels.HttpChannelFactory.HttpRequestChannel.HttpChannelRequest.WaitForReply(TimeSpan
timeout) at
System.ServiceModel.Channels.RequestChannel.Request(Message message,
TimeSpan timeout) at
System.ServiceModel.Dispatcher.RequestChannelBinder.Request(Message
message, TimeSpan timeout) at
System.ServiceModel.Channels.ServiceChannel.Call(String action,
Boolean oneway, ProxyOperationRuntime operation, Object[] ins,
Object[] outs, TimeSpan timeout) at
System.ServiceModel.Channels.ServiceChannelProxy.InvokeService(IMethodCallMessage
methodCall, ProxyOperationRuntime operation) at
System.ServiceModel.Channels.ServiceChannelProxy.Invoke(IMessage
message)
Exception rethrown at [0]: at
System.Runtime.Remoting.Proxies.RealProxy.HandleReturnMessage(IMessage
reqMsg, IMessage retMsg) at
System.Runtime.Remoting.Proxies.RealProxy.PrivateInvoke(MessageData&
msgData, Int32 type) at IOrdersService.ReturnOrders() at
OrdersServiceClient.ReturnOrders()
Inner Exception: The underlying connection was closed: An unexpected
error occurred on a receive. at
System.Net.HttpWebRequest.GetResponse() at
System.ServiceModel.Channels.HttpChannelFactory.HttpRequestChannel.HttpChannelRequest.WaitForReply(TimeSpan
timeout)
Inner Exception: Unable to read data from the transport connection: An
existing connection was forcibly closed by the remote host. at
System.Net.Sockets.NetworkStream.Read(Byte[] buffer, Int32 offset,
Int32 size) at System.Net.PooledStream.Read(Byte[] buffer, Int32
offset, Int32 size) at
System.Net.Connection.SyncRead(HttpWebRequest request, Boolean
userRetrievedStream, Boolean probeRead)
Inner Exception: An existing connection was forcibly closed by the
remote host at System.Net.Sockets.Socket.Receive(Byte[] buffer,
Int32 offset, Int32 size, SocketFlags socketFlags) at
System.Net.Sockets.NetworkStream.Read(Byte[] buffer, Int32 offset,
Int32 size)
Mark your data objects with the [DataContract(IsReference = true)] attribute. This should resolve the issue.

System.TimeoutException when calling wcf service from Biztalk

I am getting the exception shown below. BizTalk is able to call the service when there is very light load. But once I increase the load and the number of messsages to transmit increases after a while I keep getting this message for all the calls to the service.
Event Type: Error
Event Source: BizTalk Server 2006
Event Category: (1)
Event ID: 5799
Date: 11/21/2011
Time: 12:55:38 PM
User: N/A
Computer: CLBIZBTS07S
Description:
The description for Event ID ( 5799 ) in Source ( BizTalk Server 2006 ) cannot be found. The local computer may not have the necessary registry information or message DLL files to display messages from a
remote computer. You may be able to use the /AUXSOURCE= flag to retrieve this description; see Help and Support for details. The following information is part of the event: sndManageServiceEndPoint_WCF,
http://service.com:8100/ManageService.svc/ws, The adapter failed to transmit message going to send port "sndManageServiceEndPoint_WCF" with URL "http://service.com:8100/ManageService.svc/ws". It will be
retransmitted after the retry interval specified for this Send Port. Details:"System.TimeoutException: Client is unable to finish the security negotiation within the configured timeout (00:01:00). The
current negotiation leg is 3 (00:00:20.6239920). ---> System.TimeoutException: The request channel timed out attempting to send after 00:00:20.6239920. Increase the timeout value passed to the call to
Request or increase the SendTimeout value on the Binding. The time allotted to this operation may have been a portion of a longer timeout. ---> System.TimeoutException: The HTTP request to
'http://service.com:8100/ManageService.svc/ws' has exceeded the allotted timeout of 00:00:20.6230000. The time allotted to this operation may have been a portion of a longer
timeout. ---> System.Net.WebException: The operation has timed out
at System.Net.HttpWebRequest.GetRequestStream(TransportContext& context)
at System.Net.HttpWebRequest.GetRequestStream()
at System.ServiceModel.Channels.HttpOutput.WebRequestHttpOutput.GetOutputStream()
--- End of inner exception stack trace ---
at System.ServiceModel.Channels.HttpOutput.WebRequestHttpOutput.GetOutputStream()
at System.ServiceModel.Channels.HttpOutput.Send(TimeSpan timeout)
at System.ServiceModel.Channels.HttpChannelFactory.HttpRequestChannel.HttpChannelRequest.SendRequest(Message message, TimeSpan timeout)
at System.ServiceModel.Channels.RequestChannel.Request(Message message, TimeSpan timeout)
--- End of inner exception stack trace ---
at System.ServiceModel.Channels.RequestChannel.Request(Message message, TimeSpan timeout)
at System.ServiceModel.Security.IssuanceTokenProviderBase`1.DoNegotiation(TimeSpan timeout)
--- End of inner exception stack trace ---
Server stack trace:
at System.ServiceModel.Security.IssuanceTokenProviderBase`1.DoNegotiation(TimeSpan timeout)
at System.ServiceModel.Security.SspiNegotiationTokenProvider.OnOpen(TimeSpan timeout)
at System.ServiceModel.Security.WrapperSecurityCommunicationObject.OnOpen(TimeSpan timeout)
at System.ServiceModel.Channels.CommunicationObject.Open(TimeSpan timeout)
at System.ServiceModel.Security.SecurityUtils.OpenCommunicationObject(ICommunicationObject obj, TimeSpan timeout)
at System.ServiceModel.Security.SymmetricSecurityProtocol.OnOpen(TimeSpan timeout)
at System.ServiceModel.Security.WrapperSecurityCommunicationObject.OnOpen(TimeSpan timeout)
at System.ServiceModel.Channels.CommunicationObject.Open(TimeSpan timeout)
at System.ServiceModel.Channels.SecurityChannelFactory`1.ClientSecurityChannel`1.OnOpen(TimeSpan timeout)
at System.ServiceModel.Channels.CommunicationObject.Open(TimeSpan timeout)
at System.ServiceModel.Security.SecuritySessionSecurityTokenProvider.DoOperation(SecuritySessionOperation operation, EndpointAddress target, Uri via, SecurityToken currentToken, TimeSpan timeout)
at System.ServiceModel.Security.SecuritySessionSecurityTokenProvider.GetTokenCore(TimeSpan timeout)
at System.IdentityModel.Selectors.SecurityTokenProvider.GetToken(TimeSpan timeout)
at System.ServiceModel.Security.SecuritySessionClientSettings`1.ClientSecuritySessionChannel.OnOpen(TimeSpan timeout)
at System.ServiceModel.Channels.CommunicationObject.Open(TimeSpan timeout)
at System.ServiceModel.Channels.ServiceChannel.OnOpen(TimeSpan timeout)
at System.ServiceModel.Channels.CommunicationObject.Open(TimeSpan timeout)
Exception rethrown at [0]:
at System.Runtime.Remoting.Proxies.RealProxy.HandleReturnMessage(IMessage reqMsg, IMessage retMsg)
at System.Runtime.Remoting.Proxies.RealProxy.PrivateInvoke(MessageData& msgData, Int32 type)
at System.ServiceModel.ICommunicationObject.Open()
at Microsoft.BizTalk.Adapter.Wcf.Runtime.WcfClient`2.GetChannel[TChannel](IBaseMessage bizTalkMessage, ChannelFactory`1& cachedFactory)
at Microsoft.BizTalk.Adapter.Wcf.Runtime.WcfClient`2.SendMessage(IBaseMessage bizTalkMessage)". .
On the service side I have the set maxReceivedMessageSize="2147483647" in the endpoint
and in the behavior section I have set
<serviceThrottling
maxConcurrentCalls="16"
maxConcurrentInstances="2147483647"
maxConcurrentSessions="10"/>
These two settings did not help.
When I turned on the tracing on wcf service I see these type of errors
Activity Name Process action 'http://schemas.xmlsoap.org/ws/2005/02/trust/RST/SCT/Cancel'.
Time 2011-11-22 15:03:12.5301
Level Error
Source System.ServiceModel
Process w3wp
Thread 32
Computer FRA1BSHR01S
Trace Identifier/Code http://msdn.microsoft.com/en-US/library/System.ServiceModel.Diagnostics.ThrowingException.aspx
System.ServiceModel.EndpointNotFoundException, System.ServiceModel, Version=3.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089
There was no channel that could accept the message with action 'http://schemas.xmlsoap.org/ws/2005/02/trust/RST/SCT/Cancel'.
System.ServiceModel.Dispatcher.ErrorBehavior.ThrowAndCatch(Exception e, Message message)
System.ServiceModel.Channels.DatagramChannelDemuxer`2.ProcessItem(TInnerItem item)
System.ServiceModel.Channels.DatagramChannelDemuxer`2.HandleReceiveResult(IAsyncResult result)
System.ServiceModel.Channels.DatagramChannelDemuxer`2.OnReceiveComplete(IAsyncResult result)
System.ServiceModel.Channels.DatagramChannelDemuxer`2.OnReceiveCompleteStatic(IAsyncResult result)
System.ServiceModel.Diagnostics.Utility.AsyncThunk.UnhandledExceptionFrame(IAsyncResult result)
System.ServiceModel.AsyncResult.Complete(Boolean completedSynchronously)
System.ServiceModel.Channels.ReplyChannel.HelpReceiveRequestAsyncResult.OnReceiveRequest(IAsyncResult result)
System.ServiceModel.Diagnostics.Utility.AsyncThunk.UnhandledExceptionFrame(IAsyncResult result)
System.ServiceModel.AsyncResult.Complete(Boolean completedSynchronously)
System.ServiceModel.Channels.InputQueue`1.AsyncQueueReader.Set(Item item)
System.ServiceModel.Channels.InputQueue`1.EnqueueAndDispatch(Item item, Boolean canDispatchOnThisThread)
System.ServiceModel.Channels.InputQueue`1.EnqueueAndDispatch(T item, ItemDequeuedCallback dequeuedCallback, Boolean canDispatchOnThisThread)
System.ServiceModel.Channels.InputQueueChannel`1.EnqueueAndDispatch(TDisposable item, ItemDequeuedCallback dequeuedCallback, Boolean canDispatchOnThisThread)
System.ServiceModel.Channels.SingletonChannelAcceptor`3.Enqueue(QueueItemType item, ItemDequeuedCallback dequeuedCallback, Boolean canDispatchOnThisThread)
System.ServiceModel.Channels.SingletonChannelAcceptor`3.Enqueue(QueueItemType item, ItemDequeuedCallback dequeuedCallback)
System.ServiceModel.Channels.HttpChannelListener.HttpContextReceived(HttpRequestContext context, ItemDequeuedCallback callback)
System.ServiceModel.Activation.HostedHttpTransportManager.HttpContextReceived(HostedHttpRequestAsyncResult result)
System.ServiceModel.Activation.HostedHttpRequestAsyncResult.BeginRequest()
System.ServiceModel.Activation.HostedHttpRequestAsyncResult.OnBeginRequest(Object state)
System.ServiceModel.Channels.IOThreadScheduler.CriticalHelper.WorkItem.Invoke2()
System.ServiceModel.Channels.IOThreadScheduler.CriticalHelper.WorkItem.Invoke()
System.ServiceModel.Channels.IOThreadScheduler.CriticalHelper.ProcessCallbacks()
System.ServiceModel.Channels.IOThreadScheduler.CriticalHelper.CompletionCallback(Object state)
System.ServiceModel.Channels.IOThreadScheduler.CriticalHelper.ScheduledOverlapped.IOCallback(UInt32 errorCode, UInt32 numBytes, NativeOverlapped* nativeOverlapped)
System.ServiceModel.Diagnostics.Utility.IOCompletionThunk.UnhandledExceptionFrame(UInt32 error, UInt32 bytesRead, NativeOverlapped* nativeOverlapped)
System.Threading._IOCompletionCallback.PerformIOCompletionCallback(UInt32 errorCode, UInt32 numBytes, NativeOverlapped* pOVERLAP)
[TraceRecord] Severity Error
TraceIdentifier http://msdn.microsoft.com/en-US/library/System.ServiceModel.Diagnostics.ThrowingException.aspx
Description Throwing an exception.
AppDomain /LM/W3SVC/1373337114/Root-4-129664654928269938
Although increasing the timeout on both ends of the wire (BTS and WCF Service) may help avoid the error in some instances, you might just be kicking the can further down the road and you probably need to get to the root cause (does this specific web service call NORMALLY take over 20 seconds to complete?).
It is likely that under high load, or low capacity conditions, that your WCF Web Service cannot keep up with the rate of messages that BizTalk is able to send to it.
Depending on what your WCF Service does, you might look at optimising:
Database
SQL blocking while waiting for locks
DB performance may reduced during admin tasks such as Reindexing Jobs, Backups etc
IIS / ASP.NET
See if requests are being queued
Look at any throttling on the WCF Service side (e.g. MaxConcurrentCalls)
If there is no chance of further optimisation (or scale up / out) on the WCF Service side, you may need to look at throttling the number of concurrent orchestrations (using a variant of a Singleton Orch) or limiting the number of threads on your WCF Send Port Adapter (unfortunately, this will affect all messages, so you might need to split out separate SendHost
Be wary when simply 'maxing out' all of the WCF knobs - settings like maxReceivedMessageSize (or maxBufferSize) ="2147483647" will often result in OutOfMemoryExceptions and / or increase the threat surface area.
You need to change the timeout of the WCF service.
I am not sure if that solution is correct, but it is definitely in the web.config file.
I had similar problem when using SOAP adapter(I'm still in biztalk 2006)
You should try the following :
YourMessage(SOAP.ClientConnectionTimeout) = 900000;
900000 is the default value, so increase it to fit your need, the time is in milliseconds.
If this property does not apply to WCF web services, I assume there is a WCF equivalent for that property.

Categories

Resources