I'm trying to add some SOAP message logging capabilities to an old, old ATL Server web service that is running in integrated mode in IIS 7.5 on a Windows Server 2008 box, but running into a strange problem. For further background, I've added the assembly that contains the HttpModule to the modules element of the web.config for the ATL Server web service.
I've been following the answer provided here and the logging itself works great.
However, whenever I use this logging capability, the service responds with "SOAP Invalid Request", while the log file has the SOAP message as expected. I've done lots of fiddling around with it and figured out that this only happens if/when I access the request object's InputStream property in my handler for the BeginRequest event. It will fail if I even simply set a variable to the length of the InputStream like this:
private void OnBegin(object sender, EventArgs e)
{
var request = _application.Request;
//this will blow up
var foo = request.InputStream.Position;
}
If I don't touch the InputStream in my handler (which doesn't do much good when I'm only doing this to log the contents of the request, obviously), the request goes through perfectly.
I can access header values in the Request object and various other properties of the HttpApplication involved, but accessing the InputStream causes the service to choke.
Is there something intrinsic with ATL Server that will prevent me from doing this logging? Do I need to add some sort of locking or other safeguard in my BeginRequest handler to make sure this behaves? Is my handler hosing up the InputStream somehow causing it to be unusable for the service?
Another way of approaching this is to ask if there is a way to see the request as it gets to my service (i.e. after this HttpModule executes)?
It may also be worth noting that I am using SoapUI to test the service.
EDIT:
I've now done some failed request tracing in IIS and I get this error message:
ModuleName IsapiModule
Notification 128
HttpStatus 500
HttpReason Internal Server Error
HttpSubStatus 0
ErrorCode 0
ConfigExceptionInfo
Notification EXECUTE_REQUEST_HANDLER
ErrorCode The operation completed successfully. (0x0)
This comes in the handler for the ATL Server web service (i.e. the DLL for the service). Directly before that is the "GENERAL_READ_ENTITY_START" and "GENERAL_READ_ENTITY_END" messages, and the "END" has this message:
BytesReceived 0
ErrorCode 2147942438
ErrorCode Reached the end of the file. (0x80070026)
Does that mean what I think it means? That the handler isn't getting any data? Is this more evidence pointing towards my HttpModule messing with the Request's InputStream?
Are you sure your request object is valid? You're doing things slightly differently here from the sample you reference. They are extracting the stream from the sender argument whereas you obviously rely on a member variable.
So I finally determined that this wasn't a workable approach: I couldn't get the HttpModule to fire at all in IIS 6 (which I would need to have it to do for it to be an acceptable solution). I tried setting the Filter property on the Request object and all sorts of other crazy ideas, but none that led me to be able to both record the request body in the HttpModule and have the service still work.
So I did more digging and came upon this article on codeproject that talks about the inner workings of ATL Server, specifically the HandleRequest method in atlsoap.h. I mucked around in there for a while and figured out a way to get at the request body in there, and it was pretty simple to write it to a file manually from there.
For those curious, this is the final code I added to HandleRequest():
//****************************************REQUEST LOGGING**********************************************************
BYTE* bytes = pRequestInfo->pServerContext->GetAvailableData();
FILE* pFile;
pFile = fopen("C:\\IISLog\\ATL.txt", "a");
fwrite(bytes, 1, pRequestInfo->pServerContext->GetAvailableBytes(), pFile);
fclose(pFile);
//****************************************REQUEST LOGGING**********************************************************
I am going to still play around with it a bit more, but I have what appears to be a workable solution.
Related
I have a c# application that the client uses wcf to talk to the server. In the background every X seconds the client calls a Ping method to the server (through WCF). The following error has reproduced a couple of times (for different method calls):
System.ServiceModel.ProtocolException: A reply message was received for operation 'MyMethodToServer' with action 'http://tempuri.org/IMyInterface/PingServerResponse'. However, your client code requires action 'http://tempuri.org/IMyInterface/MyMethodToServerResponse'.
MyMethodToServer is not consistent and it falls on different methods.
How can this happen that a request receives a different response?
I think you have a pretty mess problem with async communication, main suggestion (as your question isn't clear very well), is try to identify every request, catch the calls and waiting for them, do asyncronic communication and getting a several work with threading.
As you present it, is a typical architecture problem.
If you present more code, can I suggest some code fixing in my answer and I'll be glad to update my answer.
If this occurs randomly and not you consistently, you might be running in a load-balanced setup, and deployed an update to only one of the servers?
Wild guess: your client uses same connection to do two requests in parallel. So what happens is:
Thread 1 sends request ARequest
Thread 2 sends request BRequest
Server sends reply BReply
Thread 1 receives reply BReply while expecting AReply
If you have request logs on the server, it'll be easy to confirm - you'll likely see two requests coming with short delay from the client host experiencing the issue
I think MaxConcurrentCall and ConcurrencyMode may be relevant here (although I did not touch WCF for a long while)
Last few days I've been building a web server application in C# that uses HttpListener. I've learned quite a lot on the way, and still am. Currently I got it all working, setting headers here and there depending on certain situations.
In most cases things are working fine, however at times a exception error is thrown. This happens on a few occasions. Most if not all of them is closing a connection before all data is send. Then the error occurs. But some of them are really caused by browsers as far as I can tell.
Like let's take Chrome. Whenever I go to a MP3 file directly, it sends 2 GET requests. And one of them is causing the error, the other is working and receiving part of the content. After this, I can listen the MP3 and there are no issues. Streaming works.
But back to the request that gives me the error, there is nothing in the headers that I could use in my code to not output data, like I do already with HEAD requests. So I'm quite puzzled here.
IE also has this problem with both opening MP3 files directly, and streaming via HTML5 audio tag. It also varies from time to time. Sometimes I open the page, and only 2 requests are made. The HTML page, and the MP3. No error there. Sometimes tho, there are 3 requests. It connects to the MP3 twice. Now sometimes one of those connections is being aborted straight after I open the page, and sometimes 2 requests to the MP3 file, doesn't even accept data. In both request headers, they want end of the file. So bytes: 123-123/124.
I've also tested it on w3school's audio element. IE also makes twice connections there, one aborted, other loading the MP3 file.
So my question is, is it possible to make the web server exception/error-proof, or maybe better question, is it bad that these exceptions are thrown? Or do you perhaps know how to fix these errors?
The error I'm getting is: I/O Operation has been aborted by either a thread exit or an application request.
The way I write to the client is:
using (Stream Output = _CResponse.OutputStream)
{
Output.Write(_FileOutput, rangeBegin, rangeLength);
}
I am not sure if there's another (better) way. This is what I came across in many topics, tutorials and pages while researching.
About headers: Default headers: Content Length, Content Type, Status Code. In some cases, like MP3 files and video's, I add a Accept-Ranges: Bytes header. In case the request header has Range in it, I add Content-Range header, and PartialContent status code.
From the server's point of view any client can disconnect at any time. This is part of the normal operation of a server. Detect this specific case, log it and swallow the exception (because it has been handled). It's not a server bug.
I have a WCF client (running on Win7) pointing to a WebSphere service.
All is good from a test harness (a little test fixture outside my web app) but when my calls to the service originate from my web project one of the calls (and only that one) is extremely slow to deserialize (it takes minutes VS seconds) and not just the first time.
I can see from fiddler that the response comes back quickly but then the WCF client hangs on the response itself for more than a minute before the next line of code is hit by the debugger, almost if the client was having trouble deserializing. This happens only if in the response I have a given pdf string (the operation generates a pdf), base64 encoded chunked. If for example the service raises a fault (thus the pdf string is not there) then the response is deserialized immediately.
Again, If I send the exact same envelope through Soap-UI or from outside the web project all is good.
I am at loss - What should I be looking for and is there some config setting that might do the trick?
Any help appreciated!
EDIT:
I coded a stub against the same service contract. Using the exact same basicHttpBinding and returning the exact same pdf string there is no delay registered. I think this rules out the string and the binding as a possible cause. What's left?
Changing transferMode="Buffered" into transferMode="Streamed" on the binding did the trick!
So the payload was apparently being chunked in small bits the size of the buffer.
I thought the same could have been achieved by increasing the buffersize (maxBufferSize="1000000") but I had that in place already and it did not help.
I have had this bite me many times. Check in your WCF client configuration that you are not trying to use the windows web proxy, that step to check on the proxy (even if there is not one configured) will eat up a lot of time during your connection.
If the tips of the other users don't help, you might want to Enable WCF Tracing and open it in the Service Trace Viewer. The information is detailed, but it has enabled me to fix a number of hard-to-identity problems in the past.
More information on WCF Tracing can be found on MSDN.
Two thing you can try:
Adjust the readerQoutas settings for your client. See http://msdn.microsoft.com/en-us/library/ms731325.aspx
Disable "Just My Code" in debugging options. Tools -> Options -> Debugging -> General "Enable Just My Code (Managed only)" and see if you can catch interal WCF exceptions.
//huusom
I had the very same issue... The problem of WCF, IMO, is in the deserialization of the base64 string returned by the service into a byte[] client side.
The easiest to solve this if you may not change your service configuration (Ex.: use a transferMode="Streamed") is to adapt your DataContract/ServiceContract client side. Replace the type "byte[]" with "string" in the Response DataContract.
Next simply decode the returned string yourself with a piece of code such as:
byte[] file = Convert.FromBase64String(pdfBase64String);
To download a PDF of 70KB, it used to required ~6 sec. With the suggested change here above, it takes now < 1 sec.
V.
PS.: Regarding the transfer mode, I did try to only change the client side (transferMode="StreamedResponse") but without improvement...
First things to check:
Is the config the same in the web project and the test project?
When you test from SOAP UI are you doing it from the same server and in the same security context as when the code is running from the web project.
-Is there any spike in the memory when the pdf comes back?
Edit
From your comments the 1 min wait, appears that it is waiting for a timeout. You also mention transactions.
I am wondering if the problem is somewhere else. The call to the WCF service goes OK, but the call is inside a transaction, and there is no complete or dispose on the transaction (I am guessing here), then the transaction / code will hang for 1 min while it waits to timeout.
Edit 2
Next things to check:
Is there any difference in the code is the test and in the web project, on how the service is being called.
Is there any differnce in the framework version, for example is one 3.0 and the other 3.5
Can it be that client side is trying to analyse what type of content is coming from server side? Try to specify mime type of the service response explicitly on the server side, e.g. Response.ContentType = "application/pdf" EDIT: By client side I mean any possible mediator like a firewall or a security suite.
I get a CommunicationException while using WCF service. The message is:
The remote endpoint no longer recognizes this sequence. This is most likely due to an abort on the remote endpoint. The value of wsrm:Identifier is not a known Sequence identifier. The reliable session was faulted.
The exception is thrown in a moment after a contract method was called. Before calling contract method the channel state is Opened. I restore my service client after catching this exception and for some time it works fine. But then this error occures again. It seems like some timeout is exceeded, but I can't understand which one exactly.
I use wsHttpBinding with reliableSession enabled. The InactivityTimeout is set to half an hour and I'm sure it's not exceeded, because exception is thrown earlier.
I solved the problem. The reason was RecieveTimeout on a server side. It was set to 1 minute, so after having no requests during 1 minute server used to close a channel, and when client tried to call a contract, channel was already crashed due to the timeout.
I found the solution after reading this article:
http://msdn.microsoft.com/en-us/library/system.servicemodel.reliablesession.inactivitytimeout.aspx
I received this error while setting up a new WCF service which returned a list of objects.
My understanding is that WCF services can only pass very simple objects back n forth.
So objects with anything other than public properties will not be transferable.
The object had a read only property doing a bit of logic.
Once I got rid of this, rebuilt, and updated the web references, the error went away.
Tip:
If you're returning a object and it has properties check the gets and sets of each one.
We had a problem around it.
I have seen this happen when an application pool gets recycled.
Look at the very last section of this blog about service recycling .
All,
I have a WCF web service (let's called service "B") hosted under IIS using a service account (VM, Windows 2003 SP2). The service exposes an endpoint that use WSHttpBinding with the default values except for maxReceivedMessageSize, maxBufferPoolSize, maxBufferSize and some of the time outs that have been increased.
The web service has been load tested using Visual Studio Load Test framework with around 800 concurrent users and successfully passed all tests with no exceptions being thrown. The proxy in the unit test has been created from configuration.
There is a sharepoint application that use the Office Sharepoint Server Search service to call web services "A" and "B". The application will get data from service "A" to create a request that will be sent to service "B". The response coming from service "B" is indexed for search. The proxy is created programmatically using the ChannelFactory.
When service "A" takes less than 10 minutes, the calls to service "B" are successfull. But when service "A" takes more time (~20 minutes) the calls to service "B" throw the following exception:
Exception Message: An unsecured or incorrectly secured fault was received from the other party. See the inner FaultException for the fault code and detail
Inner Exception Message: The message could not be processed. This is most likely because the action 'namespace/OperationName' is incorrect or because the message contains an invalid or expired security context token or because there is a mismatch between bindings. The security context token would be invalid if the service aborted the channel due to inactivity. To prevent the service from aborting idle sessions prematurely increase the Receive timeout on the service endpoint's binding.
The binding settings are the same, the time in both client server and web service server are synchronize with the Windows Time service, same time zone.
When i look at the server where web service "B" is hosted i can see the following security errors being logged:
Source: Security
Category: Logon/Logoff
Event ID: 537
User NT AUTHORITY\SYSTEM
Logon Failure:
Reason: An error occurred during logon
Logon Type: 3
Logon Process: Kerberos
Authentication Package: Kerberos
Status code: 0xC000006D
Substatus code: 0xC0000133
After reading some of the blogs online, the Status code means STATUS_LOGON_FAILURE and the substatus code means STATUS_TIME_DIFFERENCE_AT_DC. but i already checked both server and client clocks and they are syncronized.
I also noticed that the security token seems to be cached somewhere in the client server because they have another process that calls the web service "B" using the same service account and successfully gets data the first time is called. Then they start the proccess to update the office sharepoint server search service indexes and it fails. Then if they called the first proccess again it will fail too.
Has anyone experienced this type of problems or have any ideas?
Regards,
--Damian
10 mins is the default receive timeout. If you have an idled proxy for more than 10mins, the security session of that proxy is aborted by the server. Enable logging and you will see this in the diagnostics log of the server. The error message you reported fits for this behavior.
Search your system diagnostic file for "SessionIdleManager". If you find it, the above is your problem.
Give it a whirl and set the establishSecurityContext="false" for the client and the server.
Don't call the service operation in a using statement. Instead use a pattern such as...
client = new ServiceClient("Ws<binding>")
try
{
client.Operation(x,y);
client.Close();
}
catch ()
{
client.Abort();
}
I don't understand why this works but I would guess that when the proxy goes out of scope in the using statement, Close isn't called. The service then waits until receiveTimeout (on the binding) has expired and then aborts the connection causing subsequent calls to fail.
What I believe is happening here is that your channel is timing out (as you suspect).
If I understand correctly, it is not the calls to service A that are timing out, but rather to service B, before you call your operation.
I'm guessing that you are creating your channel before you call service A, rather than just in time (i.e. before calling service B). You should create the channel (proxy, service client) just before you use it like:
AResponse aResp = null;
BResponse bResp = null;
using (ServiceAProxy proxyA = new ServiceAProxy())
{
aResp = proxyA.DoServiceAWork();
using (ServiceBProxy proxyB = new ServiceBProxy())
{
bResp = proxyB.DoOtherork(aResp);
}
}
return bResp;
I believe however, that once you get over that problem (service B timing out), you'll realize that the sharepoint app's proxy (that called service A) will timeout.
To solve that, you may wish to change your service model from a request-response, to a publish-subscribe model.
With long-running services, you'll want your sharepoint app to subscribe to service A, and have service A publish its results when it is ready to do so - regardless of how long it takes.
Programming WCF Services (O'Reilly) by Juval Lowey, has a great explanation, and IDesign (Juval's company) published a great set of coding standards for WCF, as well as the code for a great Publish-Subscribe Framework.
Hope this helps,
Assaf.
I actually triggered this error just now by doing something silly. I have a unit test that modifies the system date in order to test some time-based features. And I guess the apparent time difference between when I created the context and when I called my method (because of the changes to the system date), caused something to expire.