Develop Reference

Validate SharePoint REQUESTDIGEST for a GET request?

I have a WCF REST service hosted in SharePoint that uses SSOM. The client web application sends AJAX requests containing the REQUESTDIGEST in a X-RequestDigest header. I am using the typical SPWeb.ValidateFormDigest() for my POST requests to prevent CSRF vulnerabilities. Works great.
Here's the weird part. Our client uses HP Fortify, and it is reporting that our GET requests are vulnerable to CSRF attacks. My GET requests are indempotent so this seems silly, but I must be compliant.
To get around it, I want to use ValidateFormDigest() in my WCF method for the GET request the same way I do for POST, but it throws this exception:
"Updates are currently disallowed on GET requests. To allow updates on a GET, set the 'AllowUnsafeUpdates' property on SPWeb."
I tried setting AllowUnsafeUpdates, but that just makes the request succeed without validating the digest!
Is there a way to have SPWeb validate the digest within a GET request?

HP Fortify gives recommendations, not edicts. If you think - and can prove - that the warning is spurious, then justifying the pattern should be treated as being compliant. Code analysis tools are not perfect.

ServiceStack JsonServiceClient, force traffic on the wire for localhost?

This ServiceStack client code works:
var client = new JsonServiceClient("http://localhost:32949/test");
var request = new MyRequest { ClassificationId = new ClassificationId (21300) };
var response = client.Post(request);
However, when observing the traffic in Fiddler, I see nothing. I would like to observe the traffic to get a better idea on how to build the required JSON request for other clients I have to write.
To make the above code work, I had to reference the assembly that has the service, and I am suspecting that ServiceStack is making some clever calls to avoid sending a HTTP request. Is this the case ?
Why am I not seeing any traffic in Fiddler, and how do I force it ?
HTTP traffic to localhost endpoints via the browser is shown correctly.

Edit your hosts file, located at
C:\Windows\System32\drivers\etc\hosts
and add the following entry
127.0.0.1 mymachine.com
then point your client to mymachine.com instead of localhost

I will answer my own question here - commenter #wal pointed out the problem to me:
This has nothing to do with ServiceStack, and requests actually go over the http protocol. The problem was looping back to localhost did not send the traffic through fiddler. It is actually explained on the Fiddler2 FAQ page.

The other trick is to replace your "localhost" uri with your machine name, and that should work out of the box with Fiddler.
http://machinename:port/test

Consuming a web service using POST instead of the going the usual WSDL route

This is how I have currently managed to consume a particular Microsoft web service. Notice that it is located on an HTTPS server and that it requires a username, a password, and a .cer file to be installed in the operating system's "root certificate authorities".
WSHttpBinding binding = new WSHttpBinding();
binding.Security.Mode = SecurityMode.TransportWithMessageCredential;
binding.Security.Message.ClientCredentialType = MessageCredentialType.UserName;
binding.Security.Message.NegotiateServiceCredential = true;
binding.Security.Message.AlgorithmSuite
= System.ServiceModel.Security.SecurityAlgorithmSuite.Default;
binding.Security.Message.EstablishSecurityContext = true;
EndpointAddress endpoint = new EndpointAddress("https://address.of.service");
//"GreatClient" was created for me automatically by running
//"svcutil.exe https://address.of.service?wsdl"
GreatClient client = new GreatClient(binding, endpoint);
//Username and password for the authentication. Notice that I have also installed
//the required .cer certificate into the system's "root certificate authorities".
client.ClientCredentials.UserName.UserName = "username";
client.ClientCredentials.UserName.Password = "password";
//Now I can start using the client as I wish.
My question is this: How can I obtain all the information necessary so that I can consume the web service with a direct POST to https://address.of.service, and how do I actually perform the POST with C#? I only want to use POST, where I can supply raw XML data using POST directly to https://address.of.service and get back the result as raw XML data. The question is, what is that raw XML data and how exactly should I send it using POST?
(The purpose of this question: The reason I ask is that I wish to consume this service using something other than C# and .NET (such as Ruby, or Cocoa on Mac OS X). I have no way of knowing how on earth to do that, since I don't have any easy-to-use "svcutil.exe" on other platforms to generate the required code for me. This is why I figured that just being able to consume the service using regular POST would allow me to more easily to consume the service on other platforms.)

What you are attempting to do sounds painful to do now and painful to maintain going forwards if anything changes in the server. It's really re-inventing the wheel.
If you haven't considered it already, I would:
(a) Research whether you can use the metadata you have for the service and use a proxy generator native to your target plaform. There aren't many platforms that don't have at least some tooling that might get you part of the way if not all of it. Perhaps repost a question targetting Ruby folk asking what frameworks exist to consume an HTTPS service given it's WSDL?
(b) Failing that, if your scenario allows it I would consider using a proxy written in C# that acts as a facade for the service which translates it into something easier to consume (for example, you might use something like ASP.NET MVC WebAPI which is flexible and can easily serve up standards compliant responses over which you can maintain total control).
I suspect one of these may prove easier and more valuable than the road you are on at the moment.

I had to go through something similar when porting .NET WCF code to other platforms. The easiest approach I found was to enable message logging on the WCF client. This can be configured to save both envelope and body and once everything is working on the .NET side of the house, you can use the message log to have "known-good" XML request/response to port to other platforms.
I found this approach to be more elegant since I didn't have to add an additional behavior to log messages, and it can be easily enabled/disabled/tweaked in the config. The Service Trace Viewer Tool that ships with Visual Studio is also handy for reviewing the log files.

I think when you say that the service should be consumed from other platforms, which do not have proxy class generation logic, you can go with REST services. This will allow you to create input as simple string concatenation instead of complex XML. Though its applicability depends on the situation.
Check this discussion : http://social.msdn.microsoft.com/Forums/en-US/wcf/thread/6907d765-7d4c-48e8-9e29-3ac5b4b9c405/
As far as the certificate is concerned, refer http://msdn.microsoft.com/en-us/library/ms733791.aspx on how to configure it.
I know this is not a very precise answer, but you will be the best person to evaluate above procedure, hence posted. Hope it helps.

What I'll do:
1- Create a small c# app that can post on this webservice (using svcutil). And modify it to show the XML send/received. To view the xml there are several ways: logging, wireshark etc. To add it directly to the small app there is another question here that give a good answer.
2- Once you know what you have to send, you can do it in c# like this:
// implement GetXmlString() to return the XML to post
string xml = GetXmlString();
// create the url
string url = new UriBuilder("http","address.of.service",80).ToString();
// create a client object
using(System.Net.WebClient client = new System.Net.WebClient()) {
// performs an HTTP POST
client.UploadString(url, xml);
}

I'm not a .NET programmer but I've had to interoperate with a few .NET services and have lots of SOAP/WSDL experience. Sounds like you've captured the XML for your service. The other problem you'll face is authentication. OOTB, .NET web services use NTLM for authentication. Open-source language support for NTLMv2 can be hit and miss (although a quick google search pulled up a few possibilities for ruby), and using NTLM auth over HTTP may be something that you have to wire together yourself. To answer a question above: where are the auth creds? If the service is using NTLM over the wire, authentication is happening at some layer below HTTP. If the service is using NTLM to authenticate HTTP, your NTLM creds are in the HTTP Authorization header. You should be able to tell with wireshark where they are. You'll also probably need a SOAPAction header; this can also be sniffed with wireshark. For the C# client, I'm sure there are docs explaining how to add headers to your request.

.NET Compact Framework, WCF service, compression and DIGEST authentication

I'm trying to put a number of features together, which is proving increasingly difficult due to the limitations of the .NET Compact Framework.
Specifically, I've got a WCF service and I'm writing a mobile device client for it. The catch? I want to use some sort of data compression (due to a very slow modem connected to said device) and HTTP DIGEST authentication (which is already in place on the site hosting the WCF service).
I've followed this blog entry to get the compression and generated code needed for the WCF service client.
I am, however, struggling with the HTTP DIGEST. I've no idea how to add this functionality.
Previously I didn't use compression and as such I connected to the WCF service using SOAP, using a simple WebService reference, and to add HTTP DIGEST I had to override the GetWebRequest method and add the required headers manually. This time around the generated classes seem to give very little leeway and there isn't much I can override. Also, all security or authentication parameters seem to be designed for SSL, rather than such basic authentication schemes.
To summarize: how can I create a WCF client using compression and HTTP DIGEST authentication using .NET Compact Framework?
EDIT:
Here's the code I've currently got:
System.ServiceModel.Channels.CustomBinding customBinding = new System.ServiceModel.Channels.CustomBinding();
CompressionMessageEncodingBindingElement compressionBindingElement = new CompressionMessageEncodingBindingElement();
customBinding.Elements.Add(compressionBindingElement);
HttpTransportBindingElement httpBindingElement = new HttpTransportBindingElement();
customBinding.Elements.Add(httpBindingElement);
EndpointAddress endPoint = new EndpointAddress("http://localhost:5100/Service.svc");
ServiceClient client = new ServiceClient(customBinding, endPoint);
I suspect I somehow need to add the HTTP DIGEST functionality to the CustomBinding class, but I don't know how.
I suspect I should also note, that while I am using .NET Compact Framework 3.5, I am creating a Windows CE application. As such, I didn't bother downloading Windows Mobile 6 SDKs. If these SDKs add more functionality which can be used in Window CE applications and are required for the HTTP DIGEST to work, please let me know.

We ended up disabling the DIGEST authentication for devices running .NET CF. It's not as safe, but we figured the data send and retrieved by the devices running .NET CF in our case isn't THAT sensitive, so all we really need to do is validate it.

If the client is running on the .NET Compact Framework 3.5, you can use WCF to invoke the service and use the built-in support for HTTP Digest authentication without requiring SSL.
Here's how to programmatically configure a WCF client to use Digest authentication with the BasicHttpBinding:
var binding = new BasicHttpBinding();
binding.Security.Mode = BasicHttpSecurityMode.TransportCredentialOnly;
binding.Security.Transport.ClientCredentialType = HttpClientCredentialType.Digest;
var endpoint = new EndpointAddress("http://server/myservice");
var client = new MyServiceClient(binding, endpoint);
// We have to set the actual credentials on the client proxy object
// before invoking the service:
client.ClientCredentials.HttpDigest.ClientCredential.UserName = "me";
client.ClientCredentials.HttpDigest.ClientCredential.Password = "password";
try
{
client.MyServiceOperation();
client.Close();
}
catch
{
client.Abort();
}
Related resources:
The WCF subset supported by the .NET Compact Framework 3.5
WCF Guidance for Mobile Developers (see page 66 for HTTP Digest Authentication)

The only way to achieve this is to use HttpWebRequest (manually) and specify ClientCredentials, instead of the generated classes from NetCFSvcUtil which does not support authentication.
The only WS-Security specification it supports on CF with WCF is to effectively use message security with a Mutual Certificate exchange. (Which by the way has a memory leak which a colleage and I found: http://connect.microsoft.com/VisualStudio/feedback/details/727247/native-memory-leak-in-wcf-proxy-client-with-mutual-certificate-security-in-net-compact-framework-3-5-on-windows-ce-6-0)
Of note, the generated CFClientBase also has a memory leak which can be worked around, see: http://geekswithblogs.net/GruffCode/archive/2013/03/31/memory-leak-in-cfclientbaselttgt-service-proxy-for-compact-framework-.net.aspx
For reference: The WCF subset supported by NetCF: http://blogs.msdn.com/b/andrewarnottms/archive/2007/08/21/the-wcf-subset-supported-by-netcf.aspx

ICerfiticatePolicy and ServicePoint

So I'm using the PayPal API. They require bigger companies to send an X509Certificate along with each SOAP API request. I've never heard of using a cert, it's always been just send the API signature along with an API request.
So I first created a class called Cerficate that implements the .NET ICerfiticatePolicy. One of the member methods, really the only one you have to implement is:
System.Net.ICertificatePolicy.CheckValidationResult(System.Net.ServicePoint, System.Security.Cryptography.X509Certificates.X509Certificate, System.Net.WebRequest, int)
So far I'm having trouble really understanding what to pass to this method. I guess the method simply validates that the Cerfiticate is valid. So I'm not sure what ServicePoint is and what to pass into it. I assumed it was my web service reference and a proxy class within such as the PayPalAPIAAInterfaceClient
I also see a very old example using ServicePointManager.S for something but I don't understand it, even after looking at MSDN. So I guess you are to use ServicePointManager.ServerCertificateValidationCallback and I suppose set the callback to the CheckValidationResult? If so, when do you do this? It's just very confusing to me.
Also, So I guess I create an instance of my Certificate class and set the certificate properties by reading the P12 certificate from my disk and then pass in that to this method to check if it's valid? I guess that's right.
I'm still trying to figure out this whole thing and I'm really stuck on the ServicePoint as well as WebRequest because really I'm using a proxy class in PayPal which does the under the hood sending of the request. So I don't see how I can even pass in type WebRequest because I'm using a proxy method for that anyway. So what would I even pass for the WebRequest param? I'm using a SOAP API WSDL, not NVP here so I'm not for example creating an HttpWebRequest variable like you do with REST services in order to send the API request over Http.
so far here's what I've tried:
PayPalAPIAAInterfaceClient client = new PayPalAPIAAInterfaceClient();
Certificate x509Certificate = new Certificate();
ServicePointManager.ServerCertificateValidationCallback = x509Certificate.CheckValidationResult();
client.ClientCredentials.ClientCertificate.Certificate = x509Certificate;
the problem is, what do I pass in for the ServicePiont and the rest of the params for CheckValidationResult?? I don't even know if I'm calling this right.

It's certainly not unheard of and in fact fairly common to secure SOAP services with X.509 certificates using the WS-Security spec - in fact, we do this for all of our internal and external web services. All web service frameworks including WCF are specifically designed to make this as easy as possible.
You should never have to use the ServicePointManager or ICertificatePolicy with a SOAP service using WS-Security. Unless there's something truly bizarre about PayPal's API, I think you're on the wrong track with that. All you have to do in WCF is this:
var client = new PayPalAPIInterfaceClient();
X509Certificate2 certificate = (...);
client.ClientCredentials.ClientCertificate.Certificate = certificate;
client.AddressVerify(...); // or whatever method you want to call
You don't even really need to write this code; if you have the certificate installed in the server's certificate store then you just edit the binding and behavior elements of the app.config - or use the WCF Service Configuration Editor, which is a lot easier.
Of course, in order to do this you have to have an X.509 certificate, and PayPal has to know about it. You can't just write new X509Certificate2(). You need to have a .pfx or .p12 file somewhere or, as mentioned above, have the certificate physically installed (this is the easiest way and the most secure because you're not hard-coding a password). And you need to upload the public key to PayPal.
You might be able to use OpenSSL to create a cert. PayPal's EWP page suggests that they'll accept these and gives instructions on how to create them, although it's not entirely clear whether or not the same process can be used for their SOAP API. It could be that they require a "real" certificate from Verisign, Thawte, etc. - I would try OpenSSL first and see, or just ask them.
There's a pretty comprehensive guide to the whole process here - you'll probably want to skip the sections on generating the certificate unless you have a Microsoft CA somewhere. Again, for that part, you'll probably want to try using the OpenSSL utility instead and follow PayPal's instructions, then install the cert on your server and skip to step 7 of that guide.