I have a WCF service hosted that is returning a byte[] of protobuf-net serialized data. Originally this was hosted over the default webHttpBinding settings (SOAP) and everything was working properly. I recently found out that I need to call HTTP GETs and POSTs directly from our client so I thought it'd be easier to switch to use a RESTful service. I switched to use the WebGet attribute and the WCF REST Template.
I attempted to create a simple web site to test the client and I'm having trouble deserializing the data. This is an example of how I'm calling the service:
using (WebClient client = new WebClient())
{
result = client.DownloadString(url);
}
// Deserialize
BinaryVehicles binVehs;
using (var ms = new MemoryStream(StrToByteArray(result)))
{
binVehs = Serializer.Deserialize<BinaryVehicles>(ms);
}
An example of what is returned in "result":
< base64Binary xmlns="http://schemas.microsoft.com/2003/10/Serialization/">ChsKCzEyMy00NTYtNzg5EgU0NDAwMBoFQmxhY2sKHAoLOTYzLTg1Mi03NDESBTIzMDAwGgZTaWx2ZXI=< /base64Binary>
I also attempted to deserialize the data between the < base64Binary > tags with no results. Does anyone have any idea on how I should go about sending the binary protobuf-net data from an WebGet method and how I should deserialize the data? Thanks.
protobuf-net primarily handles just the serialization aspects (the protobuf spec by Google doesn't define any further than this). So it really comes down to: how are you serializing it?
I must admit that the WCF GET approach is not something I've looked at hugely, so there is no special handling there. One simple approach may be to look at just returning a string, and handling the base-64 encoding yourself, but to be honest if you are doing HTTP GET, then WCF itself seems overkill.
I blogged here about using ASP.NET MVC at the server for protobuf via HTTP GET. The linked sample code also includes a wire-compatible ASP.NET implementation.
If there is something appropriate we can do to make WCF GET easier, I'm all ears...
Related
EDIT:
I'm based on some comments that have been deleted, I guess I need to add a service reference and create a instance of the webservice object in the client application to pass the dataset. I haven't been able to add the service reference. When the service runs, it gives a 403 forbidden error, and has done so since I first created the project. I thought this was fine because the apicontroller methods can still be accessed with an HTTP GET, but I have no way of actually sending objects to the service, and can't create the service reference. The service and the client are in separate projects. At this point even a link to a decent tutorial would help. I've found a ton of stuff for creating a full site with a few functions accessed in jquery, and plenty of stuff that includes how to send a post in C#, but nothing that combines both sides.
I saw this question asking a how to do a similar thing. Is there any difference in the client side code I have in c# and the javascript in the question?
EDIT2:
I tried using a memory stream to look at the request content to see if the entire dataset was being transferred. Its only picking up 4000 or so bytes out of 350000 being transferred. I assume this is why the dataset isn't availalbe in teh webservice parameter. but I'm not sure how to continue.
Original:
I'm building a webservice to handle the database calls for my client application. This isn't publicly facing, so I'm fine with passing a dataset instead of a custom model. Heres the client code:
public static void TestSendToDatabase(DataSet Products)
{
string xml = ToXML(Products);
byte[] bytes;
bytes = System.Text.Encoding.ASCII.GetBytes(xml);
StringBuilder urlBuilder = new StringBuilder(string.Format("{0}{1}", BaseUrl, "WalmartProducts/Update"));
string url = urlBuilder.ToString();
HttpWebRequest request = (HttpWebRequest)WebRequest.Create(url);
request.Accept = "application/xml";
request.Method = "POST";
request.ContentType = "application/x-www-form-urlencoded";
request.ContentLength = bytes.Length;
Stream requestStream = request.GetRequestStream();
requestStream.Write(bytes,0,bytes.Length);
HttpWebResponse response = (HttpWebResponse)request.GetResponse();
And the service code:
public class WalmartProductsController : ApiController
{
[HttpPost]
[ActionName("Update")]
public object UpdateWalmartProducts([FromBody] byte[] bytes)
{
string xml = System.Text.Encoding.ASCII.GetString(bytes);
DataSet productsFromClient = LoadFromXMLString(xml);
DataUpdater.UpdateWalmartData(productsFromClient.Tables[0]);
return DataUpdater.GetProducts();
}
The byte array is empty in the web service. I'm sure that I'm mixing methods that shouldn't be mixed, but I've gone through at least thirty articles trying to resolve this, and it feels like each one has a completely different method. The webservice is currently returning the dataset in an xml format, so I would prefer to stick with xml over json.
This is not a complete answer, but perhaps it is a starting point. .NET supports two main techniques for writing Web Services. The first is an older technique using .asmx files. This approach is generally called the SOAP approach, and Visual Studio generates signatures of your methods in a .wsdl file. Because the signatures are exposed, you can then ask Visual Studio to create a Web Reference file for you. This web reference exposes all the Web Services as simple classes and methods, hiding all the complexity of the communication across the web service. Any object that can be serialized, including the DataSet object, can be included as a parameter to these web methods, and .NET will take care of serializing and deserializing these objects as they pass across the HTTP boundary. I believe that MS then encapsulated a number of different communication technologies inside of a product they called WCF, which was supposed to allow you to use configuration to choose whether you wanted to use HTTP for communication or TCP/IP.
The downside to this approach is that if you use any MS class in your API's, you have effectively tied your product to MS, and it is then very difficult for any non-MS client to communicate with your web service. The DataSet object is an extremely complicated one, and I'm under the opinion that it would be quite impractical to attempt to write your own serialization/deserialization methods to send a DataSet.
The other Web Service approach that is gaining popularity is using REST API's. Recently, MS added support for these, and the ApiController is one of the starting points. If you google "ASP.NET REST API Tutorial" you should find all kinds of examples. However, with this approach I would discourage you from using the DataSet class. I've had some success with classes such as a Dictionary, provided the object classes you put into the dictionary all resolve down to fairly simple components. Attempting to use a complex MS class like DataSet is likely to lead to a lot of frustration, and will yield a Web Service that very few clients can use.
I am working on a bunch of Web APIs which must have a latency of a single digit millisecond! To produce the response I am using protobuf which is great. My question is that can protobuf be used to de-serialize the request as well?
For example for such an API:
public async Task<List<Artist>> Search(SearchArtistRequest request)
I will write a SDK (a bunch of .DLL) which will make the call to this API and third parties will only use the .DLL. So if I could send my request message in protobuf format and de-serialize 'request' using protobuf then I might gain some performance improvements (given that by default Json.NET is used and its performance is aweful). Is there a way to do so?
You can create your own serializer, test it and replace existing one with yours in WebAPI.
There are a couple of resources about custom serialization in WebAPI.
Look here ("Testing Object Serialization"), Media Formatters and Replace...serializer.
I have a WSDL definition for a SOAP service and I have successfully generated *.cs file from it using SvcUtil.
Implementing client is quite straightforward - I just need to call the necessary functions from the generated *.cs and that's it.
Implementing server seems more complicated. As I understand I need to implement an interface from the generated *.cs and then use some magic to turn it into the web server.
But I don't need a new web server, I already have a web server written in C# which already has many functionality unrelated to the SOAP service that I need to implement. I don't want to create another web server but I want my SOAP service to be just a part of my existing application (server), that is my server can answer e.g. requests http://example.com/request1, http://example.com/request2 etc. and I want this SOAP service to be just http://example.com/request3.
Since HTTP is already handled by my server I don't need .NET to handle it for me, basically my server can accept client connections and call the necessary handler based on the URL. I have a handler for SOAP request which looks approximately like this:
MyResponse HandleSOAPRequest(MyRequest request)
{
// 1. parse soap message from request.body
// 2. process it
// 3. generate response, serialize it in SOAP format and return it
}
The question is - can I rely on WSDL definition and .NET libraries to do it?
Currently I'm parsing SOAP request using XDocument and manually extract fields from it and serialize using simple string concatenation. Using .NET built-in functions to serialize or parse XML doesn't work. That is if I try to serialize response from an object of the class defined in the generated *.cs file then produced XML is different from what is expected by the protocol, similarly, if I try to parse request as an object of the class defined in the generated *.cs file I get error because XML parser expects different format. This applies to both the SoapFormatter and XmlSerializer.
Since .NET can implement client this means that everything that is necessary to parse and serialize SOAP messages is already implemented, I just need to figure out a way how to use this functionality.
The documentation for ServiceModel wasn't very helpful.
The easiest way would be to start the service via the ServiceHost:
ServiceHost host = new ServiceHost(typeof(YourService));
host.Open();
(I assumed here the configuration will come from the app.config and will not be setup in code like in the linked example below.)
How to: Expose a Contract to SOAP and Web Clients
The only drawback of this is that the application has to run with admin rights or otherwise a weird cofiguration is necessary.
I'm trialling out ServiceStack and loving what I'm seeing so far. However I've run into a bit of a brick wall.
I have a system retrieving data from another system via web services - a service at both ends. These two systems are from different vendors - so I have no control over changing them - and are configured to talk to each other via WCF web services. Let's say "Lemon" calls "Orange" to get some information about a customer.
The way we implement these two systems is slightly different to what the vendors planned - we point their service configuration to our intermediary service - let's call it "Peach" - which goes off and does some other things before returning the information. For example, "Lemon" calls what it thinks is "Orange" but is actually our intermediary service "Peach" using the same method names. "Peach" calls "Orange" for the customer information and for example overrides the email address for the customer with something else before combining all the information appropriately and returning it to "Lemon" in the format it was expecting.
I would like to get "Peach" using ServiceStack. However it's responses needs to be identical to a WCF service returning via wsHttpBinding. Is this possible with ServiceStack? Would it involve overriding the Soap 1.2 type?
Thanks for your help!
If ServiceStack's built-in SOAP Support doesn't return the response you're after, you may need to return the exact SOAP response you're after as a raw string.
Raw Access to WCF SOAP Message
To access the WCF's Raw Request in your Service you can use the IRequiresSoapMessage interface to tell ServiceStack to skip de-serialization of the request and instead pass the raw WCF Message to the Service instead for manual processing, e.g:
public class RawWcfMessage : IRequiresSoapMessage {
public Message Message { get; set; }
}
public class MyServices : Service
{
public object Post(RawWcfMessage request) {
var requestMsg = request.Message... //Raw WCF SOAP Message
}
}
Creating custom WCF Response
Some level of customization is possible by creating a custom WCF Message response and returning the raw output as a string, e.g:
var wcfResponse = wcfServiceProxy.GetPeach(...);
var responseMsg = Message.CreateMessage(
MessageVersion.Soap12, "urn:GetPeach", wcfResponse);
return responseMsg.ToString();
Otherwise you may need to use a HTTP Client like Http Utils to POST raw SOAP to the WCF Service and return the raw string Response.
This question isn't specifically related to WCF. What WCF returns is not a construct of WCF, it is returning a standards based response as specified by the WS* standards. http://en.wikipedia.org/wiki/List_of_web_service_specifications lists many of the standards.
Your question isn't specifically making ServiceStack emulate WCF, it is for ServiceStack to return responses adhering to existing published standards. Standards that WCF has already built in with the WsHttpBinding configuration.
This is how I have currently managed to consume a particular Microsoft web service. Notice that it is located on an HTTPS server and that it requires a username, a password, and a .cer file to be installed in the operating system's "root certificate authorities".
WSHttpBinding binding = new WSHttpBinding();
binding.Security.Mode = SecurityMode.TransportWithMessageCredential;
binding.Security.Message.ClientCredentialType = MessageCredentialType.UserName;
binding.Security.Message.NegotiateServiceCredential = true;
binding.Security.Message.AlgorithmSuite
= System.ServiceModel.Security.SecurityAlgorithmSuite.Default;
binding.Security.Message.EstablishSecurityContext = true;
EndpointAddress endpoint = new EndpointAddress("https://address.of.service");
//"GreatClient" was created for me automatically by running
//"svcutil.exe https://address.of.service?wsdl"
GreatClient client = new GreatClient(binding, endpoint);
//Username and password for the authentication. Notice that I have also installed
//the required .cer certificate into the system's "root certificate authorities".
client.ClientCredentials.UserName.UserName = "username";
client.ClientCredentials.UserName.Password = "password";
//Now I can start using the client as I wish.
My question is this: How can I obtain all the information necessary so that I can consume the web service with a direct POST to https://address.of.service, and how do I actually perform the POST with C#? I only want to use POST, where I can supply raw XML data using POST directly to https://address.of.service and get back the result as raw XML data. The question is, what is that raw XML data and how exactly should I send it using POST?
(The purpose of this question: The reason I ask is that I wish to consume this service using something other than C# and .NET (such as Ruby, or Cocoa on Mac OS X). I have no way of knowing how on earth to do that, since I don't have any easy-to-use "svcutil.exe" on other platforms to generate the required code for me. This is why I figured that just being able to consume the service using regular POST would allow me to more easily to consume the service on other platforms.)
What you are attempting to do sounds painful to do now and painful to maintain going forwards if anything changes in the server. It's really re-inventing the wheel.
If you haven't considered it already, I would:
(a) Research whether you can use the metadata you have for the service and use a proxy generator native to your target plaform. There aren't many platforms that don't have at least some tooling that might get you part of the way if not all of it. Perhaps repost a question targetting Ruby folk asking what frameworks exist to consume an HTTPS service given it's WSDL?
(b) Failing that, if your scenario allows it I would consider using a proxy written in C# that acts as a facade for the service which translates it into something easier to consume (for example, you might use something like ASP.NET MVC WebAPI which is flexible and can easily serve up standards compliant responses over which you can maintain total control).
I suspect one of these may prove easier and more valuable than the road you are on at the moment.
I had to go through something similar when porting .NET WCF code to other platforms. The easiest approach I found was to enable message logging on the WCF client. This can be configured to save both envelope and body and once everything is working on the .NET side of the house, you can use the message log to have "known-good" XML request/response to port to other platforms.
I found this approach to be more elegant since I didn't have to add an additional behavior to log messages, and it can be easily enabled/disabled/tweaked in the config. The Service Trace Viewer Tool that ships with Visual Studio is also handy for reviewing the log files.
I think when you say that the service should be consumed from other platforms, which do not have proxy class generation logic, you can go with REST services. This will allow you to create input as simple string concatenation instead of complex XML. Though its applicability depends on the situation.
Check this discussion : http://social.msdn.microsoft.com/Forums/en-US/wcf/thread/6907d765-7d4c-48e8-9e29-3ac5b4b9c405/
As far as the certificate is concerned, refer http://msdn.microsoft.com/en-us/library/ms733791.aspx on how to configure it.
I know this is not a very precise answer, but you will be the best person to evaluate above procedure, hence posted. Hope it helps.
What I'll do:
1- Create a small c# app that can post on this webservice (using svcutil). And modify it to show the XML send/received. To view the xml there are several ways: logging, wireshark etc. To add it directly to the small app there is another question here that give a good answer.
2- Once you know what you have to send, you can do it in c# like this:
// implement GetXmlString() to return the XML to post
string xml = GetXmlString();
// create the url
string url = new UriBuilder("http","address.of.service",80).ToString();
// create a client object
using(System.Net.WebClient client = new System.Net.WebClient()) {
// performs an HTTP POST
client.UploadString(url, xml);
}
I'm not a .NET programmer but I've had to interoperate with a few .NET services and have lots of SOAP/WSDL experience. Sounds like you've captured the XML for your service. The other problem you'll face is authentication. OOTB, .NET web services use NTLM for authentication. Open-source language support for NTLMv2 can be hit and miss (although a quick google search pulled up a few possibilities for ruby), and using NTLM auth over HTTP may be something that you have to wire together yourself. To answer a question above: where are the auth creds? If the service is using NTLM over the wire, authentication is happening at some layer below HTTP. If the service is using NTLM to authenticate HTTP, your NTLM creds are in the HTTP Authorization header. You should be able to tell with wireshark where they are. You'll also probably need a SOAPAction header; this can also be sniffed with wireshark. For the C# client, I'm sure there are docs explaining how to add headers to your request.