DocuSign Connect: Moving from XML (Legacy) to JSON - c#

Using C# and .NET (Framework)
We have a Production system for processing DocuSign Connect notifications which has been running great for over 2 years. We are currently setup to use the XML (Legacy) payload. I've taken the Connect payload schema (linked here) and generated the hierarchy of C# classes that represents the payload.
We have a Listener (webhook) and a Processor to handle the notifications:
The Listener does some security checks and a minimal parse (for validation) of the payload before queuing the notification for processing.
The Processor dequeues the notification and deserializes the XML to the DocuSign classes in order to process it.
As we understand it, in May 2023 we will no longer be able to get Connection notifications in XML, but we will have to be able to handle a JSON payload. In looking at some sample JSON payload data (displayed when setting up a Connect configuration), I'm not seeing any structural similarities to the XML format, so I doubt I'd be able to do what I'm currently doing (or similar):
var serializer = new XmlSerializer(typeof(DocuSignEnvelopeInformation));
var buffer = Encoding.UTF8.GetBytes(request.Content.ReadAsStringAsync().Result);
using (var stream = new MemoryStream(buffer))
{
var envelopeInfo = (DocuSignEnvelopeInformation)serializer.Deserialize(stream);
return envelopeInfo;
}
Question -- Is there a similar schema for the JSON payload that I can use to generate C# classes like I did with the XML payload?
I'm trying to get a rough estimate on how much effort this conversion is going to take. Any suggestions are appreciated.
If you need more information, please let me know.
thanks,
randy
UPDATE 01/05/2023: I was interested in seeing what JSON DocuSign would send me so I setup a simple Connect Configuration to use JSON, activated it, and then sent in a DocuSign request. I did not have any listener running, so I expected to see connect failures which was OK as I just wanted to see what DocuSign was sending. I completed signing the documents and then looked at the Connect Logs. There were 2 notifications: (1) was a JSON notification for the "recipient-sent" event and (2) was an XML notification for the envelope complete status. This XML notification is of the same type I'm getting currently and already know how to process. I'm surprised it wasn't a JSON equivalent of envelope complete. Would anyone know why I'm getting a mixture of JSON and XML?
Correction: I created a simplified Listener which only does some security checks, saves the payload to disk, and always returns success. I am getting a JSON payload for "envelope-completed" event. It looks like the XML version is being sent to a DocuSign "in-house" listener.
UPDATE #2 01/05/2023: I did run across several references to a handy feature of Visual Studio where you can put the JSON into the copy/paste buffer and then in Visual Studio use Edit -> Paste Special -> Paste JSON as Classes. I haven't tried deserializing any live data into these classes as I know these classes aren't complete -- they only reflect the structure that exists in the data you use at the time. I can already see that it only generated a few of the tab classes (Sign Here, Date Signed, Text, and Checkbox) but it doesn't know about all the other possible tabs. And several properties were just defined as "object" since the JSON I used didn't contain values for them. That's why I was hoping for a definitive schema for the JSON DocuSign intends to use for Connect notifications.

I was able to get back to this and I'll post what seems to be working.
I created a support ticket to DocuSign and the Support representative was helpful and passed along some information from the developers. There doesn't seem to be any "official" schemas for the JSON notification so I decided to work on my own.
I gathered as many examples of JSON notifications as I could. DocuSign has some here. Plus, as mentioned above, I could gather more by sending out DocuSign requests for the documents we primarily use and saving the JSON notifications I got back as I went through the signing process.
I then used the Visual Studio feature about generating C# classes from JSON input (also mentioned above). For each JSON notification I obtained, I generated the corresponding C# classes. After selecting one set as the most complete, I started merging each new set with the old (my favorite merge tool is Araxis Merge) when there were new classes, properties, enums, etc. to be added.
-- WARNING: I've found there are some XML property values which don't exist in the JSON version, or at least I haven't found where they are. Example: the VoidReason string.
I believe what I have will meet our requirements. My listener and processor have been updated (still in Development) to handle both XML and JSON notifications and there is still a lot of testing to do before I feel it's ready for Production.
If there's anything I can answer on this process, please let me know.
thanks, randy

Related

How do i set up a SOAP webservice

First off a little context, I work for a channel manager company that builds custom endpoints for their clients to talk to. We currently mostly code in c#.
This is the first time we are connecting with a client that asked us to support a soap web service.
Sadly, we don't have previous experience in this and I can't find any concrete explanations to what it looks like to support a SOAP call.
Currently, it looks like that I have to simply use text processing methods to run through the received XML and parse it in a non-generic way.
But this feels like there should be a more straightforward approach since the protocol is almost 20 years old.
For almost every search result I only get examples of what the XML looks like, but what I'm interested in, is what I need to implement in my application to host an endpoint for the client to successfully post and receive a SOAP message from. And if there is a way to do this without something like XmlWriters/Readers.
Anything to help me get on my way mary is appreciated!
Greetings, Davey
WCF is the least outdated library for offering SOAP.
When you can define the contracts then you're in luck, that's just writing a C# interface.
But when necessary you can also import WSDL specifications and generate C# from that.
it looks like that I have to simply use text processing methods to run through the received XML and parse it
Don't even consider that. Everything is under control of schemas. And you will also have to generate valid return packages.

Confused about what exactly a REST-based API is

I am trying to understand what exactly a REST-based API is. From what I understand it is just a convention for writing the functions within the API? All functions should be of either GET/POST/DELETE/PUT form? So, for example a function in a REST API could be
public string getLastName(User x)
{
return x.lastName;
}
I am mainly confused about how JSON/XML play a role in this?
Its more than just a convention. The concept behind a REST API is that you access it using the HTTP verbs, and that the functions those verbs have been mapped to perform the described action.
For example:
GET will return data to the caller/sender
DELETE will delete a record
And it goes further, but a lot of it is based on relying on HTTP to provide a level of consistency. For example, in a RESTful service, you might use the Accept HTTP header to request a JSON response or an XML response by supplying the application/json or application/xml values, respectively. This is just a simple example, and it is up to the implementer to decide how their API will work, but it highlights the importance placed on leveraging HTTP.
Why JSON/XML?
Along the same lines, JSON and XML are used because they are widespread and standard ways of representing and transmitting data over the web. JSON (JavaScript Object Notation) is very common in doing data transfer (especially on GET requests) due to most requests coming from JavaScript, and JS can easily interact with JSON without having to do the parsing required when dealing with XML. On the other hand, XML provides its own benefits, such as the ability to use schemas and namespaces. You may already be aware of benefits/drawbacks of each, but that's a separate discussion. The main point is that JSON/XML are the primary ways of communicating data in a REST API due to both of them being the de facto standards of the web.
There are lots of good resources for more information, this MSDN article may be helpful: http://msdn.microsoft.com/en-us/library/dd203052.aspx
There's a lot of confusion and misconceptions around REST. Unfortunately, it's a lot more common to find applications that are doing the exact opposite of what REST means and calling themselves RESTful than real REST applications.
From what I understand it is just a convention for writing the functions within the API?
No, REST is not just a convention for writing functions within the API, nor it's directly related to SOAP or HTTP as other answers here say. REST is a software architectural style inspired by the successful design decisions made for the web itself. To put it in simple terms, a REST API should work like a website does.
When you enter a website, you go to a homepage having an idea what the website is about, and the HTML document will have hyperlinks pointing you to the resources you need. The only out-of-band information are the media-types of the resources themselves, not how to find them. For instance, when you enter StackOverflow, you know what questions and answers are, and you look for links pointing you to them. How your browser render those links, how you choose them and follow them isn't different from other websites, like an email or news website. What makes it different is the media-type of the resources you're after.
That's how a REST API should work. Clients should not depend on any out-of-band information other than detailing what the resources do. They should connect to a home page that returns them links they should follow to do whatever they need. If an API doesn't do this, then it's not REST. Period.
I like to call those APIs "street-REST", because people often implement them by copying what they see in other APIs that also call themselves REST, and by what other people talk about REST.
All functions should be of either GET/POST/DELETE/PUT form?
That's a common confusion, and you'll see a lot of that, including people conflating that to CRUD operations, or that REST doesn't allow any other verbs, etc. That's bull.
REST is independent of any particular protocol, so it doesn't make sense to say functions should follow HTTP methods. REST constrain your applications into an uniform interface, meaning that whatever protocol you should using, you must stick to the standardized behavior as much as possible. If you're implementating a REST application over HTTP, this means your API must stick to the HTTP methods for the client-server interaction, meaning you can't invent other HTTP methods as some applications using HTTP do. If you change the communication protocol, clients need to know that information before entering your API, and that's more out-of-band information.
How you implement this on your code is irrelevant to REST. REST isn't a development pattern or philosophy, but an architectural style.
I am mainly confused about how JSON/XML play a role in this?
There's a lot of confusion on this too. On a REST application you should define abstract entities with states describing all the behavior you need. The API will serve as a channel to transfer media-type representations of those entities between client and server. REST means Representational State Transfer. The URI the client is requesting is an identifier for that resource, and the metadata for that request will tell the server what media-types the client is prepared to accept. JSON/XML don't play any direct role in REST, at all. They are simply one representation media-type that is easier for computers to parse and obtain information, in contrast to formats like text/html, which is meant to be rendered for human visualization by a browser.
For example, take StackOverflow itself. What's a question, in the abstract sense? It's a request for information. How that abstract resource is formally defined? There's an author, there's the actual text of the question being asked, there's the upvotes and downvotes, the comments and possible answers, etc, all of which are also abstract resources with their own definitions. The actual data is stored in a database somewhere, and when you request your homepage, it returns links with URIs identifying those questions. Take this question for instance, it has the URI http://stackoverflow.com/questions/24092517. When I want to read that question in a pretty document rendered on my browser, I will request that URI, but telling the server through the Accept header that I want a text/html representation, and my browser knows how to render the HTML into a pretty page. On the other hand, when I want to request that question to store it back on a database, for instance, I don't need all the cute stuff needed to render a pretty document page, so I ask a format that's easier to parse and doesn't contain a lot of unnecessary information, like JSON or XML.
Most people who build street-REST APIs understand this point, but they miss the most interesting part, which is that you're not limited to media-types that already exist. You can create private media-types that only exist within your API, or your company's ecosystem of applications. So, for instance, instead of calling the media type of all JSON documents application/json, no matter the content of them, we could have custom media-types that reflect more accurately that particular type of resource. So, we could have a application/vnd.stackoverflow.question.v1+json for StackOverflow questions represented in a JSON format. Once you do that, instead of wasting time documenting operations already standardized by the communication protocol, you should focus on documenting that custom media-type and how to interact with it, independently of the communication protocol. Once you have that clear, clients can interact with your services using any protocol available.
If you understand these three main points, you understand what REST is about. By using hyperlinks as the engine of your application state you're not tied to any particular point in time of your implementation. Your server can evolve at will, you can change URIs as much as you want, and clients won't break. By sticking to the standardized protocol, it's easier for generic clients to make use of your API, not to mention that it's easier for developers to understand how to integrate if they already know that you won't break the protocol. By focusing your design and documentation efforts on your media-types, not on protocol details and URI semantics, you're avoiding introducing more out-of-band information needed to drive your API, and your clients are also more resilient to changes.
REST API's act as middleman to deliver data packets between the web service and an application wanted to use some resource from web service for its operations, in order to do so Json comes into play, which helps in data transfer/communication over the channel. When one's application wishes to get list of resource, it actually get complex data or queryset from database which turn into simple data type with serialization and then turn into json to travers the channel.
RESTful API is much more than just the convention of writing functions.
The abbreviation REST stands for "REpresentational State Transfer".
REST APIs are used to call resources and allow the software to communicate based on standardized principles, properties, and constraints. REST APIs operate on a simple request/response system. You can send a request using these HTTP methods:
GET
POST
PUT
PATCH
DELETE
Also, REST APIs can include endpoints, headers, URL parameters, and the request body.
The endpoint (or route) is the URL you request for. The path determines the resource you’re requesting for. Think of it like an automated answering machine that asks you to press 1 for service, press 2 for another service, 3 for yet another service, and so on.
I am mainly confused about how JSON/XML play a role in this?
When you send a request to an endpoint, it responds with the relevant data, which is generally formatted as JSON, XML, plain text, images, HTML, and more.
I am mainly confused about how JSON/XML play a role in this?
JSON/XML are called streaming data format. There are others but over the years these two became so popular because of low latency they provide. XML is probably still little bit more popular than JSON, but JSON is more compact.
Also another main reason to use them is because they are both supported by almost all languages and their frameworks.

Usage of JSON for a daily activity journal

To keep track of my new year resolutions I created a file daily.log in the following format.
8:40 AM 1/2/2013
begin:755am
activity:enquired about 3x3 black board;bought book [beginning html 5]
waste:facebook;
meeting:old friend;mechanic
programming:none
blogpost:[asp.net deployment]
do:buy black board
done:
end:1045pm
I am in the process of creating a simple C# console application which would ask me a few questions and fill this file accordingly. One of the future features to this tool would be to display a simple dashboard style web page for measuring the progress of resolutions among other things.
I would to like to use a data serialization or configuration file format for storing daily activity information in this manner, because mature tools are available for these formats rather than for plain text.
I never used JSON before and am wondering whether the JSON format can be used independently with C# (no javascript involved), and even if I can, whether the usage of JSON is appropriate in this case.
If not JSON, its superset YAML? or are any other alternatives that suit well for this purpose?
You can use JSON.NET in C# without using javascript. And I believe this data can be modeled in JSON format.
If your goal is to work with external tools to have them recognize and be able to work with your files, a better bet than JSON would be to use XML. This format is stricter (and you can use XML Schema to validate the format) and there are way more tools that are able to work with XML than there are for JSON.
The .NET Framework also contains extensive support for XML, in the System.Xml namespace (see http://msdn.microsoft.com/en-us/library/system.xml(v=vs.100).aspx).
That being said, there is no reason why JSON would not work with C#. I have personally used the JSON.NET library for most JSON work and it works beautifully (see http://james.newtonking.com/projects/json-net.aspx). Mind you, the data you show in your example is not valid JSON.
Good luck!

Keep a WCF transferred object deflated / serialized [duplicate]

This question already has answers here:
Closed 10 years ago.
Possible Duplicate:
Get Just the Body of a WCf Message
Good Morning,
I have a WCF service that build a complex object then sends it to the consuming client. Currently the service works great but the xml is reinflated / deserialized on the client and I don't want this as the client wants to work with the XML.
So is there a way to get the "XML document" that is transmitted as a part of the WCF service response. Of course I could just serialize it again on the client but really why do extra steps if they can be avoided.
TIA
JB
Although there are possiblities to hook into the multiple WCF receive/send steps on both server and client sides, where you could influence on how the serialization is executed (not not executed in your case), I believe that would be too much overhead for your scenario.
You'd be better of to simplify the workflow and stick with the out-of-the-box functionalities.
I think the easiest approach would be to manually serialize your objects only on the server side and simply send the result as a string back to the client.
To remain flexibility (you might later want to add more data accompanying the serialized data) put this serialized XML into a property of dedicated Data Transfer Object (DTO).
This DTO is then send to the client who simply reads the XML from that property.
Under the hood there still is, of course, some serialisation going on. But this would be the built-in functionality hat is totally transparently executed by the WCF service/ client communication. If you do not have a serious reason to do influence it manually, leave it as it is. If you do not know what you are doing you might instead open the box of Pandora...
Sending a simple object like the above described DTO should not be much of a deal here. I mean, this scenario would mainly be sending a bunch of strings over the wire, which should not cause any troubles and very little overhead.

When should XML serialization be used?

I've been doing some reading up on XML serialization, and from what I understand, It is a way to take an object and persist the state in a file. As for the implementation, it looks straight forward enough, and there seems to be a load of resources for applying it. When should XML serialization be used? What are the benefits? What are situations that are best helped by using this?
The .NET XmlSerializer class isn't the only way to persist an object to XML. The newer DataContractSerializer is faster, and also allows an object to be persisted to a binary form of XML, which is more compact.
The XmlSerializer is only getting limited bug fixes these days, in part because so much code depends on the precise details of how it works, in part because it is associated with ASMX web services, which Microsoft considers to be a "legacy technology".
This is not the case with the DataContractSerializer, which continues to be a vibrant and important part of WCF.
You've answered a little bit of the question in your post. It's great for persisting the state of an object. I've used it in applications for saving user settings. It's also a great way to send data to other systems, since it is standardized. An important thing to remember is that it is easily human readable. This can either be a good or bad thing depending on your situation. You might want to consider encrypting it, or using encrypted binary serialization if you don't want someone else to be able to understand it.
EDIT:
Another gotchya worth mentioning is that the .NET implemented XMLSerializer only serializes public members in a object. If you need to persist private or protected members, you will either need to use a customized serializer or use another form of serialization.
Its good for communication between disparate systems. E.G. take a Java app and a C# app and allow them to communicate via a webservice with serializeable XML objects. Both apps understand XML and are shielded from the details of the other language. And yes while you could fire strings back and forth, XML gives us strong typing and schema validation.
this is just from personal experiences - XML serialization is good for web services.
Also, if you want to modify (or allow the modification of) the object/file that you're storing to without using the application that you're writing (i.e third party app), XML can be a good choice.
I send an array of Objects of type class I wrote, using HttpWebRequest, so I cant send it as an object , because im mixing HttpWebRequest + Soap (that Im writing), and in Soap you cant send a non predefined Objects as String, int , ... .
so I used XML serialization to convert my object to an XML string and send it through my HttpWebRequest .

Categories

Resources