We have a client/server system where all communications are done using a native protocol over a binary/SSL stream on TCP. All of our code is written in C# .NET 2.0 and some parts in 3.5. Our protocol is designed to support a variety of messaging patterns, namely Request/Response, and for lack of a better term, one-way messages from either the client or the server on an irregular basis.
Our objective is to add a feature to our system to carry our protocol over HTTP. There are several reasons for doing so, but I don't need to explain that here I think. Please tell me if I should.
The thought is to embed our protocol as application/binary in HTTP requests using the standard request methods ie., GET, PUT, POST, but not DELETE, and following the HTTP specification. This would be rather straight forward to do if our protocol was only request/response. The main concern comes from the one-way messages, and more specifically the unsolicited messages coming from the server. Another important concern is that HTTP is not oriented for persistent connections, but I believe with HTTP/1.1 this can be overcome. A third concern is that the server connections are not stateless.
We've been designing and prototyping this for a couple weeks, and we've come up with a couple ideas:
Refactor the code in the communication and protocol layers on both server and client sides. Although much of the code is shared, it is a lot of work that in all likely hood will not be a success. The question here is can this even be done with our poorly designed protocol?
Use a proxy approach. That is create an HTTP server using WCF and unwrap the HTTP messages and relay the native messages to and from our server over persistent connections. This would require a layer of abstraction on the client side that would actually maintain two connections to the proxy. One connection to perform request/response and the other to carry the unsolicited messages using a delayed response technique.
HTTP Tunneling which we haven't yet researched.
We're hoping that someone has encountered this challenge before and could lend some sound advice?
Please accept my apologies if this is the wrong place to post this question.
For the server initiated messages, you might consider WebSockets. According to Scott Guthrie's blog there is support for web sockets in ASP.Net MVC 4 beta.
Related
yes,there is so many sample for a tcp server,but I can't find one use scala future or c#/f# async/awite
Is future/async suitable for writing a simple tcp server,like echo server?
or if there is a server/client modle like smtp,the server and client will talk many times in a session(helo/ok from/ok rcpt/ok data/ok quit/ok),is future or async suitable for this modle?is there some possible that the serve A first get HELO from client A but then talk to client B with other smtp command like "mail from"?
where to find the sample code that a echo server use Future or async/awit/Task?
Thanks!
TCP servers are pretty complex beasts, and since the data is a stream (not a series of bounded packets), it isn't usually directly amenable to just an "await some data" API. The raw sockets API has an async component, but it isn't the shape you would normally expect.
Kestrel can be used to construct an async TCP server using the "pipelines" API, which deals with all the things like back-buffer management (for when you can't yet consume an incomplete frame) - I have a blog series here on that, however: no official client-side API exists (yet) for "pipelines"; again, the same blog series discusses how you can use Pipelines.Sockets.Unofficial to bridge that gap.
However! I wonder if what you really want here is a message passing library that sits on top of the TCP layer, so you can just say "send a message" and "await a reply message". Many such libraries exist, but they invariably change the shape of the underlying data protocol, as they will be introducing "framing" etc. This means that they may not be suitable choices if you intend to implement a pre-existing protocol (as the choices won't align).
I'm new to gRpc and now learning this tech
I'm wondering if gRpc can replace SignalR for updating notification bar on my client-side app (React).
Is that the case? or should I keep using SignalR for this matter?
(I'm asking it just to make sure I understand the purpose of the gRpc tech, by few articles I read it's more a web API replacement and few others compare it to SignalR)
Thanks!
y-me
Assuming you meant browser-based clients, then NO you cannot use gRPC. The gRPC protocol relies on HTTP/2 framing and in particular the ability to send and receive HTTP trailers. While browser themselves can and do use HTTP/2, current browser APIs (XHR/Fetch) don't expose HTTP/2 semantics.
There is however an alternative protocol, gRPC-web, that supports a subset of gRPC functionality you can utilize from a browser-based application. Given that it is a different protocol, your server will need to support it or you will need to employ a proxy like Envoy that can translate gRPC-web calls to gRPC.
I can't answer your question has if it is usable for Server/Client communications, just offer some thoughts. SignalR is made for the purpose of real time communications between Client and Server because of it's adaptability, and gRPC by it's constraints (HTTP/2 and HTTPS) is more reserved for backend micro-services communications.
I am trying to understand programming with sockets on a more detailed level rather than just with the API callings. I have fair understanding of C# WCF and socket programming using WinSocks in C++. Now I have 2 main questions:
Does WCF use sockets internally for the communication in all cases. In other words is WCF a wrapper around sockets and is built upon them?
Does all kind of network based communication use sockets at the end for sending/receiving data, which is something mandated by the OSI model?
A little detailed explanation will be better than just a Yes/No answer.
(With acknowledgement to the other SO users who agreed to reopen this question).
As an opening remark, remember that it's 2019 2020 and WCF is obsolete and I'm personally glad to see it gone, and I strongly recommend against using WCF for any new projects and I advise people to transition away from WCF as soon as possible.
Now, in response to your question (bold emphasis mine):
Does WCF use sockets internally for the communication in all cases. In other words is WCF a wrapper around sockets and is built upon them?
Strictly speaking, no (but in a practical sense, for inter-machine transport, yes).
WCF is a .NET platform that is concerned with "message processing". WCF tries to abstract away the underlying details of message transport (but it does so horribly, and so no-one should use it today), so it is entirely possible to build a WCF application that achieves inter-machine and inter-network communication without ever using Windows' Winsock, or whatever "Socket"-esque API is available for a given computing platform.
Now, while ostensibly WCF is all about abstraction, in practice WCF was geared around SOAP messages (and SOAP is horrible too, but that's another discussion), and SOAP primarily uses HTTP as a message transport - and HTTP primarily uses TCP/IP, and almost every single TCP/IP application on Microsoft Windows will be using the Winsock API somewhere in the process' communication stack. (It can be argued that HTTP applications on Windows will use http.sys which performs HTTP request/response processing in kernel-mode, which necessarily means bypassing Windows' user-mode Winsock API and instead http.sys uses "Winsock Kernel" which is its own thing).
In the above paragraph, note the use of the word "primarily" (as opposed to "exclusively" or "always") - because:
WCF doesn't have to use SOAP, it can use other messaging models/protocols/paradigms like net.tcp (which itself is more like a "binary SOAP") or even REST (though REST support came late in WCF's lifespan and it's a total pain to configure correctly, YMMV).
SOAP doesn't have to use HTTP, it can use other transports like SMTP. And WCF expressly supports other SOAP's other main transports like SMTP and FTP.
While HTTP is effectively tied to TCP/IP and Winsock is the only real way a user-mode application will use TCP/IP, other transports like SMTP don't have to use TCP/IP (at least, not in the way you think - see my footnote).
And of course, throughout all of this - user-mode applications are always free to use a different Networking Programming Interface besides Winsock or BSD sockets (for example, Windows' named-pipes present a streaming IPC interface just like how TCP behaves - or the vendor of a network-interface-card could have its own exclusively networking API which is somehow simply better than the Sockets API (similar to how GPU vendors in the mid-1990s were pushing their own APIs (Glide, PowerVR, Rendition, etc) until they all ended-up having to support Direct3D and OpenGL (and who uses Metal? hah).
And while WCF isn't exactly designed with testability in mind, it is still possible to host and run WCF applications inside an integration-testing environment where the actual message transport is just a thin proxy object, or a faked or mocked implementation - so Sockets are completely avoided there as well.
But in practice - in Win32, networking is accomplished using Winsock (Microsoft's implementation of the BSD Sockets API) so if you're using WCF to communicate between machines then I can say with 99% certainty that eventually your messages will pass-through Winsock.
Footnote: Regarding using WCF with SMTP without using Sockets: Many SMTP e-mail servers, including Microsoft Exchange Server, support "pickup directories" - which are filesystem directories actively monitored by the e-mail server, which detects when a new file has been added to the folder and reads each file as an SMTP envelope and processes it the same way as though it was an SMTP envelope received by the server's SMTP service endpoint - if a SOAP-in-SMTP message were to be "dropped" inside the pickup directory and it was destined for a recipient local to the pickup directory's e-mail service, then that message will not pass through Winsock at all either.
Want to have a ssl C# server that will only send and receive small JSON strings.
The clients can be anything from mobile devices or computers.
Need some feedback what to chose since some clients maybe have ssl limitations or other limitations.
Should i create an (openssl)ssl ftp socket server?
Should i create an HTTPs server?
should i create something else????
The HTTPS looks like easiest because of it's long history and width usage.
any feedback would be grate
Give a chance to WCF.
It has a very strong customizable stack, with externalizable configuration files, for both server and client side.
Moreover, it can publish services over multiples wires. This may help you to address a maximum of situations like "simple" https encryption to certificate validation.
You can even define your own channel for ultra specific contexts.
A last word from a developer perspective: the idea of WCF is to decouple the service from its connectivity. In today world, it's a huge advantage.
HTTPS is HTTP over SSL/TLS: you first establish an SSL/TLS connection and then you exchange HTTP requests and responses on top of it (as if it was a plain TCP connection).
Since you want to use SSL/TLS in both cases, what you seem to be asking is whether you should design your own protocol or use an existing one.
Should i create an (openssl)ssl ftp socket server?
There are multiple ways of securing FTP with SSL/TLS. It's certainly not the easiest way to go.
Should i create an HTTPs server? should i create something else????
Designing your own protocol means that you'll have to provide an implementation for each device yourself. HTTPS has the advantage of being widely supported. You shouldn't even have to use much of the underlying SSL/TLS API yourself (no need to learn the OpenSSL API). The only requirements might be to learn about some SSL/TLS configuration, such as setting up certificates.
Even if you're thinking of reducing the overhead due to the HTTP headers, your protocol will still have to perform similar payload management one way or another, at least to know where the requests and responses start and end.
It's unclear what your requirements are, but unless you really notice problems with HTTPS, exchanging JSON strings with HTTPS with an existing library (e.g. WCF) makes more sense than your other suggestions.
My company is experimenting with writing a proxy server using the .NET Fx 3.5 and C#. From our research I have read that HttpListener is not a good candidate for a proxy server though I am unsure as to why.
We are currently working with the Mentalis proxy example source code though that will involve, among other things, implementing our own logging and performance counters. Using HttpListener will wrap Http.sys which will give us some of the performance statistics we require out of the box.
So why is HttpListener a bad candidate for HTTP proxy work?
(And yes we are considering Squid 3.1 by writing or configuring an ICAP server as well.)
HttpListener is in .NET to provide a major building block for a simple HTTP server. Where simple includes not supporting high operation rates.
Typically HTTP proxies need to be very low overhead to support many concurrent connections as well as providing the proxy's function (which depends on the type of proxy).
Proxies are detailed in RFC 2616 ยง8.1.3) and that immediately provides one item that (if I understand HttpListener correctly) is not possible:
The proxy server MUST signal persistent connections separately with its clients and the origin servers (or other proxy servers) that it connects to. Each persistent connection applies to only one transport link.
You might also consider that the windows port of nginx was released a few days ago. Many sites that have squid and varnish experience are very pleased after converting to nginx. Then there is always whatever MS is calling ISA server these days.
Gone off to look at the Mentalis code now :D