What I have read so far, winPcap allows you to bypass OS and bypass application and transport layer processing for TCP and provides direct access to the link layer.
I am planning to use winpcap to do some user application stuff and not just sniffing. I will be receiving and sending critical information using pcap which I am currently doing via sockets.
Does bypassing OS, and according to my understanding application and transport layers on my side, involve any risks?
As a side question the winpcap documentation I have found so far talks about how to programatically implement it but does'nt tell in detail what it is bypassing and how does it do that. Any link to that will be helpful.
Also, I'd like to know if anyone is using winpcap for any purposes other than network sniffing for monitoring reasons and msn.
Sure, there are plenty of risks:
The OS won't know about your privately-managed TCP connections, so it won't know that the port(s) you've chosen is/are in use. This means that it might try to use the same port for another application's connection, leading to chaos.
The OS won't know about your privately-managed TCP connections, so unless you prevent it from seeing those packets, it will send RST packets to reset the apparently bogus connection.
Your application won't automatically be notified of changes to relevant OS-managed data, configured IP addresses and routing tables. You'll likely have to poll for updates.
Properly implementing the TCP protocol is non-trivial. Most implementations, even very well-used ones, had dormant bugs that weren't found for years. Not all the necessary information is in the RFCs, either; there are places where established practice differs from the documented behaviour, usually for good reason. There's also plenty of code in modern TCP stacks specifically to deal with historical buggy behaviour in other stacks, and replicating all that work isn't simple.
There's a substantial risk of bad interactions with third party network security software installed on the host, which will expect all TCP connections to be made via the OS.
It seems like a support nightmare to me.
Related
I have been looking around the internet to see what protocols are available for the client/server communication. I'm currently not able to find a durable solution for my problem, maybe looking the wrong places ?
The problem in short is that today I have a desktop C#, WPF application that communicates with a C# server application using WCF and TCP on dedicated ports. However, the use of TCP requires that external users needs to open the specific ports in their companies firewall to be able to use the application. This proves to be a bigger challenge than expected and therefor I'm trying to find if there are other ways to perform the communication.
I looked at websockets and signalr which uses HTTP(S), however I need at protocol that supports large amounts (sometimes 10MB at startup) of data transfer, which they don't seem to do well?
In short:
I need a protocol for client/server communication using the HTTP(s) channel and also performing well with larger amount of data.
This might be an impossible mission, due to lack of performance compared to TCP ?
I used this MS link to put together a TCP server in C# on a PC. I'm holding the port open and waiting for connections to be established by various PLC clients. The PLCs are in moving autonomous bots, so they move in and out of Wi-Fi range. I'm using this setup to acquire running variables (battery %, etc.) from the bots and display them in a UI for the system administrator to monitor.
I setup the router with port forwarding so that the data arrives on the server PC from the various clients. I'm using Siemens S7-1200 PLCs and I don't believe that they support high end security features like PCs.
So my question is this, if the admin PC is running a Windows service that constantly monitors the open port then is there a security risk? And if there are risks, can you please explain and support with links or resources to help me patch these holes (in C#)?
It seems safe to me because if the PC is off, the port is closed. If the PC is on, the port is open but is bound to the application monitoring it. If the port receives something that it does not deem valid it just dumps that data. I am not incredibly knowledgeable on software and PC security, but this is slightly different because it is a single PC interfacing with less capable hardware.
Having a port open exposes you to anyone connecting to that port and providing bad information, exposing a vulnerability on your message parsing and socket handling implementation (buffer overflow or script injection), or just swamping your application with traffic. The last one is almost impossible to protect against, someone can always DOS you at some level.
None of these are unexpected risks, but you need to be aware of them and ensure that you properly scrub incoming traffic to reject malformed requests and somehow authenticate and drop connections that aren't from the bots you expect.
If you do make an authentication step, you'll want to encrypt the channel before authentication using something like SSL or SSH. Otherwise, someone else could watch your traffic, observe the authentication transaction, and then just copy it.
Best of luck! Security is a deep rabbit hole, but a very valuable skill!
I wrote a client-server application designed to exchange files (among other things) over the local area network. In server mode, the application listens for TCP connections with a specific identification header. In client mode, it tries to establish TCP connections to IP Addresses that are provided by the user.
I now need to adapt this application to work over the internet. Without much exposure to network programming, I am not sure how to achieve this without some kind of a central server (for announcing presence, etc.). This is not an option.
Suppose I have the app running in server mode on my machine which is behind a home network. You (the reader) have the app in client mode and we need to connect. Neither of us have static IP addresses. Is there a way for the client to reach the server? Both the server and client could figure out their public IP addresses but beyond that, I'm not sure what to do.
Any guidance would be appreciated.
EDIT: Based on the answers, some clarification is in order. My question is not about discovery. Both client and server can query their public addresses and users can exchange these IPs over some other medium. The question is, how to establish a connection once each other's IPs are known but both parties are behind networks which do not have the appropriate port forwardings. My app uses port 51200 as a default over TCP for example.
I've learned that the techniques of UDP hole punching to obtain a true direct connection between 2 peers, is imperfect due to some possible combinations of NAT types involved. What this means is that UDP hole punching alone is not always going to work.
The way to claim 100% operability is to, in the cases where a p2p connection is impossible, use a relay server. The process of discovering whether a true p2p connection is possible has been standardized by ICE,TURN,and STUN. All I know is that using ICE/TURN/STUN seems to be the standardized strategy in subverting the NAT problem. It can facilitate a true p2p connection when available and offer relaying services when necessary.
Note: I use the term p2p in this answer to distinguish a truly direct connection between 2 endpoints, not to be confused with overlay networks and the like.
Advanced Users: It is possible to gain more direct p2p connections by introducing more complicated strategies to cooperate with symmetric NAT behaviour, such as port prediction, etc. But the libraries that implement these methods are costly, and seems too difficult for a DIY job.
I have not explained in detail the different types of NAT connection possibilities. You will want to research on NAT types and how they relate to UDP hole punching. There are RFCs out there and I will try to update my answer with links. But the point is that you can avert a lot of this learning by refocusing on implementing TURN/STUN/ICE, learn how they are implemented and how you can use their standardized behavior.
Potential Solutions
PJSIP - contains a stand-alone high level NAT traversal library for C/C++, also has bindings for other langauges. If you just want the NAT Traversal library sub-component, see PJNATH
LibJingle - a P2P (peer-to-peer) and RTC (real-time communication) stack that builds on XMPP. Note that many of LibJingle implementation is not interoperable with actual XMPP Jingle specification and subsidiary specifications.
These may add considerable complexity to the program, which is why it can be compelling to implement some of the mechanisms myself.
Documents
NAT on Wikipedia
NAT Traversal Fundamentals
How NAT Compatible are VoIP Applications
STUN server list
Teredo/Shipworm Doc
Symmetric NAT and UDP Hole Punching
STUN/TURN/ICE Summary
IETF
RFCs
RFC 5245 – ICE
RFC 5389 – STUN
RFC 5766 – TURN (iirc, TURN uses a STUN server.)
RFC 5768 – ICE – SIP
RFC 6336 – ICE – IANA Registry
RFC 6544 – ICE – TCP
RFC 5928 – TURN Resolution Mechanism
RFC 6062 – TURN Extensions for TCP Allocations
RFC 6156 – TURN Extension for IPv6
[Draft] Symmetric NAT Traversal using STUN
[Draft] Network Address Translation and Peer-to-Peer Applications (NATP2P)
Microsoft
MS-STUN – Microsoft STUN extensions
MS-TURN – Microsoft TURN extensions
MS-ICE – Microsoft ICE extensions
MS-ICE2 – Microsoft ICE2 extensions
Notes
I have heard that the eventual migration to IPv6 may squash the NAT traversal problem, but [probably not]. Someone could enlighten this topic. Either way, the progression to IPv6 appears slow from my non-experienced viewpoint, and I wouldn't count on it as a solution.
Few months ago I was looking for a similar solution, but unfortunately I got stuck and henceforth I dropped it.
People say you could use UDP hole punching or TCP hole punching, but I was unable to do this (I'm not a specialist in computer networks though). Whether it will work or not depends on the network itself.
Here is the question I originally asked, maybe it will be helpful to you. Honestly, I don't want to kill your hopes, but I'm afraid it may be a dead end. :(
I don't believe this is possible without some sort of central place to "find" each other. You need some way for the client the find the servers ip address to connect to.
Also, udp or tcp hole punching is an entirely different thing and uses, guess what: a central server.
If you use IPv6 it is possible - but you still need to know the other parties address. It also is possible if you use a DynDns service - but that is already a kind of central infrastructure.
I have a number of WPF clients on the same corporate network. I want these clients to share messages with each other. I don't want to run a separate server process so a brokerless solution would seem best. I have considered using PNRP but this seems to require a PNRP service to be running on each client, I'm not sure I could guarantee all the clients would or could be running this. I have also had a look at ZeroMq which looks ideal it terms of simplicity and its very lightweight, however I would need to know the endpoints for a TCP/IP style communication and each client won't be aware of the others rather they need someway to discover each other. So essentially I want a multicast style of communication but without having to use multicast since this will require me to get a range of addresses set up within the corporate network and involve infrastructure etc.
So I guess the question is are there any options I haven't considered that fit the bill?
Thanks in advance for any help.
The ZeroMQ pub-sub pattern is simple and fast until you reach hundreds of clients; you can switch to a real multicast protocol then (PGM) without modifying your application.
Perhaps you should check out NServiceBus. It is a true service bus, so no broker machines in the middle. It runs on MSMQ so your windows servers will support that out of the box. It also supports transactional messaging. It also supports a pub/sub model that should satisfy your multicast requirement.
If you don't mind using a commercial product: OMG's Data Distribution Service is a standard with several implementations that can do what you are looking for. At least one of them supports C# and does not require anything to be installed on your machines -- just the libraries. Disclosure: I work for this company.
So essentially I want a multicast style of communication but without
having to use multicast since this will require me to get a range of
addresses set up within the corporate network and involve
infrastructure etc.
DDS by default uses UDP/IP over multicast for discovery and communication, but can be instructed programmatically or via configuration files to use UDP over unicast only, or TCP. This does not affect application logics, so the conceptual multicast nature is preserved. However, if you do not have IP multicast at your disposal then you will lose some out-of-the-box automatic discovery features. In this case, you would need to know in advance the IP addresses or host names of all nodes that could potentially participate in the communication. From there, the middleware will be able to discover who of these is actually present and adjust its communications accordingly.
I think the latter is true for any solution you choose though. For fully automatic discovery, you will need either multicast/broadcast, or some known discovery server(s) running in your system.
We've written a peer-to-peer message bus, Zebus, based on ZeroMQ (transport), Cassandra (peer discovery and persistence) and Protobuf (serialisation).
It is open source and production tested https://github.com/Abc-Arbitrage/Zebus
Zebus is being actively developed and it is in heavy in-house production use. Currently there is only a .NET language binding but as the OP mentioned that he only had WCF clients this should meet his needs.
Have you considered the peer to peer protocol for WCF? See here for more details: http://msdn.microsoft.com/en-us/library/cc297274.aspx
ZeroMQ is a good choice for this. To solve the discovery problem, stand a server that every client checks in when starting and stopping. This server can also be running ZeroMQ as both a publisher and subscriber.
The clients publish to one port on the server, which binds a subscriber to that port to get check-in and check-out messages. The server in turn publishes those messages on another port (which it also binds) to which the clients subscribe.
Can anyone help, i trying to figure what i need to do, i have been given the tasks of writing a server and a client in TCP (UDP). basically multiple clients will connect to the server.. and the server sends MESSSAGES to the client.
I have no problem in creating the server and client but with tcp i am unsure whcih way to go. DOes the .net 3.5 support everything or do i need to go on the hunt for some component?
I am looking for soome good examples with c# for TCP or UDP. THis is where i am not 100% sure .. as far as i know there is UDP and TCP ... 1 is connected and 1 is not.. So which way do i go and can c# support both?? Advantages /Disadvantages?
Say if the server has to support multiple clients that i only need to open 1 port or do i need to open 2?
Also if a client crashes i need for it not to effect the SERVER hence the server can either ignore it and close connection if one is open or timeout a connection... If in fact a connection is needed again going back to tcp udp
Any ideas where i shoudl beging and choosing which protocol and amount of ports i am going to need to assign?
thanks
UDP cons:
packet size restriction means you can only send small messages (less than about 1.5k bytes).
Lack of stream makes it hard to secure UDP: hard to do an authentication scheme that works on lossy exchange, and just as hard to protect the integrity and confidentiality of individual messages (no key state to rely on).
No delivery guarantee means your target must be prepared to deal with message loss. Now is easy to argue that if the target can handle a total loss of messages (which is possible) then why bother to send them in the first place?
UDP Pros:
No need to store a system endpoint on the server for each client (ie. no socket). This is one major reason why MMO games connected to hundred of thousands of clients use UDP.
Speed: The fact that each message is routed individually means that you cannot hit a stream congestion like TCP can.
Broadcast: UDP can broadcast to all listeners on a network segment.
You shouldn't even consider UDP if you're considering TCP too. If you're considering TCP means you are thinking in terms of a stream (exactly once in order messages) and using UDP will put the burden of fragmentation, retry and acknowledgment, duplicate detection and ordering in your app. You'll be in no time reinventing TCP in your application and it took all engineers in the word 20 years to get that right (or at least as right as it is in IPv4).
If you're unfamiliar with these topics I recommend you go with the flow and use WCF, at least it gives you the advantage of switching in and out with relative ease various transports and protocols. Will be much harder to change your code base from TCP to UDP and vice versa if you made the wrong choice using raw .Net socket components.
It sounds to me like you're not clear on the distinction between TCP and UDP.
TCP is connection oriented. i.e. 2 peers will have a dedicated connection. Packet delivery and ordering is guaranteed. Typically a server will present a port, and multiple clients can connect to that port (think of a HTTP server and browsers).
UDP is connectionless. It doesn't guarantee packet delivery, nor ordering. You can implement broadcast and multicast mechanisms very easily. If you need some sort of reliability, you will have to implement this on top of UDP. Sometimes you may not care, and simply issue requests and retry on no response (SNMP does this). Because it's connectionless, you don't really worry about peers being up/down. You just have to retry if required.
So your choice of protocol is dictated by the above. e.g. does your client require a dedicated connection to the server ? Are you transmitting the same data to multiple clients ? Can you tolerate packet loss (e.g. real time price updates etc.). Perhaps it's feasible to use both TCP and UDP for different requirements within your app (e.g. TCP for registering orders, UDP for transmitting price updates/events?)
I'd consider your requirements, and familiarise yourself with the limitations and features of TCP and UDP. That should make things a little clearer.
Is there a requirement to do this at such a low level? Why not use WCF? It fully supports messaging over TCP/IP, using binary data transfer, but it's at a much higher level of abstraction than raw sockets.
Everything you need is in .Net 3.5 (and probably below). Check out the documentation and examples with the UdpClient class at MSDN for insight into how to write your client/server. A quick google found some sample code for a server and client at www.java2s.com among many other networking examples in C#. Don't be put off by the domain name.