Develop Reference

Server for handling data manipulation

I have currently built application in which I handle Products, Accounts, Orders, etc.
There are two databases one is a database which I created where I handle Users and Roles and some minor Application specific data. The other database is external and it is the one which holds all the data about Orders, Products and Accounts.
What I am trying to figure out is: how to build a server which runs parallel to the main application and handles all data manipulation with the external database.
Let's say a situation in which this would be helpful: There is an excel file which has to be created based on big amount of data and afterwards it has to be stored in the externalDB as certain type of format and sent as Email to someone. This will surely overload the main thread of the main application, hence we don't want that. Therefore, it would be a good idea to handle those kind of situations outside user's vision.
I am using ASP.NET MVC 5 and was curious what would be a good approach for this situation? I was thinking that I should make Console application, which is working as a service.

You would create a service architecture and have the services communicate with each other through controllers. These could push data back to the main application views, or the views could request data push from the services.
If you wanted to have jobs etc running on that server that send emails, you could easily just create SQL jobs or SSIS jobs or even custom service jobs that did that based on your criteria that are separate from the main view (where view is the main application that the user interacts with)
Services themselves could be configured with a light User Interface that you could call up on the server, perhaps in the service tray or through the Services component of Windows.
SOA architecture
https://www.cleverism.com/how-to-build-service-oriented-architecture-soa/
Micro Service architecture
http://microservices.io/patterns/microservices.html
Hope that helps. Good luck!

There are quite a few ways to do these kinds of things.
If your applications are for Android or iOS devices, it's common practice to have an external server running which handles intensive processing including managing your excel data, sending emails and database communication.
If your applications are cross-platform desktop applications or OS-specific desktop applications, you can still be able to use the external server, although certain other parts of the processing may be done on the client machine.
By using a web service to deal with database communication, it makes the applications more secure in regards to hard-coded database connection information, SMTP server credentials etc.
Using a RESTful web service will mean that you're adhering to the Hypertext Transfer Protocol (HTTP) which is great for exchanging data in the majority of cases.
Using a console application which acts as a TCP client server allows you to use the Transmission Control Protocol (TCP) to communicate in a different way. You're able to pass on data in a custom format if you wish. You'd be able to create a standard for communication which your applications will need to stick to.
The console application would do the same processing as the web service, however you would be in full control of logging requests, responses and data transfer.

multiple clients streamsocketlistener

For a project I have to make an application that runs in a network. In this network there are 2 or more clients(streamsockets) connected to 1 server(streamsocketlistener). Is there a way to tell the server to receive data from one client and then sending it to the other client, instead of staying on the same connection?

Do you mean as in relay the data from one client to another via a third (i.e. server)? If so then this WPF client server chat example article demonstrates that. In particular see the comments within a method called HandleIncomingChatMessage, about half way down that article.

Communicating with .net via PHP

I have a .net winform application that I want to allow users to connect to via PHP.
I'm using PHP out of personal choice and to help keep costs low.
Quick overview:
People can connect to my .net app and start a new thread that will continue running even after they close the browser. They can then login at any time to see the status of what their thread is doing.
Currently I have come up with two ways to do this:
Idea 1 - Sockets:
When a user connects for the first time and spawns a thread a GUID is associated with their "web" login details.
Next time PHP connects to the app via a socket PHP sends a "GET.UPDATE" command with their GUID which is then added to a MESSAGE IN QUEUE for the given GUID.
The .net app spawned thread is checking the MESSAGE IN QUEUE and when it sees the "GET.UPDATE" command it then endcodes the data into json and adds it to the MESSAGE OUT QUEUE
The next time there is a PHP socket request from that GUID it sends the data in the MESSAGE OUT QUEUE.
Idea 2 - Database:
Same Idea as above but commands from PHP get put into a database
the .net app thread checks for new IN MESSAGES in the database
if it gets a GET.UPDATE command it adds the json encoded data to the database
Next time PHP connects it will check the database for new messages and report the data accordingly.
I just wonderd what of the two above ideas would be best. Messing about with sockets can quicly become a pain. But i'm worried with the database ideas that if I have 1000's of users we will have a database table that could begin to slow down if there is alot of messages in the queue
Any advice would be appricated.

Either solution is acceptable, but if you are looking at a high user load, you may want to reconsider your approach. A WinForms solution is not going to be nearly as robust as a WCF solution if you're looking at thousands of requests. I would not recommend using a database solely for messaging, unless results of your processes are already stored in the database. If they are, I would not recommend directly exposing the database, but rather gating database access through an exposed API. Databases are made to be highly available/scalable, so I wouldn't worry too much on load unless you are looking at a low-end database like SQLite.
If you are looking at publicly exposing the database and using it as a messaging service for whatever reason, might I suggest Postgresql's LISTEN/NOTIFY. Npgsql has good support for this and it's very easy to implement. Postgresql is also freely available with a large community for support.

C# - Design questions about a server firing push notifications to clients - how to implement?

This is the scenario.
I have multiple clients on our application, and one server.
The server is itself disconnected from the clients, it just downloads some data from the web via a windows service (web services and FTP), processes the data and updates a database to which all the clients are connected and draw data from.
I would like to be able to actively notify the clients, and with a certain degree of granularity, when some downloading occurs (i.e. only the Traders when a price/trade update occurs, or only the Engineers when there's something for them) without polling.
The server should fire up a notification to all the connected clients instead of having them continuously "ask" if there is an update, because in this case I would have to maintain state on all the clients.
I thought about XMPP, with Matrix.
To do so each client has to open a persistent connection with the windows service, but I lack the exact details on how to implement this. MAybe with nodes!
For what I understand XMPP is perfect for what I want to accomplish and gives me the extensibility to grow to some more functionality if I have the need to.
I don't know if to implement my own server or use one of the existing one (I hear jabberd2 has an excellent windows server).
But most important: I need suggestions on A) an XMPP server to run on Windows and B) a C# library. Besides Matrix I have found very few, and above all I need notifications support (pubsub).

For simplicity, I'd consider using a WCF service that implements a long polling technique. This article gives some details on scaling the WCF service efficiently.
For notifications that there is new data in the database, if you are using SQL Server, try SqlDependency. It allows you to set up an event that fires in your code whenever the result of a given query changes. I've used it effectively for just this sort of thing.

How should server push data to rich client

I'm writing a simple accounting program consists of several C# winform clients and a java server app that read/write data into a database. One of the requirement is that all C# clients should receive updates from the server. For example, if user a create a new invoice from his C# client, other users should see this new invoice from their client.
My experience is mainly on web development and I don't know what's the best way to fulfill this requirement with C#s client and Java servlet server.
My initial though is to run ActiveMQ with Glassfish and use messaging pub/sub method so that updates can be pushed to C# client. I will create different topics like newInvoice, cancelInvoice, etc in order to differentiate the message type. Each message will simply contains the object encoded in JSON.
But it seems to me that this involves quite a lot of work. Given that my user base is very small ( just 3 or 4 concurrent user), it seems to me that there should be some simpler solutions. (I'm not familiar socket programming :) )
I know this is a client-server programming 101 questions but would be great if any experienced programmer can point me to some simple solutions.

The simplest approach here is often to simply use a poll - i.e. have the clients query for data every (your time interval). That avoids a whole family of issues (firewalls, security, line-of-sight, resolution, client-tracking, etc).
With WCF, you can have callbacks on duplex channels (allowing the server to actively send a message to clients), but this is more complex. I value simplicity, so I usually just poll.
Tricks that help here are designing the system to have an inbuilt mechanism for querying "changes since x" - for example, an audit table, perhaps fed by database triggers. The exact details vary per project, of course.
Another option that you might want to look at is ADO.NET Sync Services; this does much of what you ask for, for keeping a local copy of the database up to date with the server - but has a few complexities of its own. This is available (IIRC) in the "Local Database Cache" VS template.

Rather than pushing information from the server to 1:N Clients, would it not be easier to have the clients Poll the server for updates every so often ? Or when the client launches and creates a connection to the server, the server could dynamically generate a new Message Queue for that Client Connection, which the client could then poll for updates?

There are several push technologies available to you, like ActiveMQ (as you mentioned), or XMPP. But if you only have 3 or 4 clients to concern yourself with, polling would be the simplest solution. It doesn't scale well, but that isn't really a concern in your case, unless your server is an 8086 or something 8-)

You may want to take a look at StreamHub Push Server - its a popular Comet server written in Java that has a .NET Client SDK for receiving updates from the server in C#. It also has a Java Client SDK and the usual Ajax/Comet web browser support giving you more flexibility in the future to push data to web, Java and C# clients.