I'm currently working on a project that involves splitting large MySQL databases into multiple smaller shards. However, clients must be able to query the databases just as they had before with no change to the user interface; that is, any query they send through a MySQL client (Workbench, DBForge, etc) must return the same result set from the master database as it does from its shards.
This requires that an application be seated between the client and server to intercept queries, analyze them, modify them, and redirect them to master databases or shards as need be. I know that MySQL Proxy is particularly suited to this task, but that's where the problem comes in.
I've already written a C# application that takes a MySQL query string, modifies it, queries whichever shards it must, and aggregates the results from all shards. My problem is that I don't know how to connect MySQL Proxy to this application. Ideally, MySQL Proxy would intercept a query, determine whether it is "shardable" or not, and send it either to my sharding application or to the master database. The sharding application would then send its result set back to MySQL Proxy, which would return it to the client.
Is there any clear way to accomplish this? Perhaps if I were to turn the C# application into a WCF service? MySQL Proxy has methods for connecting to databases, but I don't know if it can connect to a simple web service, much less a WCF, considering it's written in Lua. Is there some alternative to MySQL Proxy that would better meet my needs?
Mysql allows you to script with Lua and connect to C or C++. I have no experience in C# but I supose you will have no problem.
Install mysql-proxy. You can determine one or more backend mysql DB.
Compile your C code as a module to be called from Lua: link
Depending on your code set the backend IP for your query and redirect it.
Be carefull when searching Lua documentation, because mysql-proxy implements Lua 5.1, and procedures change quite a bit from version to version.
Related
I'm working on a project that involves linking my C# application to a MySQL server. The server is running cPanel and the company does not allow all IPs to connect to it (although you can add host IPs to the remote list). The application will be running on Windows 8 Tablets on a 3G connection, so obviously I will not know the IPs if the devices to be able to add them to the list.
A collogue of mine has told me of a system, API or something like that that will allow me to interface with the Database via HTTP effectively bypassing this restriction.
I can provide more information if needed.
Does anyone know of anything similar to this or any way around this?
If you are using MySql 5.7 or higher, there is an experimental "lab" for allowing direct access to MySQL via a REST over HTTP interface, eliminating the need for a middle-tier server or database specific drivers.
You can download the plugin at the MySql Labs site. From the dropdown, select "MySQL HTTP Plugin" and you should be good.
Keep in mind that its part of the "MySql Labs" project, which means its experimental, probably buggy and should be used at your own risk. Think twice before using them on any kind of production server.
My Scenario:
I have two applications. First one is a website which is connected to MySQL Database and 2nd one is a Desktop Application which is connected to SQL Server2008 R2 Database.
The Desktop application updates records locally and the MySQL database is updated online though the website.
Problem:
Two different databases, how can we update at the spot when changes are made either in MySQL or SQL Database?
What I Want:
Databases should be synchronized to each other (e.g. if changes are made in MySQL then SQL server database should be updated, or if changes are made in SQL Database then MySQL database should be updated)
Could anybody please suggest some code, any idea, or any solution to solve this issue?
Make use of Restful API's to Update information from MS SQL server to MYSQL server.
One of the first things I would point out is that complete and perfect syncing is not possible. Unfortunately there will be data types that exist in SQL Server that don't exist in MySQL and vice versa.
But assuming the data types are pretty simple and the schemas are similar, here are some options:
Use a service bus. You can write an application that monitors both database systems and when it sees a change, it pushes an object onto the service bus. Listeners to the service bus will see the objects and write them to the appropriate destination.
Use triggers like Alex suggested. SQL Server can have CLR code execute on a trigger. The CLR code could be some C# that writes directly to MySQL. Takes some setup, but it's possible. I've investigated running a process from a trigger in MySQL and all options are ugly. It's possible, but security is a major concern. The idea is that a record is changed, trigger is fired and an external process is run.
Write an application that constantly looks for "diffs" in tables and moves data back and forth. You'll need to modify all tables to make sure there is support for date/time stamps for each record so you can track when a record has "changed".
I really don't like the concept of opening my SQL server(s) to the internet - even if I can lock down the firewall. However I've always been working directly with databases. I'm building a system now which involves 1 SQL Server database, a web application in ASP.NET/C#, and a few windows applications in Delphi XE2. But from the beginning, I'd like to put some sort of 'filter' around the database so I don't have to open it up.
I know there are many things out there for this, but don't know anything about them or what to get for my scenario. I'd like to keep it native to SQL Server; I don't plan on using any other type of database engine.
It needs to be connected from client to server by other means than the standard SQL connection, like a filter. It creates its own encrypted packets and transfers data its own way. I will have a wrapper class for both Delphi and C# which will pretty much be identical - and be able to stream its data into the DLL to interact with the DB.
Now there are three different ways I can go about this...
Complete SQL Server wrapper, most likely no source code, might even have its own language (I don't want to pick up another database language), and independent from my project as its own separate system.
Open-source wrapper, preferably in Delphi (XE2), or if not then C#, specific protocols for my system, entirely dedicated to my project, and in the final form of a DLL which can be used on both the Website (in C#) and the applications (in Delphi).
Web Service - however I only have 1 hosting spot (Paying for 1 site, 2nd site will be a double charge on me). I can't host any additional web services or windows services; it has to be integrated with the website. Otherwise, I would have done a web service for this.
I would much prefer the second option, and do not want to go anywhere close to the first one, and can't do the third one at all.
So any good libraries for database layers? And might there be some already installed in Delphi XE2? I'm thinking maybe an encrypted XML packet?
As an example, let's say I have a table for 'Customers'. In both my website and applications, I should never have any SQL script like select * from Customers or no SQL script in general. Instead, I will have a wrapper around the database. So I can call a function such as DBGetCustomers(Conditions: TGetCustomersConditions): TDBCustomers; where TGetCustomersConditions is some way of filtering the query, and TDBCustomers represents the results from the query.
There could also be a function DBAddCustomer(Item: TCustomerToAdd): TInsertSuccess; where TCustomerToAdd represents what to insert, and TInsertSuccess represents any result, such as error message(s) or rows affected. I do not intend for it to be working exactly like this, but just to explain the concept of any wrapper in general. When the app sends to request to the server, it still has not converted anything to a SQL Query. By the time the request gets to the server (which is able to connect to the database), then the server alone decodes everything to the SQL query.
What's the problem even if you have just one "hosting spot"? A web service is just a "site". And a web server can easily host multiple site even with a single IP address. Anyway, what you're looking for is an "application server" in a multi-tier design.
While Java invested heavily in that direction, MS did not. Delphi has Datasnap, which is a so-so framework, don't know if the new "restful" interface is easily callable from C#, it looks to have security flaws though. The .NET way of doing it is using WCF, as long as it uses a standard protocol you can call if from Delphi has well.
You could also look at RemObjects DataAbstract. It's not open source, but it is a mature library.
One of the 'traditional' ways to do this is via webservices (although this technique is now considered by some to be a bit dated).
One disadvantage is that it is not a generic wrapper you can throw around the database, but it has the advantage that you can limit access to the database easily to specific stored procedures for example, which will maximise security, and is a standard technique if you wish to provide limited authorised access to external applications.
If you already use a ASP.NET C# web application, you could also add a WCF (Windows Communication Foundation) based web service to your web site. This can provide database access to external applications, which need to connect the web service somehow. It should even be possible to use the same HTTP standard port for both the normal web site pages and the service, by mapping the web service to a specific context path like www.example.com/services/servicename
Kbmw allows you to make a ntier db architecture.
However since you have a web application involved, a better option would be WCF (as already suggested)
I'm going to develop a POS system for medium scale company
and the requirement for me is to make all data on time for all of their branches
while in my mind, move the server from local to web would solve this problem
but, i never done any online server for window application
may i know what is the best option for use as secure database ?
such as SQL can handle this well ?
i tried to google but all of the result return is not what i want
may i know what will you do when you facing this problem ?
my knowledge on coding is just VB and CS
also SQL for database
i would like to learn new if there is better option
i hope it is impossible to access by anonymous and it is store secure at back-end only
What you probably want to do is create a series of services exposed on the internet and accessed by your application. All database access would be mediated by these services. For security you would probably want to build them in WCF and expose them through IIS. Then your Windows application would just call these services for most of its processing.
If you design it properly you could also have it work with a local database as well so that it could work in a disconnected manner if, for example, your servers go down.
Typically you don't move the server off of the site premises.
The problem is that they will go completely down in the event your remote server is inaccessible. Things that can cause this are internet service interruption (pretty common), remote server overloaded (common enough), basically anything that can stop the traffic between the store location and your remove server will bring them to their knees. The first time this happens they'll scream. The second time and they'll want your head due to the lost sales.
Instead, leave a sql server at each location. Set up a master sql server somewhere. Then set up a VPN connection between the stores and this central office. Finally, have the store sql boxes do merge replication with the central office. Incidentally, don't use the built in replication, but an off the shelf product which specializes in replicating sql server. The built in one can be difficult to learn.
In the event their internet connection goes dark the individual stores will still be able to function. It will also remain performant as all of the desktop app traffic is purely to the local sql box.
Solving replication errors is much easier than dealing with a flaky ISP.
I would recommend you to check Viravis Platform out.
It is an application platform that also can be used just as an online database for any .NET client with the provided SDK. It has its own generic windows and web clients and some custom web solutions for some specific applications.
You may be using it as a complete solution or as a secure online database backend.
I have a silverlight application which reads data from a db and displays them in a grid.
As far as i understand it there are two ways to do it.
1) Read db with a server-side language, such as php, create a file in the server which you can later use in Silverlight to read the data.
2) Connect "directly" using Silverlight code with your db and do your job there. I have read some posts that explain more or less how this is possible, but i haven't found a working example.
I really need to go towards the 2nd approach. Can anyone provide a simple example on how to connect and query to your mysql db safely from Silverlight?
Thank you.
You could use the MySQL connector for Entity Framework and then connect to the database using .NET RIA Services.
Whatever route you go you're going to need to create some method to read data from the database on the server and return that to Silverlight as objects.
Silverlight runs on the client side in the browser, there is no way to directly hit a database. The way to get data is to hit a webservice, you could use ADO Data Services but they only run Sql Server in their cloud.