I am currently working on asp.net mvc4 web application. Part of the application, users can log in and browse the site etc. The data for the site is stored in a sql server database, contains users information etc.
A new feature to the site will be for all users to add comments to particular products shown on the site. As there could be hundreds of thousands of customers and thousands of products, this is alot of data.
So I have started looking at a NoSql option for this data and not store it in the relational sql server database. I have been looking at Mongo Db. My first question, is this a correct approach I am taking?
Next topic, how easily does c#/.net integrate with a mongo database. I havent worked with this before so my knowledge in the area is poor. Ideally, I would be querying (for the want of the correct term) the mongo db for comments based on a particular products identifier. I presume I can write a query style to get this data.
My next question is around the redundancy of a mongo db. With sql server, I have a fail over server if an issue occurs with the main db server. Is there a similar concept with mongo or how does it work? My consideration is for mongo to run on the same server as the sql server database. The data in the mongo db will not be mission critical, but the data in sql server is. My web application will run on multiple servers in a load balanced environment.
Can a mongo db be easily moved to another server? ie. how well can it be scaled out. Even can data from it be copied to another mongo db?
I appreciate my questions are of a beginner standard but I am currently researching the topic so assistance would be great.
Sql server should suffice for housing comments as long as you have some caching configured. The good thing about Sql Server is the data integrity of the foreign keys as well as the querying power.
However, working with Mongo in C# is not a huge deal. There is a slight learning curve, but this is with learning any new technology.
Connecting and Using MongoDB
MongoDB has official drivers and NuGet packages for you to use. http://www.mongodb.org/display/DOCS/CSharp+Language+Center for more information there.
Redundancy
Mongo supports replica sets where your second server would mimic all the data from the first server. Information on setting this up can be found here: http://docs.mongodb.org/manual/tutorial/deploy-replica-set/ It should be noted though that querying is a bit different in MongoDB than Sql Server.
Now I personally use mongoDB in one of my enterprise applications, but I would say as a rule of thumb: If you don't absolutely need to use it you would probably be better off sticking with one database engine. Mostly so that you only have to manage one database engine. Just my opinion though. Maybe redis for caching?
If you have not hardware memory problem(you can buy a lots of memory , you will need) Mongo can be your solution.
the thing is in mongodb design you will do a kind of denormalization...
and in my opinion hundreds of thousands user case your sql server is enough... do some more denormalizations in your db design and try implementing good cache design....
you say you are new to mongodb... so there is going to be a learning curve...
put more rams and cpus till you will have millions users...
to feel safe with mongodb you are going to need at least 3 servers
please also check this link
is this the optimal minimum setup for mongodb to allow for sharding/scaling?
try this
MVC Application With MongoDB - Part 1
MVC Application With MongoDB - Part 2
Getting Started With MongoDB in ASP.Net MVC4
Related
I have APPS Service hosted on azure using Azure SQL Database with around 15 tables:
- assistances
- users
- eventLogs
etc.
Currently I have around 150k records, and on daily based my DB is receiving around 2000 new assistances with new users related. On my app I have a cron, which is making a lot operations every 1 mins to all tables (updating, inserting etc).
Right now my aim is to create some nice dashboard, which will display data for admins (like states of assistances, number of assistances delay etc) - basicly reading data from those tables. It should give as well possibility to filter by dates (from - to date) - so in worst case scenario few users can generate report for month (aprox. 60k records) in the same time. I'm afraid doing it directly on my prod database, due to fact, that I've already cron going on with a lot operations, so I'm worry about locking.
My ideas are:
- sql database warehouse -> the biggest problem is the cost of it.
- replication to second DB, which will be used for querying data for dashboard. - I'm not convince about this solution.
- replication to noSQL database (pushing only important information) and use it for source of dashboard. - I don't have experience with such solution so far.
Do you have maybe some suggestion what will be the best?
In the end, I've used Geo-replication option from Azure, which is using snapshot isolation, so it's great! Even MS Azure recommend to use this geo-replication database as second DB used for read-only operations! I've tested and working great :)
You can use Azure automation to schedule those tasks that run every minute, instead of doing that from the application. You can know more about Azure automation here.
Instead of using Geo-replication consider using SQL Azure Data Sync. Make your primary database a “hub” database and use a replica for reporting. You can learn more about SQL Data Sync here.
You can also use Power BI to create your dashboards as explained here.
Hope this helps.
I am creating a c# wpf application that will store users information and allow users to view other users profiles in a corporation.
What would be the best way to store the data that each user is entering in order to have it accessible from a centralized location?
I'm fairly new to creating connections to databases in wpf applications and any push in the right direction would be greatly appreciated.
I've been reading a bit about SQLlite and it seems like it has its pros due to its mobile characteristics but does this mean that the SQLlite database will be local to each user? I need it to be centralized so that data entered from anywhere goes into one database.
Thank you
You could use ADO.net datasets to get the data from the server but this is done by downloading a local datatable and syncing it using a data adapter
connection between client and server
how to do in visual studio
As you stated mobile characteristics are not what you need. As for a central database consider the free SQL Express from Microsoft. You will need some type of security so they can only update their own profile.
Probably one of the more straightforward options is to create an Entity Framework model for your application. This may serve as a primer for your exploration of the topic:
Databinding with WPF
You'll probably find it easiest to develop against your local SQLExpress instance or some other local data store. But the final solution will need to be connected to an enterprise-ready database. Since there was no mention of cost constraints and the nature of this question is more of "how?" versus "what to use?", I see no reason why this shouldn't be a SQL Server solution.
Ultimately, though, this question is rather broad in scope and cannot be adequately addressed in a simple answer here.
I'm developing an ASP.NET MVC 4 commerce site but since my background is in front-end web dev with HTML/CSS/Javascript and C# with XNA all this server and database business is doing my head in.
At the moment the site uses an elasticsearch Index with NEST to perform searches for products since the products db is huge and I like the smart queries elasticsearch offers. And that all works great on my local testing environment.
My question is: is it generally a good idea to have your elasticsearch client and indices stored on a separate host from the actual site or is it OK to have it on the same server?
I understand there's the issue of space at play here, but I've heard elasticsearch queries also tie up server resources that could be better spent handling other tasks like the impending flood of payments that are sure to come through?
It really depends on your load, your data and your current server. How many users do you have on your website? How big is your index? How powerful is your current server?
It's usually best practice to put elasticsearch on a separate machine, even more than one in order to talke advantage of its distributed features. With two machines you can for example distribute your shards over them and configure one replica (default value), so that every machine contains a whole copy of the data. And if the load increases you can always add new nodes to the cluster, as long as you allocated enough shards (it's common practice to over-allocate shards a little).
On the other hand, I've also used it embedded in a Java application, as you can read in this article I recently wrote.
This is quite a long one, but I'd very much appreciate your thoughts and suggestions.
We are busy rebuilding a legacy system which was written in PHP and MySQL and replacing its components with ASP.MVC in C# and SQL Server. The legacy architecture leaves much to be desired and there is a serious issue with spaghetti code, no referential integrity in the DB, unused code and database fields and just generally bad coding.
As much as I'd love to, we can't just rip out all of the old code and replace it. The company needs to stay functional during the development process, so we will need to build new functionality while using the old databases to ensure that their data is accurate at all times. The level of data accuracy isn't real-time, but if we had 2 systems, they would have to be in sync 100% of the time. The old system uses 6 different MySQL databases, all on the same server, running Linux. We will be running Windows 2008 R2 on the new server for the new system and we are planning to use the latest version of SQL Server.
The problem I'm having to solve is: I need to somehow map all of these databases into a consolidated model that we can use through C# to develop the new system on. Once we have moved all the functionality over to C#, we need to port the data into a DB that matches our code model. This DB will be running on SQL Server. I'm not too worried about the migration just yet; my current issue is finding an ORM tool that will allow me to map these 6 MySQL databases into a single, well planned out and designed model that we can use for the new development.
The new model might have additional fields that we would have to store in a new MySQL database until we port the data across at some stage, so the ORM should support easily building entities that span multiple tables and databases.
Is what I'm trying to do possible? Is it viable in terms of effort? Is there an ORM that can do all of this? and what other way is there to maintain operational capacity of the company whilst developing on the system actively?
I have looked at these ORM options:
SubSonic (great, but I think too lightweight for what we are trying)
Entity Framework (looks like I might be able to use this if I use very dirty models with tons of stored procedures for inserts, updates and deletes)
NHibernate (the client does not want us to use this due to bad experiences in the past)
LLBLGen (seems like it can do what we need it to, but long term support could be a concern with the client)
Anything else I should look at? Is there a different approach I could try?
ORMs aren't designed to solve the problem you have. That said, a quality ORM will get you some percentage of the way toward a solution.
NHibernate is the easy choice. LLBLGen would be my second choice. I wouldn't even bother with EF or SubSonic as they are very feature poor compared to the other two and you need decent feature support in your scenario.
You'll likely have to invest a lot of time in writing custom code around your migration requirements. Your use case is not a standard, well traveled path.
For Entity Framework: if you're prepared to maintain one complete set of stored procedures with a static interface (i.e. same signature) you could implement them all in Transact-SQL on the SQL Server box, with linked servers (to the MySQL farm).
When the time comes, you could migrate the data into SQL Server and update your stored procedures.
Basically, design a nice model with nice stored procedures, and as a temporary solution implement any ugliness inside the stored procedures. Once MySQL is out of the way, you can replace the stored procedures with better ones.
SQL Server has a tendency to retrieve the entire remote table when you're running queries against a linked server, so if performance is a concern it might eventuate that all your stored procedures are wrappers around OPENROWSET (see Example A for running a query on a remote server).
I was thinking about utilizing RavenDB for some of my look-up scenarios I am doing in a high throughput application. This would replace all of the look-up calls I need to make to the DB to get things like site location, etc. Looking at a couple of options really (also .Net caching). I know that you can replicate Indexes from RavenDB to SQL Server, but wondering if anyone has done the reverse where they sync RavenDB with Sql Server?
Any suggestions / comments would be appreciated.
--S
I've done a similar scenario where data needed to be transferred in batch from a SQL Server system nightly into our RavenDB instance.
I couldn't find an off the shelf tool to do what I wanted as typically you should optimise the model you give RavenDB differently to SQL Server.
I wrote a custom console app that put the data into my RavenDB instance.
For example my console app:
Compacted several relationships into one document
Dealt with the different datatypes
TLDR: I wrote my own console app as I couldn't find a generic product that could do it.
So far the only avaible solution is write your own sync process.
I was looking for ways to improve the search scenearios using RavenDB , the RavenDB will be filled using my sql server relational database.
I think it should be a better way, however the only i can think rith now is to use a ETL process that keeps updating your NoSQL version of your structured data.