What are the best practices or pit falls that we need to be aware of when using Microsoft Oracle provider in a web service centric .NET application?
Some practices we employ based on our production experience:
Validate connections when retrieving them from the connection pool.
Write your service code to not assume that connections are valid - failure to do so can cause quite a bit of grief especially in production environments
Wherever possible, explicitly close and dispose connections after using them (using(conn){} blocks work well)
In a service, you should use connections for the shortest time possible - particularly if you are looking to create a scalable solution.
Consider using explicit timouts on requests appropriate to the typical duration of a request. The last thing you want is to have one type of request that hangs to potentially block your whole system.
Wherever possible use bind variables to avoid hard parses at the database (this can be a performance nightmare if you don't start out with this practice). Using bind variables also protect you from basic SQL-injection attacks.
Make sure you have adequate diagnostic support built into your system - consider creating a wrapper around the Oracle ADO calls so that you can instrument, log, and locate all of them.
Consider using stored procedures or views when possible to push query semantics and knowledge of the data model into the database. This allows easier profileing and query tuning.
Alternatively, consider use a good ORM library (EF, Hibernate, etc) to encapsulate data access - particularly if you perform both read and write operations.
Extending on the above - don't pepper your code with dozens of individually written SQL fragments. This quickly becomes a maintainability nightmare.
If you are committed to Oracle as a database, don't be afraid to use Oracle-specific features. The ODP library provides access to most features - such as returning table cursors, batch operations, etc.
Oracle treats empty strings ("") and NULLs as equivalent - .NET does not. Normalize your string treatment as appropriate for Oracle.
Consider using NVARCHAR2 instead of VARCHAR2 if you will store Unicode .NET string directly in your database. Otherwise, convert all unicode strings to conform to the core ASCII subset. Failure to do so can cause all sorts of confusing and evil data corruption problems.
Some more tips:
Avoid using Microsoft Oracle provider because it goes out of support (http://blogs.msdn.com/adonet/archive/2009/06/15/system-data-oracleclient-update.aspx)
If you're commited to Oracle use oracle specific features and link Oracle.DataAccess assembly to your code
If you're not sure and want to be flexible, use System.Data.Common classes and load oracle provider through
The Oracle Providers work fine in an ASP.NET application, but be aware of:
Matching the right version of the oracle client 32-bit or 64-bit with your application pool
32-bit client for a 32-bit app pool, 64-bit client for a 64-bit app pool.
Permissions - grant the app pool user rights to the oracle client directory (c:\oracle\product\10.2.0\client_1)
This doesn't have anything to do with ASP.NET, but it is important to note that Oracle stores empty string and null both as null, so if you need to know that something was empty and not null, you need to add an additional column to track that...
Related
I have a project I developed with C# windows forms using SQL Server as database but I want to make the project standalone without any database server. I found SQLite as an option to go for, Is it possible to migrate my SQL Server database to SQLite and not affecting my code? And how do I go about it?
I used entityframework code first in connecting the SQL Server database
The answer is almost certainly going to be "yes." Depending on a few things, you might have to change very little (or no) code, or you might have to change a lot.
The first consideration is your SQL code. If you were very careful to write ANSI-compliant SQL and you didn't use any of the built-in SQL Server views or T-SQL-specific functions, you may not have to re-write much code at all. In reality, you probably will have to at least write some. In particular, while SQL Server's engine is meant to handle multiple concurrent sessions and queries, SQLite is not: you will need to manage your program carefully to ensure no two threads attempt to access the SQLite database at once.
The second consideration is how your application calls the database. Again, depending on your design, you may need to re-write almost no code, or you may need to re-write a lot. In my C# applications, I create an interface for database providers that defines common functionality (select, delete, insert, etc). Then I create simple wrapper classes for different RDBMS that implement the interface. When I need to switch databases, I simply instantiate and use a different class. If you have your project setup like this, then you'd simply need to create a new class for SQLite that implements your database interface and instantiate that instead of your SQL Server class. If you wrote a lot of SQL Server specific C# code into your business logic, you might have a lot of coding to do.
Let's say I'm making an Xamarin.android (let's take it as any platform/framework). So, I have to use remote database for store and retrieve information.
I've been making PHP based APIs on my server that interact with database. I make my application hit up those API endpoints with the data and those APIs update the information in the database.
So, I was wondering that if I make my application directly connect to the remote MySQL, it'll be a bit faster than hitting up APIs.
Which way would be safer? I don't want anyone to be able to intercept the data or connection details to DB for obvious reasons. What would be the safest way? Or is there any other way to get this job done?
Forget about safer for now; A back-end layer (the API server in your case) will make your system more decoupled, and it is the least thing to do. suppose you suddenly decided to use another type of database, lets say PostgreSQL instead of MYSQL, you will need probably to use different format for connection strings, are you willing to update all your clients android apps with new database connection string each time? this will take time and cost and will make your application difficult to maintain, and break it at users end.
And yes, it is safer to use APIs (in your case); It is much easier to reverse engineer a client application such as android app than hacking a well secured and designed server. as you said in your question, knowing too much makes your database vulnerable to multiple types of attacks. therefore, it is much safer to encapsulate your database with an API.
However, it is not enough to use an API to make your system safer; you should also follow best practices. for example; use a HTTPS, use authentication techniques (passwords, Oauth2), use authorization, etc...
Still, i am not saying, it is not possible to use your database directly without API, you can secure your database with different techniques, it is possible but NOT recommended at all.
In my application programming experience, I have always worked with a SQL Server (or Access) database on the back end that stores application data. I'm now looking at some business requirements that work with data that would fit well in a relational database, but they require it to be stored in a single, portable, custom file that the application will create, and load from. I know it's a very common concept for an application to save off a single file or document that it can later load and continue to work on, but I'm not sure how to achieve this with complex data. Encrypting xml comes to mind, but that would be very slow to work with or potentially eat up a lot of memory if I had to load it all back into objects first. What are some options?
I recommend you use a SQLLite or Firebird embedded database. There are other options as well. They support single-file usage and will give you a clear upgrade path for future versions of you schema (upgrade SQL scripts).
I did not understand how encryption plays into this.
When running in a .NET environment I think that SQL Server Compact is worth looking into. It is basically a mini SQL Server that doesn't have to be installed and configured as a service, but instead is an dll that you reference. You can use normal data access tools like linq-to-sql and entity framework.
SQLite comes to my mind. Its a single file based DB. Here is a link to convert SQL server databases to SQLite. Also check out Using SQLite in your C# Application
We are making a fairly serious application that needs to remain agnostic to the DB a client wants to use. Initially we plan on supporting MySQL, Oracle & SQL Server. The tables & views are simple as are the queries (no real fancy SQL), therefore the question:
Use native DB drivers (MySQLDbConnection etc.) and encapsulate the logic of executing queries and processing results or
Use a generic OleDbConnection
Obviously option 2 involves no overhead, but I presuming the performance is not as great as with native access?
Note: This answer is relevant if you decide to use basic ADO.NET 2 functionality instead of an ORM (such as Entity Framework or NHibernate) or LINQ to SQL.
Let's assume you've got a connection string defined in your app.config:
<connectionStrings>
<add name="SomeConnection"
providerName="System.Data.SqlClient"
connectionString="..." />
</connectionStrings>
Notice the presence of the providerName attribute and its value. You could also put in a value for another DB provider, e.g. System.Data.SQLite.
(Note that non-standard providers, i.e. those that are not in the .NET Framework by default, need to be registered first, either in app.config or in the client machine's machine.config.)
Now, you can work with the specified database in a completely provider-agnostic fashion as follows:
using System.Configuration; // for ConfigurationManager
using System.Data; // for all interface types
using System.Data.Common; // for DbProviderFactories
var cs = ConfigurationManager.ConnectionStrings["SomeConnection"];
// ^^^^^^^^^^^^^^^^
var factory = DbProviderFactories.GetFactory(cs.ProviderName);
// ^^^^^^^^^^^^^^^
using (IDbConnection connection = factory.CreateConnection())
{
connection.ConnectionString = cs.ConnectionString;
// ^^^^^^^^^^^^^^^^^^^
connection.Open();
try
{
using (IDbCommand command = connection.CreateCommand())
{
... // do something with the database
}
}
finally
{
connection.Close();
}
}
Note how this code only works with interface types. The only place where you indicate a particular DB provider is through the providerName attribute value in the app.config file. (I've marked all the places where a setting from app.config is taken with ^^^s.)
Further reading:
Generic Coding with the ADO.NET 2.0 Base Classes and Factories:
similar to my answer, but goes into more detail.
ADO.NET Managed Providers and DataSet Developer Center:
includes, among other things, an index of available ADO.NET database providers.
IMHO using an ORM is a good design decision in order to have a database agnostic application. Switching database might be as easy as changing a config setting and connection string.
You don't need OleDbConnection to access nonspecific ADO.NET providers. Just use DbConnection et. al. See DbProviderFactories on MSDN for more info.
By including Oracle in that list, you've guaranteed that nothing will be simple.
Oracle uses a different prefix character (colon) for parameters, as compared to SQL Server that uses an "at" symbol.
Oracle uses a single data type (number) for long, int, short, boolean, float, and decimal; your code will have to be sure that you map these properly.
You must parameterize Oracle date and time values; if you try to use strings for dates in your SQL statements, you will go insane because of Oracle's date format. (Oracle uses a three-character month abbreviation; the format is 01-JAN-2010.)
Basic SQL functions for handling nulls can be different, particularly for null coalescing. ("NVL" versus "COALESCE") Oracle is much pickier about reserved words.
Oracle does not have native identity column support. Workarounds involve sequences, triggers, and requiring transactions just to retrieve an identity value from a new row.
In other words, your app can't be DB-agnostic. If you don't use an ORM, you will definitely want to build a data access layer that hides all these things from the rest of the application.
Voice of experience here. Just sayin'. For a common schema across SQL Server and Oracle, we've had to build most of the infrastructure of an ORM, while avoiding the aspects that can degrade performance. Interesting, but non-trivial, definitely!
LINQ is a highly regarded .NET ORM, partly because you can use it and stored procedures. Problem is, it's SQL Server only but people are working to provide similar functionality for Oracle & MySQL.
For database & query optimizations, I cringe at the idea of using an ORM. Data types, functions & overall syntax are not very portable in SQL. The most performant means of interacting with each database will be to tailor the model & queries to each one, but it means expertise, time and money. If need be, focus on one database vendor with the code setup to support vendor swap out & add support for other databases as necessary.
There's no good reason to avoid the most generic interfaces with the broadest support - OleDb and even ODBC if you're comfortable with them. Anything beyond that reduces the pool of products/languages/platforms/tools/developers you can work with. Being closest to the SQL metal, the vendor isn't going to introduce much inefficiency - certainly less than the more esoteric options. They've been around a long, long time to wring out any problems.
If you're going to add an abstraction layer (your own or someone else's), then that should be decided based on the merits of the abstractions introduced in your particular context, not just to have an abstraction layer (which is just more support unless there's an intentional benefit.)
As you can see, everyone's mileage varies. :) But in general, I think simpler is better.
Why not use the Microsoft Patterns & Practices Enterprise Library Data Access Application Block. There's minimal overhead and switching providers is a snap.
Quote:
The Data Access Application Block
takes advantage of these classes and
provides a model that further supports
encapsulation of database
type—specific features, such as
parameter discovery and type
conversions. Because of this,
applications can be ported from one
database type to another without
modifying the client code.
You can always make part of the application database agnostic by having the bulk of the application use the DAL as a bunch of interfaces. The DAL itself would then provide a concrete implementation for the target database.
This way, you get decoupling in the use of the DAL, but the benefit of performance improvements or vendor specific constructs within the DAL.
I have a WCF application that at present is using XML based file storage to store data that gets used to generate reports. Besides this processing decisions are made based on information stored in these XML files.
I'm now hitting volumes of around 30 000 text files. This is incredibly taxing, and the application at times comes to a grinding halt.
I've always wanted to swop out the XML DAL in favor of an RDBMS, but project managers simply won't allow it. But they would be willing to look at a serverless solution for example SQLLite. I am really tempted to just dive right in and start using it as a replacement DAL (Data Access Layer).
I would need no more than around 20 tables in the whole solution, and I would expect to get no more than around 20 000 - 100 000 transactions a day, however this is extreme, the real volumes would be less than this in most cases.
Update
I am not expecting a great deal of simultaneous connections, when I say transactions, I essentially mean 1 or 2 clients that make calls and execute against the database in order. At times there might be a possibility of external clients making quick calls to the DB. But the bulk of DB connections will be done by my WCF service, which is a back end scheduled task, not serving 100's of people across an organization.
Another good point is that I only need to retain data for 90 days, so the DB shouldn't grow too big.
My main concerns are:
How reliable is SQLLite? What if the DB File gets corrupted, will I loose all processing Data. How easy is the DB to back up? Will it handle my volumes? And lastly how well does the .net provider work (located here: http://sourceforge.net/projects/sqlite-dotnet2/).
If you have any experience with SQLLite, please post your experiences so I can make aan informed decision to switch or not.
Thanks in advance...
SQLite is as reliable as your OS and hardware.
Its transactional rate is similar to SQL server, and often faster because it's all in process.
The .NET ADO provider works great.
To back up the DB, stop the service and copy the file. If the journal file is present copy it too.
EDIT: SQLite uses UTF-8 by default so with the ADO-NET provider you should be able to avoid losing accents (just so long as you follow the typical XML in string rules).
You could consider Microsoft's Sql Compact Edition.
It's like sqlite, in terms of being a single file embedded database, but has better integration with the .net framework :)
SQLite seems reliable, and even with Microsoft's one, don't expect to receive much support in case of a corrupted database.
Given your transaction volume I'd say the fact that the DB itself is a single monolithic file with only file system locking available could be a problem.
There is no row based locking as far as I know.
I used SQLite with the .Net provider without problems in a monouser enviroment, except for one concern: accents, wich don't showed correcly. The backup is quite simply: the SQLite database is an plain text file. Simply copy it.
I use Sqlite for storing XML config data and have had no problems with it. I use the System.Data.Sqlite provider: http://sqlite.phxsoftware.com/. It's solid and has a good support forum. It also includes a LINQ provider. It also integrates with VS 2008 so you can use Server Explorer to query tables. The examples and documentation also show how to use parameterized commands and transactions for increased performance.
The release candidate for LinqPad now supports Sqlite: http://www.linqpad.net/Beta.aspx.
Sqlite stores everything in a single file, which can be backed up like any other binary file.
Sqlite only supports file-level locking, but shouldn't present a performance problem since it doesn't sound like you'll have a large number of simultaneous transactions.
Unicode shouldn't be a problem. This link in the forum addresses an area where someone was trying to read unicode characters with an incompatible utility http://sqlite.phxsoftware.com/forums/t/954.aspx.
This site shows how to do case-insenitive UTF8 comparisons using System.Data.Sqlite via a custom collator, with Russian characters as an example: http://www.codeproject.com/KB/database/SQLiteUTF8CIComparison.aspx.