Develop Reference

'Microsoft.Data.Sqlite' unexpected behaviour

There is a method in my code, which initializes sqlite3 db to a specified directory as you can see in the following code snippet:
using System;
using System.IO;
using Microsoft.Data.Sqlite;
namespace test
{
public static class DatabaseUtils
{
public static void InitializeDatabase(string directoryPath)
{
if (!Directory.Exists(directoryPath))
{
throw new Exception();
}
string sqliteFile = Path.Combine(Path.GetFullPath(directoryPath), "test.sqlite3");
if (File.Exists(sqliteFile))
{
throw new Exception();
}
else
{
try
{
File.Create(sqliteFile).Dispose();
using (SqliteConnection connection = new SqliteConnection($"Data Source={sqliteFile};Mode=ReadWrite"))
{
connection.Open();
using (SqliteCommand command = new SqliteCommand())
{
command.Connection = connection;
command.CommandText = "DROP TABLE IF EXISTS MyTable; CREATE TABLE MyTable (Primary_Key INTEGER PRIMARY KEY, Text_Entry NVARCHAR(2048) NULL)";
command.ExecuteNonQuery();
}
connection.Close();
}
}
catch (Exception)
{
throw;
}
}
}
}
}
Case 1:
When calling this this method for the first time, everything works as expected.
Which means => Database file is created with table(MyTable) that is specified in the code.
Case 2:
When calling this this method for the 2nd time, after manual database file deletion,
database file is created WITHOUT table(MyTable) that is specified in the code and the following error occurs => SQLite Error 8: 'attempt to write a readonly database'
It seems to me, as if connection to the database was not properly disposed after first run or database that was created during first run, stays cached in memory somehow.
Expected behaviour:
Method should create sqlite database file with specified tables, every time it's called, when the conditions above are met.
Used packages:
"Microsoft.Data.Sqlite, Version 6.0.2",
"PowerShellStandard.Library, Version 5.1.0"
Target framework:
.NET 6
Worth to mention:
When replacing 'File.Create()' method with 'Mode=ReadWriteCreate' ConnectionString, the file doesn't get created for the 2nd time, after manual deletion. Only during the first method run.
EDIT:
Swapped 'Microsoft.Data.Sqlite' for 'System.Data.SQLite.Core'
and now the method works as expected every time.
I guess It's either a BUG in 'Microsoft.Data.Sqlite', or I am just missing an important difference between those 2 libraries.
Would be great to have an official answer from MSFT devs as to what I was doing wrong.

After some deep digging in github, I have found an actual answer to my original question. There's a feature called 'connection pooling', which caused this 'Unexpected behaviour' that I have described in my question.
To DISABLE this feature, either add 'Pooling=False' to ConnectionString, or call 'SqliteConnection.ClearAllPools()' at the point where you want the connection to be disposed (using statement is not sufficient).

there might be a problem on opening and closing the connection 2 times:
Could you try it like this:
using (SQLiteConnection connection = new SQLiteConnection($"Data Source={SqliteFile};Mode=ReadWrite"))
{
using (SQLiteCommand command = new SQLiteCommand())
{
command.Connection = connection;
command.CommandText = "DROP TABLE IF EXISTS MyTable; CREATE TABLE MyTable (Primary_Key INTEGER PRIMARY KEY, Text_Entry NVARCHAR(2048) NULL)";
command.ExecuteNonQuery();
}
}

Ah ok, I missed one part. For me it always works like this (in short):
private SQLiteConnection OpenDataBase(string dataBasePath)
{
SQLiteConnection connection = new SQLiteConnection($#"Data Source = {dataBasePath}; Version = 3; New = True; Compress = True; ");
connection.Open();
return connection;
}
public void CreateTable(string dataBasePath)
{
using (SQLiteConnection? connection = OpenDataBase(dataBasePath))
{
string create = "DROP TABLE IF EXISTS MyTable; CREATE TABLE MyTable (Primary_Key INTEGER PRIMARY KEY, Text_Entry NVARCHAR(2048) NULL)"; ;
using (SQLiteCommand command = new SQLiteCommand(create, connection))
{
command.ExecuteNonQuery();
}
}
}

SQLCipher integration into Xamarin.Forms; SQLite Library doesn't support encryption exception

I am implementing SQLCipher into a Xamarin.Forms application. I thought everything was working until I noticed that the DB that was being created by the X.F. application was actually a SQLite3 DB w/o encryption or a password. After looking into it for a while, I haven't been able to find a solution. I am encountering an exception that says
System.InvalidOperationException: 'You specified a password in the connection string, but the native SQLite library you're using doesn't support encryption.'
I currently have 4 projects in this solution. The standard 3 in XamarinForms (Default PCL for cross platform stuff, Project.Android, and Project.iOS). In addition to those 3, I have a custom PCL that is labeled Project.Core. This PCL is responsible for all DataAccess since it implements the Repository Pattern, Unit Of Work, DbContext, etc.
In this 4th project, and within my DbContext.cs class, I have this:
// Added for more context
using System;
using System.IO;
using Microsoft.Data.Sqlite;
using Microsoft.EntityFrameworkCore;
using Microsoft.EntityFrameworkCore.Storage.ValueConversion;
using Xamarin.Forms;
private SqliteConnection connection;
protected override void OnConfiguring(DbContextOptionsBuilder optionsBuilder)
{
string connStr = Path.Combine(
path1: Environment.GetFolderPath(Environment.SpecialFolder.MyDocuments),
path2: "App.db");
string passStr = deviceIdentifier;
string path = Path.GetDirectoryName(connStr);
if (!Directory.Exists(path))
{
Directory.CreateDirectory(path);
}
// Check if db file exists
if (!File.Exists(connStr))
{
FileStream stream = File.Create(connStr);
stream.Close();
}
// DOCS => https://learn.microsoft.com/en-us/dotnet/standard/data/sqlite/encryption?tabs=netcore-cli
// => https://www.bricelam.net/2016/06/13/sqlite-encryption.html
var connectionString = new SqliteConnectionStringBuilder()
{
DataSource = connStr,
Mode = SqliteOpenMode.ReadWriteCreate,
Password = passStr
}.ToString();
// NOTE: THIS IS WHERE THE EXCEPTION IS THROWN!!!
// THE CODE BELOW THIS IS AN ALTERNATE ROUTE THAT DOENS'T WORK EITHER
**connection.Open();**
// This code doesn't throw anything, but it doesn't key the DB either
using (SqliteCommand command = connection.CreateCommand())
{
command.CommandText = "SELECT quote($password);";
command.Parameters.AddWithValue("$password", passStr);
string escapedPassword = (string)command.ExecuteScalar(); // Protects against SQL injection
command.CommandText = "PRAGMA key = " + escapedPassword /*+ ";"*/;
command.Parameters.Clear();
command.ExecuteNonQuery();
}
#if DEBUG
optionsBuilder.EnableSensitiveDataLogging();
#endif
optionsBuilder.UseSqlite(connection);
SQLitePCL.Batteries_V2.Init();
}
Through my research it appears there might be an issue with one of the SQLite/SQLCipher packages in this PCL (the PCL is targeting .NET Standard 2.0 for reference).
I currently have:
Microsoft.Data.Sqlite.Core 3.1.1 (w/ dependencies on Microsoft.Data.Sqlite.dll & SQLitePCLRaw.core 2.0.2)
SQLitePCLRaw.bundle_sqlcipher 1.1.14 (w dependencies on SQLitePCLRaw.core 2.0.2, SQLitePCLRaw.batteries_sqlcipher.dll, SQLitePCLRaw.batteries_v2.dll)
A couple of other things to note:
When viewing SQLitePCL namespace, it shows the package as being sqlitepclraw.bundle_e_sqlite3 instead of having a reference to sqlcipher.
\.nuget\packages\sqlitepclraw.bundle_e_sqlite3\2.0.2\lib\netstandard2.0\SQLitePCLRaw.batteries_v2.dll
I believe there may be an issue with that dependency, but I'm not sure and would appreciate any assistance!
Thanks in advance.
PS - Can provide more information as requested

Found a working solution.
After looking into the packages, I found that replacing the existing SQLitePCLRaw bundle package with SQLitePCLRaw.bundle_zetetic found here, resolved the issues connecting and maintaining an encrypted database.
Working code snippet is:
// StringBuilder here, and the SqliteConnection below are
// from the Microsoft.Data.Sqlite namespace v3.1.1
var connectionString = new SqliteConnectionStringBuilder()
{
DataSource = connStr,
Mode = SqliteOpenMode.ReadWriteCreate,
Password = passStr
}.ToString();
connection = new SqliteConnection(connectionString);
connection.Open();

SQLite is creating a disk file for In-memory DB

I tried to create a shareable in-memory database as per the documentation provided on SQLite Site. But I end up finding the solution to the problem.
var connectionString = "Data Source=sharedmemdb;Mode=Memory;Cache=Shared";
using (var connection1 = new SQLiteConnection(connectionString))
{
connection1.Open();
var command1 = connection1.CreateCommand();
command1.CommandText =
"CREATE TABLE Message ( Text TEXT );" +
"INSERT INTO Message ( Text ) VALUES ( 'Is there anybody out there?' );";
command1.ExecuteNonQuery();
using (var connection2 = new SQLiteConnection(connectionString))
{
connection2.Open();
var command2 = connection2.CreateCommand();
command2.CommandText = "SELECT Text FROM Message;";
var message = command2.ExecuteScalar() as string;
}
}
If I execute this code, it will create in-memory DB named as sharedmemdb and shared cache is enabled while making the connection, so this connection accessible to other connections also. If I run this first time this works pretty fine but if I close the application and run again it throws error "Table Message already exists" and this looks very strange as I created the table in-memory and this should not be available if application restarts.
After getting this error, I looked into the application directory and found the file "sharedmemdb" which means SQLite is not creating the shareable in-memory DB.
Any clue why this is happening?
After moving command to using block:
var connectionString = "Data Source =sharedmemdb; Mode = Memory; Cache = Shared";
using (var connection1 = new SQLiteConnection(connectionString))
{
connection1.Open();
using (var command1 = connection1.CreateCommand())
{
command1.CommandText =
"CREATE TABLE Message ( Text TEXT );" +
"INSERT INTO Message ( Text ) VALUES ( 'Is there anybody out there?' );";
command1.ExecuteNonQuery();
}
using (var connection2 = new SQLiteConnection(connectionString))
{
connection2.Open();
using (var command2 = connection2.CreateCommand())
{
command2.CommandText = "SELECT Text FROM Message;";
var message = command2.ExecuteScalar() as string;
}
}
}

System.Data.SQLite doesn't support the "Mode" parameter (that's Microsoft.Data.Sqlite).
However, in System.Data.SQLite you can use the "FullUri" parameter to pass arguments directly to SQLite, so you can achieve what you wanted by changing your connection string to
FullUri=file:mem.db?mode=memory&cache=shared
Protocol^ ^ ^ ^
DB Name -----| | |
Use memory mode ----+ |
Use shared cache ---------------+
(The first line being the actual connection string, the next couple of lines breaking it down)

using SQLLite Database without entity Framework in ASP.NET Core Project

I want to use SQlLite Database in an ASP.NET Core project but without using an Entity Framework.
I think I should use a class which derives from DbProviderFactory & which caters to SQLLite database (something like System.Data.SqlClient.SqlClientFactory.Instance which is used for sqlServer) but I am unable to find it.
All the examples which i came across suggest me to use entity framework, but I don't want to use it.

You have it in official repo:
https://github.com/aspnet/Microsoft.Data.Sqlite
And here is an example:
http://www.bricelam.net/2015/04/29/sqlite-on-corefx.html
EDIT: adding link content in case it disappears in the future.
The provider is built on top of the System.Data.Common contract. This contract is a very small subset of the ADO.NET provider model. Using the provider should feel very natural to anyone familiar with ADO.NET.
using (var connection = new SqliteConnection("" +
new SqliteConnectionStringBuilder
{
DataSource = "hello.db"
}))
{
connection.Open();
using (var transaction = connection.BeginTransaction())
{
var insertCommand = connection.CreateCommand();
insertCommand.Transaction = transaction;
insertCommand.CommandText = "INSERT INTO message ( text ) VALUES ( $text )";
insertCommand.Parameters.AddWithValue("$text", "Hello, World!");
insertCommand.ExecuteNonQuery();
var selectCommand = connection.CreateCommand();
selectCommand.Transaction = transaction;
selectCommand.CommandText = "SELECT text FROM message";
using (var reader = selectCommand.ExecuteReader())
{
while (reader.Read())
{
var message = reader.GetString(0);
Console.WriteLine(message);
}
}
transaction.Commit();
}
}
Batching
The only real feature that the library adds to the native SQLite interfaces is batching. The native interfaces only support compiling and executing one statement at a time. This library implements batching in a way that should feel completely transparent. Here is an example of using batching.
using (var connection = new SqliteConnection("Data Source=hello.db"))
{
var command = connection.CreateCommand();
command.CommandText =
"UPDATE message SET text = $text1 WHERE id = 1;" +
"UPDATE message SET text = $text2 WHERE id = 2";
command.Parameters.AddWithValue("$text1", "Hello");
command.Parameters.AddWithValue("$text2", "World");
connection.Open();
command.ExecuteNonQuery();
}
Platforms:
Currently, Microsoft.Data.Sqlite works on the following platforms.
.NET Framework
Mono
.NET Core
.NET Native
CoreCLR
Windows Universal

Error on add reference to SQLite.Interop.dll

I'm needing read some cookies of a specific site and I found a code on internet that probably can help me. This code uses some methods specifics of SQLite and for make this is necessary add references to some SQLite dlls, but the trouble is, when I will go add SQlite.Interop.dll, this generates a error that says:
A reference to 'C:\Program Files\System.Data.SQLite\2012\bin\SQLite.Interop.dll' could not be added. Please make sure that the file is accessible, and that it is a valid assembly or COM component.
So, someone can help me to solve this trouble. I already saw in several sites on internet somethings relative to it but until now, I don't had sucess.
This is code that I found, and he have need of SQLite dll file reference, like I said above.
using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.Threading.Tasks;
namespace ConsoleApplication1
{
class Program
{
static public IEnumerable<Tuple<string, string>> ReadCookies(string hostName)
{
if (hostName == null) throw new ArgumentNullException(hostName);
var dbPath = Environment.GetFolderPath(Environment.SpecialFolder.LocalApplicationData) + #"\Google\Chrome\User Data\Default\Cookies";
if (!System.IO.File.Exists(dbPath)) throw new System.IO.FileNotFoundException("Cant find cookie store", dbPath);
var connectionString = "Data Source=" + dbPath + ";pooling=false";
using (var conn = new System.Data.SQLite.SQLiteConnection(connectionString))
using (var cmd = conn.CreateCommand())
{
var prm = cmd.CreateParameter();
prm.ParameterName = hostName;
prm.Value = hostName;
cmd.Parameters.Add(prm);
cmd.CommandText = "SELECT name,encrypted_value FROM cookies WHERE host_key = " + hostName;
conn.Open();
using (var reader = cmd.ExecuteReader())
{
while (reader.Read())
{
var encryptedData = (byte[])reader[1];
var decodedData = System.Security.Cryptography.ProtectedData.Unprotect(encryptedData, null, System.Security.Cryptography.DataProtectionScope.CurrentUser);
var plainText = Encoding.ASCII.GetString(decodedData);
yield return Tuple.Create(reader.GetString(0), plainText);
}
}
conn.Close();
}
}
static void Main(string[] args)
{
var list = ReadCookies("facebook.com");
foreach (var item in list)
Console.WriteLine("{0} | {1}", item.Item1, item.Item2);
Console.WriteLine();
Console.ReadLine();
}
}
}

Add the package using NuGet
Install-Package System.Data.SQLite
It will download and add the appropriate references for SQLite. It looks like you are trying to add the wrong references. You should be adding references to binaries like the Core / EF6 / Linq from the SQLLite binaries.

I had the same error and I was able to resolve it with the following steps:
Go to the location (i.e) "C:\Program Files\System.Data.SQLite\2015\bin" and you will see SQLite.Interop.dll and SQLite.Interop
Copy these two files and then go to the your application bin location (i.e) "........\SQliteExample\SQliteExample\bin\Debug" and paste them there.