I have a quick question about the sqlite-net library which can be found here : https://github.com/praeclarum/sqlite-net.
The thing is I have no idea how collections, and custom objects will be inserted into the database, and how do I convert them back when querying, if needed.
Take this model for example:
[PrimaryKey, AutoIncrement]
public int Id { get; set; }
private string _name; // The name of the subject. i.e "Physics"
private ObservableCollection<Lesson> _lessons;
Preface: I've not used sqlite-net; rather, I spent some time simply reviewing the source code on the github link posted in the question.
From the first page on the sqlite-net github site, there are two bullet points that should help in some high level understanding:
Very simple methods for executing CRUD operations and queries safely (using parameters) and for retrieving the results of those
query in a strongly typed fashion
In other words, sqlite-net will work well with non-complex models; will probably work best with flattened models.
Works with your data model without forcing you to change your classes. (Contains a small reflection-driven ORM layer.)
In other words, sqlite-net will transform/map the result set of the SQL query to your model; again, will probably work best with flattened models.
Looking at the primary source code of SQLite.cs, there is an InsertAll method and a few overloads that will insert a collection.
When querying for data, you should be able to use the Get<T> method and the Table<T> method and there is also an Query<T> method you could take a look at as well. Each should map the results to the type parameter.
Finally, take a look at the examples and tests for a more in-depth look at using the framework.
I've worked quite a bit with SQLite-net in the past few months (including this presentation yesterday)
how collections, and custom objects will be inserted into the database
I think the answer is they won't.
While it is a very capable database and ORM, SQLite-net is targeting lightweight mobile apps. Because of this lightweight focus, the classes used are generally very simple flattened objects like:
public class Course
{
public int CourseId { get; set; }
public string Name { get; set; }
}
public class Lesson
{
public int LessonId { get; set; }
public string Name { get; set; }
public int CourseId { get; set; }
}
If you then need to Join these back together and to handle insertion and deletion of related objects, then that's down to you - the app developer - to handle. There's no auto-tracking of related objects like there is in a larger, more complicated ORM stack.
In practice, I've not found this a problem. I find SQLite-net very useful in my mobile apps.
Related
I'm building a mobile app in xamarin that has a lot of relationships between classes. For simplicity consider it to be a "University" app that has the classes: ExamPapers, Students, ExamAnswers.
The ExamPaper class would look like this
public class ExamPapers {
[ManyToMany(typeof(Student_ExamPaper))]
public List<Student> Students { get; set; }
[OneToMany]
public List<ExamAnswers> Files { get; set; }
[OneToMany(CascadeOperations = CascadeOperation.All)]
public List<ExamSection> Sections { get; set; }
public string ExamName { get; set; }
[PrimaryKey]
public string Id { get; set; }
}
So at the moment any sqlitenetextension operation (the ones that end with "WithChildren") will interact with all the relationships when I often just want to interact with one at a time. Here are some example scenarios:
A student "Lana" is just granted access to an ExamPaper "Mathematics 101". Now to grant her access I need to load; all other students with access, all the files for the exam, all the sections. Otherwise these relationships get deleted when I call "InsertOrReplaceWithChildren".
I want to find out the first question in an ExamPaper. I call "GetWithChildren(examId)". I now have an object with a lot of information I don't want (e.g. 300 students and 300 ExamAnswers).
Sorry if I missed something relevant in the documentation, but I've read it through a couple times now. https://bitbucket.org/twincoders/sqlite-net-extensions
Thanks.
Answer on your question in title: No. You cannot.
SQLite-Net-Extensions does not provide such flexible API for manipulating related data.
But there is one helpful thing that can be used in specific cases:
You can work with junction tables as simple tables through SQLite-Net methods (methods without *WithChildren postfix) if junction tables has PrimaryKey (Id).
For example if you have any data which you want to get withoud additional (related) data, just simply call Table</*class*/>() method on specific table with Where(/*your condition*/) clause for getting only data that you really need. Then you can save modified data through Update method.
Unfortunately, this thing will not work with relations update (for example, if you want to move one ExamAnswer from ExamPaper to another ExamPaper object) because all SQLite-Net-Extensions attributes inherited from Ignore SQLite-Net attribute which is ignoring in all SQLite-Net operations.
But there is another one workaround (a little hacky), you can specify second class without any SQLite-Net-Extensions attributes and that's provide you to CRUD any field in specific table.
EF Core has a "code first mentality" by default, i.e. it is supposed to be used in a code-first manner, and even though database-first approach is supported, it is described as nothing more than reverse-engineering the existing database and creating code-first representation of it. What I mean is, the model (POCO classes) created in code "by hand" (code-first), and generated from the database (by Scaffold-DbContext command), should be identical.
Surprisingly, official EF Core docs demonstrate significant differences. Here is an example of creating the model in code: https://ef.readthedocs.io/en/latest/platforms/aspnetcore/new-db.html And here is the example of reverse-engineering it from existing database: https://ef.readthedocs.io/en/latest/platforms/aspnetcore/existing-db.html
This is the entity class in first case:
public class Blog
{
public int BlogId { get; set; }
public string Url { get; set; }
public List<Post> Posts { get; set; }
}
public class Post
{
public int PostId { get; set; }
public string Title { get; set; }
public string Content { get; set; }
public int BlogId { get; set; }
public Blog Blog { get; set; }
}
and this is the entity class in second case:
public partial class Blog
{
public Blog()
{
Post = new HashSet<Post>();
}
public int BlogId { get; set; }
public string Url { get; set; }
public virtual ICollection<Post> Post { get; set; }
}
The first example is a very simple, quite obvious POCO class. It is shown everywhere in the documentation (except for the examples generated from database). The second example though, has some additions:
Class is declared partial (even though there's nowhere to be seen another partial definition of it).
Navigation property is of type ICollection< T >, instead of just List< T >.
Navigation property is initialized to new HashSet< T >() in the constructor. There is no such initialization in code-first example.
Navigation property is declared virtual.
DbSet members in a generated context class are also virtual.
I've tried scaffolding the model from database (latest tooling as of this writing) and it generates entities exactly as shown, so this is not an outdated documentation issue. So the official tooling generates different code, and the official documentation suggests writing different (trivial) code - without partial class, virtual members, construction initialization, etc.
My question is, trying to build the model in code, how should I write my code? I like using ICollection instead of List because it is more generic, but other than that, I'm not sure whether I need to follow docs, or MS tools? Do I need to declare them as virtual? Do I need to initialize them in a constructor? etc...
I know from the old EF times that virtual navigation properties allow lazy loading, but it is not even supported (yet) in EF Core, and I don't know of any other uses. Maybe it affects performance? Maybe tools try to generate future-proof code, so that when lazy-loading will be implemented, the POCO classes and context will be able to support it? If so, can I ditch them as I don't need lazy loading (all data querying is encapsulated in a repo)?
Shortly, please help me understand why is the difference, and which style should I use when building the model in code?
I try to give a short answer to each point you mentioned
partial classes are specially useful for tool-generated code. Suppose you want to implement a model-only derived property. For code first, you would just do it, wherever you want. For database first, the class file will be re-written if you update your model. So if you want to keep your extension code, you want to place it in a different file outside the managed model - this is where partial helps you to extend the class without tweaking the auto-generated code by hand.
ICollection is definitely a suitable choice, even for code first. Your database probably won't support a defined order anyway without a sorting statement.
Constructor initialization is a convenience at least... suppose you have either an empty collection database-wise or you didn't load the property at all. Without the constructor you have to handle null cases explicitely at arbitrary points in code. Whether you should go with List or HashSet is something I can't answer right now.
virtual enables proxy creation for the database entities, which can help with two things: Lazy Loading as you already mentioned and change tracking. A proxy object can track changes to virtual properties immediately with the setter, while normal objects in the context need to be inspected on SaveChanges. In some cases, this might be more efficient (not generally).
virtual IDbSet context entries allow easier design of testing-mockup contexts for unit tests. Other use cases might also exist.
I'm really looking for advice here on best practices so I will explain the situation. We have a fairly large application built on top of POCO and EF 4 with a complicated database. While we have been happy with Entity Framework there are definite performance improvements to be made for example with the following scenario (quite simplified).
We have a table called News which has a collection of users that have added it to their favourites and a collection of ratings (1 - 5) by users for example:
public class News
{
public virtual int NewsId;
public virtual string Title;
.......etc....
public virtual ICollection<User> UserFavourites { get; set; }
public virtual ICollection<Rating> Ratings { get; set; }
}
We have written a stored procedure which returns news for a user and allows us to return whether it is a favourite and whether it has already been rated by the user we are requesting the data for and the current rating for News rather than use EF to build this data from the ICollections and we end up with an object like below.
public class NewsDataModel
{
public int NewsId;
public string Title;
.......etc....
public bool IsFavourite { get; set; }
public bool IsRated { get; set; }
public double Rating { get; set; }
}
The stored procedure is much faster and a single database hit rather than EF with Lazy Loading which could be multiple calls but the data returned by the sproc does not match the POCO class for news which is above.
We have been trying to workout the best way to move forward with this as we have a INewsRepository which can either return the entity framework related class or the custom DataModel class we are populating with a stored procedure and ADO.NET. This doesn't feel right and I would appreciate any advice or insight from others experience about the best way to handle these scenarios when you want a single object with data built from multiple tables which would be a lot faster with a sproc than an entity framework call with lazy loading enabled.
Many thanks for any help
There is nothing wrong with a new method on your repository returning instances of NewsDataModel - it is still in the scope of your INewsRepository because it is data class constructed from news information. Otherwise you will have repository for every data model you defined.
I started using PetaPOCO and Dapper and they both have their own limitations. But on the contrary, they are so lightning fast than Entity Framework that I tend to let go the limitations of it.
My question is: Is there any ORM which lets us define one-to-many, many-to-one and many-to-many relationships concretely? Both Dapper.Net and PetaPOCO they kind of implement hack-ish way to fake these relationship and moreover they don't even scale very well when you may have 5-6 joins. If there isn't a single micro ORM that can let us deal with it, then my 2nd question is should I let go the fact that these micro ORMs aren't that good in defining relationships and create a new POCO entity for every single type of query that I would be executing that includes these types of multi joins? Can this scale well?
I hope I am clear with my question. If not, let me know.
I generally follow these steps.
I create my viewmodel in such a way that represents the exact data and format I want to display in a view.
I query straight from the database via PetaPoco on to my view models.
In my branch I have a
T SingleInto<T>(T instance, string sql, params object[] args);
method which takes an existing object and can map columns directly on to it matched by name. This works brilliantly for this scenario.
My branch can be found here if needed.
https://github.com/schotime/petapoco/
they don't even scale very well when you may have 5-6 joins
Yes, they don't, but that is a good thing, because when the system you will be building starts to get complex, you are free to do the joins you want, without performance penalties or headaches.
Yes, I miss when I don't needed to write all this JOINS with Linq2SQL, but then I created a simple tool to write the common joins so I get the basic SQL for any entity and then I can build from there.
Example:
[TableName("Product")]
[PrimaryKey("ProductID")]
[ExplicitColumns]
public class Product {
[PetaPoco.Column("ProductID")]
public int ProductID { get; set; }
[PetaPoco.Column("Name")]
[Display(Name = "Name")]
[Required]
[StringLength(50)]
public String Name { get; set; }
...
...
[PetaPoco.Column("ProductTypeID")]
[Display(Name = "ProductType")]
public int ProductTypeID { get; set; }
[ResultColumn]
public string ProductType { get; set; }
...
...
public static Product SingleOrDefault(int id) {
var sql = BaseQuery();
sql.Append("WHERE Product.ProductID = #0", id);
return DbHelper.CurrentDb().SingleOrDefault<Product>(sql);
}
public static PetaPoco.Sql BaseQuery(int TopN = 0) {
var sql = PetaPoco.Sql.Builder;
sql.AppendSelectTop(TopN);
sql.Append("Product.*, ProductType.Name as ProductType");
sql.Append("FROM Product");
sql.Append(" INNER JOIN ProductType ON Product.ProductoTypeID = ProductType.ProductTypeID");
return sql;
}
Would QueryFirst help here? You get the speed of micro orms, with the added comfort of every-error-a-compile-time-error, plus intellisense both for your queries and their output. You define your joins in SQL as god intended. If typing out join conditions is really bugging you, DBForge might be the answer, and because you're working in SQL, these tools are compatible, and you're not locked in.
I'm currently reading the book Pro Asp.Net MVC Framework. In the book, the author suggests using a repository pattern similar to the following.
[Table(Name = "Products")]
public class Product
{
[Column(IsPrimaryKey = true,
IsDbGenerated = true,
AutoSync = AutoSync.OnInsert)]
public int ProductId { get; set; }
[Column] public string Name { get; set; }
[Column] public string Description { get; set; }
[Column] public decimal Price { get; set; }
[Column] public string Category { get; set; }
}
public interface IProductsRepository
{
IQueryable<Product> Products { get; }
}
public class SqlProductsRepository : IProductsRepository
{
private Table<Product> productsTable;
public SqlProductsRepository(string connectionString)
{
productsTable = new DataContext(connectionString).GetTable<Product>();
}
public IQueryable<Product> Products
{
get { return productsTable; }
}
}
Data is then accessed in the following manner:
public ViewResult List(string category)
{
var productsInCategory = (category == null) ? productsRepository.Products : productsRepository.Products.Where(p => p.Category == category);
return View(productsInCategory);
}
Is this an efficient means of accessing data? Is the entire table going to be retrieved from the database and filtered in memory or is the chained Where() method going to cause some LINQ magic to create an optimized query based on the lambda?
Finally, what other implementations of the Repository pattern in C# might provide better performance when hooked up via LINQ-to-SQL?
I can understand Johannes' desire to control the execution of the SQL more tightly and with the implementation of what i sometimes call 'lazy anchor points' i have been able to do that in my app.
I use a combination of custom LazyList<T> and LazyItem<T> classes that encapsulate lazy initialization:
LazyList<T> wraps the IQueryable functionality of an IList collection but maximises some of LinqToSql's Deferred Execution functions and
LazyItem<T> will wrap a lazy invocation of a single item using the LinqToSql IQueryable or a generic Func<T> method for executing other code deferred.
Here is an example - i have this model object Announcement which may have an attached image or pdf document:
public class Announcement : //..
{
public int ID { get; set; }
public string Title { get; set; }
public AnnouncementCategory Category { get; set; }
public string Body { get; set; }
public LazyItem<Image> Image { get; set; }
public LazyItem<PdfDoc> PdfDoc { get; set; }
}
The Image and PdfDoc classes inherit form a type File that contains the byte[] containing the binary data. This binary data is heavy and i might not always need it returned from the DB every time i want an Announcement. So i want to keep my object graph 'anchored' but not 'populated' (if you like).
So if i do something like this:
Console.WriteLine(anAnnouncement.Title);
..i can knowing that i have only loaded from by db the data for the immediate Announcement object. But if on the following line i need to do this:
Console.WriteLine(anAnnouncement.Image.Inner.Width);
..i can be sure that the LazyItem<T> knows how to go and get the rest of the data.
Another great benefit is that these 'lazy' classes can hide the particular implementation of the underlying repository so i don't necessarily have to be using LinqToSql. I am (using LinqToSql) in the case of the app I'm cutting examples from, but it would be easy to plug another data source (or even completely different data layer that perhaps does not use the Repository pattern).
LINQ but not LinqToSql
You will find that sometimes you want to do some fancy LINQ query that happens to barf when the execution flows down to the LinqToSql provider. That is because LinqToSql works by translating the effective LINQ query logic into T-SQL code, and sometimes that is not always possible.
For example, i have this function that i want an IQueryable result from:
private IQueryable<Event> GetLatestSortedEvents()
{
// TODO: WARNING: HEAVY SQL QUERY! fix
return this.GetSortedEvents().ToList()
.Where(ModelExtensions.Event.IsUpcomingEvent())
.AsQueryable();
}
Why that code does not translate to SQL is not important, but just believe me that the conditions in that IsUpcomingEvent() predicate involve a number of DateTime comparisons that simply are far too complicated for LinqToSql to convert to T-SQL.
By using .ToList() then the condition (.Where(..) and then .AsQueryable() i'm effectively telling LinqToSql that i need all of the .GetSortedEvents() items even tho i'm then going to filter them. This is an instance where my filter expression will not render to SQL correctly so i need to filter it in memory. This would be what i might call the limitation of LinqToSql's performance as far as Deferred Execution and lazy loading goes - but i only have a small number of these WARNING: HEAVY SQL QUERY! blocks in my app and i think further smart refactoring could eliminate them completely.
Finally, LinqToSql can make a fine data access provider in large apps if you want it to. I found that to get the results i want and to abstract away and isolate certain things i've needed to add code here and there. And where i want more control over the actual SQL performance from LinqToSql, i've added smarts to get the desired results. So IMHO LinqToSql is perfectly ok for heavy apps that need db query optimization provided you understand how LinqToSql works. My design was originally based on Rob's Storefront tutorial so you might find it useful if you need more explanation about my rants above.
And if you want to use those lazy classes above, you can get them here and here.
Is this an efficient means of
accessing data? Is the entire table
going to be retrieved from the
database and filtered in memory or is
the chained Where() method going to
cause some LINQ magic to create an
optimized query based on the lambda?
It is efficient, if you wish to say so. The Repository exposes an IQueryable inteface, which basically represents any LINQ Data Provider (in this case Linq2Sql).
Queries are executed the moment you start iterating over the result.
IQueryable therefore supports query composition. You can add any .Where() or .GroupBy() or .OrderBy() call to a query and it will be statisfied by the database.
If you put an enumeration in your query, such as .ToList(), everything after that will happen in memory (LinqToObjects).
But I think the repository implementation is useless. I want my repository to control query execution, which is impossible when exposing IQueryable.
Yes linq2sql will generate magic to make it more efficient. It depends on you using the IQueryable interface. If you want to check clamp the SQL profiler on and you can see it generate the appropriate query.
I would recommend introducing a service layer to abstract away your dependancy on linq2sql.
I've also read that book recently and this is the SQL generated when I ran the sample code:
SELECT [t1].[Category]
FROM ( SELECT DISTINCT [t0].[Category]
FROM [Products] AS [t0] ) AS [t1] ORDER BY [t1].[Category]
I don't think you can write anything more efficient given that database. However in most real databases your Categories would be in a separate table to keep things DRY.