Is is it possible to execute multiple queries using SqlKata at the same time.
Currently I have the following code:
var latestUsersQuery = db.Query("tbl_users").Where("created_at", ">", from);
var activitiesQuery = db.Query("tbl_activities").WhereIn("user_id", latestUsersQuery.Clone());
and I am querying each one alone.
var latestUsers = latestUsersQuery.Get();
var activities = activitiesQuery().Get();
I am wondering if there is a way to execute both queries at the same time.
// for example something like this
var results = db.GetAll(latestUsersQuery, activitiesQuery);
Thanks.
Late to the game on this, but you could extend SqlKata's QueryFactory class similar to how they write the other methods; using extension methods.
using SqlKata;
using SqlKata.Execution;
using System.Collections.Generic;
using System.Linq;
// ...
public static class QueryFactoryExtensions
{
public static IEnumerable<IEnumerable<T>> GetAll<T>(this QueryFactory db, params Query[] queries)
{
return queries.Select(q =>
{
return SqlKata.Execution.QueryFactoryExtensions.Get<T>(db, q);
});
}
public static IEnumerable<IEnumerable<dynamic>> GetAll(this QueryFactory db, params Query[] queries)
{
return GetAll<dynamic>(db, queries);
}
}
The usage would look like:
var query1 = new Query();
var query2 = new Query();
var results = db.GetAll(query1, query2)
From there you could: enumerate over it, cast to an array and access by index, etc.
Bear in mind here, this is not sending one query to the database as mentioned in the comments of the OP. If performance is truly what you're after, then you might look at using the Dapper extension method QueryMultiple instead of Query (which is what SqlKata is calling on the Get() methods). I'm not familiar enough with the underlying magic that Dapper is doing to be able to confidently give you direction. Another possiblity for you could be to add extensions for GetAsync instead of Get. This may be better suited to your needs.
All of that said, at minimum I hope that this provides you with some direction in which to make a decision or potentially benchmark performance.
Related
I tried my best to explain in the title, however I am trying to achieve giving linq statements an 'alias' and still use them in dot notation. Allow me to explain further.
below we have a list that has a linq statement applied:
private List<string> _matches;
var output = _matches.Where(x => x.EntityScore == 100).ToList();
I agree that this is simple to read. However I wish to simplify it further especially when the statements start to get bigger. This is an example of linq getting longer than I care for:
private List<string> _matches;
var matchAddressList = _matches.Where(x => x.EntityDetails.Addresses.Any(x => x.Street.Equals(inputObject.Address)
&& x.StateProvinceDistrict.Equals(inputObject.State)
&& x.City.Equals(inputObject.City))).ToList();
What I am trying to do is alias certain groups of LINQ and then call that linq as a dot operator
for example:
var finalOutput = _matches.perfectMatches().addressMatches(inputObject).someOtherMatchCondition(inputObject)
I think the above line is clear and easily readable. Future devs dont necessarily have to look into the logic. They can read the business domain name and understand what it does.
I want to avoid the following line, as I believe the previous code is more clean:
var finalOutput = someOtherMatchCondition(addressMatches(perfectMatches(_matches)));
the previous line is how I feel you would go about it using functions at a basic level. However I am struggling to find a way to create an alias or encapsulate the linq logic into a business domain name and then use that as a dot operator.
I have tried expression body definitions:
public List<string> perfectMatches => _matches.Where(x => x.EntityScore == 100).ToList();
is this going to require extensions of another class? or the writing of generics? or am I perhaps unaware of a standard way of doing this?
Update: maybe this is helpfull too:
How to add custom methods for LINQ queries (C#)
It has to be an extension method to make use of the dot notation.
Do you mean something like that. It is rather pseudo code than working. You may have to play around with the types or try out some kind of generic approach:
public class ProductionCode
{
public void MyMain()
{
var myList = new List<EntityThingType>() { .... };
var newList = myList.PerfectMatches().AddressMatches(myInputObject).ToList();
}
}
public static class test
{
public static IEnumerable<EntityThingType> PerfectMatches(this IEnumerable<EntityThingType> myList)
{
return myList.Where(x => x.EntityScore == 100);
}
public static IEnumerable<EntityThingType> AddressMatches(this IEnumerable<EntityThingType> myList, MyObjectType inputObject)
{
return myList.Where(x => x.EntityDetails.Addresses.Any(x => x.Street.Equals(inputObject.Address)
&& x.StateProvinceDistrict.Equals(inputObject.State)
&& x.City.Equals(inputObject.City)));
}
}
I think what you are looking for is Extension Methods. You can have the perfectMatches() method be an extension method that takes an IEnumerable<string> and return the same. Then you can chain those together.
Code:
public static IEnumerable<TableRowModel> GetRows()
{
var to_ret = db.TableRows.select(x=> new TableRowModel(){
TableRowId = x.TableRowId,
Type = x.Type,
Name = x.Name,
CreatedAt = x.CreatedAt,
ModifiedAt = x.ModifiedAt,
Enums = x.Enums.Select(y => y.ToEnumModel()),
});
return to_ret;
}
public static EnumModel ToEnumModel(this Enum x)
{
var to_ret = new EnumModel()
{
CFPId = x.CFPId,
CreatedAt = x.CreatedAt,
ModifiedAt = x.ModifiedAt,
};
return to_ret;
}
I get the following error when using the GetRows method:
LINQ to Entities does not recognize the method
Given the error, it's understood that LINQ To Entities is not able to recognize the extension method ToEnumModel.
I would like to know if there is a way around this?
So that I would not be repeating ToEnumModel code again in GetRows extension.
Under normal circumstances, when performing an operation like a Where in EntityFramework, it isn't actually executed in memory like when you operate on an enumerable (like a List). Instead, it converted into SQL which is then executed by the Db provider. This means that doing certain things, such as using extension methods on objects, is not an option, as the converter cannot turn such things into SQL.
The quickest and easiest solution would be to load your set in-memory, where it can be operated on like any other enumerable. You can do this like so:
var result = myDbSet.AsEnumerable().Etc()
Etc() here represents all other operations you want to run. Be advised however, that this will load all data into memory, which may be prohibitively slow and expensive in some scenarios. One way to alleviate this is to put AsEnumerable() right before you need to use the extension method, thus offloading as many operations as possible to the provider. For example this:
var result = myDbSet.Where([condition]).AsEnumerable().Select(item => item.ExtensionMethod());
Might be much quicker and lighter than this:
var result = myDbSet.AsEnumerable().Where([condition]).Select(item => item.ExtensionMethod());
Your ability to use extension methods in EF is limited, but you can still extend your IQueryables
public static class Extensions
{
public static IQueryable<MyModelVM> SelectVMs(this IQueryable<MyModel> models)
{
return models.Select(x => new MyModelVM { MyModelId = x.MyModelId });
}
}
I would like to pass part of the query through as a parameter of a method.
public IEnumerable<ServiceModel> Get(/*What would the type be*/ rightQuery)
{
IEnumerable<EntityModel> leftQuery = repository.GetAll;
// the query appending might look something like this:
var outcomeBeforeMapping = leftQuery.rightQuery;
// ...rest is not important
}
The call might look like this:
service.Get((query) => query.OrderByDescending(project => project.SortOrder));
How do i go about doing this?
It sounds like you just want a Func<IEnumerable<ServiceModel>, IEnumerable<ServiceModel>> - in other words, "Given one query, I'll give you another."
public IEnumerable<ServiceModel> Get(
Func<IEnumerable<ServiceModel>, IEnumerable<ServiceModel>> transform)
{
IEnumerable<EntityModel> leftQuery = repository.GetAll;
var outcomeBeforeMapping = transform(leftQuery);
...
}
You should consider using IQueryable<T> instead of IEnumerable<T> for this if you're using EF etc, to allow the transformation to affect the SQL rather than only being applied locally.
You should also consider simply separating out the first part of your operation from the second - allow a user to call GetAll(), do what they want, then pass the result back into a second method.
I'm busy with a LINQ to SQL project that basically creates multiple threads for each entity type in my database, which constantly queries information from the DB in a thread.
Here's a pseudo example:
streamer.DefineDataExpression<Contacts>(x => x.FirstName == "Bob");
while(true)
{
List<Contacts> MyContactsResult = streamer.ResultList;
// do whatever with MyContactsResult
}
The above code doesn't exist, but this is what I have so far for the 'streamer' class (it obviously doesn't work, but you can see what I'm trying to achieve above):
public void DefineExpression(System.Linq.Expressions.Expression<System.Func<T, bool>> expression)
{
using (var db = new LINQDataContext())
{
ResultList = db.GetTable<T>().Where(expression);
}
}
How do I go about creating a method like 'DefineExpression' that will allow me to query a LINQ type dynamically?
Why not use the Dynamic LINQ provider, as mentioned by Scott Guthrie. I think that would give you everything you are looking for, because you can define the query as a string. Therefore, you can more easily build a string representation of your query, and execute on the fly.
I am just doing some experiments on Castle AR and 2nd level cache of NH. In the following two methods, I can see caching working fine but only for the repetition of the call of each. In other words if I call RetrieveByPrimaryKey twice for same PK, the object is found in cache. And if I call RetrieveAll twice, I see SQL issued only once.
But if I call RetrieveAll and then RetrieveByPrimaryKey with some PK, I see two SQL statements getting issued. My question is, Why AR does not look for that entity in cache first? Sure it would have found it there as a result of previous call to RetrieveAll.
public static T RetrieveByPrimaryKey(Guid id)
{
var res = default(T);
var findCriteria = DetachedCriteria.For<T>().SetCacheable(true);
var eqExpression = NHibernate.Criterion.Expression.Eq("Id", id);
findCriteria.Add(eqExpression);
var items = FindAll(findCriteria);
if (items != null && items.Length > 0)
res = items[0];
return res;
}
public static T[] RetrieveAll()
{
var findCriteria = DetachedCriteria.For<T>().SetCacheable(true);
var res = FindAll(findCriteria);
return res;
}
You're using caching on specific queries. that means that cache lookup is done in the following way:
search the cahce for results of a query with identical syntax AND the same parameters. If found- use cached results.
nHibernate (this has nothing to do with AR, by the way) doesn't know that logically, one query 'contains' the other. so this is why you're getting 2 db trips.
I would suggest using ISession.Get to retreive items by ID (it's the recommended method). I think (not tested it though) that Get can use items cached by other queries.
here's a nice blog post from ayende about it.