I have a number of different tables, each with some common columns and some columns unique to each table. I'm using the Entity Framework to operate on these tables, and LINQ queries for filtering, sorting, etc. The code below is a highly simplified example of a situation I'm running into.
var filter = "A";
using (var dc = new MyDataEntities())
{
var rpt1 = dc.Table1.Where(x => x.Name.StartsWith(filter));
var rpt2 = dc.Table2.Where(x => x.Name.StartsWith(filter));
var rpt3 = dc.Table3.Where(x => x.Name.StartsWith(filter));
}
If I decide I want to change the filter from StartsWith to Contains for example, I have to change it in 3 places. If I decide I want to add sorting, I have to add it in 3 places, etc. (As you can guess my actual code has this logic occurring across multiple classes, and a lot more than 3 instances.)
Is there some way to write a function that can take any table and operate on the common columns in that table? So my end result would be something like:
using (var dc = new MyDataEntities())
{
var rpt1 = Filter(dc.Table1);
var rpt2 = Filter(dc.Table2);
var rpt3 = Filter(dc.Table3);
}
That way I could put the logic to filter on the Name column on any of my tables into one Filter() function. Ideas?
Let me define all of the classes involved:
public class Table1
{
public string Name { get; set; }
}
public class Table2
{
public string Name { get; set; }
}
public class Table3
{
public string Name { get; set; }
}
as you can tell there POCO's will be more complex but for this example it should be fine:
public class Example
{
public void Test()
{
var t1 = new List<Table1>();
var t2 = new List<Table2>();
var t3 = new List<Table3>();
var filter = "hello";
Func<string, bool> filterFunc = (x) => x.StartsWith(filter);
var rpt1 = t1.Where(x => filterFunc(x.Name));
var rpt2 = t2.Where(x => filterFunc(x.Name));
var rpt3 = t3.Where(x => filterFunc(x.Name));
}
}
As you can see I've abstracted the filter out into a function delegate
Now a possible better solution, depends on if this really makes sense or not, is to put all of the shared columns into a base class that all of these derive from:
public class TableCommon
{
public string Name { get; set; }
}
public class Table1 : TableCommon
{}
public class Table2 : TableCommon
{}
public class Table3 : TableCommon
{}
public class Example
{
public void Test2()
{
var t1 = new List<Table1>();
var t2 = new List<Table2>();
var t3 = new List<Table3>();
var rpt1 = FilterData(t1);
var rpt2 = FilterData(t2);
var rpt3 = FilterData(t3);
}
public IEnumerable<T> FilterData<T>(IEnumerable<T> data) where T : TableCommon
{
var filter = "hello";
Func<T, bool> pred = (x) => x.Name.StartsWith(filter);
return data.Where(pred);
}
}
What's nice about this is now you can hide away your filter logic, or even have the ability to pass in different filter by making the pred variable a parameter and allowing this function to be a lot more generic.
Now if you are not comfortable with this way using a base class and a Type constraint on FilterData, then you will have to use reflection, I've had to do this for other reasons, and it gets quite messy and unreadable very fast. That or maybe something like dynamic linq which again can be very messy.
Related
I'm struggling with how to avoid repeating projection logic when using inheritance in EF Core.
Here's my scenario: I have three types:
Lesson (which is an abstract class) (Properties: Id, Title, etc.)
ArticleLesson (which inherits from Lesson) (Properties: Content, TotalWoddsCount, etc.)
VideoLesson (which inherits from Lesson) (Properties: VideoUrl, Duration, etc.)
Almost everything is handled properly by EF Core, and I'm using the default Table-Per-Hierarchy (TPH) approach.
The problem arises when I want to retrieve lessons from the database and I need some of the shared columns between ArticleLesson and VideoLesson (i.e. some of the properties of Lesson), PLUS, some of the properties specific to either ArticleLesson or VideoLesson. Here's the expression I've come up with:
var r1 = dbContext.Lessons.Select<Lesson, LessonDto>(l =>
l is ArticleLesson
? new ArticleLessonDto
{
Id = l.Id, // This is repeated below
Title = l.Title, // This is repeated below
// ...other properties that I would have to repeat below
Content = (l as ArticleLesson).Content,
}
: l is VideoLesson
? new VideoLessonDto
{
Id = l.Id, // This is repeated above
Title = l.Title, // This is repeated above
// ...other properties that I would have to repeat above
VideoUrl = (l as VideoLesson).VideoUrl,
}
: null
)
.ToList();
As you can see, I'm repeating the shared properties part, twice. There are just 2 properties that are being repeated in this example, Id and Title, but in the real world you can have dozens of these; and having to repeat all of them like this would be a h.
Is there any way to make this projection expression more succinct and avoid the repetition?
You could add a constructor to your LessonDto, ArticleLessonDto and VideoLessonDto that accepts the different shared properties.
public class LessonDto
{
public LessonDto(int id, ... other)
{
Id = id;
// ...
}
public int Id { get; set; }
}
public class ArticleLessonDto : LessonDto
{
public ArticleLessonDto(LessonDto dto) : base(dto.Id)
{
}
public string Content { get; set; }
}
var r1 = dbContext.Lessons
.Select(l => new
{
dto = new LessonDto(l.Id, ... other),
full = l
})
.Select(row => row.full is ArticleLesson
? new ArticleLessonDto(row.dto)
{
Content = (row.full as ArticleLesson).Content,
}
: row.full is VideoLesson
? new VideoLessonDto(row.dto)
{
VideoUrl = (row.full as VideoLesson).VideoUrl,
}
: (LessonDto)null
)
.ToList();
I have a class inheriting from another class
I am doing a query from the database
How do I fill in the static List without loop using linq lambda
If he finds a lot of data. this will not be fast
I want to escape from loop
public class Currencys
{
[Key]
[DatabaseGenerated(DatabaseGeneratedOption.None)]
public long Curr_Id { get; set; }
[StringLength(50)]
public string Curr_Name { get; set; }
[StringLength(50)]
public string CentName { get; set; }
[StringLength(50)]
public string curr_abbrivation { get; set; }
[StringLength(50)]
public string en_curr_name { get; set; }
[StringLength(50)]
public string en_centname { get; set; }
}
public class test1 : Currencys
{
static List<test1> _currenciesList;
public static void Fill()
{
if (_currenciesList != null)
{
_currenciesList.Clear();
}
_currenciesList = new List<test1>();
using (var context = new ContextFormeDb())
{
var list = context.Currencies.ToList();
list.ForEach(o=>
{
test1 _test1 = new test1();
_test1.Curr_Id = o.Curr_Id;
_test1.Curr_Name = o.Curr_Name;
_test1.CentName = o.CentName;
_test1.curr_abbrivation = o.curr_abbrivation;
_test1.en_curr_name = o.en_curr_name;
_test1.en_centname = o.en_centname;
_currenciesList.Add(_test1);
});
}
}
}
Is there anything better than this? without loop
list.ForEach(o=>
{
test1 _test1 = new test1();
_test1.Curr_Id = o.Curr_Id;
_test1.Curr_Name = o.Curr_Name;
_test1.CentName = o.CentName;
_test1.curr_abbrivation = o.curr_abbrivation;
_test1.en_curr_name = o.en_curr_name;
_test1.en_centname = o.en_centname;
_currenciesList.Add(_test1);
});
Is there anything better than this? without loop?
It depends on what you would call better. Faster? Probably not. Not much anyway. Easier to read and understand, easier to test, to debug, to change, to reuse? Probably.
Without Loop? there must be a loop somewhere, but it can be hidden inside a LINQ statement.
Whenever you want to fetch items from a database using entity framework, and you don't want to update the fetched items, always use Select, and select only the properties that you plan to use. Don't fetch the complete items, nor use Include. This will cost you overhead that you will only use if you update the fetched data.
So instead of:
var result = dbContext.Schools
.Where(school => school.Name == "Hogwarts")
.Include(school => school.Students)
.ToList();
consider to use:
var result = dbContext.Schools
.Where(school => school.Name == "Hogwarts")
.Select(school => new
{
// Select only the properties that you plan to use
Id = school.Id,
Name = school.Name,
...
students = dbContext.Students
.Where(student => student.SchoolId == school.Id)
.Select(student => new
{
Id = student.Id,
Name = student.Name,
...
// not needed, you know the value
// SchoolId = student.SchoolId,
})
.ToList(),
})
.ToList();
It will prevent the transfer of properties that you won't use
It will prevent that the fetched data will be copied to DbContext.ChangeTracker.
If you don't put data that won't be changed in the ChangeTracker, then SaveChanges will be faster.
So in your case, your code would be easier to understand, easier to reuse, easier to test and debug, and without "for each" if you use Select:
var fetchedData = dbContext.Currencies
.Where(currency => ...) // if you don't want all currencies
.Select(currency => new
{
// Select only the properties that you plan to use:
Id = currency.Id,
Name = currency.Name,
...
})
.ToList();
I used an anonymous type (new without specifying a class). This way you won't have to create a "dummy" class. The advantage is that you just write the properties and you'll have the object, you even have an "equality by value". If in future you need to add or remove a property, just do it, without any problem, no need to change your dummy class.
Disadvantage: you can't use it outside the current block, and certainly not as a return value of a procedure.
So if you need it outside your procedure:
.Select(currency => new Test1
{
// Select only the properties that you plan to use:
Id = currency.Id,
Name = currency.Name,
...
})
.ToList(),
If two lists are of the same type , you can use AddRange.
if not and for any reason you need to map properties or its diffrent object type, i would suggest configure AutoMapper in your app and like this you can easily convert you List from Type A to Type B and after that use AddRange
I don't know if I should take all the data or just the one I need :)
I am doing it for the first time.
I want to bind this data to Specific columns
See select
var SQLquery = (from artikel in db.DHH_Lagerverwaltung_Artikel
join hersteller in db.DHH_Lagerverwaltung_Hersteller on artikel.ID_Hersteller equals hersteller.ID_Hersteller
join kategorie in db.DHH_Lagerverwaltung_Kategorie on artikel.ID_Kategorie equals kategorie.ID_Kategorie
join bestand in db.DHH_Lagerverwaltung_Bestand on artikel.ID_Artikelnummer equals bestand.ID_Artikelnummer
join fach in db.DHH_Lagerverwaltung_Fach on bestand.ID_Fach equals fach.ID_Fach
join stellplatz in db.DHH_Lagerverwaltung_Stellplatz on fach.ID_Stellplatz equals stellplatz.ID_Stellplatz
join ebene in db.DHH_Lagerverwaltung_Ebene on stellplatz.ID_Ebene equals ebene.ID_Ebene
join regal in db.DHH_Lagerverwaltung_Regal on ebene.ID_Regal equals regal.ID_Regal
join lager in db.DHH_Lagerverwaltung_Lager on regal.ID_Lager equals lager.ID_Lager
//where lager.Raum == ""
select new {
ArtikelBezeichnung = artikel.Bezeichnung,
ArtikelEAN = artikel.EAN,
BestandsMenge = bestand.Menge,
MinMenge = bestand.Menge,
Lagerort = lager.Raum + regal.RegalNr + ebene.Ebene + stellplatz.Stellplatz + fach.Fach,
Hersteller = hersteller.Name,
Kategorie = kategorie.Name
});
Do this one line of code, underneath the query:
dataGridViewX.DataSource = new BindingSource(SQLquery.ToList(), null);
The BindingSource can work with the List<anonymoustype> the query will create
Alternatively, because you're working with anonymous types you could also make an extension method to generate you a BindingList instead:
static class ListExtensions
{
public static BindingList<T> ToBindingList<T>(this IList<T> source)
{
return new BindingList<T>(source);
}
}
You can bind a datagridview to a bindingList:
dataGridViewX.DataSource = SQLquery.ToList().ToBindingList();
Binding through a BindingSource gives some advantages for filtering, sorting, accessing the current item etc. It also allows you to arrange hierarchical data structures. If you're going to user BindingSource you should perhaps consider NOT using anonymous types, because they're compiler generated POCO classes that you don't really have any reliable access to if you wanted to dig your bindingSource's .Current object out and cast it to something you work with.
If instead you made your class a fully defined one in your own code, then you have:
collection.Select(c => new Whatever(){ Id = c.Id, Name = c.Name });
But you can work with it better:
var x = myBindingSource.Current as Whatever;
If you use anonymous types it's that as Whatever cast that you can't easily do, and you'll end up stuck with myBindingsource.Current being an object, needing either some dynamic workaround (which is not optimal when really this is a design time known class type) or a bit of a hack where you declare another anonymous type with the same order and type of parameters and rely on the compiler making them the same thing when it creates the anonymous types
Hi I created a small demo code. You just need to re-write your code and It should work. Hope it will helps you.
private class Data
{
public int? Id { get; set; }
public DateTime? DateTimeShipped { get;set;}
public DateTime? DontNeed1 { get; set; }
public DateTime? DontNeed2 { get; set; }
public bool? Ok { get; set; }
public Data()
{
}
public Data(int? id, DateTime? dateTime, bool? Ok, DateTime? DontNeed1, DateTime? DontNeed2)
{
this.Id = id;
this.DateTimeShipped = dateTime;
this.Ok = Ok;
this.DontNeed1 = DontNeed1;
this.DontNeed2 = DontNeed2;
}
}
Class Data holds values from linq select.
Also there is Dictionary which have list of desired columns including their types. This Dictionary is used for creating columns which should be in DataGridView.
Dictionary<string, Type> desiredCols = new Dictionary<string, Type>
{
{"Id", typeof(int)},
{"DateTimeShipped", typeof(DateTime)},
{ "Ok", typeof(bool)}
};
IEnumerable<Data> sqlList = (from data in dbContext.SomeTable
select new Data
{
Id = data.Id,
DateTimeShipped = data.DatumSuggestFrom,
Ok = data.Ok,
DontNeed1 = data.DatumSuggestFrom,
DontNeed2 = data.DatumSuggestTo
}).ToList();
var bindingList = new BindingList<Data>(sqlList.ToList());
foreach(var c in desiredCols)
{
DataGridViewColumn col = new DataGridViewTextBoxColumn();
col.Name = c.Key;
col.HeaderText = c.Key;
col.ValueType = c.Value;
dataGridView1.Columns.Add(col);
};
foreach(var col in bindingList)
{
dataGridView1.Rows.Add(col.Id, col.DateTimeShipped, col.Ok);
}
I have recently moved from coding in Java to c# and I am still learning the various elements of c#.
To access an existing database, which I cannot redesign, I am using Entity Frameworks 6 and 'Code First from database' to generate contexts and types representing the database tables. I am using Ling-To-SQL to retrieve the data from the database which is heavily denormalized.
My current task is create a report where each section is read from various tables, which all have a relationship to one base table.
This is my working example:
using(var db = new PaymentContext())
{
var out = from pay in db.Payment
join typ in db.Type on new { pay.ID, pay.TypeID } equals
new { typ.ID, typ.TypeID }
join base in db.BaseTable on
new { pay.Key1, pay.Key2, pay.Key3, pay.Key4, pay.Key5 } equals
new { base.Key1, base.Key2, base.Key3, base.Key4, base.Key5 }
where
base.Cancelled.Equals("0") &&
base.TimeStamp.CompareTo(startTime) > 0 &&
base.TimeStamp.CompareTo(endTime) < 1 &&
.
(other conditions)
.
group new { pay, typ } by new { typ.PaymentType } into grp
select new
{
name = grp.Key,
count = grp.Count(),
total = grp.Sum(x => x.pay.Amount)
};
}
There will be a large number of sections in the report and each section will generate a where clause which will contain the conditions shown. In some sections, the required data will be extracted from tables up to five levels below the BaseTable.
What I want to do is create a resuable where clause for each report section, to avoid a lot of duplicated code.
After a lot of searching, I tried to use the solution suggested here , but this has been superseded in Entity Framework 6.
How do I avoid duplicating code unnecessarily?
I did try to use the extension clauses you suggested, but my generated classes do not extend the BaseTable, so I had to explicitly define the link through the navigation property. As only a small number of tables will be common in the queries, I decided to apply the filters directly to each table as required. I will define these as required.
krillgar suggested moving to straight LINQ syntax, which seems like good advice. We intend to redesign our database in the near future and this will remove some of the SQL dependency. I merged the suggested filters and full LINQ syntax to access my data.
// A class to hold all the possible conditions applied for the report
// Can be applied at various levels within the select
public class WhereConditions
{
public string CancelledFlag { get; set; } = "0"; // <= default setting
public DateTime StartTime { get; set; }
public DateTime EndTime { get; set; }
}
// Class to define all the filters to be applied to any level of table
public static class QueryExtensions
{
public static IQueryable<BaseTable> ApplyCancellationFilter(this IQueryable<BaseTable> base, WhereConditions clause)
{
return base.Where(bse => bse.CancelFlag.Equals(clause.CancelledFlag));
}
public static IQueryable<BaseTable> ApplyTimeFilter(this IQueryable<BaseTable> base, WhereConditions clause)
{
return base.Where(bse => bse.TimeStamp.CompareTo(clause.StartTime) > 0 &&
bse.TimeStamp.CompareTo(clause.EndTime) < 1);
}
}
And the query is composed as follows:
using (var db = new PaymentContext())
{
IEnumerable<BaseTable> filter = db.BaseTable.ApplyCancellationFilter(clause).ApplyTimeFilter(clause);
var result = db.Payment.
Join(
filter,
pay => new { pay.Key1, pay.Key2, pay.Key3, pay.Key4, pay.Key5 },
bse => new { bse.Key1, bse.Key2, bse.Key3, bse.Key4, bse.Key5 },
(pay, bse) => new { Payment = pay, BaseTable = bse }).
Join(
db.Type,
pay => new { pay.Payment.TypeKey1, pay.Payment.TypeKey2 },
typ => new { typ.TypeKey1, typ.TypeKey2 },
(pay, typ) => new { name = typ.Description, amount = pay.Amount }).
GroupBy(x => x.name).
Select(y => new { name = y.Key,
count = y.Count(),
amount = y.Sum(z => z.amount)});
}
And then to finally execute composed query.
var reportDetail = result.ToArray(); // <= Access database here
As this query is the simplest I will have to apply, future queries will become much more complicated.
The nice thing about LINQ is that methods like Where() return an IEnumerable<T> that you can feed into the next method.
You could refactor the where clauses into extension methods akin to:
public static class PaymentQueryExtensions {
public static IQueryable<T> ApplyNotCancelledFilter(
this IQueryable<T> payments)
where T : BaseTable {
// no explicit 'join' needed to access properties of base class in EF Model
return payments.Where(p => p.Cancelled.Equals("0"));
}
public static IQueryable<T> ApplyTimeFilter(
this IQueryable<T> payments, DateTime startTime, DateTime endTime)
where T: BaseTable {
return payments.Where(p => p.TimeStamp.CompareTo(startTime) > 0
&& p.TimeStamp.CompareTo(endTime) < 1);
}
public static IGrouping<Typ, T> GroupByType(
this IQueryable<T> payments)
where T: BaseTable {
// assuming the relationship Payment -> Typ has been set up with a backlink property Payment.Typ
// e.g. for EF fluent API:
// ModelBuilder.Entity<Typ>().HasMany(t => t.Payment).WithRequired(p => p.Typ);
return payments.GroupBy(p => p.Typ);
}
}
And then compose your queries using these building blocks:
IEnumerable<Payment> payments = db.Payment
.ApplyNotCancelledFilter()
.ApplyTimeFilter(startTime, endTime);
if (renderSectionOne) {
payments = payments.ApplySectionOneFilter();
}
var paymentsByType = payments.GroupByType();
var result = paymentsByType.Select(new
{
name = grp.Key,
count = grp.Count(),
total = grp.Sum(x => x.pay.Amount)
}
);
Now that you have composed the query, execute it by enumerating. No DB access has happened until now.
var output = result.ToArray(); // <- DB access happens here
Edit After the suggestion of Ivan, I looked at our codebase. As he mentioned, the Extension methods should work on IQueryable instead of IEnumerable. Just take care that you only use expressions that can be translated to SQL, i.e. do not call any custom code like an overriden ToString() method.
Edit 2 If Payment and other model classes inherit BaseTable, the filter methods can be written as generic methods that accept any child type of BaseTable. Also added example for grouping method.
Here is simplified SQL of my tables, which is converted to LINQ to SQL model.
CREATE TABLE Campaign (
Id int PRIMARY KEY,
Name varchar NOT NULL
);
CREATE TABLE Contract (
Id int PRIMARY KEY,
CampaignId int NULL REFERENCES Campaign(Id)
);
Now i have classes like this (these are in different namespace, not entity classes from datamodel).
public class CampaignInfo {
public static CampaignModel Get(DataModel.CampaignInfo campaign) {
return new CampaignInfo {
Id = campaign.Id,
Name = campaign.Name,
Status = CampaignStatus.Get( c )
};
}
public int Id {get; set;}
public int Name {get; set;}
public CampaignStatus { get; set;}
}
public class CampaignStatus {
public static CampaignStatus Get(DataModel.Campaign campaign) {
return new CampaignStatus {
Campaign = campaign.Id, // this is just for lookup on client side
ContractCount = campaign.Contracts.Count()
// There is much more fields concerning status of campaign
};
}
public int Campaign { get; set; }
public int ContractCount {get; set;}
}
And than i am running query:
dataContext.Campaigns.Select( c => CampaignInfo.Get( c ) );
Other piece of code can do something like this:
dataContext.Campaigns.Where( c => c.Name == "DoIt" ).Select( c => CampaignInfo.Get( c ) );
Or i want to get list of statuses for campaigns:
dataContext.Campaigns.Select( c => CampaignStatus.Get( c ) );
Note: Results from those calls are converted to JSON, so there is no need to keep track on original db entities.
As you can see, there are two goals. To have control over what data are taken from database and reuse those structures in other places. However this approach is a huge mistake, because of that classes returning object and using it in expression tree.
Now i understand, it cannot magically create expression to make whole thing with one query. Instead it's getting count for every campaign separately. In rather complex scenarios it's ugly slowdown.
Is there some simple way how to achieve this ? I guess, those classes should return some expressions, but i am totally new to this field and i am not sure what to do.
The general problem, if I understand correctly, is that you have some business logic that you don't want to have repeated throughout your code (DRY). You want to be able to use this logic inside your LINQ methods.
The general solution with LINQ (over Expression trees) is to create a filter or transformation function that returns an IQueryable so you can do further processing on that query, before it is sent to the database.
Here is a way to do this:
// Reusable method that returns a query of CampaignStatus objects
public static IQueryable<CampaignStatus>
GetCampaignStatusses(this IQueryable<Compaign> campaigns)
{
return
from campaign in campaigns
new CampaignStatus
{
Campaign = campaign.Id,
ContractCount = compaign.Contracts.Count()
};
}
With this in place, you can write the following code:
using (var db = new DataModel.ModelDataContext() )
{
return
from campaign in db.Campaigns.WithContractCount()
from status in db.Campaigns.GetCampaignStatusses()
where campaign.Id == status.Campaign
select new
{
Id = campaign.Id,
Name = campaign.Name,
Status = status
};
}
Having an method that returns an IQueryable allows you to do all sorts of extra operations, without that method knowing about it. For instance, you can add filtering:
from campaign in db.Campaigns.WithContractCount()
from status in db.Campaigns.GetCampaignStatusses()
where campaign.Id == status.Campaign
where campaign .Name == "DoIt"
where status .ContractsCount < 10
select new
{
Id = campaign.Id,
Name = campaign.Name,
Status = status
};
or add extra properties to the output:
from campaign in db.Campaigns.WithContractCount()
from status in db.Campaigns.GetCampaignStatusses()
where campaign.Id == status.Campaign
select new
{
OtherProp = campaign.OtherProp,
Id = status.Campaign,
Name = campaign.Name,
Status = status
};
This will be translated to an efficient SQL query. The query will not get more records or columns from the database than strictly needed.
You could use something like:
using( var dataContext = new DataModel.ModelDataContext() )
{
dataContext.Campaigns.Select( c => new {
Id = c.Id,
Name = c.Name,
Status = new CampaignStatus()
{
ContractCount = c.Contracts.Count()
}
})
}
in your query.