This is easy for me to perform in TSQL, but I'm just sitting here banging my head against the desk trying to get it to work in EF4!
I have a table, lets call it TestData. It has fields, say: DataTypeID, Name, DataValue.
DataTypeID, Name, DataValue
1,"Data 1","Value1"
1,"Data 1","Value2"
2,"Data 1","Value3"
3,"Data 1","Value4"
I want to group on DataID/Name, and concatenate DataValue into a CSV string. The desired result should contain -
DataTypeID, Name, DataValues
1,"Data 1","Value1,Value2"
2,"Data 1","Value3"
3,"Data 1","Value4"
Now, here's how I'm trying to do it -
var query = (from t in context.TestData
group h by new { DataTypeID = h.DataTypeID, Name = h.Name } into g
select new
{
DataTypeID = g.Key.DataTypeID,
Name = g.Key.Name,
DataValues = (string)g.Aggregate("", (a, b) => (a != "" ? "," : "") + b.DataValue),
}).ToList()
The problem is that LINQ to Entities does not know how to convert this into SQL. This is part of a union of 3 LINQ queries, and I'd really like it to keep it that way. I imagine that I could retrieve the data and then perform the aggregate later. For performance reasons, that wouldn't work for my app. I also considered using a SQL server function. But that just doesn't seem "right" in the EF4 world.
Anyone care to take a crack at this?
If the ToList() is part of your original query and not just added for this example, then use LINQ to Objects on the resulting list to do the aggregation:
var query = (from t in context.TestData
group t by new { DataTypeID = t.DataTypeID, Name = t.Name } into g
select new { DataTypeID = g.Key.DataTypeID, Name = g.Key.Name, Data = g.AsEnumerable()})
.ToList()
.Select (q => new { DataTypeID = q.DataTypeID, Name = q.Name, DataValues = q.Data.Aggregate ("", (acc, t) => (acc == "" ? "" : acc + ",") + t.DataValue) });
Tested in LINQPad and it produces this result:
Some of the Answers suggest calling ToList() and then perform the calculation as LINQ to OBJECT. That's fine for a little amount of data, but what if I have a huge amount of data that I do not want to load into memory too early, then, ToList() may not be an option.
So, the better idea would be to process/format the data in the presentation layer and let the Data Access layer do only loading or saving raw data that SQL likes.
Moreover, in your presentation layer, most probably you are filtering the data by paging, or maybe you are showing one row in the details page, so, the data you will load into the memory is likely smaller than the data you load from the database. (Your situation/architecture may be different,.. but I am saying, most likely).
I had a similar requirement. My problem was to get the list of items from the Entity Framework object and create a formatted string (comma separated value)
I created a property in my View Model which will hold the raw data from the repository and when populating that property, the LINQ query won't be a problem because you are simply querying what SQL understands.
Then, I created a get-only property in my ViewModel which reads that Raw entity property and formats the data before displaying.
public class MyViewModel
{
public IEnumerable<Entity> RawChildItems { get; set; }
public string FormattedData
{
get
{
if (this.RawChildItems == null)
return string.Empty;
string[] theItems = this.RawChildItems.ToArray();
return theItems.Length > 0
? string.Format("{0} ( {1} )", this.AnotherRegularProperty, String.Join(", ", theItems.Select(z => z.Substring(0, 1))))
: string.Empty;
}
}
}
Ok, in that way, I loaded the Data from LINQ to Entity to this View Model easily without calling.ToList().
Example:
IQueryable<MyEntity> myEntities = _myRepository.GetData();
IQueryable<MyViewModel> viewModels = myEntities.Select(x => new MyViewModel() { RawChildItems = x.MyChildren })
Now, I can call the FormattedData property of MyViewModel anytime when I need and the Getter will be executed only when the property is called, which is another benefit of this pattern (lazy processing).
An architecture recommendation: I strongly recommend to keep the data access layer away from all formatting or view logic or anything that SQL does not understand.
Your Entity Framework classes should be simple POCO that can directly map to a database column without any special mapper. And your Data Access layer (say a Repository that fetches data from your DbContext using LINQ to SQL) should get only the data that is directly stored in your database. No extra logic.
Then, you should have a dedicated set of classes for your Presentation Layer (say ViewModels) which will contain all logic for formatting data that your user likes to see. In that way, you won't have to struggle with the limitation of Entity Framework LINQ. I will never pass my Entity Framework model directly to the View. Nor, I will let my Data Access layer creates the ViewModel for me. Creating ViewModel can be delegated to your domain service layer or application layer, which is an upper layer than your Data Access Layer.
Thanks to moi_meme for the answer. What I was hoping to do is NOT POSSIBLE with LINQ to Entities. As others have suggested, you have to use LINQ to Objects to get access to string manipulation methods.
See the link posted by moi_meme for more info.
Update 8/27/2018 - Updated Link (again) - https://web.archive.org/web/20141106094131/http://www.mythos-rini.com/blog/archives/4510
And since I'm taking flack for a link-only answer from 8 years ago, I'll clarify just in case the archived copy disappears some day. The basic gist of it is that you cannot access string.join in EF queries. You must create the LINQ query, then call ToList() in order to execute the query against the db. Then you have the data in memory (aka LINQ to Objects), so you can access string.join.
The suggested code from the referenced link above is as follows -
var result1 = (from a in users
b in roles
where (a.RoleCollection.Any(x => x.RoleId = b.RoleId))
select new
{
UserName = a.UserName,
RoleNames = b.RoleName)
});
var result2 = (from a in result1.ToList()
group a by a.UserName into userGroup
select new
{
UserName = userGroup.FirstOrDefault().UserName,
RoleNames = String.Join(", ", (userGroup.Select(x => x.RoleNames)).ToArray())
});
The author further suggests replacing string.join with aggregate for better performance, like so -
RoleNames = (userGroup.Select(x => x.RoleNames)).Aggregate((a,b) => (a + ", " + b))
You are so very close already. Try this:
var query = (from t in context.TestData
group h by new { DataTypeID = h.DataTypeID, Name = h.Name } into g
select new
{
DataTypeID = g.Key.DataTypeID,
Name = g.Key.Name,
DataValues = String.Join(",", g),
}).ToList()
Alternatively, you could do this, if EF doesn't allow the String.Join (which Linq-to-SQL does):
var qs = (from t in context.TestData
group h by new { DataTypeID = h.DataTypeID, Name = h.Name } into g
select new
{
DataTypeID = g.Key.DataTypeID,
Name = g.Key.Name,
DataValues = g
}).ToArray();
var query = (from q in qs
select new
{
q.DataTypeID,
q.Name,
DataValues = String.Join(",", q.DataValues),
}).ToList();
Maybe it's a good idea to create a view for this on the database (which concatenates the fields for you) and then make EF use this view instead of the original table?
I'm quite sure it's not possible in a LINQ statement or in the Mapping Details.
Related
I have a database that contains 3 tables:
Phones
PhoneListings
PhoneConditions
PhoneListings has a FK from the Phones table(PhoneID), and a FK from the Phone Conditions table(conditionID)
I am working on a function that adds a Phone Listing to the user's cart, and returns all of the necessary information for the user. The phone make and model are contained in the PHONES table, and the details about the Condition are contained in the PhoneConditions table.
Currently I am using 3 queries to obtain all the neccesary information. Is there a way to combine all of this into one query?
public ActionResult phoneAdd(int listingID, int qty)
{
ShoppingBasket myBasket = new ShoppingBasket();
string BasketID = myBasket.GetBasketID(this.HttpContext);
var PhoneListingQuery = (from x in myDB.phoneListings
where x.phonelistingID == listingID
select x).Single();
var PhoneCondition = myDB.phoneConditions
.Where(x => x.conditionID == PhoneListingQuery.phonelistingID).Single();
var PhoneDataQuery = (from ph in myDB.Phones
where ph.PhoneID == PhoneListingQuery.phonePageID
select ph).SingleOrDefault();
}
You could project the result into an anonymous class, or a Tuple, or even a custom shaped entity in a single line, however the overall database performance might not be any better:
var phoneObjects = myDB.phoneListings
.Where(pl => pl.phonelistingID == listingID)
.Select(pl => new
{
PhoneListingQuery = pl,
PhoneCondition = myDB.phoneConditions
.Single(pc => pc.conditionID == pl.phonelistingID),
PhoneDataQuery = myDB.Phones
.SingleOrDefault(ph => ph.PhoneID == pl.phonePageID)
})
.Single();
// Access phoneObjects.PhoneListingQuery / PhoneCondition / PhoneDataQuery as needed
There are also slightly more compact overloads of the LINQ Single and SingleOrDefault extensions which take a predicate as a parameter, which will help reduce the code slightly.
Edit
As an alternative to multiple retrievals from the ORM DbContext, or doing explicit manual Joins, if you set up navigation relationships between entities in your model via the navigable join keys (usually the Foreign Keys in the underlying tables), you can specify the depth of fetch with an eager load, using Include:
var phoneListingWithAssociations = myDB.phoneListings
.Include(pl => pl.PhoneConditions)
.Include(pl => pl.Phones)
.Single(pl => pl.phonelistingID == listingID);
Which will return the entity graph in phoneListingWithAssociations
(Assuming foreign keys PhoneListing.phonePageID => Phones.phoneId and
PhoneCondition.conditionID => PhoneListing.phonelistingID)
You should be able to pull it all in one query with join, I think.
But as pointed out you might not achieve alot of speed from this, as you are just picking the first match and then moving on, not really doing any inner comparisons.
If you know there exist atleast one data point in each table then you might aswell pull all at the same time. if not then waiting with the "sub queries" is nice as done by StuartLC.
var Phone = (from a in myDB.phoneListings
join b in myDB.phoneConditions on a.phonelistingID equals b.conditionID
join c in ph in myDB.Phones on a.phonePageID equals c.PhoneID
where
a.phonelistingID == listingID
select new {
Listing = a,
Condition = b,
Data = c
}).FirstOrDefault();
FirstOrDefault because single throws error if there exists more than one element.
Can the select new MobileTeamModel be refactored into a reusable method and still be read by Entity Framework? I have quite a bit of requests that need this same data and would like to reuse it but I know Entity Framework complains about this type of stuff.
var teams = new MobileListResponse<MobileTeamModel>
{
List = (from e in _divisionsRepository.DataContext.DivisionTeams.Where(#where.Expand())
orderby e.Team.Name
select new MobileTeamModel
{
Id = e.Id,
Name = e.Team.Name,
Status = e.Status,
Paid = e.Paid,
Division = e.Division.Name,
City = e.Team.TeamAddress.Address.City,
StateRegion =
e.Team.TeamAddress.Address.StateRegionId.HasValue
? e.Team.TeamAddress.Address.StateRegion.Name
: null
}).ToList()
};
EDIT
The idea is to implement the select new MobileTeamModel { ... } in a reusable way, while having EF only query the required columns.
Probably a nice extension method:
public static class MobileTeamModelExtensions
{
public static IEnumerable<MobileTeamModel> ToMobileTeamModels
(this IQueryable<DivisionTeam> instance)
{
var result = instance.Select(e =>
select new MobileTeamModel
{
Id = e.Id,
Name = e.Team.Name,
Status = e.Status,
Paid = e.Paid,
Division = e.Division.Name,
City = e.Team.TeamAddress.Address.City,
StateRegion =
e.Team.TeamAddress.Address.StateRegionId.HasValue
? e.Team.TeamAddress.Address.StateRegion.Name
: null
}).ToList()
return result;
}
}
So you could:
var query = _divisionsRepository.DataContext.DivisionTeams
.Where(#where.Expand());
var list = query.ToMobileTeamModels();
var query = query.Where(<more where>);
var list2 = query.ToMobileTeamModels();
You simply have to add a constructor or a factory (static method that return a MobileTeamModel) and receives a Team.
Then you can make a simpler query like this:
select new MobileTeamModel(e.Team) // parametreized constructor
or this
select MobileTeamModel.FromTeam(e.Team) // factory
Finallyyou can use something like AutoMapper or ValueInjecter to project the data returned by the query to another class by using conventions or mapping.
NOTE: I can't see how your clasees look like, but this is the basic idea.
EDIT: getting EF to query only the necessary columns
As for your comment, you want to reuse the construction part, and have EF only query the required columns, and not he whole entities.
Another good explanation here: Los Techies. Efficient querying with LINQ, AutoMapper and Future queries
To do so you must use Automapper, with Queryable Extensions.
WHen you use this extensions, you can use a mapping from a query result to the destination class, and have EF only query the mapped columns.
I'm currently using EF Code-First, and I'd like to SELECT data from many tables in the database. This query is a customizable query, and hence I can't predict what kind of data I will be retrieving.
At first, when I tried running ctx.Database.SqlQuery<string>(sql, param), I ran into an exception when facing a DateTime value. I'd like to do this without casting it on the server side.
Does anybody have any idea how I can go about doing it? It can be in LINQ, LINQ-SQL, or purely SQL--so long as it gets the job done! Thanks guys...
You will not get it. Linq-to-entities will not make transformation to list of strings. Your best chance is executing normal queries and do conversion and transformation your application.
Argument that you don't know which columns user selects just means you need more dynamic solution - Linq-to-entities is not a good tool for you (except if you try to use Dynamic Linq or build expression trees manually). Use ESQL or SQL directly.
When selecting data from many tables, use anonymous types to encapsulate the properties(fields) you want to select from these tables into a new entity, something like:
var query = _db.Categories.Join(
_db.Products,
c => c.CategoryId,
p => p.CategoryId,
(category, product) =>
new
{
ProductName = product.Name,
CategoryName = category.Name,
ExpiryDate = product.ExpiryDate
});
You can achieve string values by casting your data fields to string in this way:
var query = _db.Categories.Join(
_db.Products,
c => c.CategoryId,
p => p.CategoryId,
(category, product) =>
new
{
ProductName = product.Name.toString(),
CategoryName = category.Name.toString(),
ExpiryDate = product.ExpiryDate.toString()
});
I have three tables in database :
Customer, SalesManTabels, CustomerSalesManTabels
Now I use this code in Entity Framework and C# to get all customer from the Customer table
except customers that are contained in the CustomerSalesmansTabel for the same salesman:
List<CustomerSalesManTabel> CustomerSalesManList = new List<CustomerSalesManTabel>();
List<Customer> CustomerList = new List<Customer>();
MedicalCustomersDBEntities PuplicEntityForSave = new MedicalCustomersDBEntities();
private void LoadCustomerSalesManToList()
{
IEnumerable<CustomerSalesManTabel> Cus = from a in PuplicEntityForSave.CustomerSalesManTabels.Include("SalesManTabel") select a;
CustomerSalesManList.AddRange(Cus);
}
private void LoadCustomerToList()
{
MedicalCustomersDBEntities md = new MedicalCustomersDBEntities();
IEnumerable<Customer> Cus = from a in md.Customers select a;
CustomerList.AddRange(Cus);
}
IEnumerable<Guid?> CustomerSalesManIEnumerable = CustomerSalesManList.AsEnumerable().Where(s => s.SalesManId == SalesManId).Select(s => s.CustomerId);
var Cus = from nb in CustomerList
where CustomerSalesManIEnumerable.Contains(nb.Id) == false
select nb;
checkedListBoxControlNonSelected.ValueMember = "Id";
checkedListBoxControlNonSelected.DisplayMember = "FirstName";
checkedListBoxControlNonSelected.DataSource = Cus.ToList<Customer>();
This code works, but my problem with Contains because I have a huge data, I have 12000 Customer when I use Contains it takes too long when I assigned "cus" to DataSourceof checklistbox
I want another way to do code like this but with high performance ?
You're downloading all of the lists to the client, then filtering them in-memory.
This defeats the purpose of the Entity Framework.
You should run your queries directly against your DataContext:
from c in entities.Customers
where !entites.CustomerSalesManTabels.Any(s => c.Id == s.CustomerId)
select c
To start with, you are looping through a lot more than you need, because you are not realising the result in CustomerSalesManIEnumerable, so each time you use Contains it looks in the entire result from CustomerSalesManList (which is realised by AsEnumerable).
Realising the result as a list gives you less data to wade through. As you want to look for items in the result, you want a collection which uses a hash so that you get a fast lookup, like a HashSet:
HashSet<Guid?> CustomerSalesManIEnumerable = new HashSet(CustomerSalesManList.AsEnumerable().Where(s => s.SalesManId == SalesManId).Select(s => s.CustomerId));
You should however consider if this is possible to do in the database instead.
I am looking for a way to query for products in a catalog using filters on properties which have been assigned to the product based on the category to which the product belongs. So I have the following entities involved:
Products
-Id
-CategoryId
Categories
[Id, Name, UrlName]
Properties
[Id, CategoryId, Name, UrlName]
PropertyValues
[Id, PropertyId, Text, UrlText]
ProductPropertyValues
[ProductId, PropertyValueId]
When I add a product to the catalog, multiple ProductPropertyValues will be added based on the category and I would like to be able to filter all products from a category by selecting values for one or more properties. The business logic and SQL indexes and constraints make sure that all UrlNames and texts are unique for values properties and categories.
The solution will be a MVC3 EF code first based application and the routing is setup as followed:
/products/{categoryUrlName}/{*filters}
The filter routing part has a variable length so multiple filters can be applied. Each filter contains the UrlName of the property and the UrlText of the value separated by an underscore.
An url could look like this /products/websites/framework_mvc3/language_csharp
I will gather all filters, which I will hold in a list, by reading the URL. Now it is time to actually get the products based on multiple properties and I have been trying to find the right strategy.
Maybe there is another way to implement the filters. All larger web shops use category depending filters and I am still looking for the best way to implement the persistence part for this type of functionality. The suggested solutions result in an "or" resultset if multiple filters are selected. I can imagine that adding a text property to the product table in which all property values are stores as a joined string can work as well. I have no idea what this would cost performance wise. At leased there will be no complex join and the properties and their values will be received as text anyway.
Maybe the filtering mechanism can be done client side ass well.
The tricky part about this is sending the whole list into the database as a filter. Your approach of building up more and more where clauses can work:
productsInCategory = ProductRepository
.Where(p => p.Category.Name == category);
foreach (PropertyFilter pf in filterList)
{
PropertyFilter localVariableCopy = pf;
productsInCategory = from product in productsInCategory
where product.ProductProperties
.Any(pp => pp.PropertyValueId == localVariableCopy.ValueId)
select product;
}
Another way to go is to send the whole list in using the List.Contains method
List<int> valueIds = filterList.Select(pf => pf.ValueId).ToList();
productsInCategory = ProductRepository
.Where(p => p.Category.Name == category)
.Where(p => p.ProductProperties
.Any(pp => valueIds.Contains(pp.PropertyValueId)
);
IEnumerable<int> filters = filterList.Select(pf => pf.ValueId);
var products = from pp in ProductPropertyRepository
where filters.Contains(pp.PropertyValueId)
&& pp.Product.Category.Name == category
select pp.Product;
Bear in mind that as Contains is used, the filters will be passed in as sproc parameters, this means that you have to be careful not to exceed the sproc parameter limit.
I came up with a solution that even I can understand... by using the 'Contains' method you can chain as many WHERE's as you like. If the WHERE is an empty string, it's ignored (or evaluated as a select all). Here is my example of joining 2 tables in LINQ, applying multiple where clauses and populating a model class to be returned to the view.
public ActionResult Index()
{
string AssetGroupCode = "";
string StatusCode = "";
string SearchString = "";
var mdl = from a in _db.Assets
join t in _db.Tags on a.ASSETID equals t.ASSETID
where a.ASSETGROUPCODE.Contains(AssetGroupCode)
&& a.STATUSCODE.Contains(StatusCode)
&& (
a.PO.Contains(SearchString)
|| a.MODEL.Contains(SearchString)
|| a.USERNAME.Contains(SearchString)
|| a.LOCATION.Contains(SearchString)
|| t.TAGNUMBER.Contains(SearchString)
|| t.SERIALNUMBER.Contains(SearchString)
)
select new AssetListView
{
AssetId = a.ASSETID,
TagId = t.TAGID,
PO = a.PO,
Model = a.MODEL,
UserName = a.USERNAME,
Location = a.LOCATION,
Tag = t.TAGNUMBER,
SerialNum = t.SERIALNUMBER
};
return View(mdl);
}
I know this an old answer but if someone see's this I've built this project:
https://github.com/PoweredSoft/DynamicLinq
Which should be downloadable on nuget as well:
https://www.nuget.org/packages/PoweredSoft.DynamicLinq
You could use this to loop through your filter coming from query string and do
something in the lines of
query = query.Query(q =>
{
q.Compare("AuthorId", ConditionOperators.Equal, 1);
q.And(sq =>
{
sq.Compare("Content", ConditionOperators.Equal, "World");
sq.Or("Title", ConditionOperators.Contains, 3);
});
});