I met situation I have to write app where I'm taking bunch of records from tableA then for each of record I have to do lookup against tableB to pull extra information (get another 3 columns).
TableA is a small table (<1000 records), but tableB is much bigger. Also, these resides in separate DB on the same DB server.
What would be best approach to get it optimized?
There is no option to get all records into list of objects from tableB then operate on it, rather I would need to run LINQ query for each of tableA element(object) against tableB. This is part of my MVC so could you please provide me an draft of solution, described at high level, rather than providing code.
EDIT
The tableA records need to be "enriched" all against tableB before they are displayed, in effecitve this may be +/- 500 tableA records to be lookup against tableB. Also, limitation is I have only read access to the tableB..no option to write procedures, etc
You could create a view in one of the databases that combines data in table A and B. Then map your entity to that view. Check this out https://www.mssqltips.com/sqlservertip/1990/how-to-use-sql-server-views-with-the-entity-framework/
Without seeing your models and query, it is hard to provide an accurate answer. However, the best start is within the database (I would assume SQL Server 2012+).
From your description, the generated query should look, in a very simplified way, like the following:
SELECT A.*, B.Col1, B.Col2, B.Col3
FROM Db1.dbo.TableA AS A
JOIN Db2.dbo.TableB AS B ON B.Id = A.FkToBId
According to this question and its accepted answer, there is no big difference between selecting from the same database vs. selecting from another database within the same instance.
If TableB is big, you should avoid table scans, so the following index should be a good start:
CREATE INDEX IDX_TableB_Id ON TableB (Id) INCLUDE (Col1, Col2, Col3)
However, if the schema is properly normalized, the lookup key should also be a primary key and this index should not be required. I think that if it is clustered, it might bring extra benefit.
Of course, there is a chance that your LINQ generates a slightly different query. If this is the case, edit your question and include table schema, LINQ and generated query.
[EDIT]
Using SqlQuery is an option, but I am thinking about another way:
1) Generate a database context for each database. Lets call them DbContextA and DbContextB
2) Get only required information from TableB, store it in a dictionary for fast lookups and use in an aggregated query.
var ids = DbContextA.TableA.AsNoTracking().Select(item => item.FkToBId).ToList();
var bInfo = DbContextB.TableB.AsNoTracking()
.Where(item => ids.Contains(item.id))
.ToDictionary(
item => item.Id,
item => new { item.Col1, item.Col2, item.Col3 }
);
var aggrInfo = DbContextA.TableA.AsNoTracking()
.ToList()
.Select(item => new
{
AField = item,
Col1 = bInfo[item.FkToBId],
Col2 = bInfo[item.FkToBId],
Col3 = bInfo[item.FkToBId]
};
If this does not provide the required efficiently, SqlQuery remains the only option, as a database context cannot work with two databases at once (source).
You should create a one class means .cs file and add all the columns of TableA and TableB which is required.
Let see and Example Here i am having two tables category and sub category i want the name of the category and want to show the list.
public class SubCategory
{
public int Id { get; set; }
public string Name { get; set; }
public string Image { get; set; }
public Nullable<bool> Isdisplay_ { get; set; }
public Nullable<int> CatId { get; set; }
public string CatName { get; set; }
}
var data = (from t in db.SubCategories
join ts in db.Categories on t.CatId equals ts.CategoryId
select new { a1 = t.SubId, a2 = t.SubImage, a3 = t.SubIsdisplay_, a4 =
t.SubName, a5 = ts.CategoryName }).ToList();
List<SubCategory> p1 = new List<SubCategory();
foreach (var i in data)
{
SubCategory p2 = new SubCategory();
p2.Id = i.a1;
p2.Name = i.a4;
p2.Image = i.a2;
p2.Isdisplay_ = i.a3;
p2.CatName = i.a5;
p1.Add(p2);
}
return p1;
Now you can use the list p1 to show your data.
Related
Here is a truncated example of what I'm trying to do:
var stuffTOSave = new List<SomeObject> {
public int OtherTableId { get; set; }
public List<Guid> ComponentIds { get; set; }
};
var sql = #"CREATE TABLE Components( ComponentId uniqueidentifier PRIMARY KEY )
INSERT INTO Components VALUES (#WhatGoesHere?)
SELECT * FROM OtherTable ot
JOIN Components c on c.ComponentId = ot.ComponentId
WHERE Id = #OtherTableId
DROP TABLE Components"
Connection.Execute(sql, stuffToSave);
I know from other SO questions that you can pass a list into an insert statement with Dapper, but I can't find any examples that pass a list as well as another parameter (in my example, OtherTableId), or that have a non-object list (List<Guid> as opposed to a List<SomeObject> that has properties with names to reference).
For the second issue, I could select the ComponentIds into a list to give them a name like:
stuffToSave.ComponentIds.Select(c => new { ComponentId = c })
but then I'm not sure what to put in my sql query so that dapper understands to get the ComponentId property from my list of ComponentIds (Line 7)
I would still like to know the real way of accomplishing this, but I have this workaround that uses string interpolation:
var sql = $#"CREATE TABLE Components( ComponentId uniqueidentifier PRIMARY KEY )
INSERT INTO Components VALUES ('{string.Join($"'),{Environment.NewLine}('", request.ComponentIds)}')
SELECT * FROM OtherTable ot
JOIN Components c on c.ComponentId = ot.ComponentId
WHERE Id = #OtherTableId
DROP TABLE Components"
I'm not worried about SQL Injection since this is just interpolating a list of Guids, but I'd rather avoid this method if possible.
I have three simple tables:
Employees
EmployeePhones
EmployeeEmails
Each employee has multiple emails and phones. What are the options for getting all Employees and all of their phones and emails using ADO.NET and stored procedures. I'm basically trying to fill a structure similar to this for every employee (very simplified):
public class Recipient
{
public Employee RecipientEmployee { get; set; }
public List<EmployeeEmail> RecipientEmails { get; set; }
public List<EmployeePhone> RecipientPhones { get; set; }
}
One option is to make an initial database call to get all employees. Then iterate over each employee and make additional calls to get each employees phones and emails. Obviously in doing this, I will have an initial database call plus at least one additional call for each employee returned. There could be a lot of employees.
Another option is making three DB calls to get all employees, all phones, all emails, and intersecting the three datasets somehow.
What are the other options?
Project is written using C# ADO.NET calling stored procedures in SQL Server 2008. Solution should do the same.
Why can't you do a JOIN between all the table something like below and then return a single record set from your procedure which then you can store in your application in DataSet or DataTable (Assuming that all those 3 tables do have relationship between them defined)
select e.name, e.age, e.city,
ep.primaryphone, ee.primaryemail
from Employees e join EmployeePhones ep on e.id = ep.eid
join EmployeeEmails ee on e.id = ee.eid;
Per your latest comment a sample query with group by:
select e.name, e.age, e.city,
ep.primaryphone, ee.primaryemail
from Employees e join
( select eid, primaryphone
from EmployeePhones
group by eid ) ep on e.id = ep.eid
join (
select eid, primaryemail
from EmployeeEmails
group by eid ) ee on e.id = ee.eid;
I have a database that gets loaded based on Template T. However, now I want to join other tables based on strings or passing in a "T2" Template.
How can I create a function like this to generate an IQueryable?
public void createJoinedTable<T, T2>(T2 join_table, string join_on_this_property, string order, string order_by)
where T : class
where T2 : class
{
var table = GetGenericTable<T>(); // I have the IQueryable<T> of the main table.
// now join the joined table.
var id = 1;
table = table // your starting point - table in the "from" statement
.Join(join_table, // the source table of the inner join
firsttable => post.myid, // Select the primary key (the first part of the "on" clause in an sql "join" statement)
secondtable => meta.othertableid, // Select the foreign key (the second part of the "on" clause)
(firsttable, secondtable) => new { Unknown = firsttable , Unknown2 = secondtable}) // selection
.Where(x => x.Unknown.ID == id); // where statement
table = table.CustomOrderByDescending(order_by, direction); // custom ordering by string
m_queryable = table; // record results.
}
The problem is, that I cannot do a .Join() because it is not constrained by the Entity class. It's constrained as a generic "class".
Where T : class instead of where T: MyEntityTable
Well, if I did that in the arguments, then what's the point of having a "generic join table function"?
I want to be able to join whatever I want and based on text-based arguments.
How would I use "join_on_this_property" to help me accomplish this?
BONUS Challenge: Join unlimited amounts of tables based on "List tables, List join_ON_properties"--but that could be very complicated.
I have a process that extracts customer info from multiple databases (MySql) based on a timestamp. I store this data into a DataTable. The data table represents updates to existing customer info as well as new customer info.
I want to delete any dupes in the destination database (SqlServer) based on one constant value, CompanyID, and the CustomerID. So, I thought a join would give me the RecordIDs of the dupes in the destination DB, pass the List<int> (or some collection mechanism) to the DELETE method.
What I have:
using (var context = new DataContext(SqlConnection))
{
var tblSource = context.GetTable<tblCustomerInfo>();
var dupeIDs = from currCust in tblSource
join newCust in myTable.AsEnumerable() on currCust.CompanyID equals newCust.Field<string>("CompanyID")
where currCust.CustomerID.Equals(newCust.Field<int>("CustomerID")
select currCust.RecordID;
}
This obviously does not work. I will update with the exact error messages in a bit, but this doesn't compile.
First, is my join syntax even correct for what I am wanting to achieve?
Second, how can I write this Linq to join between a DataTable and the destination SqlServer database?
Afterthought - is it possible to, once I have a collection of dupe RecordIDs, use Linq to DELETE records from the destination database?
Edit
To clarify the process, I have incoming data tables like so and contained in a DataSet:
Table1
CompanyID CustomerID Field1 Field2 ....
1 5 ... ...
1 15 ... ...
Table2
CompanyID CustomerID Field1 Field2 ....
10 125 ... ...
10 145 ... ...
Which will all go into a single database:
Destination DB
CompanyID CustomerID Field1 Field2 ....
1 5 ... ...
1 15 ... ...
1 27 ... ...
5 15 ... ...
10 125 ... ...
10 145 ... ...
11 100 ... ...
So, in this case I would delete from the destination table the items that match from tables 1 & 2. The destination database will be growing constantly so creating a List of CustomerID does not seem feasible. However, I expect daily imports of new and updated customer info to be relatively small (in the hundreds, maybe near 1000 records).
If I cannot write a single join what other method for completing this process would be appropriate? I am trying to figure something out since it looks like I cannot actually mix Linq-to-Sql and Linq-to-Objects.
Is it possible to somehow map my data table to the entity datamap, tbl_CustomerInfo, filling an otherwise immutable var, then perform the join?
Update
Here is what I have accomplished at this point and I get the results I expect from dupes:
using (DataContext context = new DataContext(SqlConnection)
{
var custInfo = context.GetTable<tbl_CustomerInfo>();
string compID = ImportCust.Rows[0]["CompanyID"].ToString();
var imports = from cust in ImportCust.AsEnumerable()
select cust.Field<int>("CustomerID");
var dupes = from cust in custInfo
join import in imports
on cust.CustomerID equals import
where cust.CompanyID == compID
select cust;
custInfo.DeleteOnSubmit(/* what goes here */);
context.SubmitChanges();
}
My question now is, what goes into the DeleteOnSubmit(...)? I feel like I have gotten so close only to be foiled by this.
I usually tackle all of this in a stored proc for efficiency.
Add an identity field to your destination table to uniquely identify the records, then use a query like this:
DELETE d
FROM DestinationTable d JOIN (
Select CompanyID, CustomerID, Min(UniqueID) AS FirstRecID
FROM DestinationTable
GROUP BY CompanyID, CustomerID) u on u.CompanyID=d.CompanyID AND u.CustomerID=d.CustomerID
WHERE d.UniqueID <> u.FirstRecID
Alternatively, you could create two lists of List<int>, containing id's from your two sources, then use the Intersect LINQ operator to find the common items.
List<int> a = new List<int>{1,2,3,4,5,6,8, 10};
List<int> b = new List<int>{1,2,99,5,6,8, 10};
var c= a.Intersect(b); //returns the items common to both lists
Here is what I have that works:
using (DataContext context = new DataContext(SqlConnection)
{
var custInfo = context.GetTable<tbl_CustomerInfo>();
string compID = ImportCust.Rows[0]["CompanyID"].ToString();
var imports = from cust in ImportCust.AsEnumerable()
select cust.Field<int>("CustomerID");
var dupes = from import in imports
join cust in custInfo
on import equals cust.CustomerID
where cust.CompanyID== pivnum
select cust;
var records = dupes.GetEnumerator();
while (records.MoveNext())
{ custInfo.DeleteOnSubmit(records.Current); }
context.SubmitChanges();
}
If there is a more efficient method, I'm interested in options.
I want to be able to do the following:
I have a model and inside there I do have an entity.
This entity has the following structure:
public class Client
{
public int Id { get; set; }
public string Name { get; set; }
public string Description { get; set; }
}
What I want now, is to just get the client name based on the id.
Therefore I wrote a stored procedure which is doing this.
CREATE PROCEDURE [Client].[GetBasics]
#Id INT
AS
BEGIN
-- SET NOCOUNT ON added to prevent extra result sets from
-- interfering with SELECT statements.
SET NOCOUNT ON;
SELECT
Name
FROM Client.Client
INNER JOIN Client.Validity ON ClientId = Client.Id
WHERE
Client.Id = #Id;
END
Now, going back to VS, I do update the model from the database with the stored procedure included.
Next step is to map this stored procedure to the client entity as a function import.
This also works fine.
Trying now to load one client's name results into an error during runtime...
"The data reader is incompatible with
the specified 'CSTestModel.Client'. A
member of the type, 'Id', does not
have a corresponding column in the
data reader with the same name."
I am OK with the message. I know how to fix this (returning as result set Id, Name, Description).
My idea behind this question is the following:
I just want to load parts of the entity, not the complete entity itself.
I have a restriction here to just use stored procedures for the entire communication towards/from the database.
Is there a solution to my problem (except creating complex types, LINQ on the result set itself)?
And if yes, can someone point me to the right direction?
Many thanks,
Dimi
Just project onto a POCO:
var q = from c in Context.Clients
select new NameOnlyPresentation
{
Id = c.Id,
Name = c.Name
};
... or just the name:
public string ClientName(int id)
{
return (from c in Context.Clients
where c.Id == id
select c.Name).FirstOrDefault();
}