Comparing two collection lists - c#

I've been at this for few hours now and can't seem to find a solution. I have 2 inventory lists, one a spreadsheet and the other a data table. I need to match the spreadsheet against the data table to find out if I have missing inventory. The spreadsheet should match with what I have in the db, ie the spreadsheet is like a master so when I have missing inventory in DB I need to add it an list and build a report.
I thought by looping throught the spreadsheet and for each inventory in the spreadsheet loop through the data table I can achieve my goal but that proved to be wrong. Any ideas how I would do this?
Thanks,
Eric
Here is the method:
public void Reconcile()
{
ObjectDataSource ods = new ObjectDataSource();
ods.ID = "ods";
ods.TypeName = "";
ods.SelectMethod = "GetAssets";
ods.TypeName = "dsAssetsTableAdapters.AssetsTableAdapter";
ods.SelectParameters.Clear();
ReportDataSource rds = new ReportDataSource("dsAssets_Assets", ods);
reportViewer1.LocalReport.DataSources.Clear();
reportViewer1.LocalReport.DataSources.Add(rds);
string _list = "";
string _list_missing_SN = "";
string filename = Server.MapPath("XLS/reconcile.xls");
string sheetname = GetExcelSheetNames(filename)[0].ToString();
String sConnectionString = "Provider=Microsoft.Jet.OLEDB.4.0;" +
"Data Source=" + filename + ";" +
"Extended Properties=Excel 8.0;";
OleDbConnection objConn = new OleDbConnection(sConnectionString);
objConn.Open();
OleDbCommand objCmdSelect = new OleDbCommand("SELECT * FROM [" + sheetname + "]", objConn);
OleDbDataAdapter objAdapter1 = new OleDbDataAdapter();
objAdapter1.SelectCommand = objCmdSelect;
DataSet objDataset1 = new DataSet();
objAdapter1.Fill(objDataset1, "XLData");
string m_AssetManagement = System.Configuration.ConfigurationManager.ConnectionStrings["Asset_Management"].ToString();
List<string> SN_list = new List<string>();
SqlDataReader Assets_rd;
SqlCommand cmdMyAssets = new SqlCommand();
cmdMyAssets.Connection = new SqlConnection(m_AssetManagement);
cmdMyAssets.CommandType = CommandType.StoredProcedure;
cmdMyAssets.CommandText = "sp_Assets_Hardware_Select_by_Serial_Number";
try
{
cmdMyAssets.Connection.Open();
Assets_rd = cmdMyAssets.ExecuteReader();
string strString;
while (Assets_rd.Read())
{
strString = Assets_rd.GetSqlString(0).ToString().Trim() + "^" + Assets_rd.GetInt32(1).ToString().Trim() + "^" + Assets_rd.GetInt32(2).ToString().Trim();
SN_list.Add(strString);
}
}
catch (SqlException dbError)
{
Trace.Write("Database unavailable with Message: ", dbError.Message);
Trace.Write("Stack Trace: ", dbError.StackTrace);
throw;
}
bool record_match = false;
foreach (DataRow drXCL in objDataset1.Tables[0].Rows)
{
if (drXCL.ItemArray[1].ToString() != string.Empty)
{
try
{
string[] assetInfo = null;
assetInfo = SN_list[0].Split('^');
if (assetInfo[0].Contains(drXCL.ItemArray[1].ToString()))
{
_list += "|" + drXCL.ItemArray[1].ToString();
}
else
{
_list_missing_SN += drXCL.ItemArray[1].ToString().Trim() + "<br>";
}
}
catch (Exception SqlEx)
{
// Throw Sqw Exception
clAppExceptions.buildEmailNotification(SqlEx.Message.ToString());
}
}
else
{
//_list += "|*** NO SERIAL NUMBER ***";
}
}
if (_list_missing_SN != "")
{
Page.ClientScript.RegisterClientScriptBlock(this.Page.GetType(), "myAlert", "<script language='javascript'>alert('Following Serial Numbers were not on the spreasheet: " + _list_missing_SN + "');</script>");
}
_list += "|";
ods.SelectMethod = "GetAssetsBySerialNumbers";
ods.SelectParameters.Add("list", _list);
reportViewer1.LocalReport.ReportPath = Server.MapPath("~/Reports/Asset_List.rdlc");
ReportParameter rpCategory = new ReportParameter("ReportParameter", "These assets are gone.");
ReportParameter[] _rpCategory = { rpCategory };
reportViewer1.LocalReport.SetParameters(_rpCategory);
reportViewer1.LocalReport.Refresh();
}

I would load the master list into an array and create a second array of bools that correspond to the positions of the first array. Then looping through the datatable, when you find the element, flip the bool to true. If you cannot find it, store that element in a not found array. Once the datatable loop is finished, you can produce 2 lists. The first list is the items in the datatable but not in the master list... the not found array. The second list is created by looping through the bool array, any value of false means that the master list element was not found in the datatable.
This can then be expanded to include counts or other pieces of information that should match but do not.

I'd suggest a different approach. You could copy the data from the database and put it in the spreadsheet on a different worksheet and use the match function. You could also take the data from the spreadsheet and put it in a new table. Then use a query to find discrepencies. I don't think a programming solution is required unless this isn't a one time thing. If this is required for an application of some sort, ignore my answer:)

Don't know if this is of any use but if you have the two lists in IEnumerable sequeneces you could do something simple with LINQ.
I have an extension method I wrote for IEnumerable that I use for this purpose:
public static IEnumerable<T> NotIn<T>(this IEnumerable<T> inputSequence, IEnumerable<T> secondSequence)
{
return secondSequence == null ? new List<T>(inputSequence) : inputSequence.Where(element => !secondSequence.Contains(element));
}
If I recall correctly I ended up finding a native LINQ function that accomplished the same thing but I, of course, forgot what it was

If your just looking for a quick solution, I would just do everything in Excel. It's easy to link Excel to a DB and to link lists.
Link your DB to your Excel file (this way it's always linked to the DB)
Insert a formula to check if the (part, key, etc...) in your master list exists in your list from the DB.
Use this link to see how to link lists in Excel.

Ultimately you have many options. To make a sound decision you need to answer a few questions.
How often will this task need to be performed?
What level of resources do you have available to utilize?
How quickly does this task need to run?
How much data needs to be compared?
Once you have answered these questions, we can suggest a solid solution to you more accurately.

Keep it simple... ADO.Net will probably the simplest approach for this problem. If you fill a DataTable with the values from the spreadsheet (hopefully using OleDb) you will be able to also pull information from the Database (using either OleDb or the correct ADO.Net client.) You can then update the values back into the database for fields such as location or last seen time. These Fill and Update commands can be queries or stored procs.
If you provide more detail such as table schema I could expand my answer further.
Edit...
If you already have one of the sources in a DataTable in .Net you could put both of them in the same DataSet and write a DataView query that would do an outer join. The Outer Join would allow you to see the matched and unmatched values.
Updated...
Sorry it took so long to get back to this. (Started a new job so I have been rather busy.) I am using two spreedsheets, but there is not reason that you couldn't use thie same concept between different databases and even different ADO.Net providers. The basic idea behind this example is to create a LastSeen timestamp in your database. Then instead of looking for what isn't there, you post the latest inventroy back to the database and then query for what hasn't been updated.
var inventoryFile = "Inventory.xlsx"; //ID,Item
var databaseFile = "Database.xlsx"; //ID,Item,Type,SN,LastSeen
var connectionFormatter = "Provider=Microsoft.ACE.OLEDB.12.0;" +
"Data Source=\"{0}\";Mode=ReadWrite;" +
"Extended Properties=\"Excel 12.0 Xml;HDR=Yes;\";";
var inventoryConnectionString = string.Format(connectionFormatter,
inventoryFile);
var databaseConnectionString = string.Format(connectionFormatter,
databaseFile);
using (var inventoryConnection =
new OleDbConnection(inventoryConnectionString))
using (var databaseConnection =
new OleDbConnection(databaseConnectionString))
{
if (inventoryConnection.State != ConnectionState.Open)
inventoryConnection.Open();
if (databaseConnection.State != ConnectionState.Open)
databaseConnection.Open();
var lastSeenCmdString = "SELECT MAX(LastSeen) FROM [Sheet1$]";
var lastSeenCommand = new OleDbCommand(lastSeenCmdString,
databaseConnection);
var lastSeen = lastSeenCommand.ExecuteScalar();
var inventorySelectCmdString = "SELECT ID, Item FROM [Sheet1$]";
var inventoryCmd = new OleDbCommand(inventorySelectCmdString,
inventoryConnection);
var table = new DataTable();
var idCol = table.Columns.Add("ID", typeof(int));
var itemCol = table.Columns.Add("Item", typeof(int));
var inventoryDataAdapter = new OleDbDataAdapter(inventoryCmd);
var databaseDataAdapter = new OleDbDataAdapter();
var updateLastSeenCmdString =
"UPDATE [Sheet1$] SET LastSeen=NOW() WHERE Item=?";
var updateCmd = new OleDbCommand(updateLastSeenCmdString,
databaseConnection);
var parameter = updateCmd.Parameters.Add("Item",
OleDbType.Integer,
0,
"Item");
databaseDataAdapter.UpdateCommand = updateCmd;
inventoryDataAdapter.Fill(table);
table.AcceptChanges();
foreach (var row in table.Rows.OfType<DataRow>())
row.SetModified();
databaseDataAdapter.Update(table);
var notSeenCmdString = "SELECT ID,Item,Type,SN,LastSeen " +
"FROM [Sheet1$]" +
"WHERE LastSeen <= ?";
var notSeenCmd = new OleDbCommand(notSeenCmdString,
databaseConnection);
notSeenCmd.Parameters.Add("LastSeen", OleDbType.Date).Value = lastSeen;
databaseDataAdapter.SelectCommand = notSeenCmd;
var missingInventory = new DataTable();
databaseDataAdapter.Fill(missingInventory);
foreach (var row in missingInventory.Rows.OfType<DataRow>())
Console.WriteLine("ID: {0} Item:{1} Type:{2} SN:{3} LastSeen:{4}",
row.ItemArray);
}

Related

C# OleDbDataAdapter does not always return a value

Im currently reading an excel file with the OleDbDataAdapter, everything worked fine. But now i wanted to read another file and now some fields are empty, but there is clearly a value in that specific field in excel.
I just need informations from the file, so i read the data into a two dimensional array, later on i use the data for further calculations.
The field im trying to read is a merged cell, but this worked fine in the last file...
There is also another strange behaviour, if the same excel is already opened, the OleDbDataAdapter returns another number of rows... (I fixed this with starting an excel process before.)
This is my code:
// import the data from excel into a two dimensional list so it will be easy to navigate through
var excelData = new List<List<string>>();
var worksheetName = "theNameOfMyWorksheet";
var connectionString = "Provider = Microsoft.ACE.OLEDB.12.0; Data Source = " + pathToFile + "; Extended Properties = 'Excel 12.0'";
OleDbConnection oledbConn = new OleDbConnection(connectionString);
oledbConn.Open();
var command = new OleDbCommand("SELECT * FROM [" + worksheetName.Key + "$]", oledbConn);
// Create new Database connection
OleDbDataAdapter oleda = new OleDbDataAdapter
{
SelectCommand = command
};
// Create a DataSet which will hold the data extracted from the worksheet.
DataSet ds = new DataSet();
// Fill the DataSet from the data extracted from the excel worksheet.
oleda.Fill(ds);
foreach (var table in ds.Tables)
{
var dataTable = (DataTable)table;
for (int e = 0; e < dataTable.Rows.Count; e++)
{
var rowItems = new List<string>();
var dataRow = (DataRow)(dataTable.Rows[e]);
for (int i = 0; i < dataRow.ItemArray.Length; i++)
{
var entry = dataRow.ItemArray[i];
rowItems.Add(entry.ToString());
}
excelData.Add(rowItems);
}
}
oledbConn.Close();
return excelData;
Here you can see that there is no value after "Maschine"
And this is how my excel looks like:
As #PanagiotisKanavos recommended in the comments, i will now use ExcelDataReader.
Works perfectly for me, it is also way faster than doing it with OleDb.
Thanks!

C# get many rows in one line

in C# I'm currently getting some data from SQL, my query returns me only one column due to my select (see code below) but in some case, it could be possible that I have more than one row, so I'd like to know if is there a way to return these row in only one line of text?
Here my current code:
public bool getDBValue(string m_strValue, string m_strFilter, string m_strDBTable, out DataSet m_objDs, out string m_strError)
try
{
using (SqlConnection oConn = new SqlConnection(getConnectionString()))
{
oConn.Open();
SqlDataAdapter l_objAdapter = new SqlDataAdapter("Select [" + m_strValue + "] FROM "+ m_strDBTable+" Where ProjectUID =" + "'" + m_strFilter + "'", oConn);
m_objDs = new DataSet();
l_objAdapter.Fill(m_objDs);
}
return (true);
}
catch (Exception e)
{
m_strError = e.Message;
return (false);
}
Because for the moment, I used my data with with line:
string value = l_objDs.Tables[0].Rows[0]["ColumnName"].ToString();
but as I could have many values, it returns only one values. So any idea of how could I do that?
Thanks.
No it is not possible since you have taking value at Row[0] of Table[0] so you will get value of first row of your first Table.
string value = l_objDs.Tables[0].Rows[0]["ColumnName"].ToString();
i would rather suggest you to iterate over table rows and add it to a StringBuilder and then assign it to the string value.
Something like Below.
StringBuilder sb = new StringBuilder();
foreach (DataRow dr in l_objDs.Tables[0].Rows)
{
sb.Append(dr["ColumnName"]);
}
string value= sb.ToString();
Hope This Helps!
Just use foreach to loop through all rows:
string value = "";
foreach (DataRow row in m_objDs.Tables[0].Rows)
value += row["ColumnName"];

Bulk Update in SQL Server from C#

I have to update multiple records in SQL Server table from C#. Below are the steps I have to follow and below is the code.
The code is working but the process is taking much longer than expected.
I need a quick way to update 10000 records, not sure whether Bulk Copy would work for Update.
I have seen the other answers which has Bulk insert to temp and then update..But that update has a single statement and here I need to update the records in DB based on Excel data and for this I have to loop each excel record.So how can I achieve faster update.
1) Read the Excel Data and copied the data into a data table
string strDirectory = string. Empty;
strDirectory = System.IO.Directory.GetCurrentDirectory() + "\\" + "Filename.xlsx";
string Connection String = "Provider=Microsoft.ACE.OLEDB.12.0; Data Source = " + strDirectory + "; Extended Properties = \"Excel 12.0;HDR=YES;IMEX=1\"";
using (OleDbConnection conn = new OleDbConnection(Connection String))
{
conn.Open();
DataTable schemaTable = conn.GetOleDbSchemaTableOleDbSchemaGuid.Tables, new object[] { null, null, null, "TABLE" });
DataRow schemaRow = schemaTable. Rows[0];
string sheet = schemaRow["TABLE_NAME"].ToString();
string query = "SELECT * FROM [" + sheet + "]";
OleDbDataAdapter daexcel = new OleDbDataAdapter(query, conn);
daexcel.Fill(dt);
conn.Close();
}
2) Doing some manipulations on datatable data before updating into table.
string strsqlst = string. Empty;
using (SqlConnection sqlConn = new SqlConnection(Connectionstring))
{
sqlConn.Open();
SqlCommand cmd;
StringBuilder sb = new StringBuilder();
sb.AppendLine("DataTable content:");
foreach (DataRow row in dt.Rows)
{
if (row.ItemArray[0].ToString() == "")
break;
strsqlst = "Update table Set col1= " + row.ItemArray[4].ToString() + " ,col2= " + row.ItemArray[5].ToString() + " where <Condition>'";
cmd = new SqlCommand(strsqlst, sqlConn);
cmd.CommandType = CommandType.Text;
cmd.ExecuteNonQuery();
}
sqlConn.Close();
}
The SqlCommand can be a whole SQL batch and is not limited to a single statement. So you can create a single large batch with 10,000 UPDATE statements, or divide it into for example 20 batches of 500 each.
In other words, you can create a single command with CommandText like this:
UPDATE [T] SET Col1='Value1', Col2='Value2' WHERE [Id] = 1;
...
UPDATE [T] SET Col1='Value999', Col2='Value1000' WHERE [Id] = 500;
That said, you should use parameters for all data values (to ensure SQL injection is not possible).
If you want to handle any errors (updates failing due to invalid data) you will need something a bit more sophisticated.

query to read a column excel worksheet with OLEDB

I'm totally new with OleDB and reading excel files. I have a worksheet with 3 columns (Name - Surname - E-mail Address) and I need to:
know the rows number
read all the addresses in the third columns
extract one by one each address
I use an OpenFileDialog object (ofd) and a TextBox (excel) to display the selected file. This is my code:
if (ofd.ShowDialog() == DialogResult.OK)
{
excel.Text = ofd.FileName;
connection = "Provider=Microsoft.ACE.OLEDB.12.0;Data Source=" + excel.Text + ";Extended Properties=\"Excel 12.0 Xml;HDR=NO;IMEX=1\"";
conn.ConnectionString = connection;
conn.Open();
string name_query = "SELECT A FROM[" + ofd.SafeFileName + "]";
OleDbDataAdapter da = new OleDbDataAdapter(name_query, conn);
da.Fill(table);
conn.Close();
j = table.Rows.Count;
}
It doesn't work, a query problem in the "FROM...". I usually read this type of query:
"SELECT * FROM [Sheet1$]"
but I can't find what Sheet1$ exactly is. Someone could explain me the right query?
2) To access to each element of the table (it would contain only the third column) and save it in a string variable what I have to do?
Thanks a lot!
To get the sheetnames, you can use the default getschema functionality from the data providers (connection.getschema).
Without column headers (HDR=NO), the columns are named F1,F2,etc., so you for the third field, you could query on F3. If you want to be completely sure, you can also use getschema to get the column names of the sheet/table found with the first getschema.
Finally, to get the values in a string list, you can use a bit of Linq (see the stringlist in the example). Not sure if you meant in a single string value, but if that's the case, you can use a string.join on the linq select.
Combined code starting form connection opening:
conn.Open();
var tableschema = conn.GetSchema("Tables");
var firstsheet = tableschema.Rows[0]["TABLE_NAME"].ToString();
string name_query = "SELECT F3 FROM [" + firstsheet + "]";
OleDbDataAdapter da = new OleDbDataAdapter(name_query, conn);
da.Fill(table);
conn.Close();
j = table.Rows.Count;
var stringlist = table.Rows.Cast<DataRow>().Select(dr => dr[0].ToString()).ToList();

How can I extract an MDB file's table contents to text in C#?

A project I'm working on contains an MDB (acecss database) file. I'd like to export the contents of the tables to text, but am having a hard time finding a way to do it easily using C#. Is there a faster way than using OLEDB and queries?
Update:
Ideally I'd like to not have to statically name each table (there are hundreds) and I have to use .NET 2.0 or below.
There might be a more efficient way, but you could populate the data into a DataTable, and then export to a text file:
Getting data into the DataTable:
string connString = "Provider=Microsoft.ACE.OLEDB.12.0;data source=C:\\marcelo.accdb";
DataTable results = new DataTable();
using(OleDbConnection conn = new OleDbConnection(connString))
{
OleDbCommand cmd = new OleDbCommand("SELECT * FROM Clientes", conn);
conn.Open();
OleDbDataAdapter adapter = new OleDbDataAdapter(cmd);
adapter.Fill(results);
}
Exporting the DataTable to CSV:
EDIT I haven't tested this, but something like this should work for .NET 2.0.
//initialize the strinbuilder
StringBuilder sb = new StringBuilder();
//append the columns to the header row
string[] columns = new string[dt.Columns.Count - 1];
for (int i = 0; i < dt.Columns.Count; i++)
columns[i] = dt.Columns[i].ColumnName;
sb.AppendLine(string.Join(",", columns));
foreach (DataRow row in dt.Rows)
{
//append the data for each row in the table
string[] fields = new string[row.ItemArray.Length];
for (int x = 0; x < myDataRow.ItemArray.Length; x++)
arr[x] = row[x].ToString();
sb.AppendLine(string.Join(",", fields));
}
File.WriteAllText("test.csv", sb.ToString());
No obvious way comes to mind. Just write something that iterates through the tables and spits out the data in whatever text format you want (.csv, tab delimited, etc).
You could always write it in VBA inside of Access, but I don't know if that would make it faster or slower.
If you want to go the Interop route, you can do it in a single command with the Access TransferText method:
using Access = Microsoft.Office.Interop.Access;
using System.Runtime.InteropServices;
static void ExportToCsv(string databasePath, string tableName, string csvFile) {
Access.Application app = new Access.Application();
app.OpenCurrentDatabase(databasePath);
Access.DoCmd doCmd = app.DoCmd;
doCmd.TransferText(Access.AcTextTransferType.acExportDelim, Type.Missing, tableName, csvFile, true);
app.CloseCurrentDatabase();
Marshal.FinalReleaseComObject(doCmd);
doCmd = null;
app.Quit();
Marshal.FinalReleaseComObject(app);
app = null;
}
I do not know C#, but here is another idea, but quite rough. It uses Microsoft.Office.Interop.Access.Dao
DBEngine dbEng = new DBEngine();
Workspace ws = dbEng.CreateWorkspace("", "admin", "",
WorkspaceTypeEnum.dbUseJet);
Database db = ws.OpenDatabase("z:\\docs\\test.accdb", false, false, "");
foreach (TableDef tdf in db.TableDefs)
{
string tablename=tdf.Name;
if (tablename.Substring(0,4) != "MSys")
{
string sSQL = "SELECT * INTO [Text;FMT=Delimited;HDR=Yes;DATABASE=Z:\\Docs].[out_"
+ tablename + ".csv] FROM " + tablename;
db.Execute(sSQL);
}
}

Categories

Resources