I have read the Mirosoft Document.When we open a connection and then close it, it is possible to use the session.
I have written this block of code to run a command but I get an error message, which says there is no connection. Do you have any Idee how can I close the connection, but use the session to run a cammand:
try
{
using (AdomdConnection adomdConnection = new AdomdConnection("MY Connection String"))
{
adomdConnection.Open();
adomdConnection.Close(false);
while (true)
{
String query = #"EVALUATE { BLANK()}";
AdomdCommand adomdCommand = new AdomdCommand(query);
Console.WriteLine(adomdConnection.SessionID.ToString() + " " + DateTime.Now.ToString());
AdomdDataReader reader = adomdCommand.ExecuteReader();
reader.Close();
System.Threading.Thread.Sleep(30000);
}
}
}
catch(AdomdConnectionException ex)
{
Console.WriteLine(ex.Message.ToString());
}
In the examples shown in the document you list, it has:
/*First, try to connect to the specified data source.
If the connection string is not valid, or if the specified
provider does not support sessions, an exception is thrown. */
objConnection.ConnectionString = connectionString;
objConnection.Open();
// Now that the connection is open, retrieve the new
// active session ID.
strSessionID = objConnection.SessionID;
// Close the connection, but leave the session open.
objConnection.Close(false);
return strSessionID;
And in your code specifically, you have:
adomdConnection.Open();
adomdConnection.Close(false);
while (true)
{
String query = #"EVALUATE { BLANK()}";
AdomdCommand adomdCommand = new AdomdCommand(query);
Console.WriteLine(adomdConnection.SessionID.ToString() + " " +
DateTime.Now.ToString());
AdomdDataReader reader = adomdCommand.ExecuteReader();
reader.Close();
System.Threading.Thread.Sleep(30000);
}
Wouldn't you want to have this instead (based on the example given)?
adomdConnection.Open();
while (true)
{
String query = #"EVALUATE { BLANK()}";
AdomdCommand adomdCommand = new AdomdCommand(query);
Console.WriteLine(adomdConnection.SessionID.ToString() + " " +
DateTime.Now.ToString());
AdomdDataReader reader = adomdCommand.ExecuteReader();
reader.Close();
System.Threading.Thread.Sleep(30000);
}
adomdConnection.Close(false);
It seems as though it's complaining because you're closing the connection before you even use it, according to the order in which your code looks to be operating. Try moving the adomdConnection.Close(false); after your while loop.
Related
I have built a C# console app that accepts TCP connections from GPS reporting devices I have. I built this app to collect that data and dump it into a SQL Server table.
Currently, I have the application working, but it has a bug I can't seem to figure out. As the GPS devices make connections, one out of random 1-10 successful connections give me an index out of range exception.
When I dump the raw data it does not look like something the device is sending me. Would any of you happen to know what is causing this? Also, once I get this application working correctly, it could be receiving up to 3-5k connections a minute, do you think this code could handle this?
This is the error I receive every so often, with the dump of misc data:
Image of error
This is my code:
namespace GPS2DB
{
class Program
{
static void Main(string[] args)
{
try
{
IPAddress ipAddress = IPAddress.Parse("10.71.150.253");
Console.WriteLine("Waiting for Tracker Connections...");
TcpListener listener = new TcpListener(ipAddress, 10000);
listener.Start();
while (true)
{
Socket client = listener.AcceptSocket();
Console.WriteLine("Connection accepted.");
var childSocketThread = new Thread(() =>
{
byte[] data = new byte[1024];
int size = client.Receive(data);
string gpsData = "";
for (int i = 0; i < size; i++)
{
Console.Write(Convert.ToChar(data[i]));
gpsData = gpsData + Convert.ToChar(data[i]);
}
string txt = gpsData;
string txt2 = (txt.Trim(new Char[] { '$', '#' }));
String[] values = txt2.Split(',');
//Console.WriteLine(txt2);
/*
Console.WriteLine("Unit ID: " + values[0]);
Console.WriteLine("Event Code: " + values[1]);
Console.WriteLine("UTC Date: " + values[2]);
Console.WriteLine("UTC Time: " + values[3]);
Console.WriteLine("Lat: " + values[4]);
Console.WriteLine("Long: " + values[5]);
Console.WriteLine("Speed: " + values[7]);
Console.WriteLine("Heading: " + values[11]);
Console.WriteLine("V+: " + values[16]);
Console.WriteLine("Cell Strength: " + values[17]);
Console.WriteLine("GPS Status: " + values[18]);
Console.WriteLine("Fuel Level: " + values[20]);
*/
//dump 2 database
string connectionString = "Data Source=DVT501;Initial Catalog=VehicleTracking;Persist Security Info=True;User ID=TABLE;Password=PASS";
using (SqlConnection connection = new SqlConnection(connectionString))
{
SqlCommand cmd = new SqlCommand("INSERT INTO Data_Dump (uid, eventCode, utcDate, utcTime, lat, long, speed, heading, voltage, cellStrength, gpsStatus, fuelLevel) VALUES (#uid, #eventCode, #utcDate, #utcTime, #lat, #long, #speed, #heading, #voltage, #cellStrength, #gpsStatus, #fuelLevel)");
cmd.CommandType = System.Data.CommandType.Text;
cmd.Connection = connection;
try
{
cmd.Parameters.AddWithValue("#uid", values[0]);
cmd.Parameters.AddWithValue("#eventCode", values[1]);
cmd.Parameters.AddWithValue("#utcDate", values[2]);
cmd.Parameters.AddWithValue("#utcTime", values[3]);
cmd.Parameters.AddWithValue("#lat", values[4]);
cmd.Parameters.AddWithValue("#long", values[5]);
cmd.Parameters.AddWithValue("#speed", values[7]);
cmd.Parameters.AddWithValue("#heading", values[11]);
cmd.Parameters.AddWithValue("#voltage", values[16]);
cmd.Parameters.AddWithValue("#cellStrength", values[17]);
cmd.Parameters.AddWithValue("#gpsStatus", values[18]);
cmd.Parameters.AddWithValue("#fuelLevel", values[20]);
connection.Open();
cmd.ExecuteNonQuery();
}
catch (System.IndexOutOfRangeException e)
{
Console.WriteLine("IndexOutOfRangeException caught" + e);
Console.WriteLine(txt);
}
}
//end dump
Console.WriteLine();
client.Close();
});
childSocketThread.Start();
}
listener.Stop();
}
catch (Exception e)
{
Console.WriteLine("Error: " + e.StackTrace);
Console.ReadLine();
}
}
}
}
The error is with the incoming data, it's either a gps device that's configured to send data differently or some random tcp event. Check the port number you're using in a google search and make sure it's not reserved for something else.
This code will definitely not handle that many connections, you iterate through a byte array and convert one character at a time (use System.Text.Encoding.ASCII.GetString(byte[]) instead), you open and close a connection to the sql server within the receive block and so on. In order to handle that kind of activity you need to just read the data and put it in a bus or temp storage to be bulk processed.
You are assuming that you will read one message at a time. TCP provides a boundaryless stream of bytes. You can very well read a partial message or multiple messages.
How to deal with that depends on the format of the stream. If it is line based StreamReader.ReadLine() is a great solution.
Title explains a small part so let me explain 2 scenarios. Scenario 1 is raising errors, scenario 2 works like a charm.
Scenario 1:
I checkout a document with the method below, when the document is saved to a location where already is a file with that name it gets overwritten, But surprisingly it also locks the file for some reason:
public bool SaveDocument(int bestandsId, string fileName, string path)
{
//Initialize the Sql Query
var sql = "SELECT DATA FROM Documenten WHERE BESTAND_ID = " + bestandsId;
//Initialize SqlConnection
var connection = new SqlConnection(Instellingen.Instance.DmsConnectionString);
//Initialize SqlCommand
var command = new SqlCommand(sql, connection);
try
{
//Open Connection
connection.Open();
//Fill 'data' from command.ExecuteScalar()
var data = (byte[]) command.ExecuteScalar();
//Write 'data' to file.
File.WriteAllBytes(path + #"\" + fileName, data);
//Return true if no exceptions are raised.
return true;
}
catch (Exception ex)
{
//Initialize Dms Exception
var dmsEx = new DmsException(ex);
//Write Dms Exception to Log File.
DmsException.WriteErrorsToLog(dmsEx);
//Return false, because something went wrong...
return false;
}
finally
{
//Close Sql Connection
connection.Close();
}
}
The method runs smoothly. No problems occur. But when I check in the document with the method below, I get this exception:
Scenario 2:
When I use the SaveDocument method to save the document to a location where there isn't a file with the same name, the file is newly created and is ready to be edited or what ever you want to do with it.
Using scenario 2 is working perfect. The document is ready to be checked in again without receiving an error as shown in the picture above.
Request for code by: #CodeCaster
---------------------------------BEGIN EDIT---------------------------------
public static bool InsertDocument(Document document)
{
try
{
//Exception is thrown when Initializing the FileStream
var fileStream = new FileStream(document.Fileinfo.FullName, FileMode.Open, FileAccess.Read);
var binaryReader = new BinaryReader(fileStream);
var totalNumberOfBytes = new FileInfo(document.Fileinfo.FullName).Length;
var data = binaryReader.ReadBytes((Int32) totalNumberOfBytes);
fileStream.Close();
fileStream.Dispose();
binaryReader.Close();
binaryReader.Dispose();
var pdftext = string.Empty;
try
{
if (document.DocumentType == ".pdf")
{
var reader = new PdfReader(document.Fileinfo.FullName);
var text = string.Empty;
for (var page = 1; page <= reader.NumberOfPages; page++)
{
text += PdfTextExtractor.GetTextFromPage(reader, page);
}
reader.Close();
pdftext = text;
}
}
catch (Exception ex)
{
var dmsEx = new DmsException(ex);
DmsException.WriteErrorsToLog(dmsEx);
}
return InsertIntoDatabase(document.BestandsNaam, document.Eigenaar, document.Omschrijving,
document.DatumToevoeg.ToString(), document.DatumIncheck.ToString(),
document.DatumUitcheck.ToString(), document.UitgechecktDoor,
document.DocumentType, data, pdftext, document.Versie, document.Medewerker,
document.DossierNummer, document.PersonalFolderId.ToString(),
document.DossierFolderId, -1, document.DocumentProgres,
document.OriBestandId.ToString(), 0);
}
catch (Exception ex)
{
var dmsEx = new DmsException("Fout bij inlezen voor toevoeging van nieuw document",
"Klasse Document (InsertDocument)", ex);
ExceptionLogger.LogError(dmsEx);
return false;
}
}
---------------------------------END EDIT---------------------------------
My questions:
What is the cause for the file being locked when it gets overwritten?
How can I prevent this from happening?
Is there some sort of function or parameter that I can set so it doesn't get locked?
Using a tool called "Unlocker" I managed to see what program is locking the file, and YES -> DMS.exe is my application.......:
using(var stream = File.Create(newPath)){}
File.WriteAllBytes(newPath, item.File);
With StreamWriter
using (FileStream fs = File.Create(newPath))
{
fs.Write(item.File, 0, item.File.Length);
}
Or:
File.WriteAllBytes(newPath, item.File);
Reference: "The process cannot access the file because it is being used by another process" with Images
We have been using C# and mysql with .net connector but with individual commit where at times it fail to commit all. So we are now moving to this tool http://www.devart.com/dotconnect/mysql/ which supports distributed transaction. All works fine just that we are not too sure how to put the multiple connection. Method 1 is nesting each of the connection into one another. Method 2 is separately. Where we should be closing the connection or is handled by the transScope.Complete(); and transScope.Dispose();
Method 1.
using (TransactionScope transScope = new TransactionScope())
{
string myConnStringLocal = "User Id=***;Password=****;Host=" + globalSettings.settingLocalIP + ";Database=" + globalSettings.settingLocalDB;
using (MySqlConnection connectionLocal = new MySqlConnection(myConnStringLocal))
{
connectionLocal.open()
string myConnStringCentral = "User Id=***;Password=*****;Host=" + globalSettings.settingCentralIP + ";Database=" + globalSettings.settingCentralDB;
using (MySqlConnection connectionCentral = new MySqlConnection(myConnStringCentral))
{
connectionCentral.Open();
string myConnStringCentralCopy = "User Id=*****;Password=*****;Host=" + globalSettings.settingCentralCopyIP + ";Database=" + globalSettings.settingCentralCopyDB;
using (MySqlConnection connectionCentralCopy = new MySqlConnection(myConnStringCentralCopy))
{
connectionCentralCopy.Open();
}
}
}
if (rollbackBoolean == 0)
transScope.Complete();
else
transScope.Dispose();
}
Method 2
using (TransactionScope transScope = new TransactionScope())
{
string myConnStringLocal = "User Id=***;Password=****;Host=" + globalSettings.settingLocalIP + ";Database=" + globalSettings.settingLocalDB;
using (MySqlConnection connectionLocal = new MySqlConnection(myConnStringLocal))
{
connectionLocal.open()
}
string myConnStringCentral = "User Id=***;Password=*****;Host=" + globalSettings.settingCentralIP + ";Database=" + globalSettings.settingCentralDB;
using (MySqlConnection connectionCentral = new MySqlConnection(myConnStringCentral))
{
connectionCentral.Open();
}
string myConnStringCentralCopy = "User Id=*****;Password=*****;Host=" + globalSettings.settingCentralCopyIP + ";Database=" + globalSettings.settingCentralCopyDB;
using (MySqlConnection connectionCentralCopy = new MySqlConnection(myConnStringCentralCopy))
{
connectionCentralCopy.Open();
}
if (rollbackBoolean == 0)
transScope.Complete();
else
transScope.Dispose();
}
Both methods are correct. TransactionScope will be used together with dotConnect for MySQL in both cases. The following code is not required:
else
transScope.Dispose();
because the Dispose method is called automatically when exiting the using (TransactionScope transScope = new TransactionScope()) block.
... I went through it meaning in my case the transactionScope will close it when I call the complete rite...
No, the transScope.Complete() does not close the connection. If the connection is closed inside the using block for TransactionScope, the connection object will be closed, however the internal connection will stay open for sending changes to the database when calling transScope.Dispose() if it is preceeded by the transScope.Complete() call. transScope.Dispose() closes the internal connection if the Close or Dispose method was called for the connection object. If the connection object was not closed, transScope.Dispose() does nothing to the connection.
... Another thing in each of my connection I keep track of try and catch and if there is any error I flagged the rollbackBoolean to 1 so then it wont complete and whole transaction should be rollbacked is that a correct mechanism? ...
If an error occurred, just don't call Complete. If the Complete method was not called, the transaction will be rolled back when executing the Dispose method.
Here is the example with a try/catch block and rolling back the transaction in case of error:
using (TransactionScope transScope = new TransactionScope())
{
try
{
string myConnStringLocal = "User Id=***;Password=****;Host=" + globalSettings.settingLocalIP + ";Database=" + globalSettings.settingLocalDB;
using (MySqlConnection connectionLocal = new MySqlConnection(myConnStringLocal))
{
connectionLocal.Open();
}
string myConnStringCentral = "User Id=***;Password=*****;Host=" + globalSettings.settingCentralIP + ";Database=" + globalSettings.settingCentralDB;
using (MySqlConnection connectionCentral = new MySqlConnection(myConnStringCentral))
{
connectionCentral.Open();
}
string myConnStringCentralCopy = "User Id=*****;Password=*****;Host=" + globalSettings.settingCentralCopyIP + ";Database=" + globalSettings.settingCentralCopyDB;
using (MySqlConnection connectionCentralCopy = new MySqlConnection(myConnStringCentralCopy))
{
connectionCentralCopy.Open();
}
transScope.Complete();
Console.WriteLine("Transaction is completed");
}
catch (Exception)
{
Console.WriteLine("Transaction is rolled back");
}
}
I'm new to C# (worked in PHP, Python, and Javascript) and I'm trying to more or less make a duplicate of another page and change some things - to make a form and database submission.
Anyway, here's the code:
public partial class commenter : System.Web.UI.Page
{
string employee_reviewed;
//public Commenter();
public void SaveBtn_Click(object sender, EventArgs e)
{
if (CommentTB.Text == "Please enter a comment.")
{
String csname = "Email Error";
Type cstype = this.GetType();
ClientScriptManager cs = Page.ClientScript;
if (!cs.IsStartupScriptRegistered(cstype, csname))
{
String cstext = "alert('Please submit at least one comment.');";
cs.RegisterStartupScript(cstype, csname, cstext, true);
}
FormMessage.Text = "Please submit at least one comment.";
return;
}
string comment = CommentTB.Text;
comment = comment.Replace("'", "''");
comment = comment.Replace("’", "''");
comment = comment.Replace("`", "''");
try
{
//myCommand.Connection.Open();
//myCommand.ExecuteNonQuery();
//myCommand.Connection.Close();
MySqlCommand myCommand;
MySqlConnection connection;
string connStringName = "server=localhost;database=hourtracking;uid=username;password=password";
connection = new MySqlConnection(connStringName);
string sql_query;
sql_query = "insert into peer_review_comment " + " (emp_id, comment)" + " values(?employeeid, ?comment) ";
//String csname = "Email Error";
//Type cstype = this.GetType();
//ClientScriptManager cs = Page.ClientScript;
//cs.RegisterStartupScript(cstype, csname, sql_query, true);
myCommand = new MySqlCommand(sql_query, connection);
//FormMessage.Text = sql_query;
//return;
Trace.Write("comment = ", comment);
myCommand.Parameters.Add(new MySqlParameter("?employeeid", ViewState["employeeid"].ToString()));
myCommand.Parameters.Add(new MySqlParameter("?comment", comment));
try
{
myCommand.Connection.Open();
myCommand.ExecuteNonQuery();
myCommand.Connection.Close();
}
catch (Exception ex)
{
FormMessage.Text = "Error:SaveBtn_Click - " + ex.Message;
}
//SendNotification(from, to, cc, subject, body, attach);
FormMessage.Text = "\n Thank you for leaving anonymous feedback for " + employee_reviewed; ;
ThankyouDiv.Visible = true;
FormFieldDiv.Visible = false;
reviewHeader.Visible = false;
}
catch (Exception ex)
{
FormMessage.Text = "Error:SaveBtn_Click - " + ex.Message;
}
}
}
I really have little idea what I'm doing - I'm reading the tutorials, but C# is a significantly different language than I am used to.
I get the Javascript alert when I do not change the text currently, but submission isn't working - I want it to submit to peer_review_comment database table, and fill in employeeid as well as the submitted comment.
Sorry if my understanding is so spotty, I am a TOTAL C# newbie (currently reading http://www.csharp-station.com/Tutorial/CSharp/)
My guess is the problem is here:
try
{
myCommand.Connection.Open();
myCommand.ExecuteNonQuery();
myCommand.Connection.Close();
}
catch (Exception ex)
{
FormMessage.Text = "Error:SaveBtn_Click - " + ex.Message;
// no "return;" !!
}
//SendNotification(from, to, cc, subject, body, attach);
FormMessage.Text = "\n Thank you for leaving anonymous feedback for " +
employee_reviewed; ;
Your catch block is setting the FormMessage.Text value bot not exiting the method, so the method keeps executing where the catch block finishes off, resetting the Text value and appearing that no exception was thrown.
add a return; at the end of your catch block to see the excpetion message.
Some general guidelines to make these kinds of problems easier to trap:
Don't try to do too much in one method. Have one method that validates the message (or do it client-side using Validators, another to do the DB call, etc.
Learn to use the debugger. You can step through code and get a better idea of what causes these kinds of errors.
Unless you can DO something about an exception, there's no harm in letting them bubble up to a higher level event handler (like Elmah) so exceptions don't get accidentally swallowed like it does here. In general it's preferrable to re-throw exceptions in lower-level methods (maybe adding some context or a user-friendly message) so the higher level exception handling can decide what to do (show a message, log, etc.)
I have taken the liberty of refactoring your code. This shows some better code practices but may also show you the problem. Along with these code changes I would also recommend reading D. Stanley's answer; there are some helpful tips in there as well.
if (CommentTB.Text == "Please enter a comment.")
{
String csname = "Email Error";
Type cstype = this.GetType();
ClientScriptManager cs = Page.ClientScript;
if (!cs.IsStartupScriptRegistered(cstype, csname))
{
String cstext = "alert('Please submit at least one comment.');";
cs.RegisterStartupScript(cstype, csname, cstext, true);
}
FormMessage.Text = "Please submit at least one comment.";
return;
}
// This helps some but very little, just wanted to show an alternative to writing three statements
string comment = CommentTB.Text.Replace("'", "''").Replace("’", "''").Replace("`", "''");
//string comment = CommentTB.Text;
//comment = comment.Replace("'", "''");
//comment = comment.Replace("’", "''");
//comment = comment.Replace("`", "''");
try
{
// No need to do string concatenation...just make it one string.
// sql_query = "insert into peer_review_comment " + " (emp_id, comment)" + " values(?employeeid, ?comment) ";
string sql_query = "insert into peer_review_comment (emp_id, comment) values (?employeeid, ?comment) ";
string connStringName = "server=localhost;database=hourtracking;uid=username;password=password";
// Use a "using" clause because it guarantees the connection is closed even when an exception occurs.
using (MySqlConnection connection = new MySqlConnection(connStringName))
{
connection.Open();
// Again, use a "using" clause
using (MySqlCommand myCommand = new MySqlCommand(sql_query, connection))
{
Trace.Write("comment = ", comment);
myCommand.Parameters.Add(new MySqlParameter("?employeeid", ViewState["employeeid"].ToString()));
myCommand.Parameters.Add(new MySqlParameter("?comment", comment));
myCommand.ExecuteNonQuery();
// No need for a Close statement with "using" clause.
//myCommand.Connection.Close();
}
}
FormMessage.Text = "\n Thank you for leaving anonymous feedback for " + employee_reviewed;
ThankyouDiv.Visible = true;
FormFieldDiv.Visible = false;
reviewHeader.Visible = false;
}
catch (Exception ex)
{
FormMessage.Text = "Error:SaveBtn_Click - " + ex.Message;
}
My program is now still running to import data from a log file into a remote SQL Server Database. The log file is about 80MB in size and contains about 470000 lines, with about 25000 lines of data. My program can import only 300 rows/second, which is really bad. :(
public static int ImportData(string strPath)
{
//NameValueCollection collection = ConfigurationManager.AppSettings;
using (TextReader sr = new StreamReader(strPath))
{
sr.ReadLine(); //ignore three first lines of log file
sr.ReadLine();
sr.ReadLine();
string strLine;
var cn = new SqlConnection(ConnectionString);
cn.Open();
while ((strLine = sr.ReadLine()) != null)
{
{
if (strLine.Trim() != "") //if not a blank line, then import into database
{
InsertData(strLine, cn);
_count++;
}
}
}
cn.Close();
sr.Close();
return _count;
}
}
InsertData is just a normal insert method using ADO.NET. It uses a parsing method:
public Data(string strLine)
{
string[] list = strLine.Split(new[] {'\t'});
try
{
Senttime = DateTime.Parse(list[0] + " " + list[1]);
}
catch (Exception)
{
}
Clientip = list[2];
Clienthostname = list[3];
Partnername = list[4];
Serverhostname = list[5];
Serverip = list[6];
Recipientaddress = list[7];
Eventid = Convert.ToInt16(list[8]);
Msgid = list[9];
Priority = Convert.ToInt16(list[10]);
Recipientreportstatus = Convert.ToByte(list[11]);
Totalbytes = Convert.ToInt32(list[12]);
Numberrecipient = Convert.ToInt16(list[13]);
DateTime temp;
if (DateTime.TryParse(list[14], out temp))
{
OriginationTime = temp;
}
else
{
OriginationTime = null;
}
Encryption = list[15];
ServiceVersion = list[16];
LinkedMsgid = list[17];
MessageSubject = list[18];
SenderAddress = list[19];
}
InsertData method:
private static void InsertData(string strLine, SqlConnection cn)
{
var dt = new Data(strLine); //parse the log line into proper fields
const string cnnStr =
"INSERT INTO LOGDATA ([SentTime]," + "[client-ip]," +
"[Client-hostname]," + "[Partner-Name]," + "[Server-hostname]," +
"[server-IP]," + "[Recipient-Address]," + "[Event-ID]," + "[MSGID]," +
"[Priority]," + "[Recipient-Report-Status]," + "[total-bytes]," +
"[Number-Recipients]," + "[Origination-Time]," + "[Encryption]," +
"[service-Version]," + "[Linked-MSGID]," + "[Message-Subject]," +
"[Sender-Address]) " + " VALUES ( " + "#Senttime," + "#Clientip," +
"#Clienthostname," + "#Partnername," + "#Serverhostname," + "#Serverip," +
"#Recipientaddress," + "#Eventid," + "#Msgid," + "#Priority," +
"#Recipientreportstatus," + "#Totalbytes," + "#Numberrecipient," +
"#OriginationTime," + "#Encryption," + "#ServiceVersion," +
"#LinkedMsgid," + "#MessageSubject," + "#SenderAddress)";
var cmd = new SqlCommand(cnnStr, cn) {CommandType = CommandType.Text};
cmd.Parameters.AddWithValue("#Senttime", dt.Senttime);
cmd.Parameters.AddWithValue("#Clientip", dt.Clientip);
cmd.Parameters.AddWithValue("#Clienthostname", dt.Clienthostname);
cmd.Parameters.AddWithValue("#Partnername", dt.Partnername);
cmd.Parameters.AddWithValue("#Serverhostname", dt.Serverhostname);
cmd.Parameters.AddWithValue("#Serverip", dt.Serverip);
cmd.Parameters.AddWithValue("#Recipientaddress", dt.Recipientaddress);
cmd.Parameters.AddWithValue("#Eventid", dt.Eventid);
cmd.Parameters.AddWithValue("#Msgid", dt.Msgid);
cmd.Parameters.AddWithValue("#Priority", dt.Priority);
cmd.Parameters.AddWithValue("#Recipientreportstatus", dt.Recipientreportstatus);
cmd.Parameters.AddWithValue("#Totalbytes", dt.Totalbytes);
cmd.Parameters.AddWithValue("#Numberrecipient", dt.Numberrecipient);
if (dt.OriginationTime != null)
cmd.Parameters.AddWithValue("#OriginationTime", dt.OriginationTime);
else
cmd.Parameters.AddWithValue("#OriginationTime", DBNull.Value);
//if OriginationTime was null, then insert with null value to this column
cmd.Parameters.AddWithValue("#Encryption", dt.Encryption);
cmd.Parameters.AddWithValue("#ServiceVersion", dt.ServiceVersion);
cmd.Parameters.AddWithValue("#LinkedMsgid", dt.LinkedMsgid);
cmd.Parameters.AddWithValue("#MessageSubject", dt.MessageSubject);
cmd.Parameters.AddWithValue("#SenderAddress", dt.SenderAddress);
cmd.ExecuteNonQuery();
}
How can my program run faster?
Thank you so much!
Use SqlBulkCopy.
Edit: I created a minimal implementation of IDataReader and created a Batch type so that I could insert arbitrary in-memory data using SqlBulkCopy. Here is the important bit:
IDataReader dr = batch.GetDataReader();
using (SqlTransaction tx = _connection.BeginTransaction())
{
try
{
using (SqlBulkCopy sqlBulkCopy =
new SqlBulkCopy(_connection, SqlBulkCopyOptions.Default, tx))
{
sqlBulkCopy.DestinationTableName = TableName;
SetColumnMappings(sqlBulkCopy.ColumnMappings);
sqlBulkCopy.WriteToServer(dr);
tx.Commit();
}
}
catch
{
tx.Rollback();
throw;
}
}
The rest of the implementation is left as an exercise for the reader :)
Hint: the only bits of IDataReader you need to implement are Read, GetValue and FieldCount.
Hmmm, let's break this down a little bit.
In pseudocode what you did is the ff:
Open the file
Open a connection
For every line that has data:
Parse the string
Save the data in SQL Server
Close the connection
Close the file
Now the fundamental problems in doing it this way are:
You are keeping a SQL connection open while waiting for your line parsing (pretty susceptible to timeouts and stuff)
You might be saving the data line by line, each in its own transaction. We won't know until you show us what the InsertData method is doing
Consequently you are keeping the file open while waiting for SQL to finish inserting
The optimal way of doing this is to parse the file as a whole, and then insert them in bulk. You can do this with SqlBulkCopy (as suggested by Matt Howells), or with SQL Server Integration Services.
If you want to stick with ADO.NET, you can pool together your INSERT statements and then pass them off into one large SQLCommand, instead of doing it this way e.g., setting up one SQLCommand object per insert statement.
You create the SqlCommand object for every row of data. The simplest improvement would therefore to create a
private static SqlCommand cmdInsert
and declare the parameters with the Parameters.Add() method. Then for each data row, set the parameter values using
cmdInsert.Parameters["#paramXXX"].Value = valueXXX;
A second performance improvement might be to skip creation of Data objects for each row, and assign Parameter values directly from the list[] array.