I would like to know if it's possible to ignore a file or a part of my code during a commit. To develop my solution I actually use two different computers, each using his own connection string to the database.
//Informations - DB
private const String _SERVER = "localhost";
private const String _DATABASE = "dbname";
private const String _UID = "";
private const String _PASSWORD = "";
//DB connection
public static MySqlConnection v_DBConnection;
public static void InititializeDB()
{
String v_ConnectionString = null;
MySqlConnectionStringBuilder v_MySqlConnectionStringBuilder;
try
{
v_ConnectionString = null;
v_MySqlConnectionStringBuilder = new MySqlConnectionStringBuilder();
v_MySqlConnectionStringBuilder.Server = _SERVER;
v_MySqlConnectionStringBuilder.UserID = _UID;
v_MySqlConnectionStringBuilder.Password = _PASSWORD;
v_MySqlConnectionStringBuilder.Database = _DATABASE;
//CONNECTION CHAIN
v_ConnectionString = v_MySqlConnectionStringBuilder.ToString();
v_DBConnection = new MySqlConnection(v_ConnectionString);
}
catch (Exception exception)
{
MessageBox.Show("InititializeDB fail due to:" + exception.ToString());
}
}
I would like during a commit ignore the file containing the string or exclude a part of the code (connection string).
I tried to add the file containing my connection chain in my .gitignore file but I'm not sure, I did'nt understandhow the .gitignore file work.
# Ignored files
DBConnection.cs
I am creating a Excel Addin through which i want to access a database. code is as follows
[ExcelFunction("My First Excel-DNA Function")]
public static string GreetFunction(string name)
{
GetConnection();
return "Hello" + " " + name;
}
public static void GetConnection()
{
//db = new SQLiteConnection("Data Source="+System.IO.Path.GetDirectoryName(System.Reflection.Assembly.GetExecutingAssembly().GetName().CodeBase)+"\\Database\\XLSQLiteDemo.sqlite");
db = new SQLiteConnection("Data Source=Database/XLSQLiteDemo.sqlite");
try
{
db.Open();
cmd = db.CreateCommand();
System.Windows.MessageBox.Show("Connection created");
}
catch (SQLiteException ex)
{
System.Windows.MessageBox.Show(ex.ToString());
}
}
so when i give absolute path like c:/test/firstlibrary.../XLSQLiteDemo.sqlite it works.
but when i use relative path like db = new SQLiteConnection("Data Source=Database/XLSQLiteDemo.sqlite");
it throws an exception: unable to open database file error code 14.
the code which is in comment i.e.
//db = new SQLiteConnection("Data Source="+System.IO.Path.GetDirectoryName(System.Reflection.Assembly.GetExecutingAssembly().GetName().CodeBase)+"\\Database\\XLSQLiteDemo.sqlite");
also doesn't work i.e. it calculates the absolute path but when i tried to debug; debugging is automatically terminated after db.Open();
and output in excel sheet is also #Value which indicates some error.
#adrino may be the "file" word in your string is the problem.remove it.
string relativePath = #"Database\XLSQLiteDemo.sqlite";
string currentPath = System.IO.Path.GetDirectoryName(System.Reflection.Assembly.GetExecutingAssembly().GetName().CodeBase);
string absolutePath = System.IO.Path.Combine(currentPath, relativePath);
absolutePath=absolutePath.Remove(0, 6);//this code is written to remove file word from absolute path
string connectionString = string.Format("Data Source={0}", absolutePath);
this works on my machine.tell me if its correct.
Strange, I can't get ASP.Net project (using C# mode) to log result into my D:\Web.log file and also no exception was thrown.
Is there something wrong with my simple code?
private static XmlDocument _doc = new XmlDocument();
private static string LogFileName = "D:\\Web.log";
static void LogToFile(string WebAddress, string IPAddress, string Title)
{
if (File.Exists(#"" + LogFileName + ""))
_doc.Load(#"" + LogFileName + "");
else
{
var root = _doc.CreateElement("Web");
_doc.AppendChild(root);
}
try
{
var el = (XmlElement)_doc.DocumentElement.AppendChild(_doc.CreateElement("Web"));
el.SetAttribute("Title", Title);
el.AppendChild(_doc.CreateElement("WebAddress")).InnerText = WebAddress;
el.AppendChild(_doc.CreateElement("IPAddress")).InnerText = IPAddress;
_doc.Save(#"" + LogFileName + "");
}
catch (Exception ex)
{
}
}
By removing the file D:\Web.log that I created, it is now able to append the file.
Remove the #"" and leading "" from all instances of #"" + LogFileName + "".
They are not needed as you have already escaped the variable LogFileName.
I have a database (mdf file) which I'm approaching with the Entity Framework. Is it possible to make a backup of the MDF file.
I tried already but SMO but the problem is because I'm using a mdf file the database name is empty. I've read that it's autogenerated.
Piece of my backup code:
String destinationPath = "C:\\";
Backup sqlBackup = new Backup();
sqlBackup.Action = BackupActionType.Database;
sqlBackup.BackupSetDescription = "ArchiveDataBase:" + DateTime.Now.ToShortDateString();
sqlBackup.BackupSetName = "Archive";
BackupDeviceItem deviceItem = new BackupDeviceItem(destinationPath, DeviceType.File);
ServerConnection connection = new ServerConnection(".\\SQLEXPRESS");
Server sqlServer = new Server(connection);
StringCollection sc = new StringCollection();
sc.Add(Environment.CurrentDirectory + "\\db\\Xmain.mdf"); //Bin directory
sc.Add(Environment.CurrentDirectory + "\\db\\Xmain_log.ldf");
sqlServer.AttachDatabase("Xmain", sc);
Database db = sqlServer.Databases["Xmain"];
sqlBackup.Initialize = true;
sqlBackup.Checksum = true;
sqlBackup.ContinueAfterError = true;
sqlBackup.Devices.Add(deviceItem);
sqlBackup.Incremental = false;
sqlBackup.ExpirationDate = DateTime.Now.AddDays(3);
sqlBackup.LogTruncation = BackupTruncateLogType.Truncate;
sqlBackup.FormatMedia = false;
sqlBackup.SqlBackup(sqlServer);
I am missing something here and need more context, but I will rant on for a second and see if anything is helpful.
Do you mean actually back up the file, not the data? If so, the easy answer is no. The problem is SQL Server will lock the file when it is attached to the database server (SQL Express in this case). You can detatch and copy and then attach, but the application will be down during that time. This can also be done manually.
If you want to backup the data, I would consider scheduling it within SQL Server rather than programmatic, unless you cannot do it that way. Backup is more of a maintenance function than a part of the program.
As for your database name being empty, that is impossible. In fact, it looks like you are trying to set up a database called XMain.
You must set initial catalog in connection string in app.config:
<add name="SalaryAndBenefitsEntities" connectionString="metadata=res://*/SalaryAndBenefitsModel.csdl|res://*/SalaryAndBenefitsModel.ssdl|res://*/SalaryAndBenefitsModel.msl;provider=System.Data.SqlClient;provider connection string="data source=.\sqlexpress;attachdbfilename=|DataDirectory|SalaryAndBenefits.mdf;Initial Catalog=SalaryAndBenefit;user instance=true;password=ca;integrated security=SSPI;MultipleActiveResultSets=True;App=EntityFramework"" providerName="System.Data.EntityClient" />
For backUp Follow below:
create sql command content
public string BackUpCommand(string databaseName, string fileAddress)
{
string command = #"BACKUP DATABASE " + databaseName + #"
TO DISK = '" + fileAddress + "' WITH FORMAT";
return command;
}
Write Backup Method:
public class ActionResult
{
public bool Result { get; set; }
public string Message { get; set; }
}
public ActionResult BackUpDatabase(string filePath)
{
ActionResult res = new ActionResult { Result = true };
using (SalaryAndBenefitsEntities _context = new SalaryAndBenefitsEntities())
{
string command = "select db_name()";
string databaseName = _context.Database.SqlQuery(typeof(string), command).ToListAsync().Result.FirstOrDefault().ToString();
string backUpQuery = BackUpCommand(databaseName, filePath);
var result = _context.Database.SqlQuery<List<string>>(backUpQuery).ToList();
if (result.Count() > 0)
{
res.Result = false;
result.ForEach(x =>
{
res.Message += x.ToString();
});
}
return res;
}
}
if return true database backup is successful else not
For restore Follow below:
create sql command content
public string RestoreCommand(string databaseName, string fileAddress)
{
string command = #"use [master]
ALTER DATABASE " + databaseName + #"
SET SINGLE_USER
WITH ROLLBACK IMMEDIATE
RESTORE DATABASE " + databaseName + #"
FROM DISK = N'" + fileAddress + "'";
return command;
}
Write Restore Method:
public ActionResult RestoreDatabase(string filePath)
{
ActionResult res = new ActionResult { Result = true };
using (SalaryAndBenefitsEntities _context = new SalaryAndBenefitsEntities())
{
string command = "select db_name()";
string databaseName = _context.Database.SqlQuery(typeof(string), command).ToListAsync().Result.FirstOrDefault().ToString();
string restoreQuery = RestoreCommand(databaseName, filePath);
var result = _context.Database.SqlQuery<List<string>>(restoreQuery).ToList();
if (result.Count() > 0)
{
res.Result = false;
result.ForEach(x =>
{
res.Message += x.ToString();
});
}
return res;
}
}
if return true database restore is successful else not
the filePath Like: C:\Temp\backup.bak
the Directory of filePath (C:\Temp) must created manually before use of this methods
My program is now still running to import data from a log file into a remote SQL Server Database. The log file is about 80MB in size and contains about 470000 lines, with about 25000 lines of data. My program can import only 300 rows/second, which is really bad. :(
public static int ImportData(string strPath)
{
//NameValueCollection collection = ConfigurationManager.AppSettings;
using (TextReader sr = new StreamReader(strPath))
{
sr.ReadLine(); //ignore three first lines of log file
sr.ReadLine();
sr.ReadLine();
string strLine;
var cn = new SqlConnection(ConnectionString);
cn.Open();
while ((strLine = sr.ReadLine()) != null)
{
{
if (strLine.Trim() != "") //if not a blank line, then import into database
{
InsertData(strLine, cn);
_count++;
}
}
}
cn.Close();
sr.Close();
return _count;
}
}
InsertData is just a normal insert method using ADO.NET. It uses a parsing method:
public Data(string strLine)
{
string[] list = strLine.Split(new[] {'\t'});
try
{
Senttime = DateTime.Parse(list[0] + " " + list[1]);
}
catch (Exception)
{
}
Clientip = list[2];
Clienthostname = list[3];
Partnername = list[4];
Serverhostname = list[5];
Serverip = list[6];
Recipientaddress = list[7];
Eventid = Convert.ToInt16(list[8]);
Msgid = list[9];
Priority = Convert.ToInt16(list[10]);
Recipientreportstatus = Convert.ToByte(list[11]);
Totalbytes = Convert.ToInt32(list[12]);
Numberrecipient = Convert.ToInt16(list[13]);
DateTime temp;
if (DateTime.TryParse(list[14], out temp))
{
OriginationTime = temp;
}
else
{
OriginationTime = null;
}
Encryption = list[15];
ServiceVersion = list[16];
LinkedMsgid = list[17];
MessageSubject = list[18];
SenderAddress = list[19];
}
InsertData method:
private static void InsertData(string strLine, SqlConnection cn)
{
var dt = new Data(strLine); //parse the log line into proper fields
const string cnnStr =
"INSERT INTO LOGDATA ([SentTime]," + "[client-ip]," +
"[Client-hostname]," + "[Partner-Name]," + "[Server-hostname]," +
"[server-IP]," + "[Recipient-Address]," + "[Event-ID]," + "[MSGID]," +
"[Priority]," + "[Recipient-Report-Status]," + "[total-bytes]," +
"[Number-Recipients]," + "[Origination-Time]," + "[Encryption]," +
"[service-Version]," + "[Linked-MSGID]," + "[Message-Subject]," +
"[Sender-Address]) " + " VALUES ( " + "#Senttime," + "#Clientip," +
"#Clienthostname," + "#Partnername," + "#Serverhostname," + "#Serverip," +
"#Recipientaddress," + "#Eventid," + "#Msgid," + "#Priority," +
"#Recipientreportstatus," + "#Totalbytes," + "#Numberrecipient," +
"#OriginationTime," + "#Encryption," + "#ServiceVersion," +
"#LinkedMsgid," + "#MessageSubject," + "#SenderAddress)";
var cmd = new SqlCommand(cnnStr, cn) {CommandType = CommandType.Text};
cmd.Parameters.AddWithValue("#Senttime", dt.Senttime);
cmd.Parameters.AddWithValue("#Clientip", dt.Clientip);
cmd.Parameters.AddWithValue("#Clienthostname", dt.Clienthostname);
cmd.Parameters.AddWithValue("#Partnername", dt.Partnername);
cmd.Parameters.AddWithValue("#Serverhostname", dt.Serverhostname);
cmd.Parameters.AddWithValue("#Serverip", dt.Serverip);
cmd.Parameters.AddWithValue("#Recipientaddress", dt.Recipientaddress);
cmd.Parameters.AddWithValue("#Eventid", dt.Eventid);
cmd.Parameters.AddWithValue("#Msgid", dt.Msgid);
cmd.Parameters.AddWithValue("#Priority", dt.Priority);
cmd.Parameters.AddWithValue("#Recipientreportstatus", dt.Recipientreportstatus);
cmd.Parameters.AddWithValue("#Totalbytes", dt.Totalbytes);
cmd.Parameters.AddWithValue("#Numberrecipient", dt.Numberrecipient);
if (dt.OriginationTime != null)
cmd.Parameters.AddWithValue("#OriginationTime", dt.OriginationTime);
else
cmd.Parameters.AddWithValue("#OriginationTime", DBNull.Value);
//if OriginationTime was null, then insert with null value to this column
cmd.Parameters.AddWithValue("#Encryption", dt.Encryption);
cmd.Parameters.AddWithValue("#ServiceVersion", dt.ServiceVersion);
cmd.Parameters.AddWithValue("#LinkedMsgid", dt.LinkedMsgid);
cmd.Parameters.AddWithValue("#MessageSubject", dt.MessageSubject);
cmd.Parameters.AddWithValue("#SenderAddress", dt.SenderAddress);
cmd.ExecuteNonQuery();
}
How can my program run faster?
Thank you so much!
Use SqlBulkCopy.
Edit: I created a minimal implementation of IDataReader and created a Batch type so that I could insert arbitrary in-memory data using SqlBulkCopy. Here is the important bit:
IDataReader dr = batch.GetDataReader();
using (SqlTransaction tx = _connection.BeginTransaction())
{
try
{
using (SqlBulkCopy sqlBulkCopy =
new SqlBulkCopy(_connection, SqlBulkCopyOptions.Default, tx))
{
sqlBulkCopy.DestinationTableName = TableName;
SetColumnMappings(sqlBulkCopy.ColumnMappings);
sqlBulkCopy.WriteToServer(dr);
tx.Commit();
}
}
catch
{
tx.Rollback();
throw;
}
}
The rest of the implementation is left as an exercise for the reader :)
Hint: the only bits of IDataReader you need to implement are Read, GetValue and FieldCount.
Hmmm, let's break this down a little bit.
In pseudocode what you did is the ff:
Open the file
Open a connection
For every line that has data:
Parse the string
Save the data in SQL Server
Close the connection
Close the file
Now the fundamental problems in doing it this way are:
You are keeping a SQL connection open while waiting for your line parsing (pretty susceptible to timeouts and stuff)
You might be saving the data line by line, each in its own transaction. We won't know until you show us what the InsertData method is doing
Consequently you are keeping the file open while waiting for SQL to finish inserting
The optimal way of doing this is to parse the file as a whole, and then insert them in bulk. You can do this with SqlBulkCopy (as suggested by Matt Howells), or with SQL Server Integration Services.
If you want to stick with ADO.NET, you can pool together your INSERT statements and then pass them off into one large SQLCommand, instead of doing it this way e.g., setting up one SQLCommand object per insert statement.
You create the SqlCommand object for every row of data. The simplest improvement would therefore to create a
private static SqlCommand cmdInsert
and declare the parameters with the Parameters.Add() method. Then for each data row, set the parameter values using
cmdInsert.Parameters["#paramXXX"].Value = valueXXX;
A second performance improvement might be to skip creation of Data objects for each row, and assign Parameter values directly from the list[] array.