Develop Reference

Create list of arrays from text file in C#

I have a number of text files that all follow the same content format:
"Title section","Version of the app"
10
"<thing 1>","<thing 2>","<thing 3>","<thing 4>","<thing 5>","<thing 6>","<thing 7>","<thing 8>","<thing 9>","<thing 10>"
'Where:
' first line never changes, it always contains exactly these 2 items
' second line is a count of how many "line 3s" there are
' line 3 contains a command to execute and (up to) 9 parameters
' - there will always be 10 qoute-delimited entries, even if some are blank
' - there can be N number of entries (in this example, there will be 10 commands to read)
I am reading each of these text files in, using StreamReader, and want to set each file up in its own class.
public class MyTextFile{
public string[] HeaderLine { get; set; }
public int ItemCount { get; set; }
List<MyCommandLine> Commands { get; set;}
}
public class MyCommandLine{
public string[] MyCommand { get; set; }
}
private void btnGetMyFilesiles_Click(object sender, EventArgs e){
DirectoryInfo myFolder = new DirectoryInfo(#"C:\FileSpot");
FileInfo[] myfiles = myfolder.GetFiles("*.ses");
string line = "";
foreach(FileInfo file in Files ){
str = str + ", " + file.Name;
// Read the file and display it line by line.
System.IO.StreamReader readingFile = new System.IO.StreamReader(file.Name);
MyTextFile myFileObject = new MyTextFile()
while ((line = readingFile.ReadLine()) != null){
' create the new MyTextFile here
}
file.Close();
}
}
}
The objective is to determine what the actual command being called is (""), and if any of the remaining parameters point to a pre-existing file, determine if that file exists. My problem is that I can't figure out how to read N number of "line 3" into their own objects and append these objects to the MyTextFile object. I'm 99% certain that I've led myself astray in reading each file line-by-line, but I don't know how to get out of it.

So, addressing the specific issue of getting N number of line 3 items into your class, you could do something like this (obviously you can make some changes so it is more specific to your application).
public class MyTextFile
{
public List<Array> Commands = new List<Array>();
public void EnumerateCommands()
{
for (int i = 0; i < Commands.Count; i++)
{
foreach (var c in Commands[i])
Console.Write(c + " ");
Console.WriteLine();
}
}
}
class Program
{
static void Main(string[] args)
{
string line = "";
int count = 0;
MyTextFile tf = new MyTextFile();
using (StreamReader sr = new StreamReader(#"path"))
{
while ((line = sr.ReadLine()) != null)
{
count += 1;
if (count >= 3)
{
object[] Arguments = line.Split(',');
tf.Commands.Add(Arguments);
}
}
}
tf.EnumerateCommands();
Console.ReadLine();
}
}
At least now you have a list of commands within your 'MyTextFile' class that you can enumerate through and do stuff with.
** I added the EnumerateCommands method so that you could actually see the list is storing the line items. The code should run in a Console application with the appropriate 'using' statements.
Hope this helps.

If all of the is separated with coma sign , you can just do something like :
int length = Convert.ToInt32 (reader.ReadLine ());
string line = reader.ReadLine ();
IEnumerable <string> things = line.Split (',').Select (thing => thing. Replace ('\"'', string.Empty).Take(length);
Take indicates how many things to take from the line.

C# Webservice stucks. Need to override published file to restart

Please help me, I'm facing a fatal problem here. If someone could fix this, I swear I will treat u to a huge drink whenever u step into my country (Vietnam). Ok here's the problem: I'm coding a webservice for multi connection simultaneously from tablet (around 100 clients). It ran well but recently whenever high traffic occurs, my webservice seems to stuck somehow and I need to copy - override the published file of webservice in order for it to run again (restart website in IIS is no use) ...
This is my w/s code for handling the data:
public string Info_Handling(string id, string name, string strDetails)
{
string checkExist = "";
string str = "";
string str2 = "";
MLL_Customer _customerClient = new MLL_Customer();
MLL_CustomerCategory _categoryClient = new MLL_CustomerCategory();
MLL_Product _productClient = new MLL_Product();
MLL_SampleProduct _sampleClient = new MLL_SampleProduct();
if (_customerClient.CheckExistCustomer(id, name.ToUpper(), 2) == 1) // SID & NAME
{
checkExist = "EXIST";
}
using (SqlConnection connection = new SqlConnection(ConfigurationSettings.AppSettings["Main.ConnectionString"]))
{
connection.Open();
SqlTransaction trans = connection.BeginTransaction("XXX");
try
{
// ID Example: 11 means VIP - 12 means Normal - 13 means ples... jkg
// First - Insert Customer
string strCustomerCategory = _categoryClient.SelectCategoryByID(id).ToString();
if (!checkExist.Equals("EXIST"))
{
Customer businessObject = new Customer();
businessObject.ID = sid;
businessObject.Name = name.ToUpper();
businessObject.CategoryID = strCustomerCategory;
str = "" + _customerClient.Insert(businessObject, connection, trans);
}
// Second Insert Product spliting from a string Ex: "TV&Laptop&CD"
string[] productDetails = strDetails.Split(new char[] { '&' });
object obj3;
SampleProduct objSample;
Product objProduct;
for (int j = 0; j < productDetails.Length; j++)
{
if (_productClient.CheckExist(id, productDetails[j])) == null) // Check if customer already owns this product
{
// Get the properties of sample product.
objSample = _sampleClient.SelectSampleProduct(productDetails[j]);
objProduct = new Product();
objProduct.SID = sid;
objProduct.Testcode = objSample.TestCode;
objProduct.Category = objSample.Category;
objProduct.Unit = objSample.Unit;
objProduct.Price = objSample.Price;
if (_productClient.Insert(objProduct, connection, trans) != 0)
{
str2 = str2 + "&" + objProduct.Testcode;
// return the code of product in order to see which product has been inserted successfully
}
}
}
trans.Commit();
SqlConnection.ClearAllPools();
}
catch (Exception exception)
{
str = "0";
str2 = exception.Message + exception.Source;
try
{
trans.Rollback();
}
catch (Exception)
{
}
}
}
if (!str2.Equals(""))
{
return (str + "&" + id + str2);
}
return ("0&" + sid + str);
}
I modified the code but this is basically how i roll. Could anyone plz tell me some solution. Deeply thank u.
1 more thing about ClearAllPools() method: I know how it works but I dont even know why I need it. Without this, my data will be messed up terrible. CategoryID of one customer will be assigned for another customer sometimes. ???? How could it happened ?? HELP

Unable to stop SQL injection errors

I finally get it. It's not just the code I use to execute the ExecuteScalar method but it is mainly the code up stream that is executing the class. It is everything calling your code. That said, can someone please see if the code executing the my SQL class has faults. I still cant pass the scans. First I will show you two examples of the code calling my code, then the calling code, and finally the executing code, which I formulated and displayed from a previous post.
Calling code with three parameters:
public bool isTamAsp(int aspKey, int fy, string accountCode)
{
MyParam myParam;
string sqlQuery = "select isTamMacom = count(macom_key) FROM hier_fy " +
"WHERE hier_key = #aspKey AND fy = #fy AND #accountCode NOT IN (3,4,7,8) AND macom_key IN (select hier_key from lkup_e581_MacomThatRequireTAM) AND is_visible = 1 AND is_active = 1";
QueryContainer Instance = new QueryContainer(sqlQuery);
myParam = new MyParam();
myParam.SqlParam = new SqlParameter("#aspKey", Instance.AddParameterType(_DbTypes.Int));
myParam.SqlParam.Value = aspKey;
Instance.parameterList.Add(myParam);
myParam = new MyParam();
myParam.SqlParam = new SqlParameter("#fy", Instance.AddParameterType(_DbTypes.Int));
myParam.SqlParam.Value = fy;
Instance.parameterList.Add(myParam);
myParam = new MyParam();
myParam.SqlParam = new SqlParameter("#accountCode", Instance.AddParameterType(_DbTypes._string));
myParam.SqlParam.Value = accountCode;
Instance.parameterList.Add(myParam);
if (Convert.ToInt32(ExecuteScaler(Instance)) < 1)
return false;
return true;
}
Calling code with no parameters:
public long GetMarinesUploadNextUploadKey()
{
string query = "SELECT MAX(upload_key) FROM temp_auth_usmc_upload";
QueryContainer Instance = new QueryContainer(query);
string result = Convert.ToString(ExecuteScaler(Instance));
if (string.IsNullOrEmpty(result))
return 1;
else
return Convert.ToInt64(result) + 1;
}
Code calling my previous code with three parameters:
public bool isTamAsp(int aspKey, int fy, string accountCode)
{
return e581provider.isTamAsp(aspKey, fy, accountCode);
}
Method calling the SQL executing my code:
DbCommand command = _provider.CreateCommand();
command.Connection = _connection;
{
command.CommandText = Instance.Query;
command.CommandType = CommandType.Text;
if (Instance.parameterList.Count > 0)
{
foreach (var p in Instance.parameterList)
{
command.Parameters.Add(p.SqlParam);
}
}
if (_useTransaction) { command.Transaction = _transaction; }
try
{
returnValue = command.ExecuteScalar();
}
My Class containing the SQL string and the cmd parameter List
public enum _DbTypes
{
Int = 1, _string = 2, _long = 3, _bool = 4, _DateTime = 5,
_decimal = 6, _float = 7, _short = 8, _bite = 9
}
public class MyParam
{
public SqlParameter SqlParam { get; set; }
}
/// <summary>
/// Summary description for QueryContainer SGH
/// </summary>
public class QueryContainer
{
string _query;
public List<MyParam> parameterList = new List<MyParam>();
public QueryContainer(string query) { _query = query; }
public SqlDbType AddParameterType(_DbTypes id)
{
switch (id)
{
case _DbTypes.Int:
return (SqlDbType)Enum.Parse(typeof(SqlDbType), "int", true);
case _DbTypes._string:
return (SqlDbType)Enum.Parse(typeof(SqlDbType), "NVarChar", true);
case _DbTypes._long:
return (SqlDbType)Enum.Parse(typeof(SqlDbType), "SqlDbType.BigInt", true);
case _DbTypes._bool:
return (SqlDbType)Enum.Parse(typeof(SqlDbType), "SqlDbType.Bit", true);
}
return SqlDbType.VarChar;
}
public string Query
{
get
{
return _query;
}
set { _query = value; }
}
}

I don't see a vulnerability in that code, but I have an idea what the scan may be asking for. The problem could be that this code makes it too easy for developers to ignore the parameterList collection in your class. If I'm a new developer in your organization who hasn't discovered Sql injection yet, I'd be tempted to ignore all that complicated query parameter stuff and just use string concatenation before setting your Query property.
Instead of wrapping this in a class, what I'm more used to seeing is a single method that has a signature like this:
IEnumerable<T> GetData<T>(string query, IEnumerable<Sqlparameter> parameters)
...or some permutation of that method signature that may use arrays or lists instead of IEnumerable. This forces downstream developers to deal with that parameters argument to the method. They can't ignore it, and so the temptation to use a quick, lazy string concatenation call to substitute some user-provided data into the query is reduced.

How to take a CSV field and write to columns in SQL

I have the following code which takes a CSV and writes to a console:
using (CsvReader csv = new CsvReader(
new StreamReader("data.csv"), true))
{
// missing fields will not throw an exception,
// but will instead be treated as if there was a null value
csv.MissingFieldAction = MissingFieldAction.ReplaceByNull;
// to replace by "" instead, then use the following action:
//csv.MissingFieldAction = MissingFieldAction.ReplaceByEmpty;
int fieldCount = csv.FieldCount;
string[] headers = csv.GetFieldHeaders();
while (csv.ReadNextRecord())
{
for (int i = 0; i < fieldCount; i++)
Console.Write(string.Format("{0} = {1};",
headers[i],
csv[i] == null ? "MISSING" : csv[i]));
Console.WriteLine();
}
}
The CSV file has 7 headers for which I have 7 columns in my SQL table.
What is the best way to take each csv[i] and write to a row for each column and then move to the next row?
I tried to add the ccsv[i] to a string array but that didn't work.
I also tried the following:
SqlCommand sql = new SqlCommand("INSERT INTO table1 [" + csv[i] + "]", mysqlconnectionstring);
sql.ExecuteNonQuery();
My table (table1) is like this:
name address city zipcode phone fax device

your problem is simple but I will take it one step further and let you know a better way to approach the issue.
when you have a problem to sold, always break it down into parts and apply each part in each own method. For example, in your case:
1 - read from the file
2 - create a sql query
3 - run the query
and you can even add validation to the file (imagine your file does not even have 7 fields in one or more lines...) and the example below it to be taken, only if your file never passes around 500 lines, as if it does normally you should consider to use a SQL statement that takes your file directly in to the database, it's called bulk insert
1 - read from file:
I would use a List<string> to hold the line entries and I always use StreamReader to read from text files.
using (StreamReader sr = File.OpenText(this.CsvPath))
{
while ((line = sr.ReadLine()) != null)
{
splittedLine = line.Split(new string[] { this.Separator }, StringSplitOptions.None);
if (iLine == 0 && this.HasHeader)
// header line
this.Header = splittedLine;
else
this.Lines.Add(splittedLine);
iLine++;
}
}
2 - generate the sql
foreach (var line in this.Lines)
{
string entries = string.Concat("'", string.Join("','", line))
.TrimEnd('\'').TrimEnd(','); // remove last ",'"
this.Query.Add(string.Format(this.LineTemplate, entries));
}
3 - run the query
SqlCommand sql = new SqlCommand(string.Join("", query), mysqlconnectionstring);
sql.ExecuteNonQuery();
having some fun I end up doing the solution and you can download it here, the output is:
The code can be found here. It needs more tweaks but I will left that for others. Solution written in C#, VS 2013.
The ExtractCsvIntoSql class is as follows:
public class ExtractCsvIntoSql
{
private string CsvPath, Separator;
private bool HasHeader;
private List<string[]> Lines;
private List<string> Query;
/// <summary>
/// Header content of the CSV File
/// </summary>
public string[] Header { get; private set; }
/// <summary>
/// Template to be used in each INSERT Query statement
/// </summary>
public string LineTemplate { get; set; }
public ExtractCsvIntoSql(string csvPath, string separator, bool hasHeader = false)
{
this.CsvPath = csvPath;
this.Separator = separator;
this.HasHeader = hasHeader;
this.Lines = new List<string[]>();
// you can also set this
this.LineTemplate = "INSERT INTO [table1] SELECT ({0});";
}
/// <summary>
/// Generates the SQL Query
/// </summary>
/// <returns></returns>
public List<string> Generate()
{
if(this.CsvPath == null)
throw new ArgumentException("CSV Path can't be empty");
// extract csv into object
Extract();
// generate sql query
GenerateQuery();
return this.Query;
}
private void Extract()
{
string line;
string[] splittedLine;
int iLine = 0;
try
{
using (StreamReader sr = File.OpenText(this.CsvPath))
{
while ((line = sr.ReadLine()) != null)
{
splittedLine = line.Split(new string[] { this.Separator }, StringSplitOptions.None);
if (iLine == 0 && this.HasHeader)
// header line
this.Header = splittedLine;
else
this.Lines.Add(splittedLine);
iLine++;
}
}
}
catch (Exception ex)
{
if(ex.InnerException != null)
while (ex.InnerException != null)
ex = ex.InnerException;
throw ex;
}
// Lines will have all rows and each row, the column entry
}
private void GenerateQuery()
{
foreach (var line in this.Lines)
{
string entries = string.Concat("'", string.Join("','", line))
.TrimEnd('\'').TrimEnd(','); // remove last ",'"
this.Query.Add(string.Format(this.LineTemplate, entries));
}
}
}
and you can run it as:
class Program
{
static void Main(string[] args)
{
string file = Ask("What is the CSV file path? (full path)");
string separator = Ask("What is the current separator? (; or ,)");
var extract = new ExtractCsvIntoSql(file, separator);
var sql = extract.Generate();
Output(sql);
}
private static void Output(IEnumerable<string> sql)
{
foreach(var query in sql)
Console.WriteLine(query);
Console.WriteLine("*******************************************");
Console.Write("END ");
Console.ReadLine();
}
private static string Ask(string question)
{
Console.WriteLine("*******************************************");
Console.WriteLine(question);
Console.Write("= ");
return Console.ReadLine();
}
}

Usually i like to be a bit more generic so i'll try to explain a very basic flow i use from time to time:
I don't like the hard coded attitude so even if your code will work it will be dedicated specifically to one type. I prefer i simple reflection, first to understand what DTO is it and then to understand what repository should i use to manipulate it:
For example:
public class ImportProvider
{
private readonly string _path;
private readonly ObjectResolver _objectResolver;
public ImportProvider(string path)
{
_path = path;
_objectResolver = new ObjectResolver();
}
public void Import()
{
var filePaths = Directory.GetFiles(_path, "*.csv");
foreach (var filePath in filePaths)
{
var fileName = Path.GetFileName(filePath);
var className = fileName.Remove(fileName.Length-4);
using (var reader = new CsvFileReader(filePath))
{
var row = new CsvRow();
var repository = (DaoBase)_objectResolver.Resolve("DAL.Repository", className + "Dao");
while (reader.ReadRow(row))
{
var dtoInstance = (DtoBase)_objectResolver.Resolve("DAL.DTO", className + "Dto");
dtoInstance.FillInstance(row.ToArray());
repository.Save(dtoInstance);
}
}
}
}
}
Above is a very basic class responsible importing the data. Nevertheless of how this piece of code parsing CSV files (CsvFileReader), the important part is thata "CsvRow" is a simple List.
Below is the implementation of the ObjectResolver:
public class ObjectResolver
{
private readonly Assembly _myDal;
public ObjectResolver()
{
_myDal = Assembly.Load("DAL");
}
public object Resolve(string nameSpace, string name)
{
var myLoadClass = _myDal.GetType(nameSpace + "." + name);
return Activator.CreateInstance(myLoadClass);
}
}
The idea is to simple follow a naming convetion, in my case is using a "Dto" suffix for reflecting the instances, and "Dao" suffix for reflecting the responsible dao. The full name of the Dto or the Dao can be taken from the csv name or from the header (as you wish)
Next step is filling the Dto, each dto or implements the following simple abstract:
public abstract class DtoBase
{
public abstract void FillInstance(params string[] parameters);
}
Since each Dto "knows" his structure (just like you knew to create an appropriate table in the database), it can easily implement the FillInstanceMethod, here is a simple Dto example:
public class ProductDto : DtoBase
{
public int ProductId { get; set; }
public double Weight { get; set; }
public int FamilyId { get; set; }
public override void FillInstance(params string[] parameters)
{
ProductId = int.Parse(parameters[0]);
Weight = double.Parse(parameters[1]);
FamilyId = int.Parse(parameters[2]);
}
}
After you have your Dto filled with data you should find the appropriate Dao to handle it
which is basically happens in reflection in this line of the Import() method:
var repository = (DaoBase)_objectResolver.Resolve("DAL.Repository", className + "Dao");
In my case the Dao implements an abstract base class - but it's not that relevant to your problem, your DaoBase can be a simple abstract with a single Save() method.
This way you have a dedicated Dao to CRUD your Dto's - each Dao simply knows how to save for its relevant Dto. Below is the corresponding ProductDao to the ProductDto:
public class ProductDao : DaoBase
{
private const string InsertProductQuery = #"SET foreign_key_checks = 0;
Insert into product (productID, weight, familyID)
VALUES (#productId, #weight, #familyId);
SET foreign_key_checks = 1;";
public override void Save(DtoBase dto)
{
var productToSave = dto as ProductDto;
var saveproductCommand = GetDbCommand(InsertProductQuery);
if (productToSave != null)
{
saveproductCommand.Parameters.Add(CreateParameter("#productId", productToSave.ProductId));
saveproductCommand.Parameters.Add(CreateParameter("#weight", productToSave.Weight));
saveproductCommand.Parameters.Add(CreateParameter("#familyId", productToSave.FamilyId));
ExecuteNonQuery(ref saveproductCommand);
}
}
}
Please ignore the CreateParameter() method, since it's an abstraction from the base classs. you can just use a CreateSqlParameter or CreateDataParameter etc.
Just notice, it's a real naive implementation - you can easily remodel it better, depends on your needs.

From the first impression of your questionc I guess you would be having hugely number of records (more than lacs). If yes I would consider the SQL bulk copies an option. If the record would be less go ahead single record. Insert. The reason for you insert not working is u not providing all the columns of the table and also there's some syntax error.

How to increment the string value during run time without using dr.read() and store it in the oledb db?

Here is my code, everything is working fine the only problem is with the ReadData() method in which i want the string value to be increment i.e AM0001,AM0002,AM0003 etc. This is happening till the execution of the program once i stop the execution of program, the second time when i run the program the same value i.e AM0001 is getting return. Due to this i am getting a error from oledb because of AM0001 is a primary key field.
This is my code:
class Jewellery : Connectionstr
{
string lmcode;
public string LM_code
{
get { return lmcode;}
set { lmcode = ReadData();}
}
string mname;
public string M_Name
{
get { return mname; }
set { mname = value;}
}
string desc;
public string Desc
{
get { return desc; }
set { desc = value; }
}
public string ReadData()
{
string jid = string.Empty;
string displayString = string.Empty;
String query = "select max(LM_code)from Master_Accounts";
Datamanager.RunExecuteReader(Constr,query);
jid = LM_code;// this is working on first execution, the second time when i run the program the value null defined in LM_code.
if (string.IsNullOrEmpty(jid))
{
jid = "AM0000";//This string value has to increment at every time, but it is getting increment only one time.
}
int len = jid.Length;
string split = jid.Substring(2, len - 2);
int num = Convert.ToInt32(split);
num++;
displayString = jid.Substring(0, 2) + num.ToString("0000");
return displayString;
}
public void add()
{
String query ="insert into Master_Accounts values ('" + LM_code + "','" + M_Name + "','" + Desc + "')";
Datamanager.RunExecuteNonQuery(Constr , query);
}
Any help will be appreciated.

Do not declare jid in method, declare it in class level:
private string jid = string.Empty;