I would like to write the result of a SELECT query in Cassandra into a CSV file. Any help, please. I am just new to C#. Here is what I did. Any help to write the result into CSV file? Thanks
Cluster cluster = Cluster.Builder().AddContactPoint("127.0.0.1").Build();
List<string> lstOfKeyspaces = cluster.Metadata.GetKeyspaces().ToList<string>();
ISession session = cluster.Connect("test");
//get tables in the keysapce
List<string> lstString = cluster.Metadata.GetTables("test").ToList<string>();
Console.WriteLine("Connection succeeded");
// Console.ReadLine();
// RowSet resultRequest = session.Execute(" select * from integrationobjects.emp");
//Execute a query on a connection synchronously
var rs = session.Execute("SELECT * FROM emp");
Here is the solution
List<string> lstColumnName = new List<string>();
string strPath = System.IO.Directory.GetCurrentDirectory();
try
{
Cluster cassandraCluster = Cluster.Builder().AddContactPoint(strIpAddress).Build();
ISession session = cassandraCluster.Connect(strKeyspace);
string strCqlRequest = "SELECT * FROM" + " " + strTable;
RowSet rs = session.Execute(strCqlRequest);
using (var w = new StreamWriter(strPathToSave))
{
//get table columns with types
TableMetadata t = cassandraCluster.Metadata.GetTable(strKeyspace,strTable);
TableColumn[] arrcol = t.TableColumns;
foreach (var strCol in arrcol)
{
lstColumnName.Add(strCol.Name);
}
IDictionary<string,TableColumn> dic =t.ColumnsByName;
//Add column liste to the file
var strColumnLine = String.Join(",", lstColumnName.ToArray());
w.WriteLine(strColumnLine);
//Iterate through the RowSet and add rows to the file
foreach (Row row in rs)
{
List<string> values = new List<string>();
IEnumerator<Object> colEnumerator = row.GetEnumerator();
while (colEnumerator.MoveNext())
{
values.Add(colEnumerator.Current.ToString());
}
var line = String.Join(",", values.ToArray());
w.WriteLine(line);
w.Flush();
}
}
Related
How can I print all the column names?
Console.WriteLine(row.RawRow.ETag); does not work it prints blank lines
This is my code so far
static void Main(string[] args)
{
List<string> rows = new List<string>();
string projectId = "project-id 123";
var client = BigQueryClient.Create(projectId);
string sql = #"SELECT * FROM table";
var res = client.ExecuteQuery(sql, parameters: null);
foreach (var row in res)
Console.WriteLine(row["id"])
//rows.Add(row["id"])
}
One of the examples has your answer
Here
public List<T> Execute<T>(string sql)
{
var client = BigQueryClient.Create(projectId);
List<T> result = new List<T>();
try
{
string query = sql;
BigQueryResults results = client.ExecuteQuery(query, parameters: null);
List<string> fields = new List<string>();
foreach (var col in results.Schema.Fields)
{
fields.Add(col.Name);
}
i am currently working on a small Project and i got stuck with a Problem i currently can not manage to solve...
I have multiple ".CSV" Files i want to read, they all have the same Data just with different Values.
Header1;Value1;Info1
Header2;Value2;Info2
Header3;Value3;Info3
While reading the first File i Need to Create the Headers. The Problem is they are not splited in Columns but in rows (as you can see above Header1-Header3).
Then it Needs to read the Value 1 - Value 3 (they are listed in the 2nd Column) and on top of that i Need to create another Header -> Header4 with the data of "Info2" which is always placed in Column 3 and Row 2 (the other values of Column 3 i can ignore).
So the Outcome after the first File should look like this:
Header1;Header2;Header3;Header4;
Value1;Value2;Value3;Info2;
And after multiple files it sohuld be like this:
Header1;Header2;Header3;Header4;
Value1;Value2;Value3;Value4;
Value1b;Value2b;Value3b;Value4b;
Value1c;Value2c;Value3c;Value4c;
I tried it with OleDB but i get the Error "missing ISAM" which i cant mange to fix. The Code i Used is the following:
public DataTable ReadCsv(string fileName)
{
DataTable dt = new DataTable("Data");
/* using (OleDbConnection cn = new OleDbConnection("Provider=Microsoft.Jet.OLEDB.4.0;Data Source=\"" +
Path.GetDirectoryName(fileName) + "\";Extendet Properties ='text;HDR=yes;FMT=Delimited(,)';"))
*/
using (OleDbConnection cn = new OleDbConnection("Provider=Microsoft.Jet.OLEDB.4.0;Data Source=" +
Path.GetDirectoryName(fileName) + ";Extendet Properties ='text;HDR=yes;FMT=Delimited(,)';"))
{
using(OleDbCommand cmd = new OleDbCommand(string.Format("select *from [{0}]", new FileInfo(fileName).Name,cn)))
{
cn.Open();
using(OleDbDataAdapter adapter = new OleDbDataAdapter(cmd))
{
adapter.Fill(dt);
}
}
}
return dt;
}
Another attempt i did was using StreamReader. But the Headers are in the wrong place and i dont know how to Change this + do this for every file. the Code i tried is the following:
public static DataTable ReadCsvFilee(string path)
{
DataTable oDataTable = new DataTable();
var fileNames = Directory.GetFiles(path);
foreach (var fileName in fileNames)
{
//initialising a StreamReader type variable and will pass the file location
StreamReader oStreamReader = new StreamReader(fileName);
// CONTROLS WHETHER WE SKIP A ROW OR NOT
int RowCount = 0;
// CONTROLS WHETHER WE CREATE COLUMNS OR NOT
bool hasColumns = false;
string[] ColumnNames = null;
string[] oStreamDataValues = null;
//using while loop read the stream data till end
while (!oStreamReader.EndOfStream)
{
String oStreamRowData = oStreamReader.ReadLine().Trim();
if (oStreamRowData.Length > 0)
{
oStreamDataValues = oStreamRowData.Split(';');
//Bcoz the first row contains column names, we will poluate
//the column name by
//reading the first row and RowCount-0 will be true only once
// CHANGE TO CHECK FOR COLUMNS CREATED
if (!hasColumns)
{
ColumnNames = oStreamRowData.Split(';');
//using foreach looping through all the column names
foreach (string csvcolumn in ColumnNames)
{
DataColumn oDataColumn = new DataColumn(csvcolumn.ToUpper(), typeof(string));
//setting the default value of empty.string to newly created column
oDataColumn.DefaultValue = string.Empty;
//adding the newly created column to the table
oDataTable.Columns.Add(oDataColumn);
}
// SET COLUMNS CREATED
hasColumns = true;
// SET RowCount TO 0 SO WE KNOW TO SKIP COLUMNS LINE
RowCount = 0;
}
else
{
// IF RowCount IS 0 THEN SKIP COLUMN LINE
if (RowCount++ == 0) continue;
//creates a new DataRow with the same schema as of the oDataTable
DataRow oDataRow = oDataTable.NewRow();
//using foreach looping through all the column names
for (int i = 0; i < ColumnNames.Length; i++)
{
oDataRow[ColumnNames[i]] = oStreamDataValues[i] == null ? string.Empty : oStreamDataValues[i].ToString();
}
//adding the newly created row with data to the oDataTable
oDataTable.Rows.Add(oDataRow);
}
}
}
//close the oStreamReader object
oStreamReader.Close();
//release all the resources used by the oStreamReader object
oStreamReader.Dispose();
}
return oDataTable;
}
I am thankful for everyone who is willing to help. And Thanks for reading this far!
Sincerely yours
If I understood you right, there is a strict parsing there like this:
string OpenAndParse(string filename, bool firstFile=false)
{
var lines = File.ReadAllLines(filename);
var parsed = lines.Select(l => l.Split(';')).ToArray();
var header = $"{parsed[0][0]};{parsed[1][0]};{parsed[2][0]};{parsed[1][0]}\n";
var data = $"{parsed[0][1]};{parsed[1][1]};{parsed[2][1]};{parsed[1][2]}\n";
return firstFile
? $"{header}{data}"
: $"{data}";
}
Where it would return - if first file:
Header1;Header2;Header3;Header2
Value1;Value2;Value3;Value4
if not first file:
Value1;Value2;Value3;Value4
If I am correct, rest is about running this against a list file of files and joining the results in an output file.
EDIT: Against a directory:
void ProcessFiles(string folderName, string outputFileName)
{
bool firstFile = true;
foreach (var f in Directory.GetFiles(folderName))
{
File.AppendAllText(outputFileName, OpenAndParse(f, firstFile));
firstFile = false;
}
}
Note: I missed you want a DataTable and not an output file. Then you could simply create a list and put the results into that list making the list the datasource for your datatable (then why would you use semicolons in there? Probably all you need is to simply attach the array values to a list).
(Adding as another answer just to make it uncluttered)
void ProcessMyFiles(string folderName)
{
List<MyData> d = new List<MyData>();
var files = Directory.GetFiles(folderName);
foreach (var file in files)
{
OpenAndParse(file, d);
}
string[] headers = GetHeaders(files[0]);
DataGridView dgv = new DataGridView {Dock=DockStyle.Fill};
dgv.DataSource = d;
dgv.ColumnAdded += (sender, e) => {e.Column.HeaderText = headers[e.Column.Index];};
Form f = new Form();
f.Controls.Add(dgv);
f.Show();
}
string[] GetHeaders(string filename)
{
var lines = File.ReadAllLines(filename);
var parsed = lines.Select(l => l.Split(';')).ToArray();
return new string[] { parsed[0][0], parsed[1][0], parsed[2][0], parsed[1][0] };
}
void OpenAndParse(string filename, List<MyData> d)
{
var lines = File.ReadAllLines(filename);
var parsed = lines.Select(l => l.Split(';')).ToArray();
var data = new MyData
{
Col1 = parsed[0][1],
Col2 = parsed[1][1],
Col3 = parsed[2][1],
Col4 = parsed[1][2]
};
d.Add(data);
}
public class MyData
{
public string Col1 { get; set; }
public string Col2 { get; set; }
public string Col3 { get; set; }
public string Col4 { get; set; }
}
I don't know if this is the best way to do this. But what i would have done in your case, is to rewrite the CSV's the conventionnal way while reading all the files, then create a stream containing the new CSV created.
It would look like something like this :
var csv = new StringBuilder();
csv.AppendLine("Header1;Header2;Header3;Header4");
foreach (var item in file)
{
var newLine = string.Format("{0},{1},{2},{3}", item.value1, item.value2, item.value3, item.value4);
csv.AppendLine(newLine);
}
//Create Stream
MemoryStream stream = new MemoryStream();
StreamReader reader = new StreamReader(stream);
//Fill your data table here with your values
Hope this will help.
I want to use the linq where condition multi value by array string is split(',')
I list data from data file in folder. (not in database)
Code c#
public List<sFile> GettingFiles(string path)
{
//Read File in folder
List<sFile> allfile = new List<sFile>();
DirectoryInfo di = new DirectoryInfo(path);
FileInfo[] fileinfo = di.GetFiles("*.*");
foreach (FileInfo item in fileinfo)
{
allfile.Add(new sFile
{
FileName = item.Name,
Seq = int.Parse(item.Name.Substring(12, item.Name.Length - 12)),
PmnCode = item.Name.Substring(7, item.Name.Length - 12),
Path = item.DirectoryName,
Size = formatSize(item.Length),
SizeInt = int.Parse(item.Length.ToString())
});
}
return allfile;
}
public void btnQuery_Click(object sender, EventArgs e)
{
List<sFile> allFiles = GettingFiles(path); //List file in Folder
string pmnCode = txtPMNCode.Text.ToString(); //AAAA, BBBBB, CCCCC, DDDDD
string[] subPmnCode = pmnCode.Split(',');
string totalPmnCode = string.Empty;
foreach (string item2 in subPmnCode)
{
var queryData = from d in allFiles.AsQueryable()
where (d.PmnCode.Contains(item2))
select d;
//Add Column
DataTable dt = new DataTable();
dt.Columns.Add(enmField.NAME.ToString());
dt.Columns.Add(enmField.SIZE.ToString());
dt.Columns.Add(enmField.MODIFY_DATE.ToString());
dt.Columns.Add(enmField.PATH.ToString());
DataRow myRow = dt.NewRow();
foreach (sFile item in queryData.ToList())
{
myRow = dt.NewRow();
myRow[enmField.NAME.ToString()] = item.FileName.Trim();
myRow[enmField.SIZE.ToString()] = item.Size.Trim();
myRow[enmField.MODIFY_DATE.ToString()] = item.Date;
myRow[enmField.PATH.ToString()] = item.Path.Trim() + "\\" + item.FileName.Trim();
dt.Rows.Add(myRow);
}
gvDetail.DataSource = dt;
gvDetail.DataBind();
}
}
Example Data
Pmn Code
AAAAA
BBBBB
CCCCC
DDDDD
I want query wheer condition by pmn_code is AAAAA,BBBBB, DDDDD
I want show data
var queryData = from d in allFiles.AsQueryable()
where (d.PmnCode.Contains("AAAAA") &&
d.PmnCode.Contains("BBBBB") &&
d.PmnCode.Contains("DDDDD")
)
select d;
But i can not query array string by result this.
How can i use array linq?
please help me. Thanks advance ;)
Maybe you can try:
var queryData = from p in allFiles.AsQueryable()
where subPmnCode.Any(val => p.PmnCode.Contains(val))
select p;
var queryData = from d in allFiles.AsQueryable()
where (subPmnCode.Any(s => s.Trim().Equals(d)))
select d;
Checks if d matches any of the elements in the subPmnCode array. I use trim to ensure that we ignore blank spaces generated by splitting the string into an array using the comma delimiter
I have two list (1st with values from a website, 2nd with values from a .csv file) and I'd like to join them in another list, starting two equals values, and display it in a datagridview.
Before to post the code, I'd like to say that I tried to fill my datagridview with these two lists separately and they work.
I didn't get any error, but I can't see my datagridview with values.
I'm going to post my code and explain it.
First List Code:
var url = textBox5.Text;
//var url = "http://www.betexplorer.com/soccer/norway/tippeligaen/results/";
var web = new HtmlWeb();
var doc = web.Load(url);
Bets = new List<Bet>();
// Lettura delle righe
var Rows = doc.DocumentNode.SelectNodes("//tr");
foreach (var row in Rows)
{
if (!row.GetAttributeValue("class", "").Contains("rtitle"))
{
if (string.IsNullOrEmpty(row.InnerText))
continue;
var rowBet = new Bet();
foreach (var node in row.ChildNodes)
{
var data_odd = node.GetAttributeValue("data-odd", "");
if (string.IsNullOrEmpty(data_odd))
{
if (node.GetAttributeValue("class", "").Contains("first-cell"))
{
rowBet.Match = node.InnerText.Trim();
var matchTeam = rowBet.Match.Split(new[] { " - " }, StringSplitOptions.RemoveEmptyEntries);
rowBet.Home = matchTeam[0];
rowBet.Host = matchTeam[1];
}
if (node.GetAttributeValue("class", "").Contains("result"))
{
rowBet.Result = node.InnerText.Trim();
var matchPoints = rowBet.Result.Split(new[] { ':' }, StringSplitOptions.RemoveEmptyEntries);
int help;
if (int.TryParse(matchPoints[0], out help))
{
rowBet.HomePoints = help;
}
if (matchPoints.Length == 2 && int.TryParse(matchPoints[1], out help))
{
rowBet.HostPoints = help;
}
}
if (node.GetAttributeValue("class", "").Contains("last-cell"))
rowBet.Date = node.InnerText.Trim();
}
else
{
rowBet.Odds.Add(data_odd);
}
}
if (!string.IsNullOrEmpty(rowBet.Match))
Bets.Add(rowBet);
}
}
Second List & Combined List Code:
string FileName = #"C:\mydir\testcsv.csv";
OleDbConnection conn = new OleDbConnection
("Provider=Microsoft.Jet.OleDb.4.0; Data Source = " +
Path.GetDirectoryName(FileName) +
"; Extended Properties = \"Text;HDR=YES;FMT=Delimited\"");
conn.Open();
OleDbDataAdapter adapter = new OleDbDataAdapter
("SELECT * FROM " + Path.GetFileName(FileName), conn);
DataSet ds = new DataSet("Temp");
adapter.Fill(ds);
conn.Close();
// DataTable dt = new DataTable();
DataTable dt = ds.Tables[0];
//dataGridView2.DataSource = dt;
// dataGridView2.DataMember = "Table";
List<HT> matchlist = new List<HT>();
matchlist = (from DataRow dr in dt.Rows
select new HT()
{
Home = dr["Home"].ToString().Replace("Milan", "AC Milan").Replace("Roma", "AS Roma"),
Host = dr["Host"].ToString().Replace("Milan", "AC Milan").Replace("Roma", "AS Roma"),
ScoreHome = dr["ScoreHome"].ToString(),
ScoreAway = dr["ScoreAway"].ToString(),
//Segno = dr["Segno"].ToString(),
//odd1 = dr["odd1"].ToString(),
//oddx = dr["oddx"].ToString(),
//odd2 = dr["odd2"].ToString()
}).ToList();
// dataGridView2.DataSource = matchlist;
var combinedDataList = (from d1 in Bets
//join d2 in dataList2 on d1.Home equals d2.Home
join d2 in matchlist on new { d1.Home, d1.Host } equals new { d2.Home, d2.Host }
select new CombinedData
{
Data = d1.Date,
Home = d1.Home,
Away = d1.Host,
HSFT = d1.HomePoints,
ASFT = d1.HostPoints,
HSHT = d2.ScoreHome,
ASHT = d2.ScoreAway,
HODD = d1.odd1,
XODD = d1.oddX,
AODD = d1.odd2,
RisFin = d1.RisFin,
Over05SH = d1.over05sh,
Over05FT = d1.Over05FT,
Over15FT = d1.Over15FT,
Over25FT = d1.Over25FT,
Over35FT = d1.Over35FT,
Over45FT = d1.Over45FT
}).OrderBy(p => p.HODD);
dataGridView2.DataSource = combinedDataList;
Thank you for your attention. Have a fantastic sunday!
EDIT: I delete unnecessary code
EDIT2: I add the screen of my single list output. Let's see:
First List:
Second List:
So, I'd like to merge "ScoreHome" and "ScoreAway" from the second list in my first list based on "Home" and "Host" that I have in both lists.
how insert data array to database with linq to sql?
DataClassesDataContext db = new DataClassesDataContext();
simpleTbl tbl = new simpleTbl();
string[] str =File.ReadAllLines(Server.MapPath("~/str.txt"));
for (int i = 0; i <= 10; i++)
{
tbl.Name =str[i];
}
db.simpleTbl.InsertOnSubmit(tbl);
db.SubmitChanges();
but dosent work
using(DataClassesDataContext db = new DataClassesDataContext())
{
string[] strings = File.ReadAllLines(Server.MapPath("~/str.txt"));
foreach (var str in strings )
{
db.simpleTbl.InsertOnSubmit(new simpleTbl(){ Name = str });
}
db.SubmitChanges();
}
You need to insert a new item for each entry in the string array.
this code is correct but uses for one array if i have more one array how implement this?
using(DataClassesDataContext db = new DataClassesDataContext())
{
string[] strings = File.ReadAllLines(Server.MapPath("~/str.txt"));
foreach (var str in strings )
{
db.simpleTbl.InsertOnSubmit(new simpleTbl(){ Name = str });
}
db.SubmitChanges();
}