Develop Reference

Fill a MDF Database with CSV data

Well this is how my CSV data looks like:
Artistname;RecordTitle;RecordType;Year;SongTitle
999;Concrete;LP;1981;Mercy Mercy
999;Concrete;LP;1981;Public Enemy No.1
999;Concrete;LP;1981;So Greedy
999;Concrete;LP;1981;Taboo
10cc;Bloody Tourists;LP;1978;Dreadlock Holiday
10cc;Bloody Tourists;LP;1978;Everyhing You've Ever Wanted To Know About!!!
10cc;Bloody Tourists;LP;1978;Shock On The Tube
This is my code where I save this data in the Database:
private void FillDatabase()
{
var firstTime = true;
var lines = File.ReadAllLines("musicDbData.csv");
var list = new List<string>();
foreach (var line in lines)
{
var split = line.Split(";");
if (!firstTime)
{
var artist = new Artist()
{
ArtistName = split[0],
};
db.Artists.Add(artist);
db.SaveChanges();
}
else
{
firstTime = false;
}
}
}
The problem is that every artist should be in the Database only once. Right now there is 4 times Artist 999 and 3 times 10cc and if everything is correct there should only be one row for 999 and one row for 10cc. What do I have to add to my code to get the expected result.

First, a CSV is a comma-separated values file, rather than semicolon.
Besides, the parameter in method String.Split can be type of Char. So you need to modify it like line.Split(';').
And your csv file contains column name line, you need to exclude it when reading the file.
if everything is correct there should only be one row for 999 and one row for 10cc
Do you want to just save the first data of 999 and 10cc to the database? If so, you can first use LINQ to check whether the Artistname already exists in the database.
private void FillDatabase()
{
var lines = File.ReadAllLines("musicDbData.csv");
int count = 0; // line count
foreach (var line in lines)
{
count++;
if (count == 1) // remove first line
continue;
var split = line.Split(';');
string artistname = split[0];
var artistIndb = db.ArtistTables
.Where(c => c.Artistname == artistname)
.SingleOrDefault();
if (artistIndb == null) // check if exists, if not ...
{
var artist = new ArtistTable()
{
Artistname = split[0],
SongTitle = split[4]
};
db.ArtistTables.Add(artist);
db.SaveChanges();
}
}
}
If you want to merge lines with the same Artistname, you can refer to the following code.
if (artistIndb == null)
{
// code omitted
// ...
}
else
{
artistIndb.SongTitle += " ," + split[4]; // Modify the data in SongTitle column
try
{
db.SaveChanges();
}
catch { }
}

Join one-dimesional array to write over a CSV file with multiple records

I am at the final step of a Save function and I need help figuring out how to structure the data back into the format of my CSV so that I can write over the original. This is a link to my other code/post helping me get to this point: StackOverflow, first question I have separated the records and changed the right values but now I need to join the records back in the same format as the CSV file to then write over the original CSV.
Here is my code:
string txt = System.IO.File.ReadAllText("Assets/Resources/characters.csv.txt");
string[] data = txt.Split(new char[] { '\n' });
foreach(string record in data){
var dataValues = record.Split(',');
if(dataValues.Length >= 6 && dataValues[1] == trickName)
{
dataValues[3] = currentXP.ToString();
dataValues[4] = currentLevel.ToString();
dataValues[5] = maximumXP.ToString();
}
foreach(string row in dataValues)
{
//prints out a one-dimesional array, single lines no commas
Debug.Log(row);
}
}

Use string.Join() to concatenate the separated and updated values back into a single line.
// string txt = System.IO.File.ReadAllText("Assets/Resources/characters.csv.txt");
// string[] data = txt.Split(new char[] { '\n' });
// use ReadAllLines function instead of ReadAllText
string[] data = File.ReadAllLines("Assets/Resources/characters.csv.txt");
foreach(string record in data)
{
var dataValues = record.Split(',');
if(dataValues.Length >= 6 && dataValues[1] == trickName)
{
dataValues[3] = currentXP.ToString();
dataValues[4] = currentLevel.ToString();
dataValues[5] = maximumXP.ToString();
}
// concatenate values into comma delimited line
string outputLine = string.Join(",", dataValues);
Debug.Log(outputLine);
}

How to read and separate segments of a txt file?

I have a txt file, that has headers and then 3 columns of values (i.e)
Description=null
area = 100
1,2,3
1,2,4
2,1,5 ...
... 1,2,1//(these are the values that I need in one list)
Then another segment
Description=null
area = 10
1,2,3
1,2,4
2,1,5 ...
... 1,2,1//(these are the values that I need in one list).
In fact I just need one list per "Table" of values, the values always are in 3 columns but, there are n segments, any idea?
Thanks!
List<double> VMM40xyz = new List<double>();
foreach (var item in VMM40blocklines)
{
if (item.Contains(','))
{
VMM40xyz.AddRange(item.Split(',').Select(double.Parse).ToList());
}
}
I tried this, but it just work with the values in just one big list.

It looks like you want your data to end up in a format like this:
public class SetOfData //Feel free to name these parts better.
{
public string Description = "";
public string Area = "";
public List<double> Data = new List<double>();
}
...stored somewhere in...
List<SetOfData> finalData = new List<SetOfData>();
So, here's how I'd read that in:
public static List<SetOfData> ReadCustomFile(string Filename)
{
if (!File.Exists(Filename))
{
throw new FileNotFoundException($"{Filename} does not exist.");
}
List<SetOfData> returnData = new List<SetOfData>();
SetOfData currentDataSet = null;
using (FileStream fs = new FileStream(Filename, FileMode.Open))
{
using (StreamReader reader = new StreamReader(fs))
{
while (!reader.EndOfStream)
{
string line = reader.ReadLine();
//This will start a new object on every 'Description' line.
if (line.Contains("Description="))
{
//Save off the old data set if there is one.
if (currentDataSet != null)
returnData.Add(currentDataSet);
currentDataSet = new SetOfData();
//Now, to make sure there is something after "Description=" and to set the Description if there is.
//Your example data used "null" here, which this will take literally to be a string containing the letters "null". You can check the contents of parts[1] inside the if block to change this.
string[] parts = line.Split('=');
if (parts.Length > 1)
currentDataSet.Description = parts[1].Trim();
}
else if (line.Contains("area = "))
{
//Just in case your file didn't start with a "Description" line for some reason.
if (currentDataSet == null)
currentDataSet = new SetOfData();
//And then we do some string splitting like we did for Description.
string[] parts = line.Split('=');
if (parts.Length > 1)
currentDataSet.Area = parts[1].Trim();
}
else
{
//Just in case your file didn't start with a "Description" line for some reason.
if (currentDataSet == null)
currentDataSet = new SetOfData();
string[] parts = line.Split(',');
foreach (string part in parts)
{
if (double.TryParse(part, out double number))
{
currentDataSet.Data.Add(number);
}
}
}
}
//Make sure to add the last set.
returnData.Add(currentDataSet);
}
}
return returnData;
}

Removing specified text from CSV file

it's my first attempt at doing this and I have no idea if I'm on the right lines.
Basically I want to remove text from a CSV file that contains a specific keyword but I can't figure out how to remove the line.
static void Main(string[] args)
{
var searchItem = "running";
var lines = File.ReadLines("C://Users//Pete//Desktop//testdata.csv");
foreach (string line in lines)
{
if (line.Contains(searchItem))
{
//Remove line here?
}
}
}

Try this one to remove one or a few multiple words.
static void sd(string[] args)
{
string contents = File.ReadAllText("C://Users//Pete//Desktop//testdata.csv");
string output = contents.Replace("running", string.Empty).Replace("replaceThis", string.Empty).Replace("replaceThisToo", string.Empty);
//string output = contents.Replace("a", "b").Replace("b", "c").Replace("c", "d");
}
To remove multiple string, you can use this...
static void Main(string[] args)
{
string[] removeTheseWords = { "aaa", "bbb", "ccc" };
string contents = File.ReadAllText("C://Users//Pete//Desktop//testdata.csv");
string output = string.Empty;
foreach (string value in removeTheseWords)
{
output = contents.Replace(value, string.Empty);
}
}
More info: https://learn.microsoft.com/en-us/dotnet/api/system.string.replace

The simple way if you'd like to remove a whole line:
var searchItem = "running";
var pathToYourFile = #"C://Users//Pete//Desktop//testdata.csv";
var lines = File.ReadAllLines(pathToYourFile);
lines = lines.Where(line => !line.Contains(searchItem)).ToArray();
File.WriteAllLines(pathToYourFile, lines);
For multiple search items:
var searchItems = "running;walking;waiting;any";
var pathToYourFile = #"..\..\items.csv";
var lines = File.ReadAllLines(pathToYourFile);
// split with your separator, actually is ';' character
foreach(var searchItem in searchItems.Split(';'))
lines = lines.Where(line =>!line.Contains(searchItem)).ToArray();
File.WriteAllLines(pathToYourFile, lines);

if you are using foreach and removing from lines its will through an exception called collection modified exception so go with for
for(int i=lines.Count - 1; i > -1; i--)
{
if (lines[i].Contains(searchItem))
{
lines.RemoveAt(i);
}
}

You don't need to remove line just skip those lines that contain your search term
foreach (string line in lines)
{
if (!line.Contains(searchItem)) //<= Notice here I added exclamation mark (!)
{
//Do your work when line does not contains search term
}
else
{
//Do something if line contains search term
}
}
Or alternative is to filtered your lines that does not contains your search term before loop like
lines = lines.Where(line => !line.Contains(searchItem));
foreach (string line in lines)
{
//Here are those line that does not contain search term
}
If your search term contains multiple words separated with comma(,) then you can skip those line by
lines = lines.Where(line => searchItem.Split(',').All(term => !line.Contains(term)));

C# compare fields from different lines in csv

I am trying to compare the value in the 0 index of an array on one line and the 0 index on the following line. Imagine a CSV where I have a unique identifier in the first column, a corresponding value in the second column.
USER1, 1P
USER1, 3G
USER2, 1P
USER3, 1V
I would like to check the value of [0] the next line (or previous if that's easier) to compare and if they are the same (as they are in the example) concatenate it to index 1. That is, the data should read as
USER1, 1P, 3G
USER2, 1P
USER3, 1V
before it gets passed onto the next function. So far I have
private void csvParse(string path)
{
using (TextFieldParser parser = new TextFieldParser(path))
{
parser.Delimiters = new string[] { "," };
while (!parser.EndOfData)
{
string[] parts = parser.ReadFields();
if (parts == null)
{
break;
}
contact.ContactId = parts[0];
long nextLine;
nextLine = parser.LineNumber+1;
//if line1 parts[0] == line2 parts[0] etc.
}
}
}
Does anyone have any suggestions? Thank you.

How about saving the array into a variable:
private void csvParse(string path)
{
using (TextFieldParser parser = new TextFieldParser(path))
{
parser.Delimiters = new string[] { "," };
string[] oldParts = new string[] { string.Empty };
while (!parser.EndOfData)
{
string[] parts = parser.ReadFields();
if (parts == null || parts.Length < 1)
{
break;
}
if (oldParts[0] == parts[0])
{
// concat logic goes here
}
else
{
contact.ContactId = parts[0];
}
long nextLine;
nextLine = parser.LineNumber+1;
oldParts = parts;
//if line1 parts[0] == line2 parts[0] etc.
}
}
}

If I understand you correctly, what you are asking is essentially "how do I group the values in the second column based on the values in the first column?".
A quick and quite succinct way of doing this would be to Group By using LINQ:
var linesGroupedByUser =
from line in File.ReadAllLines(path)
let elements = line.Split(',')
let user = new {Name = elements[0], Value = elements[1]}
group user by user.Name into users
select users;
foreach (var user in linesGroupedByUser)
{
string valuesAsString = String.Join(",", user.Select(x => x.Value));
Console.WriteLine(user.Key + ", " + valuesAsString);
}
I have left out the use of your TextFieldParser class, but you can easily use that instead. This approach does, however, require that you can afford to load all of the data into memory. You don't mention whether this is viable.

The easiest way to do something like this is to convert each line to an object. You can use CsvHelper, https://www.nuget.org/packages/CsvHelper/, to do the work for you or you can iterate each line and parse to an object. It is a great tool and it knows how to properly parse CSV files into a collection of objects. Then, whether you create the collection yourself or use CsvHelper, you can use Linq to GroupBy, https://msdn.microsoft.com/en-us/library/bb534304(v=vs.100).aspx, your "key" (in this case UserId) and Aggregate, https://msdn.microsoft.com/en-us/library/bb549218(v=vs.110).aspx, the other property into a string. Then, you can use the new, grouped by, collection for your end goal (write it to file or use it for whatever you need).

You're basically finding all the unique entries so put them into a dictionary with the contact id as the key. As follows:
private void csvParse(string path)
{
using (TextFieldParser parser = new TextFieldParser(path))
{
parser.Delimiters = new string[] { "," };
Dictionary<string, List<string>> uniqueContacts = new Dictionary<string, List<string>>();
while (!parser.EndOfData)
{
string[] parts = parser.ReadFields();
if (parts == null || parts.Count() != 2)
{
break;
}
//if contact id not present in dictionary add
if (!uniqueContacts.ContainsKey(parts[0]))
uniqueContacts.Add(parts[0],new List<string>());
//now there's definitely an existing contact in dic (the one
//we've just added or a previously added one) so add to the
//list of strings for that contact
uniqueContacts[parts[0]].Add(parts[1]);
}
//now do something with that dictionary of unique user names and
// lists of strings, for example dump them to console in the
//format you specify:
foreach (var contactId in uniqueContacts.Keys)
{
var sb = new StringBuilder();
sb.Append($"contactId, ");
foreach (var bit in uniqueContacts[contactId])
{
sb.Append(bit);
if (bit != uniqueContacts[contactId].Last())
sb.Append(", ");
}
Console.WriteLine(sb);
}
}
}