Develop Reference

Dictionary<string, string> Value lookup performance

I am working on a small project but have run into a performance roadblock.
I have a Dictionary<string, string>()
I have a string[].
Lets say my Dictionary has 50,000 entries, and my string[] has 30,000 entries.
I want to collect the Keys from my Dictionary where the value.ToCharArray().OrderBy(x => x) equals a value.ToCharArray().OrderBy(x => x) of my string[].
I have tried reducing the number of KeyValue pairs I have to look through by comparing the length of my string[] value to the values in the Dictionary, but that has not really gained me any performance.
Does anyone have an ideas how I can improve the performance of this lookup?
Thanks!
To expand the pseudocode:
var stringToLookUp = GetSomeStrings(s.ToString()).Select(x => x).OrderBy(x => x).ToArray();
var aDictionaryOfStringString = GetDictionary(Resources.stringList);
var results = new List<string>();
foreach (var theString in stringToLookUp.Where(aString=> aString.Length > 0))
{
if (theString.Length > 0)
{
var theStringClosure = theString;
var filteredKeyValuePairs = aDictionaryOfStringString.Where(w => w.Value.Length == theStringClosure.Length && !results.Contains(w.Key)).ToArray();
var foundStrings = filteredKeyValuePairs.Where(kv => kv.Value.ToCharArray().OrderBy(c => c).ToArray().SequenceEqual(theStringClosure))
.Select(kv => kv.Key)
.ToArray();
if (foundStrings.Any()) results.AddRange(foundStrings);
}
}

I think principal problem is you iterate over whole dictionary in every single iteration - this is O(N^2). Better build hashset based on your modified key (either from dictionary or from array) and iterate over the second. This is O(N).
// some values
var dictionary = new Dictionary<string, string>();
var fields = new string[]{};
string[] modifiedFields = new string[fields.Length];
for(var i =0; i < fields.Length; i++)
{
modifiedFields[i] = new string(fields[i].ToCharArray().OrderBy(x =>x).ToArray());
}
var set = new HashSet<string>(modifiedFields);
var results = new List<string>();
foreach(var pair in dictionary)
{
string key = new string(pair.Value.ToCharArray().OrderBy(x =>x).ToArray());
if (set.Contains(key))
{
results.Add(pair.Key);
}
}

You can try this
var stringToLookUp = GetSomeStrings(s.ToString()).Select(x => x).OrderBy(x => x).ToArray();
var aDictionaryOfStringString = GetDictionary(Resources.stringList);
var results = aDictionaryOfStringString.Where(kvp => stringToLookUp.Select(s => s.OrderBy(x => x)).Contains(kvp.Value.OrderBy(x => x))).Select(kvp => kvp.Key).ToList();

Split string into custom List<T>

I need to split a custom string into the following format using C#.
The following string: AD=Demo,OU=WEB,OU=IT,L=MyCity,C=MyCountry, i want to split it at comma into a
List<CustomDictionary> myList = new
List<CustomDictionary>();
Based on the text above and after the split, the myList list should contain 5 objects of type CustomDictionary.
object1.Key = AD
object1.Value = Demo
object2.Key = OU
object2.Value = WEB
object3.Key = OU
object3.Value = IT
object4.Key = L
object4.Value = MyCity
object5.Key = C
object5.Value = MyCountry
Here is the CustomObject class
public class CustomDictionary
{
public string Key { get; set; }
public string Value { get; set; }
public CustomDictionary(string key, string value)
{
this.Key = key;
this.Value = value;
}
}
So far I tried this:
Here I am stuck!
List<CustomDictionary> keyVal = new List<CustomDictionary>val.Split(',').Select(x=>x.Split('=')).Select(x=>x.));
where val is the actual string ...

With linq:
var query = from x in str.Split(',')
let p = x.Split('=')
select new CustomDictionary(p[0], p[1]);
var list = query.ToList();
Also seems like you want to get a dictionary as a result. If so, try this code:
var dict = str.Split(',').Select(x => x.Split('='))
.ToDictionary(x => x[0], x => x[1]);
To handle duplicate keys, you can store objects in Lookup. Just call ToLookup instead of ToDictionaty.

After splitting the second time you create a CustomDictionary from the items in that array, then use ToList to make a list of the result.
List<CustomDictionary> keyVal =
val.Split(',')
.Select(x => x.Split('='))
.Select(a => new CustomDictionary(a[0], a[1]))
.ToList();
There is already a class in the framework having a key and value, which you can use instead:
List<KeyValuePair<string, string>> keyVal =
val.Split(',')
.Select(x => x.Split('='))
.Select(a => new KeyValuePair<string, string>(a[0], a[1]))
.ToList();
You can also use a Dictionary<string, string> instead of a list of key-value pairs. It stores the value based on the hash code of the key, so getting a value by key is much faster than looking through a list (but it doesn't retain the order of the items):
Dictionary<string, string> keyVal =
val.Split(',')
.Select(x => x.Split('='))
.ToDictionary(a => a[0], a => a[1]);

This is how you would do it:
var parts = theString.Split(',');
var myList = new List<CustomDictionary>();
foreach(string part in parts)
{
var kvp = part.Split('=');
myList.Add(new CustomDictionary(kvp[0], kvp[1]));
}
This can also be done using LINQ.

Since you have 2 OUs you can't use Dictionary. Instead use Lookup
string input = "AD=Demo,OU=WEB,OU=IT,L=MyCity,C=MyCountry";
var dict = Regex.Matches(input, #"(\w+)=([^,$]+)").Cast<Match>()
.ToLookup(m => m.Groups[1].Value, m => m.Groups[2].Value);

what about MyString.split(','); and the on each string you get:
CO.key = SubString.split('=')[0];
CO.value = SubString.split('=')[1];

With LINQ:
List<CustomDictionary> myList = (from x in input.Split(new char[] { ',' })
select
new CustomDictionary (x.Substring(0, x.IndexOf('=')), x.Substring(x.IndexOf('=') + 1))
).ToList();

string str = "AD=Demo,OU=WEB,OU=IT,L=MyCity,C=MyCountry";
var result = str.Split(',').Select(s =>
{
var tmp = s.Split('=');
return new CustomDictionary(tmp[0], tmp[1]);
}).ToList();

Sorting a string based on prefixes

If you are given an array with random prefixes, like this:
DOG_BOB
CAT_ROB
DOG_DANNY
MOUSE_MICKEY
DOG_STEVE
HORSE_NEIGH
CAT_RUDE
HORSE_BOO
MOUSE_STUPID
How would i go about sorting this so that i have 4 different arrays/lists of strings?
So the end result would give me 4 string ARRAYS or lists with
DOG_BOB,DOG_DANNY,DOG_STEVE <-- Array 1
HORSE_NEIGH, HORSE_BOO <-- Array 2
MOUSE_MICKEY, MOUSE_STUPID <-- Array 3
CAT_RUDE, CAT_ROB <-- Array 4
sorry about the names i just made them up lol
var fieldNames = typeof(animals).GetFields()
.Select(field => field.Name)
.ToList();
List<string> cats = new List<string>();
List<string> dogs = new List<string>();
List<string> mice= new List<string>();
List<string> horse = new List<string>();
foreach (var n in fieldNames)
{
var fieldValues = typeof(animals).GetField(n).GetValue(n);"
//Here's what i'm trying to do, with if statements
if (n.ToString().ToLower().Contains("horse"))
{
}
}
So i need them to be splitted into STRING ARRAYS/STRING LISTS and NOT just strings

string[] strings = new string[] {
"DOG_BOB",
"CAT_ROB",
"DOG_DANNY",
"MOUSE_MICKEY",
"DOG_STEVE",
"HORSE_NEIGH",
"CAT_RUDE",
"HORSE_BOO",
"MOUSE_STUPID"};
string[] results = strings.GroupBy(s => s.Split('_')[0])
.Select(g => String.Join(",",g))
.ToArray();
Or maybe something like this
List<List<string>> res = strings.ToLookup(s => s.Split('_')[0], s => s)
.Select(g => g.ToList())
.ToList();

var groups = fieldNames.GroupBy(n => n.Split('_')[0]);
Usage
foreach(var group in groups)
{
// group.Key (DOG, HORSE, CAT, etc)
foreach(var name in group)
// all names groped by prefix
}

foreach (String s in strings)
{
if (s.StartsWith("CAT_")
cats.Add(s);
else if (s.StartsWith("HORSE_")
horses.Add(s);
// ...
}
Or:
foreach (String s in strings)
{
String[] split = s.Split(new Char [] { '_' });
if (split[0].Equals("CAT")
cats.Add(s);
else if (split[0].Equals("HORSE")
horses.Add(s);
// ...
}
But I would prefer the first one.

Algorithmically, I'd do the following:
Parse out all unique prefixes by using the "_" as your delimeter.
Loop through your list of prefixes.
2a. Retrieve any values that have your prefix (loop/find/regex/depends on structure)
2b. Place retrieved values in a List.
2c. Sort list.
Output your results, or do what you need with your collections.

You can order the list up front and sort by prefix:
string[] input = new string[] {"DOG_BOB","CAT_ROB","DOG_DANNY","MOUSE_MICKEY","DOG_STEVE","HORSE_NEIGH","CAT_RUDE","HORSE_BOO","MOUSE_STUPID"};
string[] sortedInput = input.OrderBy(x => x).ToArray();
var distinctSortedPrefixes = sortedInput.Select(item => item.Split('_')[0]).Distinct().ToArray();
Dictionary<string, string[]> orderedByPrefix = new Dictionary<string, string[]>();
for (int prefixIndex = 0; prefixIndex < distinctSortedPrefixes.Length; prefixIndex++)
{
string prefix = distinctSortedPrefixes[prefixIndex];
var group = input.Where(item => item.StartsWith(prefix)).ToArray();
orderedByPrefix.Add(prefix, group);
}

With LINQ, using something like
names.GroupBy(s => s.Substring(0, s.IndexOf("_"))) // group by prefix
.Select(g => string.Join(",", g)) // join each group with commas
.ToList(); // take the results
See it in action (some extra .ToArray() calls included for .NET 3.0 compatibility)

This LINQ expression does what you want.
var result = data.GroupBy(data.Split('_')[0])
.Select(group => String.Join(", ", group))
.ToList();
For a list of lists of strings use this expression.
var result = data.GroupBy(data.Split('_')[0])
.Select(group => group.ToList())
.ToList();

How to find the number of each elements in the row and store the mean of each row in another array using C#?

I am using the below code to read data from a text file row by row. I would like to assign each row into an array. I must be able to find the number or rows/arrays and the number of elements on each one of them.
I would also like to do some manipulations on some or all rows and return their values.
I get the number of rows, but is there a way to to loop something like:
*for ( i=1 to number of rows)
do
mean[i]<-row[i]
done
return mean*
var data = System.IO.File.ReadAllText("Data.txt");
var arrays = new List<float[]>();
var lines = data.Split(new[] {'\r', '\n'}, StringSplitOptions.RemoveEmptyEntries);
foreach (var line in lines)
{
var lineArray = new List<float>();
foreach (var s in line.Split(new[] {','}, StringSplitOptions.RemoveEmptyEntries))
{
lineArray.Add(Convert.ToSingle(s));
}
arrays.Add(lineArray.ToArray());
}
var numberOfRows = lines.Count();
var numberOfValues = arrays.Sum(s => s.Length);

var arrays = new List<float[]>();
//....your filling the arrays
var averages = arrays.Select(floats => floats.Average()).ToArray(); //float[]
var counts = arrays.Select(floats => floats.Count()).ToArray(); //int[]

Not sure I understood the question. Do you mean something like
foreach (string line in File.ReadAllLines("fileName.txt")
{
...
}

Is it ok for you to use Linq? You might need to add using System.Linq; at the top.
float floatTester = 0;
List<float[]> result = File.ReadLines(#"Data.txt")
.Where(l => !string.IsNullOrWhiteSpace(l))
.Select(l => new {Line = l, Fields = l.Split(new[] { ',' }, StringSplitOptions.RemoveEmptyEntries) })
.Select(x => x.Fields
.Where(f => Single.TryParse(f, out floatTester))
.Select(f => floatTester).ToArray())
.ToList();
// now get your totals
int numberOfLinesWithData = result.Count;
int numberOfAllFloats = result.Sum(fa => fa.Length);
Explanation:
File.ReadLines reads the lines of a file (not all at once but straming)
Where returns only elements for which the given predicate is true(f.e. the line must contain more than empty text)
new { creates an anonymous type with the given properties(f.e. the fields separated by comma)
Then i try to parse each field to float
All that can be parsed will be added to an float[] with ToArray()
All together will be added to a List<float[]> with ToList()

Found an efficient way to do this. Thanks for your input everybody!
private void ReadFile()
{
var lines = File.ReadLines("Data.csv");
var numbers = new List<List<double>>();
var separators = new[] { ',', ' ' };
/*System.Threading.Tasks.*/
Parallel.ForEach(lines, line =>
{
var list = new List<double>();
foreach (var s in line.Split(separators, StringSplitOptions.RemoveEmptyEntries))
{
double i;
if (double.TryParse(s, out i))
{
list.Add(i);
}
}
lock (numbers)
{
numbers.Add(list);
}
});
var rowTotal = new double[numbers.Count];
var rowMean = new double[numbers.Count];
var totalInRow = new int[numbers.Count()];
for (var row = 0; row < numbers.Count; row++)
{
var values = numbers[row].ToArray();
rowTotal[row] = values.Sum();
rowMean[row] = rowTotal[row] / values.Length;
totalInRow[row] += values.Length;
}

Convert a IList<int> collection to a comma separated list

Any elegant ways of converting a IList collection to a string of comma separated id's?
"1,234,2,324,324,2"

IList<int> list = new List<int>( new int[] { 1, 2, 3 } );
Console.WriteLine(string.Join(",", list));

You can do:
// Given: IList<int> collection;
string commaSeparatedInts = string.Join(",",collection.Select(i => i.ToString()).ToArray());

// list = IList<MyObject>
var strBuilder = new System.Text.StringBuilder();
foreach(var obj in list)
{
strBuilder.Append(obj.ToString());
strBuilder.Append(",");
}
strBuilder = strBuilder.SubString(0, strBuilder.Length -1);
return strBuilder.ToString();

This will do it
IList<int> strings = new List<int>(new int[] { 1,2,3,4 });
string[] myStrings = strings.Select(s => s.ToString()).ToArray();
string joined = string.Join(",", myStrings);
OR entirely with Linq
string aggr = strings.Select(s=> s.ToString()).Aggregate((agg, item) => agg + "," + item);

List<int> intList = new List<int>{1,234,2,324,324,2};
var str = intList.Select(i => i.ToString()).Aggregate( (i1,i2) => string.Format("{0},{1}",i1,i2));
Console.WriteLine(str);

mstrickland has a good idea on using string builder because of its speed with larger lists. However, you can't set a stringbuilder as a string. Try this instead.
var strBuilder = new StringBuilder();
foreach (var obj in list)
{
strBuilder.Append(obj.ToString());
strBuilder.Append(",");
}
return strBuilder.ToString(0, strBuilder.Length - 1);