Develop Reference

use linq to aggregate text file contents so that they are grouped

Given the following query.
var query = files
.SelectMany(file => File.ReadAllLines(file))
.Where(_ => !_.StartsWith("*"))
.Select(line => new {
Order = line.Substring(32, 7),
Delta = line.Substring(40, 3),
Line = new String[] { line }
});
This clearly produces a list of objects with the properties Order: string, Delta: string and Line: string[]
I have a list of items that looks like this.
{ 1, 'A', {'line1'} },
{ 1, 'A', {'line2'} },
{ 2, 'B', {'line3'} },
{ 1, 'B', {'line4 } }
is it possible to use the Linq Aggregate or similar functional construct to collect all the adjacent Order and Delta combinations together whilst accumulating the lines.
So that the aggregate is a list of items containing all it's 'lines'
{ 1, 'A', {'line1', 'line2'} }
{ 2, 'B', {'line3'} }
{ 1, 'B', {'line4'} }
Since aggregation iterates sequentially it should be possible to collect all the adjacent lines that have the same fields equal.
It's easy to do in a loop, but I am trying to do it with a set of lambdas.

You'll need the following variation of GroupBy:
public static class EnumerableExtensions
{
public class AdjacentGrouping<K, T> : List<T>, IGrouping<K, T>
{
public AdjacentGrouping(K key) { Key = key; }
public K Key { get; private set; }
}
public static IEnumerable<IGrouping<K, T>> GroupByAdjacent<T, K>(
this IEnumerable<T> sequence, Func<T, K> keySelector)
{
using (var it = sequence.GetEnumerator())
{
if (!it.MoveNext())
yield break;
T curr = it.Current;
K currKey = keySelector(curr);
var currentCluster = new AdjacentGrouping<K, T>(currKey) { curr };
while (it.MoveNext())
{
curr = it.Current;
currKey = keySelector(curr);
if (!EqualityComparer<K>.Default.Equals(currKey, currentCluster.Key))
{
// start a new cluster
yield return currentCluster;
currentCluster = new AdjacentGrouping<K, T>(currKey);
}
currentCluster.Add(curr);
};
// currentCluster is never empty
yield return currentCluster;
}
}
}
Having this adjacent grouping, your code can be the same as in Chris's answer:
var query = files
.SelectMany(file => File.ReadAllLines(file))
.Where(_ => !_.StartsWith("*"))
.Select(line => new
{
Order = line.Substring(32, 7),
Delta = line.Substring(40, 3),
Line = new String[] { line }
})
.GroupByAdjacent(o => new { o.Order, o.Delta })
.Select(g => new { g.Key.Order, g.Key.Delta, Lines = g.Select(o => o.Line).ToList() });
Disclaimer: the function GroupByAdjacent is from my own pet project and not copied from anywhere.

Note: Does not group items by adjacency
You can produce the desired results using a simple GroupBy combined with a SelectMany:
var query = new[] {
new { order = 1, delta = "A", line = new[] { "line1" } },
new { order = 1, delta = "A", line = new[] { "line2" } },
new { order = 2, delta = "B", line = new[] { "line3" } },
new { order = 1, delta = "B", line = new[] { "line4" } },
};
query
.GroupBy(q => new { q.order, q.delta })
.Select(q => new {
order = q.Key.order,
delta = q.Key.delta,
lines = q.SelectMany(l => l.line)
});
Produces:

Sort by numbers first

I have a list as under
var x = new List<string>() { "a", "1", "b", "2" };
var sortedResultByNumeric = x
.Select(s => new { OriginalString = s,
ExtractNumbers = String.Join("", s.Where(Char.IsDigit)) })
.OrderBy(o => o.ExtractNumbers).ToList();
The output is
a
b
1
2
But expected is
1
2
a
b
How can I do so?

The output that you see and expected is based on OriginalString because ExtractNumbers is "","","1","2" then you should OrderBy OriginalString:
var x = new List<string>() { "a", "1", "b", "2" };
var sortedResultByNumeric = x
.Select(s => new { OriginalString = s, ExtractNumbers = String.Join("", s.Where(Char.IsDigit)) })
.OrderBy(o => o.OriginalString).ToList();
The output:
1
2
a
b

You can try this:
var myList = new List<string>() { "a", "1", "b", "2", "123", "cd", "12346", "657" };
var nonNumericItems = myList.Where(item => !item.Any(i => Char.IsDigit(i)))
.OrderBy(item => item);
var numericItems = myList
.Select(item => String.Join("", item.Where(Char.IsDigit)))
.Where(item => !String.IsNullOrEmpty(item))
.Select(item => Convert.ToInt32(item))
.OrderBy(item => item)
.Select(item => item.ToString());
var result = numericItems
.Union(nonNumericItems);
result.ToList()
.ForEach(Console.WriteLine);
The output is:
1, 2, 123, 657, 12346, a, b, cd
P.S: You didn't tell us any additional explanation about your logic here String.Join("", item.Where(Char.IsDigit). So, I didn't ask any additional question about that.

If your list contains only 1-char strings you can order them by their char numerical values:
var x = new List<string>() { "a", "1", "b", "2" };
var sorted = x.OrderBy(c => Convert.ToChar(c))
.ToList();
foreach (var c in sorted)
Console.WriteLine(c);
// 1
// 2
// a
// b

This solution may seem complicated but it will work with any Net built in sort method.
using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.Data;
namespace ConsoleApplication63
{
class Program
{
const string FILENAME = #"c:\temp\test.csv";
static void Main(string[] args)
{
List<AlphaNumberic> aN = new List<AlphaNumberic>() { "a", "1", "b", "2" };
aN.Sort((x, y) => x.CompareTo(y));
}
}
public class AlphaNumberic
{
int results = 0;
public string alphaNumeric { get; set; }
public AlphaNumberic(string value)
{
this.alphaNumeric = value;
}
public static implicit operator string(AlphaNumberic d)
{
return d.alphaNumeric;
}
public static implicit operator AlphaNumberic(string d)
{
return new AlphaNumberic(d);
}
public int CompareTo(AlphaNumberic compareItem)
{
int thisNum;
int num;
if (int.TryParse(compareItem.alphaNumeric, out num))
{
if (int.TryParse(this.alphaNumeric, out thisNum))
{
//alphaNumeric and compareItem both integers;
results = thisNum.CompareTo(num);
}
else
{
//alphaNumeric not an integer and compareItem integers;
results = 1;
}
}
else
{
if (int.TryParse(this.alphaNumeric, out thisNum))
{
//alphaNumeric is an integer and compareItem not an integers;
results = -1;
}
else
{
//alphaNumeric not an integer and compareItem not an integers;
results = alphaNumeric.CompareTo(compareItem);
}
}
return results;
}
}
}

Determine List<string[]> Pivot

i have the following simplified input:
List<string[]> sList = new List<string[]>();
sList.Add(new string[]{ "Product", "ProductID"});
sList.Add(new string[] { "Fork", "1" });
sList.Add(new string[] { "Spoon", "2" });
sList.Add(new string[] { "Knife", "3" });
and i want the following output
ResultList[0] == { "Product" ,"Fork","Spoon","Knife"};
ResultList[1] == { "ProductID" ,"1","2","3"};
first i solved this with 2 loops, then i changed my approach to this linq (both working):
List<string[]> Result = sList.SelectMany(x => x)
.Select((x, index) => new KeyValuePair<int, string>(index % sList.First().Length, x))
.GroupBy(s => s.Key).Select(g => g.Select(x => x.Value)
.ToArray()).ToList();
it works but it seems to be a bit circuitous to me. isn't there a simpler approach (built in?) i've overlooked?

There's a better solution for this: use Dictionary<TKey, TValue>.
Dictionary<string, string> products = new Dictionary<string, string>()
{
{ "Fork", "1" },
{ "Spoon", "2" }
};
Now you can access products.Keys (which will get product names) and products.Values (which will get product identifiers):
ResultList[0] = new [] { "Product" }.Union(products.Values);
ResultList[1] = new [] { "ProductID" }.Union(products.Keys);
Possible automatization:
List<List<string>> resultList = new [] { "Product", "ProductID" }
.Select((g, index) => (index == 0 ? new [] { g }.Union(products.Keys) : new [] { g }.Union(products.Values)).ToList())
.ToList();
BTW, as long as you've stored your data in a dictionary, I don't find a reason to add "column names" to the result if you know that product names are products.Keys and identifiers products.Values. That is, you can work with the dictionary directly...

So if you know for sure that each array contains exactly 2 elements, problem becomes a combination of two trivial selecs. However if you only know that all arrays are of the same size, but do not want to rely on exact numbers, this approach should do it:
List<string[]> Result =
sList.First()
.Select((x, i) => i)
.Select(i => sList.Select(x => x[i]).ToArray())
.ToList();
We are basically using first item in sList to figure out how many members there are in the array, and then doing a select for each of them.

What about something like this:
List<string[]> sList = new List<string[]>();
sList.Add(new string[] { "Product", "ProductID" });
sList.Add(new string[] { "Fork", "1" });
sList.Add(new string[] { "Spoon", "2" });
sList.Add(new string[] { "Knife", "3" });
var res = new List<string[]> { sList.Select(a => a[0]).ToArray(), sList.Select(b => b[1]).ToArray() };

Linq: Sum() non-integer values

This is a continuation from my previos question:
Linq (GroupBy and Sum) over List<List<string>>
I have a query like so:
var content = new List<List<string>>
{
new List<string>{ "book", "code", "columnToSum" },
new List<string>{ "abc", "1", "10" },
new List<string>{ "abc", "1", "5" },
new List<string>{ "cde", "1", "6" },
};
var headers = content.First();
var result = content.Skip(1)
.GroupBy(s => new { Code = s[headers.IndexOf("code")], Book = s[headers.IndexOf("book")]})
.Select(g => new
{
Book = g.Key.Book,
Code = g.Key.Code,
Total = g.Select(s => int.Parse(s[headers.IndexOf("columnToSum")])).Sum()
});
This works fine but I'm just wondering how I can handle the case there the columnToSum is empty? So for example this gives me the error "Input string was not in a correct format" as the int.Parse fails
var content = new List<List<string>>
{
new List<string>{ "book", "code", "columnToSum" },
new List<string>{ "abc", "1", "10" },
new List<string>{ "abc", "1", "" },
new List<string>{ "cde", "1", "6" },
};
How can I handle this scenario gracefully?

Why don't you just add a zero onto the front of the string?
s => int.Parse("0" + s[headers.IndexOf("columnToSum")])
Of course, it's a big hack. But it will solve your problem quickly and (quite) readably if the only exceptional case you're really worried about is the empty string.
I wonder where you're getting these empty strings from. If it's something you have control over like a SQL query, why don't you just change your query to give "0" for no value? (As long as the empty column isn't used in a different sense somewhere else in your code.)

One option, use string.All(Char.IsDigit) as pre-check:
Total = g.Select(s => !string.IsNullOrEmpty(s[headers.IndexOf("columnToSum")]) &&
s[headers.IndexOf("columnToSum")].All(Char.IsDigit) ?
int.Parse(s[headers.IndexOf("columnToSum")]) : 0).Sum())
another would be to use int.TryParse:
int val = 0;
// ...
Total = g.Select(s => int.TryParse(s[headers.IndexOf("columnToSum")], out val) ?
int.Parse(s[headers.IndexOf("columnToSum")]) : 0).Sum())

That code assumes that empty string is 0:
Total = g.Where(s => !String.IsNullOrEmpty(s)).Select(s => int.Parse(s[headers.IndexOf("columnToSum")])).Sum()

Unfortunately, this isn't going to look very nice...
g.Select(s => !s[headers.IndexOf("columnToSum")].Any(Char.IsDigit) ?
0 : Int32.Parse(s[headers.IndexOf("columnToSum")])).Sum()
However, you could wrap this up in a nice extension method
public static class StrExt
{
public static int IntOrDefault(this string str, int defaultValue = 0)
{
return String.IsNullOrEmpty(str) || !str.Any(Char.IsDigit) ? defaultValue : Int32.Parse(str);
}
}
...
g.Select(s => s[headers.IndexOf("columnToSum")].IntOrDefault()).Sum();
The extension method give you the flexibility to set whatever default value you want if the str is not a number - it defaults to 0 if the parameter is ommitted.

Using lists here is problematic, and I would parse this into a proper data structure (like a Book class), which I think will clean up the code a bit. If you're parsing CSV files, take a look at FileHelpers, it's great library for these types of tasks, and it can parse into a data structure for you.
That being said, if you'd still like to continue using this paradigm, I think you can get the code fairly clean by creating two custom methods: one for dealing with the headers (one of the few places I'd use dynamic types to get rid of ugly strings in your code) and one for parsing the ints. You then get something like this:
var headers = GetHeaders(content.First());
var result = from entry in content.Skip(1)
group entry by new {Code = entry[headers.code], Book = entry[headers.book] } into grp
select new {
Book = grp.Key.Book,
Code = grp.Key.Code,
Total = grp.Sum(x => ParseInt(x[headers.columnToSum]))
};
public dynamic GetHeaders(List<string> headersList){
IDictionary<string, object> headers = new ExpandoObject();
for (int i = 0; i < headersList.Count; i++)
headers[headersList[i]] = i;
return headers;
}
public int ParseInt(string s){
int i;
if (int.TryParse(s, out i))
return i;
return 0;
}

You can use multiple lines in a lambda expression and return a value at end.
So, instead of
Total = g.Select(s => int.Parse(s[headers.IndexOf("columnToSum")])).Sum()
I would write
Total = g.Select(s => {
int tempInt = 0;
int.TryParse(s[headers.IndexOf("columnToSum")], out tempInt);
return tempInt;
}).Sum()

t = new List<List<string>>
{
new List<string>{ "book", "code", "columnToSum" },
new List<string>{ "abc", "1", "10" },
new List<string>{ "abc", "1", "5" },
new List<string>{ "cde", "1", "6" },
};
var headers = content.First();
var result = content.Skip(1)
.GroupBy(s => new { Code = s[headers.IndexOf("code")], Book = s[headers.IndexOf("book")]})
.Select(g => new
{
Book = g.Key.Book,
Code = g.Key.Code,
Total = g.Select(s => int.Parse(s[headers.IndexOf("columnToSum")]!=""?s[headers.IndexOf("columnToSum")]:0)).Sum()
});

How to filter a jagged array by using the where function

I have a jagged array that looks like this:
string[][] list = new string[d.Rows.Count + 1][];
int c = 0;
while (c < d.Rows.Count)
{
list[c] = new string[]
{
d.Rows[c].ItemArray[2].ToString(),
d.Rows[c].ItemArray[1].ToString(),
d.Rows[c].ItemArray[4].ToString(),
d.Rows[c].ItemArray[5].ToString(),
d.Rows[c].ItemArray[7].ToString(),
d.Rows[c].ItemArray[3].ToString(),
d.Rows[c].ItemArray[14].ToString()
};
c += 1;
}
return list;
Now, for a new requirement, i need only the items from this array whose value at this location: list[x][0] are equal to any of the following strings: "Text", "FullText", "FullMatch"
I got started with a regular array i could do this: but it obvioulsy won't work for a jagged array.
string[][] newlist = list.where(item => item.equals("Text");
Does any one know how to extend this for my situation?

You can do a where on list which will iterate over each one-dimensional array, then compare element 0 to the strings given.
string[][] newlist = list
.Where(item => item[0].Equals("Text")
|| item[0].Equals("FullText")
|| item[0].Equals("FullMatch"))
.ToArray();
Tested this on some sample data as shown below:
var list = new string[][]
{
new string[] { "Text", "A", "B", "C", "D" },
new string[] { "None", "Z", "C" },
new string[] { "FullText", "1", "2", "3" },
new string[] { "FullMatch", "0", "A", "C", "Z" },
new string[] { "Ooops", "Nothing", "Here" },
};
string[][] newlist = list.Where(item => item[0].Equals("Text")
|| item[0].Equals("FullText")
|| item[0].Equals("FullMatch")).ToArray();
// now display all data...
foreach (string[] row in newlist)
{
Console.Write("Row: ");
foreach (string item in row)
{
Console.Write(item + " ");
}
Console.WriteLine();
}
This worked correctly with output being:
Row: Text A B C D
Row: FullText 1 2 3
Row: FullMatch 0 A C Z

If you wanted a fully LINQ-based solution, then I think the following should do the trick (although I haven't tested it, because I'm not usre what the variable d refers to):
var res =
(from c in Enumerable.Range(0, d.Rows.Count)
let list = new string[] {
d.Rows[c].ItemArray[2].ToString(),
d.Rows[c].ItemArray[1].ToString(),
d.Rows[c].ItemArray[4].ToString(),
d.Rows[c].ItemArray[5].ToString(),
d.Rows[c].ItemArray[7].ToString(),
d.Rows[c].ItemArray[3].ToString(),
d.Rows[c].ItemArray[14].ToString()
}
where list[0] == "Text" || list[0] == "FullText" || list[0] == "FullMatch"
select list).ToArray();
A jagged array is just an array of arrays, so you can process it using LINQ. The only trick is that individual items will be arrays (representing your columns). After using Where, you can turn the result back to an array using ToArray:
string[][] newlist = list.Where(item => item[0] == "Text" || ... ).ToArray();