Develop Reference

Creating and Displaying multiple arrays simultaniously C#

I'm new to C# and programming as a whole and I've been unable to come up with a solution to what I want to do. I want to be able to create a way to display several arrays containing elements from three external text files with values on each line (e.g. #"Files\Column1.txt", #"Files\Column2.txt" #"Files\Column3.txt"). They then need to be displayed like this in the command line:
https://www.dropbox.com/s/0telh1ils201wpy/Untitled.png?dl=0
I also need to be able to sort each column individually (e.g. column 3 from lowest to highest).
I've probably explained this horribly but I'm not sure how else to put it! Any possible solutions will be greatly appreciated!

One way to do it would be to store the corresponding items from each file in a Tuple, and then store those in a List. This way the items will all stay together, but you can sort your list on any of the Tuple fields. If you were doing anything more detailed with these items, I would suggest creating a simple class to store them, so the code would be more maintainable.
Something like:
public class Item
{
public DayOfWeek Day { get; set; }
public DateTime Date { get; set; }
public string Value { get; set; }
}
The example below could easily be converted to use such a class, but for now it uses a Tuple<string, string, string>. As an intermediate step, you could easily convert the items as you create the Tuple to get more strongly-typed versions, for example, you could have Tuple<DayOfWeek, DateTime, string>.
Here's the sample code for reading your file items into a list, and how to sort on each item type:
public static void Main()
{
// For testing sake, I created some dummy files
var file1 = #"D:\Public\Temp\File1.txt";
var file2 = #"D:\Public\Temp\File2.txt";
var file3 = #"D:\Public\Temp\File3.txt";
// Validation that files exist and have same number
// of items is intentionally left out for the example
// Read the contents of each file into a separate variable
var days = File.ReadAllLines(file1);
var dates = File.ReadAllLines(file2);
var values = File.ReadAllLines(file3);
var itemCount = days.Length;
// The list of items read from each file
var fileItems = new List<Tuple<string, string, string>>();
// Add a new item for each line in each file
for (int i = 0; i < itemCount; i++)
{
fileItems.Add(new Tuple<string, string, string>(
days[i], dates[i], values[i]));
}
// Display the items in console window
fileItems.ForEach(item =>
Console.WriteLine("{0} {1} = {2}",
item.Item1, item.Item2, item.Item3));
// Example for how to order the items:
// By days
fileItems = fileItems.OrderBy(item => item.Item1).ToList();
// By dates
fileItems = fileItems.OrderBy(item => item.Item2).ToList();
// By values
fileItems = fileItems.OrderBy(item => item.Item3).ToList();
// Order by descending
fileItems = fileItems.OrderByDescending(item => item.Item1).ToList();
// Show the values based on the last ordering
fileItems.ForEach(item =>
Console.WriteLine("{0} {1} = {2}",
item.Item1, item.Item2, item.Item3));
}

c# List Query without using Linq

I'm after some help with how to query a list and return back the index, but not using Linq. I've seen many example where Linq is used, but the software I'm writing the query into doesn't support Linq. :(
So example to get us going:
List<string> location = new List<string>();
location.add(#"C:\test\numbers\FileName_IgnoreThis_1.jpg");
location.add(#"C:\test\numbers\FileName_IgnoreThis_2.jpg");
location.add(#"C:\test\numbers\FileName_ImAfterThis_3.jpg");
location.add(#"C:\test\numbers\FileName_IgnoreThis_4.jpg");
location.add(#"C:\test\numbers\FileName_ImAfterThis_5.jpg");
So this list will be populated with probably a few hundred records, what I need to do is query the list for the text "ImAfterThis" then return the index number location for this item into a string array but without using Linq.
The desired result would be 2 and 4 being added to the string array.
I was thinking of doing a for loop through the list, but is there a better way to achieve this?
List<int> results = new List<int>();
int i = 0;
foreach (string value in location)
{
if (value.Contains("IAfterThis"))
{
results.Add(i);
Console.WriteLine("Found in Index: " + i);
}
i++;
}
Console.ReadLine();
Thanks in advance.

If you want to get just the first occurrence you could simply use the IndexOf() method.
If you want all occurrences of string "whatever" then a for loop would certainly work for you. For the sake of argument here I've capture the indexes in another list:
string MyString = "whatever";
List<int> indexes = new List();
for (int i = 0; i < location.Count; i++)
{
if (location[i] == MyString)
{
indexes.Add(i);
}
}

Fast lookup in a multidimensional System.Collection.IList

I need your advice on the following.
I have a multi-dimensional IList containing items which have an index, Id and Text. Normally I know the value of Id and based on that I need to get the Text. Both Id and Text values are read from a database.
What we are currently using to get the value of Text field is:
foreach (Object myObj in List)
{
if (((MessageType)myObj).Id == id)
{
return ((MessageType)myObj).Text;
}
}
When count in IList becomes large (more than 32K), it takes some time to process.
Question: Is there a way to efficiently get the Text value without iterating through the IList?
Things I tried without success:
Use List.IndexOf(Id) - did not work because IndexOf applies to text only.
Converting List to multi-dimensional array - failed on List.CopyTo(array,0) my guess because it is multi-dimensional:
string[] array=new string[List.Count,List.Count];
List.CopyTo(array,0);
I can not use a AJAX/JQuery solution because it is an existing(live) project and it will take too much to re-code.
Thanks

If you want fast searching by some identifier in a collection with 32k elements, you should use Dictionary<K,V> as your collection.
var dict = new Dictionary<IDType, MessageType>();
A Dictionary is basically a search tree where the elements are stored in a sorted way so an element with a specific key (in your case Id) can be found without looking at all elements. For more information see MSDN.
If you cannot refactor the collection to be a dictionary, you may initially fill the dictionary (slow) and then search in the dictionary (fast). This will only be faster if you do multiple searches before you fill the dictionary again, i.e. if your list does not change often.
foreach(object o in List)
{
var msg = (MessageType)o;
dict.Add(msg.Id, msg);
}
Searching then is easy:
MessageType msg = dict[id];
EDIT: Well, I was curious and wrote a test routine which compares the linear search and the dictionary approach. Here's what I used:
using System;
using System.Collections;
using System.Collections.Generic;
using System.Diagnostics;
namespace ConsoleApplication1
{
class MessageType
{
public string Id;
public string Text;
}
class Program
{
static void Main(string[] args)
{
var rand = new Random ();
// filling a list with random text messages
List<MessageType> list = new List<MessageType>();
for (int i = 0; i < 32000; i++)
{
string txt = rand.NextDouble().ToString();
var msg = new MessageType() {Id = i.ToString(), Text = txt };
list.Add(msg);
}
IList List = (IList)list;
// doing some random searches
foreach (int some in new int[] { 2, 10, 100, 1000 })
{
var watch1 = new Stopwatch();
var watch2 = new Stopwatch();
Dictionary<string, MessageType> dict = null;
for (int i = 0; i < some; i++)
{
string id = rand.Next(32000).ToString();
watch1.Start();
LinearLookup(List, id);
watch1.Stop();
watch2.Start();
// fill once
if (dict == null)
{
dict = new Dictionary<string, MessageType>();
foreach (object o in List)
{
var msg = (MessageType)o;
dict.Add(msg.Id, msg);
}
}
// lookup
DictionaryLookup(dict, id);
watch2.Stop();
}
Console.WriteLine(some + " x LinearLookup took "
+ watch1.Elapsed.TotalSeconds + "s");
Console.WriteLine("Dictionary fill and " + some
+ " x DictionaryLookup took "
+ watch2.Elapsed.TotalSeconds + "s");
}
}
static string LinearLookup(IList List, string id)
{
foreach (object myObj in List)
{
if (((MessageType)myObj).Id == id)
{
return ((MessageType)myObj).Text;
}
}
throw new Exception();
}
static string DictionaryLookup(Dictionary<string, MessageType> dict,
string id)
{
return dict[id].Text;
}
}
}
The results I got in Release / x86:
Number of | Time [ms] with | Time[ms] with | Speedup (approx.)
searches | linear search | dictionary(*) | with dictionary
----------+----------------+---------------+-----------------
2 | 1.161 | 2.006 | 0.6
----------+----------------+---------------+-----------------
10 | 2.834 | 2.060 | 1.4
----------+----------------+---------------+-----------------
100 | 25.39 | 1.973 | 13
----------+----------------+---------------+-----------------
1000 | 261.4 | 5.836 | 45
----------+----------------+---------------+-----------------
(*) including filling the dictionary once.
So, I was a bit optimistic to say that searching twice would already pay off. In my test application I have to search 10 times for the dictionary to be faster.
I'm sorry I could not make a more realistic example, my Ids are all sorted. Feel free to try modifying and experimenting though ;-)

From the looks of it you have a List<MessageType> here, which is not multi-dimensional. Rather the objects inside the list have multiple properties.
You could easily get them out with LINQ much faster than a loop most likely:
var text = (from MessageType msgType in myList
where msgType.Id == id
select msgType.Text).FirstOrDefault();
Or even easier with an inline LINQ statement:
var text = myList.Where(s => s.Id == id).Select(s => s.Text).FirstOrDefault();
NOTE: As mentioned in comments above, the speed of these LINQ statements are only as good as the object's position in the List. If it is the last object in the list, you will likely see the same performance discrepancy. Dictionary<Index, MessageType> is going to be much more performant.

Better way is to use ILookup.
For example:
var look = query.ToLookup(x => x.SomeID, y=> y.Name)
and use:
if (look.Contains(myID)){
var name = look[myID].First();
}

C# dedupe List based on split

I'm having a hard time deduping a list based on a specific delimiter.
For example I have 4 strings like below:
apple|pear|fruit|basket
orange|mango|fruit|turtle
purple|red|black|green
hero|thor|ironman|hulk
In this example I should want my list to only have unique values in column 3, so it would result in an List that looks like this,
apple|pear|fruit|basket
purple|red|black|green
hero|thor|ironman|hulk
In the above example I would have gotten rid of line 2 because line 1 had the same result in column 3. Any help would be awesome, deduping is tough in C#.
how i'm testing this:
static void Main(string[] args)
{
BeginListSet = new List<string>();
startHashSet();
}
public static List<string> BeginListSet { get; set; }
public static void startHashSet()
{
string[] BeginFileLine = File.ReadAllLines(#"C:\testit.txt");
foreach (string begLine in BeginFileLine)
{
BeginListSet.Add(begLine);
}
}
public static IEnumerable<string> Dedupe(IEnumerable<string> list, char seperator, int keyIndex)
{
var hashset = new HashSet<string>();
foreach (string item in list)
{
var array = item.Split(seperator);
if (hashset.Add(array[keyIndex]))
yield return item;
}
}

Something like this should work for you
static IEnumerable<string> Dedupe(this IEnumerable<string> input, char seperator, int keyIndex)
{
var hashset = new HashSet<string>();
foreach (string item in input)
{
var array = item.Split(seperator);
if (hashset.Add(array[keyIndex]))
yield return item;
}
}
...
var list = new string[]
{
"apple|pear|fruit|basket",
"orange|mango|fruit|turtle",
"purple|red|black|green",
"hero|thor|ironman|hulk"
};
foreach (string item in list.Dedupe('|', 2))
Console.WriteLine(item);
Edit: In the linked question Distinct() with Lambda, Jon Skeet presents the idea in a much better fashion, in the form of a DistinctBy custom method. While similar, his is far more reusable than the idea presented here.
Using his method, you could write
var deduped = list.DistinctBy(item => item.Split('|')[2]);
And you could later reuse the same method to "dedupe" another list of objects of a different type by a key of possibly yet another type.

Try this:
var list = new string[]
{
"apple|pear|fruit|basket",
"orange|mango|fruit|turtle",
"purple|red|black|green",
"hero|thor|ironman|hulk "
};
var dedup = new List<string>();
var filtered = new List<string>();
foreach (var s in list)
{
var filter = s.Split('|')[2];
if (dedup.Contains(filter)) continue;
filtered.Add(s);
dedup.Add(filter);
}
// Console.WriteLine(filtered);

Can you use a HashSet instead? That will eliminate dupes automatically for you as they are added.

May be you can sort the words with delimited | on alphabetical order. Then store them onto grid (columns). Then when you try to insert, just check if there is column having a word which starting with this char.

If LINQ is an option, you can do something like this:
// assume strings is a collection of strings
List<string> list = strings.Select(a => a.Split('|')) // split each line by '|'
.GroupBy(a => a[2]) // group by third column
.Select(a => a.First()) // select first line from each group
.Select(a => string.Join("|", a))
.ToList(); // convert to list of strings
Edit (per Jeff Mercado's comment), this can be simplified further:
List<string> list =
strings.GroupBy(a => a.split('|')[2]) // group by third column
.Select(a => a.First()) // select first line from each group
.ToList(); // convert to list of strings

Compare adjacent list items

I'm writing a duplicate file detector. To determine if two files are duplicates I calculate a CRC32 checksum. Since this can be an expensive operation, I only want to calculate checksums for files that have another file with matching size. I have sorted my list of files by size, and am looping through to compare each element to the ones above and below it. Unfortunately, there is an issue at the beginning and end since there will be no previous or next file, respectively. I can fix this using if statements, but it feels clunky. Here is my code:
public void GetCRCs(List<DupInfo> dupInfos)
{
var crc = new Crc32();
for (int i = 0; i < dupInfos.Count(); i++)
{
if (dupInfos[i].Size == dupInfos[i - 1].Size || dupInfos[i].Size == dupInfos[i + 1].Size)
{
dupInfos[i].CheckSum = crc.ComputeChecksum(File.ReadAllBytes(dupInfos[i].FullName));
}
}
}
My question is:
How can I compare each entry to its neighbors without the out of bounds error?
Should I be using a loop for this, or is there a better LINQ or other function?
Note: I did not include the rest of my code to avoid clutter. If you want to see it, I can include it.

Compute the Crcs first:
// It is assumed that DupInfo.CheckSum is nullable
public void GetCRCs(List<DupInfo> dupInfos)
{
dupInfos[0].CheckSum = null ;
for (int i = 1; i < dupInfos.Count(); i++)
{
dupInfos[i].CheckSum = null ;
if (dupInfos[i].Size == dupInfos[i - 1].Size)
{
if (dupInfos[i-1].Checksum==null) dupInfos[i-1].CheckSum = crc.ComputeChecksum(File.ReadAllBytes(dupInfos[i-1].FullName));
dupInfos[i].CheckSum = crc.ComputeChecksum(File.ReadAllBytes(dupInfos[i].FullName));
}
}
}
After having sorted your files by size and crc, identify duplicates:
public void GetDuplicates(List<DupInfo> dupInfos)
{
for (int i = dupInfos.Count();i>0 i++)
{ // loop is inverted to allow list items deletion
if (dupInfos[i].Size == dupInfos[i - 1].Size &&
dupInfos[i].CheckSum != null &&
dupInfos[i].CheckSum == dupInfos[i - 1].Checksum)
{ // i is duplicated with i-1
... // your code here
... // eventually, dupInfos.RemoveAt(i) ;
}
}
}

I have sorted my list of files by size, and am looping through to
compare each element to the ones above and below it.
The next logical step is to actually group your files by size. Comparing consecutive files will not always be sufficient if you have more than two files of the same size. Instead, you will need to compare every file to every other same-sized file.
I suggest taking this approach
Use LINQ's .GroupBy to create a collection of files sizes. Then .Where to only keep the groups with more than one file.
Within those groups, calculate the CRC32 checksum and add it to a collection of known checksums. Compare with previously calculated checksums. If you need to know which files specifically are duplicates you could use a dictionary keyed by this checksum (you can achieve this with another GroupBy. Otherwise a simple list will suffice to detect any duplicates.
The code might look something like this:
var filesSetsWithPossibleDupes = files.GroupBy(f => f.Length)
.Where(group => group.Count() > 1);
foreach (var grp in filesSetsWithPossibleDupes)
{
var checksums = new List<CRC32CheckSum>(); //or whatever type
foreach (var file in grp)
{
var currentCheckSum = crc.ComputeChecksum(file);
if (checksums.Contains(currentCheckSum))
{
//Found a duplicate
}
else
{
checksums.Add(currentCheckSum);
}
}
}
Or if you need the specific objects that could be duplicates, the inner foreach loop might look like
var filesSetsWithPossibleDupes = files.GroupBy(f => f.FileSize)
.Where(grp => grp.Count() > 1);
var masterDuplicateDict = new Dictionary<DupStats, IEnumerable<DupInfo>>();
//A dictionary keyed by the basic duplicate stats
//, and whose value is a collection of the possible duplicates
foreach (var grp in filesSetsWithPossibleDupes)
{
var likelyDuplicates = grp.GroupBy(dup => dup.Checksum)
.Where(g => g.Count() > 1);
//Same GroupBy logic, but applied to the checksum (instead of file size)
foreach(var dupGrp in likelyDuplicates)
{
//Create the key for the dictionary (your code is likely different)
var sample = dupGrp.First();
var key = new DupStats() {FileSize = sample.FileSize, Checksum = sample.Checksum};
masterDuplicateDict.Add(key, dupGrp);
}
}
A demo of this idea.

I think the for loop should be : for (int i = 1; i < dupInfos.Count()-1; i++)
var grps= dupInfos.GroupBy(d=>d.Size);
grps.Where(g=>g.Count>1).ToList().ForEach(g=>
{
...
});

Can you do a union between your two lists? If you have a list of filenames and do a union it should result in only a list of the overlapping files. I can write out an example if you want but this link should give you the general idea.
https://stackoverflow.com/a/13505715/1856992
Edit: Sorry for some reason I thought you were comparing file name not size.
So here is an actual answer for you.
using System;
using System.Collections.Generic;
using System.Linq;
public class ObjectWithSize
{
public int Size {get; set;}
public ObjectWithSize(int size)
{
Size = size;
}
}
public class Program
{
public static void Main()
{
Console.WriteLine("start");
var list = new List<ObjectWithSize>();
list.Add(new ObjectWithSize(12));
list.Add(new ObjectWithSize(13));
list.Add(new ObjectWithSize(14));
list.Add(new ObjectWithSize(14));
list.Add(new ObjectWithSize(18));
list.Add(new ObjectWithSize(15));
list.Add(new ObjectWithSize(15));
var duplicates = list.GroupBy(x=>x.Size)
.Where(g=>g.Count()>1);
foreach (var dup in duplicates)
foreach (var objWithSize in dup)
Console.WriteLine(objWithSize.Size);
}
}
This will print out
14
14
15
15
Here is a netFiddle for that.
https://dotnetfiddle.net/0ub6Bs
Final note. I actually think your answer looks better and will run faster. This was just an implementation in Linq.