Develop Reference

Reduce duplication - Create delegate from an instance method

I am working with an existing, slightly complex relationship between objects. After some refactoring for the sake of understandability, my code now looks like this:
TemplateColorList colors = new TemplateColorList();
var template = new Template {
TextElements = CreateTextElements(textElementData,
/* -> */ (textElement, colorParam1, colorParam2) => colors.AddTextColor(textElement, colorParam1, colorParam2)
),
ClipArtElements = CreateClipArtElements(clipArtElementData,
/* -> */ (clipArtElement, colorParam) => colors.AddClipArtColor(clipArtElement, colorParam)
),
Colors = colors,
};
I see duplication that I think can be removed, though - The delegates are just passing parameters as-is to colors methods. Is there a way to create a delegate from an instance method (not static) without duplicating the parameters?

Converting comments into an answer...
"Isn't it just CreateTextElements( textElementData, colors.AddTextColor ) without this extra delegate?" - #WiktorZychla
"If you've ever forgotten the parenthesis on a method call, you might have gotten a compiler error: Cannot convert method group to something. This is the opposite. Now you want to refer to the method without parenthesis or arguments because the method is the argument." - #ScottHannen (emphasis mine)
Thanks for the help! I was wrongly making an assumption that the types wouldn't work out here. Taking their advice, my code now looks like:
TemplateColorList colors = new TemplateColorList();
var template = new Template {
TextElements = CreateTextElements(textElementData, colors.AddTextColor),
ClipArtElements = CreateClipArtElements(clipArtElementData, colors.AddClipArtColor),
Colors = colors,
};

Importing different files from Excel with different rules

I have recently been tasked with writing a piece of software that will import Excel files.
The problem I am trying to solve is that my company has c100 clients and each supply a file in a different layout, in so much as the columns in a file will differ between clients but the pertinent information is there in each file.
This process is complicated due to the fact that certain operations need to be done to different files.
In 1 file, for example, a column needs to be inserted after a specifc column and then the result of a calculation needs to be placed into that column. In that same sheet an address is supplied across 9 columns, this address needs to be moved into the last 6 of the 9 columns and then have the first 3 columns removed.
What I don't want to do is write the processing logic for each file (c 100 as mentioned) and thereby get trapped into the drudge of having to maintain this code and be responsible for adding new customer files as they come in.
What I want to do is create a Rule or Processing engine of sorts whereby I can have basic rules like "Insert Column", "Remove Column", "Insert Calculation", "Format a, b, c, d, e & f Columns To Use d, e & f" - the reason being so that configuring the read and process of any new file can be done through a front-end piece of software by an end user (obviously with some training on what to do).
Is there a pattern or strategy that might fit this? I have read about Rules engines but the best examples of these are simple boolean comparisons like "Age = 15" or "Surname = 'Smith'" but can't find a decent example of doing something like "Insert Column after Column G" then "Put G - 125 in to Column H".
Any help here, or a pointer to a good approach, would be greatly appreciated.

Let me see if I can help you out here.
Correct me if I am wrong, but it seems like all your input and output files contain data in columns and columns only.
In that case, you should imagine your problem as a transformation of X input columns to Y output columns. For each client, you will need a configuration that will specify the transform. The configuration might look like below
Y1 = X1
Y2 = X1 + X2
Y3 = X3 + " some string"
As you can see, your configuration lines are simply C# expressions. You can use the LINQ Expression class to build an expression tree from your transformation formulas. You can learn about Expressions here. These expressions can then be compiled and used to do the actual transform. If you think in terms of C#, you will build a static transform method that takes a list as input and returns a list as output for each client. When you use Expressions, you will have to parse the configuration files yourself.
You can also use the Roslyn Compiler Services, which can support proper C# syntax. This way, you can literally have a static method which can do the transform. This also relieves you of the parsing duties.
In either case, you will still have to deal with things like: should I expect the columns to be a string (which means your support needs to know explicitly instruct the configuration GUI to parse needed columns into numbers) or should I automatically convert number like fields into numbers (now support doesn't have to do extra configuration, but they might hit issues when dealing with columns which have numbers, like ID, but should be treated as a string to avoid any improper handling), etc.
In Summary, my approach is:
Create config file per client.
Convert the config file into C# method dynamically using Expressions or Roslyn
Provide a GUI for generating this config - this way the support person can easily specify the transform without knowing your special syntax (Expressions) or C# syntax (Roslyn). When saving config, you can generate one method per client in a single assembly (or separate assembly per client) and persist it. Let's call it client library.
Your main application can do all the standard stuff of reading from excel, validating, etc and then call the client library method to generate the output in a standard format, which can be further processed in your main application.
Hope you got the gist.
Edit: Adding some code to demonstrate. The code is a bit long-winded, but commented for understanding.
// this data represents your excel data
var data = new string[][] {
new string [] { "col_1_1", "10", "09:30" },
new string [] { "col_2_1", "12", "09:40" }
};
// you should read this from your client specific config file/section
// Remember: you should provide a GUI tool to build this config
var config = #"
output.Add(input[0]);
int hours = int.Parse(input[1]);
DateTime date = DateTime.Parse(input[2]);
date = date.AddHours(hours);
output.Add(""Custom Text: "" + date);
";
// this template code should be picked up from a
// non client specific config file/section
var code = #"
using System;
using System.Collections.Generic;
using System.Linq;
namespace ClientLibrary {
static class ClientLibrary {
public static List<string> Client1(string[] input) {
var output = new List<string>();
<<code-from-config>>
return output;
}
}
}
";
// Inject client configuration into template to form full code
code = code.Replace(#"<<code-from-config>>", config);
// Compile your dynamic method and get a reference to it
var references = new MetadataReference[] {
MetadataReference.CreateFromFile(typeof(object).Assembly.Location),
MetadataReference.CreateFromFile(typeof(Enumerable).Assembly.Location)
};
CSharpCompilation compilation = CSharpCompilation.Create(
null,
syntaxTrees: new[] { CSharpSyntaxTree.ParseText(code) },
references: references,
options: new CSharpCompilationOptions(OutputKind.DynamicallyLinkedLibrary));
MethodInfo clientMethod = null;
using (var ms = new MemoryStream()) {
EmitResult result = compilation.Emit(ms);
if (!result.Success) {
foreach (Diagnostic diagnostic in result.Diagnostics) {
Console.Error.WriteLine("{0}: {1}", diagnostic.Id, diagnostic.GetMessage());
}
} else {
ms.Seek(0, SeekOrigin.Begin);
Assembly assembly = Assembly.Load(ms.ToArray());
clientMethod = assembly.GetType("ClientLibrary.ClientLibrary").GetMethod("Client1");
}
}
if (clientMethod == null)
return;
// Do transformation
foreach (string[] row in data) {
var output = clientMethod.Invoke(null, new object[] { row }) as List<string>;
Console.WriteLine(string.Join("|", output));
}
You will need some nuget libraries to compile this, and their corresponding using clauses
nuget install Microsoft.Net.Compilers # Install C# and VB compilers
nuget install Microsoft.CodeAnalysis # Install Language APIs and Services
using System;
using System.Collections.Generic;
using System.IO;
using System.Linq;
using System.Reflection;
using Microsoft.CodeAnalysis;
using Microsoft.CodeAnalysis.CSharp;
using Microsoft.CodeAnalysis.Emit;
As you notice, the only piece to worry about is the GUI to auto-generate the code for the transformation - which I have not provided here. If you want simple transforms, that should be very easy, but for a complex transform, it will be more involved

It sounds like you're expecting your end user to be technical-savvy enough to understand this configuration mechanism that you're going to write. If they can handle that level of technical detail, it might be simpler to give them an Excel book and an official excel template that contains all the columns that your import app needs and they can manually massage the data to the spec.
Otherwise, I would suggest some strategy design based pattern solution to build a library of "data massager" classes for known formats, and just add new classes as new formats are encountered. e.g.
public interface IClientDataImporter
{
List<MyCustomRowStructure> Import(string filename);
}
// client 1 importer
public class ClientOneImporter : IClientDataImporter
{
public List<MyCustomRowStructure> Import(string filename)
{
var result = new List<MyCustomRowStructure>();
// ..... insert custom logic here
return result;
}
}
// client 2 importer
public class ClientTwoImporter : IClientDataImporter
{
public List<MyCustomRowStructure> Import(string filename)
{
var result = new List<MyCustomRowStructure>();
// ..... insert custom logic here
return result;
}
}
// repeat up to however many formats you need
// then.....
public class ExcelToDatabaseImporter
{
public void ImportExcelFile(string filename, string clientName)
{
var myValidData = GetClientDataImporter(clientName).Import(filename);
StickMyDataToMyDatabase(myValidData); // this is where you would load the structure into the db... won't need to touch every time a new format is encountered
}
public IClientDataImporter GetClientDataImporter(string clientName)
{
switch (clientName):
case "ClientOne":
return new ClientOneImporter();
break;
case "ClientTwo":
return new ClientTwoImporter();
break;
default:
throw new ArgumentException("No importer for client");
break;
}
}

I would suggest you to maintain an xml configuration file for each excel file. The xml configuration has to read by a tool, may be a console application, and generate new CSV file, based on the xml configuration.
As XML configuration file can be easily edited by any text editor, users can update the same.

How do I initialize an empty array in C#?

Is it possible to create an empty array without specifying the size?
For example, I created:
String[] a = new String[5];
Can we create the above string array without the size?

If you are going to use a collection that you don't know the size of in advance, there are better options than arrays.
Use a List<string> instead - it will allow you to add as many items as you need and if you need to return an array, call ToArray() on the variable.
var listOfStrings = new List<string>();
// do stuff...
string[] arrayOfStrings = listOfStrings.ToArray();
If you must create an empty array you can do this:
string[] emptyStringArray = new string[0];

In .NET 4.6 the preferred way is to use a new method, Array.Empty:
String[] a = Array.Empty<string>();
The implementation is succinct, using how static members in generic classes behave in .Net:
public static T[] Empty<T>()
{
return EmptyArray<T>.Value;
}
// Useful in number of places that return an empty byte array to avoid
// unnecessary memory allocation.
internal static class EmptyArray<T>
{
public static readonly T[] Value = new T[0];
}
(code contract related code removed for clarity)
See also:
Array.Empty source code on Reference Source
Introduction to Array.Empty<T>()
Marc Gravell - Allocaction, Allocation, Allocation - my favorite post on tiny hidden allocations.

Try this:
string[] a = new string[] { };

You could inititialize it with a size of 0, but you will have to reinitialize it, when you know what the size is, as you cannot append to the array.
string[] a = new string[0];

There is not much point in declaring an array without size. An array is about size. When you declare an array of specific size, you specify the fixed number of slots available in a collection that can hold things, and accordingly memory is allocated. To add something to it, you will need to anyway reinitialize the existing array (even if you're resizing the array, see this thread). One of the rare cases where you would want to initialise an empty array would be to pass array as an argument.
If you want to define a collection when you do not know what size it could be of possibly, array is not your choice, but something like a List<T> or similar.
That said, the only way to declare an array without specifying size is to have an empty array of size 0. hemant and Alex Dn provides two ways. Another simpler alternative is to just:
string[] a = { };
[The elements inside the bracket should be implicitly convertible to type defined, for instance, string[] a = { "a", "b" };]
Or yet another:
var a = Enumerable.Empty<string>().ToArray();
Here is a more declarative way:
public static class Array<T>
{
public static T[] Empty()
{
return Empty(0);
}
public static T[] Empty(int size)
{
return new T[size];
}
}
Now you can call:
var a = Array<string>.Empty();
//or
var a = Array<string>.Empty(5);

string[] a = new string[0];
or short notation:
string[] a = { };
The preferred way now is:
var a = Array.Empty<string>();
I have written a short regular expression that you can use in Visual Studio if you want to replace zero-length allocations e.g. new string[0].
Use Find (search) in Visual Studio with Regular Expression option turned on:
new[ ][a-zA-Z0-9]+\[0\]
Now Find All or F3 (Find Next) and replace all with Array.Empty<…>() !

Simple and elegant!
string[] array = {}

You can define array size at runtime.
This will allow you to do whatever to dynamically compute the array's size. But, once defined the size is immutable.
Array a = Array.CreateInstance(typeof(string), 5);

I had tried:
string[] sample = new string[0];
But I could only insert one string into it, and then I got an exceptionOutOfBound error, so I just simply put a size for it, like
string[] sample = new string[100];
Or another way that work for me:
List<string> sample = new List<string>();
Assigning Value for list:
sample.Add(your input);

As I know you can't make array without size, but you can use
List<string> l = new List<string>()
and then l.ToArray().

Performance Rule CA1825: Avoid allocating zero-length arrays.
Rule discription:
Initializing a zero-length array leads to an unnecessary memory allocation. Instead, use the statically allocated empty array instance by calling the Array.Empty method.
In your case:
var a = Array.Empty<string>();

Combining #nawfal & #Kobi suggestions:
namespace Extensions
{
/// <summary> Useful in number of places that return an empty byte array to avoid unnecessary memory allocation. </summary>
public static class Array<T>
{
public static readonly T[] Empty = new T[0];
}
}
Usage example:
Array<string>.Empty
UPDATE 2019-05-14
(credits to #Jaider ty)
Better use .Net API:
public static T[] Empty<T> ();
https://learn.microsoft.com/en-us/dotnet/api/system.array.empty?view=netframework-4.8
Applies to:
.NET Core: 3.0 Preview 5 2.2 2.1 2.0 1.1 1.0
.NET Framework: 4.8 4.7.2 4.7.1 4.7 4.6.2 4.6.1 4.6
.NET Standard: 2.1 Preview 2.0 1.6 1.5 1.4 1.3
...
HTH

you can use the Array.Empty method (in .Net Core, at least)
string ToCsv(int[] myArr = null) { // null by default
// affect an empty array if the myArr is null
myArr ??= Array.Empty<int>();
//... do stuff
string csv = string.Join(",", myArr);
return csv;
}

You can do:
string[] a = { String.Empty };
Note: OP meant not having to specify a size, not make an array sizeless

Here is a real world example. In this it is necessary to initialize the array foundFiles first to zero length.
(As emphasized in other answers: This initializes not an element and especially not an element with index zero because that would mean the array had length 1. The array has zero length after this line!).
If the part = string[0] is omitted, there is a compiler error!
This is because of the catch block without rethrow. The C# compiler recognizes the code path, that the function Directory.GetFiles() can throw an Exception, so that the array could be uninitialized.
Before anyone says, not rethrowing the exception would be bad error handling: This is not true. Error handling has to fit the requirements.
In this case it is assumed that the program should continue in case of a directory which cannot be read, and not break- the best example is a function traversing through a directory structure. Here the error handling is just logging it. Of course this could be done better, e.g. collecting all directories with failed GetFiles(Dir) calls in a list, but this will lead too far here.
It is enough to state that avoiding throw is a valid scenario, and so the array has to be initialized to length zero. It would be enough to do this in the catch block, but this would be bad style.
The call to GetFiles(Dir) resizes the array.
string[] foundFiles= new string[0];
string dir = #"c:\";
try
{
foundFiles = Directory.GetFiles(dir); // Remark; Array is resized from length zero
}
// Please add appropriate Exception handling yourself
catch (IOException)
{
Console.WriteLine("Log: Warning! IOException while reading directory: " + dir);
// throw; // This would throw Exception to caller and avoid compiler error
}
foreach (string filename in foundFiles)
Console.WriteLine("Filename: " + filename);

Importing data files using generic class definitions

I am trying to import a file with multiple record definition in it. Each one can also have a header record so I thought I would define a definition interface like so.
public interface IRecordDefinition<T>
{
bool Matches(string row);
T MapRow(string row);
bool AreRecordsNested { get; }
GenericLoadClass ToGenericLoad(T input);
}
I then created a concrete implementation for a class.
public class TestDefinition : IRecordDefinition<Test>
{
public bool Matches(string row)
{
return row.Split('\t')[0] == "1";
}
public Test MapColumns(string[] columns)
{
return new Test {val = columns[0].parseDate("ddmmYYYY")};
}
public bool AreRecordsNested
{
get { return true; }
}
public GenericLoadClass ToGenericLoad(Test input)
{
return new GenericLoadClass {Value = input.val};
}
}
However for each File Definition I need to store a list of the record definitions so I can then loop through each line in the file and process it accordingly.
Firstly am I on the right track
or is there a better way to do it?

I would split this process into two pieces.
First, a specific process to split the file with multiple types into multiple files. If the files are fixed width, I have had a lot of luck with regular expressions. For example, assume the following is a text file with three different record types.
TE20110223 A 1
RE20110223 BB 2
CE20110223 CCC 3
You can see there is a pattern here, hopefully the person who decided to put all the record types in the same file gave you a way to identify those types. In the case above you would define three regular expressions.
string pattern1 = #"^TE(?<DATE>[0-9]{8})(?<NEXT1>.{2})(?<NEXT2>.{2})";
string pattern2 = #"^RE(?<DATE>[0-9]{8})(?<NEXT1>.{3})(?<NEXT2>.{2})";
string pattern3 = #"^CE(?<DATE>[0-9]{8})(?<NEXT1>.{4})(?<NEXT2>.{2})";
Regex Regex1 = new Regex(pattern1);
Regex Regex2 = new Regex(pattern2);
Regex Regex3 = new Regex(pattern3);
StringBuilder FirstStringBuilder = new StringBuilder();
StringBuilder SecondStringBuilder = new StringBuilder();
StringBuilder ThirdStringBuilder = new StringBuilder();
string Line = "";
Match LineMatch;
FileInfo myFile = new FileInfo("yourFile.txt");
using (StreamReader s = new StreamReader(f.FullName))
{
while (s.Peek() != -1)
{
Line = s.ReadLine();
LineMatch = Regex1.Match(Line);
if (LineMatch.Success)
{
//Write this line to a new file
}
LineMatch = Regex2.Match(Line);
if (LineMatch.Success)
{
//Write this line to a new file
}
LineMatch = Regex3.Match(Line);
if (LineMatch.Success)
{
//Write this line to a new file
}
}
}
Next, take the split files and run them through a generic process, that you most likely already have, to import them. This works well because when the process inevitably fails, you can narrow it to the single record type that is failing and not impact all the record types. Archive the main text file along with the split files and your life will be much easier as well.
Dealing with these kinds of transmitted files is hard, because someone else controls them and you never know when they are going to change. Logging the original file as well as a receipt of the import is very import and shouldn't be overlooked either. You can make that as simple or as complex as you want, but I tend to write a receipt to a db and copy the primary key from that table into a foreign key in the table I have imported the data into, then never change that data. I like to keep a unmolested copy of the import on the file system as well as on the DB server because there are inevitable conversion / transformation issues that you will need to track down.
Hope this helps, because this is not a trivial task. I think you are on the right track, but instead of processing/importing each line separately...write them to a separate file. I am assuming this is financial data, which is one of the reasons I think provability at every step is important.

I think the FileHelpers library solves a number of your problems:
Strong types
Delimited
Fixed-width
Record-by-Record operations
I'm sure you could consolidate this into a type hierarchy that could tie in custom binary formats as well.

Have you looked at something using Linq? This is a quick example of Linq to Text and Linq to Csv.
I think it would be much simpler to use "yield return" and IEnumerable to get what you want working. This way you could probably get away with only having 1 method on your interface.

Simplest possible key/value pair file parsing in .NET

My project requires a file where I will store key/value pair data that should be able to be read and modified by the user. I want the program to just expect the keys to be there, and I want to parse them from the file as quickly as possible.
I could store them in XML, but XML is way to complex, and it would require traversing nodes, and child nodes and so on, all I want is some class that takes a file and generates key value pairs. I want as little error handling as possible, and I want it done with as little code as possible.
I could code a class like that myself, but I'd rather learn how it's don'e in the framework than inventing the wheel twice. Are there some built in magic class in .NET (3.5) that are able to do so?
MagicClass kv = new MagicClass("Settings.ini"); // It doesn't neccesarily have to be an INI file, it can be any simple key/value pair format.
string Value1 = kv.get("Key1");
...

If you're looking for a quick easy function and don't want to use .Net app\user config setting files or worry about serialization issues that sometimes occur of time.
The following static function can load a file formatted like KEY=VALUE.
public static Dictionary<string, string> LoadConfig(string settingfile)
{
var dic = new Dictionary<string, string>();
if (File.Exists(settingfile))
{
var settingdata = File.ReadAllLines(settingfile);
for (var i = 0; i < settingdata.Length; i++)
{
var setting = settingdata[i];
var sidx = setting.IndexOf("=");
if (sidx >= 0)
{
var skey = setting.Substring(0, sidx);
var svalue = setting.Substring(sidx+1);
if (!dic.ContainsKey(skey))
{
dic.Add(skey, svalue);
}
}
}
}
return dic;
}
Note: I'm using a Dictionary so keys must be unique, which is usually that case with setting.
USAGE:
var settingfile = AssemblyDirectory + "\\mycustom.setting";
var settingdata = LoadConfig(settingfile);
if (settingdata.ContainsKey("lastrundate"))
{
DateTime lout;
string svalue;
if (settingdata.TryGetValue("lastrundate", out svalue))
{
DateTime.TryParse(svalue, out lout);
lastrun = lout;
}
}

Use the KeyValuePair class for you Key and Value, then just serialize a List to disk with an XMLSerializer.
That would be the simplest approach I feel. You wouldn't have to worry about traversing nodes. Calling the Deserialize function will do that for you. The user could edit the values in the file if they wish also.

I don't know of any builtin class to parse ini file. I've used nini when needed to do so. It's licensed under the MIT/X11 license, so doesn't have any issue to be included in a closed source program.
It's very to use. So if you have a Settings.ini file formatted this way:
[Configuration]
Name = Jb Evain
Phone = +330101010101
Using it would be as simple as:
var source = new IniConfigSource ("Settings.ini");
var config = source.Configs ["Configuration"];
string name = config.Get ("Name");
string phone = config.Get ("Phone");

if you want the user to be able to read and modify the file, i suggest a comma-delimited pair, one per line
key1,value1
key2,value2
...
parsing is simple: read the file, split at newline or comma, then take the elements in pairs

Format the file this way:
key1=value1
key2=value2
Read the entire file into a string (there is a simple convenience function that does that, maybe in the File or string class), and call string.Split('='). Make sure you also call string.Trim() on each key and value as you traverse the list and pop each pair into a hashtable or dictionary.