Develop Reference

How can I chain methods depending on user input?

I have a method that I'm using to process images (rotate, filter, resize, etc). It looks like this:
public Image ProcessImage(Image image, Func<ImageFactory, ImageFactory> process)
{
using (var imageFactory = new ImageFactory(preserveExifData: true))
{
using (var imageStream = new MemoryStream())
{
var loadResult = imageFactory.Load(image);
var processResult = process(loadResult);
processResult.Save(imageStream);
return Image.FromStream(imageStream);
}
}
}
Since I'm using Func<> in this way I can just call the editing method I want like so:
_imageEditor.ProcessImage(_image, im => im.Resize(size))
Which means I can chain methods like:
_imageEditor.ProcessImage(_image, im => im.Resize(size).Rotate(54).Flip(true))
My question is, how can I chain these methods depending on the using input? So if my user wants to rotate and resize at the same time, I could just add the .Resize and .Rotate methods (which by the way, take different parameters).
Sure, I could use a bunch of bools but if I had a ton of editing methods it would become impossible, very very ugly to use.
Is there a way to add methods to this chain, and if so, how would you go about it?

Your question isn't 100% clear. You've left a lot of details out. But, it sounds like you are either asking how to successively invoke your methods, passing the result from one invocation to the next, or how to use inputs created at runtime to create this list of invocations, or both.
Without a Minimal, Reproducible Example, I don't have any good way to reproduce your scenario, nor to provide a solution that is specific to that scenario. But, here's a demonstration of the basic techniques you might use to accomplish those goals:
class Program
{
private static readonly Dictionary<string, Func<int[], Func<A, A>>> _parser =
new Dictionary<string, Func<int[], Func<A, A>>>()
{
{ "Init", MakeInitFunc },
{ "Add", MakeAddFunc },
{ "Multiply", MakeMultiplyFunc },
{ "Negate", MakeNegateFunc },
};
static void Main(string[] args)
{
(string, int[])[] ops =
{
("Init", new [] { 17 }),
("Add", new [] { 5 }),
("Multiply", new [] { 2 }),
("Negate", new int[0]),
};
Console.WriteLine(Chain(new A(), OpsToDelegates(ops)).Value);
Console.WriteLine(Chain(new A(), OpsToDelegates(ops).Reverse()).Value);
}
private static IEnumerable<Func<A,A>> OpsToDelegates(IEnumerable<(string Name, int[] Args)> ops)
{
foreach (var op in ops)
{
yield return _parser[op.Name](op.Args);
}
}
private static A Chain(A a, IEnumerable<Func<A, A>> ops)
{
foreach (Func<A, A> op in ops)
{
a = op(a);
}
return a;
}
private static Func<A, A> MakeInitFunc(int[] args)
{
return a => a.Init(args[0]);
}
private static Func<A, A> MakeAddFunc(int[] args)
{
return a => a.Add(args[0]);
}
private static Func<A, A> MakeMultiplyFunc(int[] args)
{
return a => a.Add(args[0]);
}
private static Func<A, A> MakeNegateFunc(int[] args)
{
return a => a.Negate();
}
}
class A
{
public int Value { get; private set; }
public A Init(int value) { Value = value; return this; }
public A Add(int value) { Value += value; return this; }
public A Multiply(int value) { Value *= value; return this; }
public A Negate() { Value *= -1; return this; }
}
There are two key elements to the above:
Given a sequence of Func<A, A> delegates, it is possible to chain them together with a simple loop. See the Chain() method for how that can be done.
Given some user input, it is possible to transform that into a sequence of Func<A, A> delegates. There is actually a very wide range of possible ways to approach that particular problem. My example shows a very simple technique, using a dictionary that maps input string values to helper methods that do the actual work of generating the elements of the sequence. See OpsToDelegates() for that.
Combining those two into this simple program, you can see how you can start with just a list of names of operations and parameters to apply them, and turn that into a functional sequence of operations actually applied to an object.
I trust you can take these general ideas and apply them to your particular scenario.

The easiest way to chain methods together is to use the Aggregate LINQ method.
You just need to change your method signature to Image ProcessImage(Image image, params Func<ImageFactory, ImageFactory>[] processes) and then you can do this:
public Image ProcessImage(Image image, params Func<ImageFactory, ImageFactory>[] processes)
{
using (var imageFactory = new ImageFactory(preserveExifData: true))
{
using (var imageStream = new MemoryStream())
{
var loadResult = imageFactory.Load(imageStream);
var processResult = processes.Aggregate(loadResult, (r, p) => p(r));
processResult.Save(imageStream);
return Image.FromStream(imageStream);
}
}
}
Now you just have to build your Func<ImageFactory, ImageFactory>[] processes from the user's selections.

Change normal loop to Parallel loop

I have the following code:
static void Main(string[] args)
{
TaskExecuter.Execute();
}
class Task
{
int _delay;
private Task(int delay) { _delay = delay; }
public void Execute() { Thread.Sleep(_delay); }
public static IEnumerable GetAllTasks()
{
Random r = new Random(4711);
for (int i = 0; i < 10; i++)
yield return new Task(r.Next(100, 5000));
}
}
static class TaskExecuter
{
public static void Execute()
{
foreach (Task task in Task.GetAllTasks())
{
task.Execute();
}
}
}
I need to change the loop in Execute method to paralle with multiple threads, I tried the following, but it isn't working since GetAllTasks returns IEnumerable and not a list
Parallel.ForEach(Task.GetAllTasks(), task =>
{
//Execute();
});

Parallel.ForEach works with IEnumerable<T>, so adjust your GetAllTasks to return IEnumerable<Task>.
Also .net has widely used Task class, I would avoid naming own class like that to avoid confusion.

Parallel.ForEach takes an IEnumerable<TSource>, so your code should be fine. However, you need to perform the Execute call on the task instance that is passed as parameter to your lambda statement.
Parallel.ForEach(Task.GetAllTasks(), task =>
{
task.Execute();
});
This can also be expressed as a one-line lambda expression:
Parallel.ForEach(Task.GetAllTasks(), task => task.Execute());
There is also another subtle bug in your code that you should pay attention to. Per its internal implementation, Parallel.ForEach may enumerate the elements of your sequence in parallel. However, you are calling an instance method of the Random class in your enumerator, which is not thread-safe, possibly leading to race issues. The easiest way to work around this would be to pre-populate your sequence as a list:
Parallel.ForEach(Task.GetAllTasks().ToList(), task => task.Execute());

This worked on my linqpad. I just renamed your Task class to Work and also returned an IEnumerable<T> from GetAllTasks:
class Work
{
int _delay;
private Work(int delay) { _delay = delay; }
public void Execute() { Thread.Sleep(_delay); }
public static IEnumerable<Work> GetAllTasks()
{
Random r = new Random(4711);
for (int i = 0; i < 10; i++)
yield return new Work(r.Next(100, 5000));
}
}
static class TaskExecuter
{
public static void Execute()
{
foreach (Work task in Work.GetAllTasks())
{
task.Execute();
}
}
}
void Main()
{
System.Threading.Tasks.Parallel.ForEach(Work.GetAllTasks(), new Action<Work>(task =>
{
//Execute();
}));
}

TPL with multiple Method calls Console.Write issue

If i use TPL i run into problems in Parse.. Methods i do use Console.Write to build some Line but somtimes one is to fast and writes in the other Methods row. How do i lock or is there some better way?
Parallel.Invoke(
() => insertedOne = Lib.ParseOne(list),
() => insertedTwo = Lib.ParseTwo(list),
() => insertedThree = Lib.ParseThree(list));
Example for Parse.. Methods.
public static int ParseOne(string[] _list) {
Console.Write("blabla");
Console.Write("blabla");
return 0;
}
public static int ParseTwo(string[] _list) {
Console.Write("hahahah");
Console.Write("hahahah");
return 0;
}
public static int ParseThree(string[] _list) {
Console.Write("egegege");
Console.Write("egegege");
return 0;
}

To be able to print your blablas, hahahahs and egegeges as a single entity(indivisible)
you can write your method as:
public static int ParseThree(string[] _list)
{
lock (Console.Out)
{
Console.Write("egegege");
Console.Write("egegege");
}
return 0;
}

Why don't you run all the tasks in one thread, one after the other?
System.Threading.Tasks.Task.Factory.StartNew(()=>
{
insertedOne = Lib.ParseOne(list);
insertedTwo = Lib.ParseTwo(list);
insertedThree = Lib.ParseThree(list);
});
This way you won't have that much of a race condition.

IObservable TakeLast(n) and blocking

I am trying to get the 20 latest values of an observable and exposing it as a property without blocking occurring. At the moment, my code looks like:
class Foo
{
private IObservable<int> observable;
public Foo(IObservable<int> bar)
{
this.observable = bar;
}
public IEnumerable<int> MostRecentBars
{
get
{
return this.observable.TakeLast(20).ToEnumerable();
}
}
}
However, when the MostRecentBars getter is called, this is blocking, presumably because ToEnumerable will not return until there are at least 20 observed values.
Is there a built-in way to expose up to a maximum of 20 most recent values of the observable without blocking? If there are less than 20 observed values then it should just return all of them.

I'll give you two choices. One uses the Rx Scan operator, but I think that one makes it a little more complicated to read. The other uses a standard Queue with locking. You can choose.
(1)
class Foo
{
private int[] bars = new int[] { };
public Foo(IObservable<int> bar)
{
bar
.Scan<int, int[]>(
new int[] { },
(ns, n) =>
ns
.Concat(new [] { n, })
.TakeLast(20)
.ToArray())
.Subscribe(ns => bars = ns);
}
public IEnumerable<int> MostRecentBars
{
get
{
return bars;
}
}
}
(2)
class Foo
{
private Queue<int> queue = new Queue<int>();
public Foo(IObservable<int> bar)
{
bar.Subscribe(n =>
{
lock (queue)
{
queue.Enqueue(n);
if (queue.Count > 20)
{
queue.Dequeue();
}
}
});
}
public IEnumerable<int> MostRecentBars
{
get
{
lock (queue)
{
return queue.ToArray();
}
}
}
}
I hope these help.

I can't think of a built-in Rx operator(s) that fits your requirements. You could implement it this way:
class Foo
{
private IObservable<int> observable;
private Queue<int> buffer = new Queue<int>();
public Foo(IObservable<int> bar)
{
this.observable = bar;
this.observable
.Subscribe(item =>
{
lock (buffer)
{
if (buffer.Count == 20) buffer.Dequeue();
buffer.Enqueue(item);
}
});
}
public IEnumerable<int> MostRecentBars
{
get
{
lock (buffer)
{
return buffer.ToList(); // Create a copy.
}
}
}
}

Although you have already got your answer, I was thinking of solving this using Replay Subject with buffer and came up with something like:
class Foo
{
private ReplaySubject<int> replay = new ReplaySubject<int>(20);
public Foo(IObservable<int> bar)
{
bar.Subscribe(replay);
}
public IEnumerable<int> MostRecentBars
{
get
{
var result = new List<int>();
replay.Subscribe(result.Add); //Replay fill in the list with buffered items on same thread
return result;
}
}
}
Let me know if this fits into your problem.

I have a few extensions I tend to attach to any project I build with the reactive extensions, one of them is a sliding window:
public static IObservable<IEnumerable<T>> SlidingWindow<T>(this IObservable<T> o, int length)
{
Queue<T> window = new Queue<T>();
return o.Scan<T, IEnumerable<T>>(new T[0], (a, b) =>
{
window.Enqueue(b);
if (window.Count > length)
window.Dequeue();
return window.ToArray();
});
}
This returns an array of the most recent N items (or less, if there have not been N items yet).
For your case, you should be able to do:
class Foo
{
private IObservable<int> observable;
private int[] latestWindow = new int[0];
IDisposable slidingWindowSubscription;
public Foo(IObservable<int> bar)
{
this.observable = bar;
slidingWindowSubscription = this.observable.SlidingWindow(20).Subscribe(a =>
{
latestWindow = a;
});
}
public IEnumerable<int> MostRecentBars
{
get
{
return latestWindow;
}
}
}

Seriously, doesn't this all depend on how long you willing to wait? How do you know the observable has completed when you have 19 items in hand? One second later you are sure it's done? ten years later? You are sure it's done? How do you know? It's Observable, so you have to keep observing it until your operators or whatever monadic transforms you apply do some useful transform with the incoming stream.
I would think the (possibly new addition) Window or Buffer with the TimeSpan overload would work. Especially Window, it releases an Observable of Observable so once the outer Observable is created, you can actually listen for the First of 20 items, and then you need to listen carefully for the OnCompleted, or you lose the whole point of the Window operator, but you get the idea.

Which is the best way to improve memory usage when you collect a large data set before processing it? (.NET)

When I have to get GBs of data, save it on a collection and process it, I have memory overflows. So instead of:
public class Program
{
public IEnumerable<SomeClass> GetObjects()
{
var list = new List<SomeClass>();
while( // get implementation
list.Add(object);
}
return list;
}
public void ProcessObjects(IEnumerable<SomeClass> objects)
{
foreach(var object in objects)
// process implementation
}
void Main()
{
var objects = GetObjects();
ProcessObjects(objects);
}
}
I need to:
public class Program
{
void ProcessObject(SomeClass object)
{
// process implementation
}
public void GetAndProcessObjects()
{
var list = new List<SomeClass>();
while( // get implementation
Process(object);
}
return list;
}
void Main()
{
var objects = GetAndProcessObjects();
}
}
There is a better way?

You ought to leverage C#'s iterator blocks and use the yield return statement to do something like this:
public class Program
{
public IEnumerable<SomeClass> GetObjects()
{
while( // get implementation
yield return object;
}
}
public void ProcessObjects(IEnumerable<SomeClass> objects)
{
foreach(var object in objects)
// process implementation
}
void Main()
{
var objects = GetObjects();
ProcessObjects(objects);
}
}
This would allow you to stream each object and not keep the entire sequence in memory - you would only need to keep one object in memory at a time.

Don't use a List, which requires all the data to be present in memory at once. Use IEnumerable<T> and produce the data on demand, or better, use IQueryable<T> and have the entire execution of the query deferred until the data are required.
Alternatively, don't keep the data in memory at all, but rather save the data to a database for processing. When processing is complete, then query the database for the results.

public IEnumerable<SomeClass> GetObjects()
{
foreach( var obj in GetIQueryableObjects
yield return obj
}

You want to yield!
Delay processing of your enumeration. Build a method that returns an IEnumerable but only returns one record at a time using the yield statement.

The best methodology in this case would be to Get and Process in chunks. You will have to find out how big a chunk to Get and Process by trial and error. So the code would be something like :
public class Program
{
public IEnumerable GetObjects(int anchor, int chunkSize)
{
var list = new List();
while( // get implementation for given anchor and chunkSize
list.Add(object);
}
return list;
}
public void ProcessObjects(IEnumerable<SomeClass> objects)
{
foreach(var object in objects)
// process implementation
}
void Main()
{
int chunkSize = 5000;
int totalSize = //Get Total Number of rows;
int anchor = //Get first row to process as anchor;
While (anchor < totalSize)
(
var objects = GetObjects(anchor, chunkSize);
ProcessObjects(objects);
anchor += chunkSize;
}
}
}