Develop Reference

How to properly partition code in a C# functional library?

As a premise one of a key difference of FP design about reusable libraries (for what I'm learning), is that these are more data-centric that corresponding OO (in general).
This seems confirmed also from emerging techniques like TFD (Type-First-Development), well explained by Tomas Petricek in this blog post.
Nowadays language are multi-paradigm and the same Petricek in its book explains various functional techniques usable from C#.
What I'm interested here and, hence the question, is how to properly partition code.
So I've defined library data structures, using the equivalent of discriminated unions (as shown in Petricek book), and I project to use them with immutable lists and/or tuples according to the domain logic of mine requirements.
Where do I place operations (methods ... functions) that acts on data structures?
If I want define an high-order function that use a function value embodied in a standard delegates Func<T1...TResult>, where do I place it?
Common sense says me to group these methods in static classes, but I'd like a confirmation from people that already wrote functional libs in C#.
Assuming that this is correct and I've an high-order function like this:
static class AnimalTopology {
IEnumerable<Animal> ListVertebrated(Func<Skeleton, bool> selector) {
// remainder omitted
}
}
If choosing vertebrated animal has N particular cases that I want to expose in the library, what's the more correct way to expose them.
static class VertebratedSelectorsA {
// this is compatible with "Func<Skeleton, bool> selector"
static bool Algorithm1(Skeleton s) {
//...
}
}
or
static class VertebratedSelectorsB {
// this method creates the function for later application
static Func<Skeleton, bool> CreateAlgorithm1Selector(Skeleton s) {
// ...
}
}
Any indication will be very appreciated.
EDIT:
I want to quote two phrases from T. Petricek, Real World Functional Programming foreword by Mads Torgersen:
[...] You can use functional programming techniques in C# to great benefit,
though it is easier and more natural to do so in F#.
[...]
Functional programming is a state of mind. [...]
EDIT-2:
I feel there's a necessity to further clarify the question. The functional mentioned in the title strictly relates to Functional Programming; I'm not asking the more functional way of grouping methods, in the sense of more logic way or the the way that make more sense in general.
This implies that the implementation will try to follow as more as possible founding concepts of FP summarized by NOOO manifesto and quoted here for convenience and clarity:
Functions and Types over classes
Purity over mutability
Composition over inheritance
Higher-order functions over method dispatch
Options over nulls
The question is around how to layout a C# library wrote following FP concepts, so (for example) it's absolutely not an option putting methods inside data structure; because this is a founding Object-Oriented paradigm.
EDIT-3:
Also if the question got response (and various comments), I don't want give the wrong impression that there has been said that one programming paradigm is superior than another.
As before I'll mention an authority on FP, Don Syme, in its book Expert F# 3.0 (ch.20 - Designing F# Libraries - pg.565):
[...] It's a common misconception that the functional and OO programming methodologies compete; in fact, they're largely orthogonal. [...]

Note: If you want a shorter, more-to-the-point answer, see my other answer. I am aware that this one here might seem to ramble & go on forever & talk past your issue, but perhaps it will give you a few ideas.
It is difficult to answer your question without knowing the exact relationship between Animal and Skeleton. I will make a recommendation about this relationship in the second half of my answer, but before I do that, I will simply go along with what I see in your post.
First I will try to infer a few things from your code:
static class AnimalTopology
{
// Note: I made this function `static`... or did you omit the keyword on purpose?
static IEnumerable<Animal> ListVertebrated(Func<Skeleton, bool> selector)
{
…
}
}
If you have designed that function according to functional principles, it should have no side-effects. That is, its output relies only on its arguments. (And in a semi-object-oriented setting, perhaps on other static members of AnimalTopology; but since you didn't show any, let us ignore that possibility.)
If the function is indeed side-effect-free (and does not access static members of AnimalTopology), then the function's type signature suggests that it is possible to derive an Animal from a Skeleton, because it accepts something that acts on Skeletons and returns Animals.
If this is also true, then let me assume the following for the sake of being able to give an answer:
class Skeleton
{
…
public Animal Animal { get { … } } // Skeletons have animals!? We'll get to that.
}
Now it is obvious that your function is impossible to implement, since it could derive Animals from Skeletons, but it doesn't receive any Skeleton at all; it only receives a predicate function that acts on a Skeleton. (You could fix this by adding a second parameter of type Func<IEnumerable<Skeleton>> getSkeletons, but...)
In my opinion, something like the following would make more sense:
static IEnumerable<Animal> GetVertebrates(this IEnumerable<Skeleton> skeletons,
Func<Skeleton, bool> isVertebrate)
{
return skeletons
.Where(isVertebrate)
.Select(s => s.Animal);
}
Now, one might wonder why you are guessing animals from their skeletons; and isn't the bool property "is vertebrate" an inherent property of an animal (or skeleton)? Are there really several ways to decide on this?
I would suggest the following:
class Animal
{
Skeleton Skeleton { get; } // not only vertebrates have skeletons!
}
class Vertebrate : Animal { … } // vertebrates are a kind of animal
static class AnimalsExtensions
{
static IEnumerable<Vertebrate> ThatAreVertebrates(this IEnumerable<Animal> animals)
{
return animals.OfType<Vertebrate>();
}
}
Please note the use of extension methods above. Here's an example how to use it:
List<Animal> animals = …;
IEnumerable<Vertebrate> vertebrates = animals.ThatAreVertebrates();
Now suppose your extension method did more complex work. In that case, it might be a good idea to put it inside its own designated "algorithm type":
interface IVertebrateSelectionAlgorithm
{
IEnumerable<Vertebrate> GetVertebrates(IEnumerable<Animal> animals);
}
This has the advantage that it can be set up / parameterized e.g. via a class constructor; and you could split up the algorithm into several methods that all reside in the same class (but are all private except for GetVertebrates.)
Of course you can do the same kind of parameterization with functional closures, but in my experience that quickly gets messy in a C# setting. Here, classes are a good means to group a set of functions together as one logical entity.

Where do I place operations (methods ... functions) that acts on data structures?
I see four common approaches (in no particular order):
Put the functions inside the data structures. (This is the object-oriented "method" approach. It is suitable when a function acts only on an instance of that type. It is perhaps less appropriate e.g. when a function "draws together" several objects of different types, and spits out an object of yet another type. In this case, I would...)
Put the functions inside their own designated "algorithm classes". (This seems reasonable when the functions do much or complex work, or need to be parameterized/configured, or where you might want to split the algorithm into several functions that you can then logically "group" together by putting them in a class type.)
Turn the functions into lambdas (a.k.a. anonymous delegates, closures, etc.). (This works well if they're small and you only need them in one specific place; the code won't be easily reusable in a different place.)
Put the functions in a static class and make them extension methods. (That's how LINQ to Objects works. It is a hybrid functional & object-oriented approach. It takes some extra care to get the discoverability / namespacing issue right. Many people will think this approach breaks "encapsulation" when taken too far. For a counter-argument, read the excellent C++ article "How Non-Member Functions Improve Encapsulation"; substitute "extension method" for "non-member friend function".)
Note: I could go into each of these in more detail if people want, but before I do that, I'll wait and see what kind of feedback this answer receives.

Implement method decorators in C#

In python is possible to implement function decorators to extend the behavior of functions and methods.
In particular I'm migrating a device lib from python to C#. The communication with device can generate errors which should reraised with custom exception.
In python I would write like this:
#device_error_wrapper("Device A", "Error while setting output voltage.")
def set_voltage(self, voltage):
"""
Safely set the output voltage of device.
"""
self.__handle.write(":source:voltage:level {0}".format(voltage))
This method call would expand to
try:
self.__handle.write(":source:voltage:level {0}".format(voltage))
except Error:
raise DeviceError("Error while setting output voltage.", "DeviceA")
With this pattern you can easily wrap and extend methods without having to write every try-except clause in every method.
Is it to possible to implement a similar pattern using C#?
If the implementation of the decorator (device_error_wrapper) is needed, please tell.

As others have pointed out, tools like PostSharp allow you to weave in the cross cutting logic during (actually, after) compilation.
The alternative is to do it in runtime. Some IoC tools allow you to define the interceptors which are then added to proxy classes to your implementation. This sounds much more complex then it really is, so I will show an example based on Castle DynamicProxy.
First you define your class which needs to be wrapped.
[Interceptor(typeof(SecurityInterceptor))]
public class OrderManagementService : IOrderManagementService
{
[RequiredPermission(Permissions.CanCreateOrder)]
public virtual Guid CreateOrder(string orderCode)
{
Order order = new Order(orderCode);
order.Save(order); // ActiveRecord-like implementation
return order.Id;
}
}
RequiredPermission serves as a decorator here. The class itself is adorned with Interceptor attribute specifying the handler for the interface method calls. This can also be put into configuration, so it is hidden from the class.
The interceptor implementation contains the decorator logic
class SecurityInterceptor : IMethodInterceptor
{
public object Intercept(IMethodInvocation invocation, params object[] args)
{
MethodInfo method = invocation.Method;
if (method.IsDefined(typeof(RequiredPermission), true) // method has RequiredPermission attribute
&& GetRequiredPermission(method) != Context.Caller.Permission) {
throw new SecurityException("No permission!");
}
return invocation.Proceed(args);
}
private Permission GetRequiredPermission(MethodInfo method)
{
RequiredPermission attribute = (RequiredPermission)method.GetCustomAttributes(typeof(RequiredPermission), false)[0];
return attribute.Permission;
}
}
There are some drawbacks, however:
with DynamicProxy you can only wrap interfaces and virtual methods.
you need to instantiate the object via IoC container and not directly (which is not a problem if you already use IoC container)

You can achieve something similar using Aspect Oriented Programming. I've only used PostSharp in the past but it's not free for commercial use though.
There are other AOP solutions out there and you can certainly achieve something similar using Mono.Cecil, but it would require more work.
Reza Ahmadi wrote a nice little introduction article called Aspect Oriented Programming Using C# and PostSharp. It can give you a clear enough idea of what to expect and how it works.

There's no easy way to implement such decorators in C# - custom Attributes are by default only descriptive. There are however projects that extend C# compiler or runtime so that you can actually use this. I think the best one is PostSharp. With it you can define such method decorator ("aspect" in general) and the method gets wrapped during compilation like you need.
I've also seen this implemented by actually wrapping your classes by decorator classes, but that's a lot of work and I don't think it can be done in a really general way. Wikipedia shows this in Decorator Pattern article

As others have mentioned you are looking for AOP. PostSharp is a good post compile solution, but Castle DynamicProxy is a runtime AOP solution.

What is the intent of dynamics in C# 4.0? What are some "neat" uses of it?

Just as the title states... is it primarily for instances where you don't know precisely what type will be returned from a method call (COM interop maybe)?

Here is an example that i consider a nice improvement, it doesn't change a great deal, but it's the little things in life that make a difference :)
http://weblogs.asp.net/gunnarpeipman/archive/2010/07/27/asp-net-mvc-3-new-viewmodel-is-dynamic-viewdata.aspx

As you mention, it makes it easier to interoperate with dynamic languages.
For me, that usually means javascript. One of my favorite uses is consuming JSON on the server without defining DTOs - which also enables LINQ to JSON:
JsonObject body;
string[] favoriteToys =
(from child in (JsonValue)body.AsDynamic().Children
where child.Value.AsDynamic().Name.ReadAs<string>(string.Empty).StartsWith("J")
select child.Value.AsDynamic().BestToy.ReadAs<string>("No favorite toy"))
.ToArray();
(Sample code lifted from "Okay, WCF, we can be friends now", and be sure to see "WCF support for jQuery on wcf.codeplex.com")

If you didn't exactly know the type of a variable, or you wanted to allow the user to specify a class to load at runtime and use the C# reflection API to load it, then you could use dynamic typing so that the user does not need their class to derive from anything, but just needs to define certain methods (an exception would be thrown if not and an error printed from the exception to tell the user what happened).
The advantage of this is that there would be less typing by the user, and therefore less space for user error.

They are useful in scenarios where you don't know the return type, or you don't know at compile-time what members will be available on an object, so you can defer it until run-time. Another neat use of dynamic is ExpandoObject. This allows you to added members to an object at run-time. I've used it to replace Dictionary<string, object> style code.

Yes, COM interop is probably the most common place where dynamics will be incredibly useful. I've used it in conjunction with IronPython to make calls to python methods (which are dynamic scripts by nature) much cleaner and easier. These are the kinds of things it was intended for.
That said, the overall effect of introducing dynamics is to enable C# to become a kind of scripting language, and there is a strong likelihood that it will be used for a number of other purposes along those lines (much to the horror of language purists). For example, the ViewBag property in MVC 3 is just a dynamic wrapper for the ViewData collection. It makes syntax look just a tad "cleaner" (ViewBag.FirstName instead of ViewData["FirstName"]), but at the potential cost of clarity: At a glance, a C# developer might assume that the former is an access to a property on a strongly-typed object of some kind, which can give them a false sense of security when there are no compiler errors.
I've also heard of frameworks that can deserialize a JSON string into a dynamic object, so your .NET system could handle dynamic javascript objects from AJAX requests in much the same way javascript handles them. It's really cool to some people and at least a little creepy to others.

one of the "cool" things you get with dynamic is multiple dispatch without the visitor pattern.
public class Fruit {}
public class Apple : Fruit {}
public class GrannySmith : Apple {}
public class FruitSalad
{
public void Mix(Apple apple) {
Console.WriteLine("mixing a apple");
}
public void Mix(GrannySmith apple) {
Console.WriteLine("mixing a Granny-Smith apple");
}
public void Mix(Fruit fruit) {
Console.WriteLine("unknown fruit, can't mix!");
}
}
public static void Main(string[] args)
{
var salad = new FruitSalad();
Fruit fruit = new Apple();
salad.Mix(fruit); // unknown fruit, can't mix
// (can't resolve onm the type of parameter)
dynamic fruit2 = new Apple();
salad.Mix(fruit2); // mixing a apple
// with dynamic, the parameter's resolution
// is deferred to runtime, so now we know it's an apple
}

Is Reflection breaking the encapsulation principle?

Okay, let's say we have a class defined like
public class TestClass
{
private string MyPrivateProperty { get; set; }
// This is for testing purposes
public string GetMyProperty()
{
return MyPrivateProperty;
}
}
then we try:
TestClass t = new TestClass { MyPrivateProperty = "test" };
Compilation fails with TestClass.MyPrivateProperty is inaccessible due to its protection level, as expected.
Try
TestClass t = new TestClass();
t.MyPrivateProperty = "test";
and compilation fails again, with the same message.
All good until now, we were expecting this.
But then one write:
PropertyInfo aProp = t.GetType().GetProperty(
"MyPrivateProperty",
BindingFlags.NonPublic | BindingFlags.Instance);
// This works:
aProp.SetValue(t, "test", null);
// Check
Console.WriteLine(t.GetMyProperty());
and here we are, we managed to change a private field.
Isn't it abnormal to be able to alter some object's internal state just by using reflection?
Edit:
Thanks for the replies so far. For those saying "you don't have to use it": what about a class designer, it looks like he can't assume internal state safety anymore?

Reflection breaks encapsulation principles by giving access to private fields and methods, but it's not the first or only way in which encapsulation can be circumvented; one could argue that serialization exposes all the internal data of a class, information which would normally be private.
It's important to understand that encapsulation is only a technique, one that makes designing behaviour easier, provided consumers agree use an API you have defined. If somebody chooses to circumvent your API using reflection or any other technique, they no longer have the assurance that your object will behave as you designed it. If somebody assigns a value of null to a private field, they'd better be ready to catch a NullReferenceException the next time they try to use your class!
In my experience, programming is all about assertions and assumptions. The language asserts constraints (classes, interfaces, enumerations) which make creating isolated behaviour much easier to produce, on the assumption that a consumer agrees to not violate those boundaries.
This is a fair assertion to make given it makes a divide-and-conquer approach to software development more easy than any technique before it.

Reflection is a tool. You may use it to break encapsulation, when it gives you more than it takes away.
Reflection has a certain "pain" (or cost -- in performance, in readability, in reliability of code) associated with it, so you won't use it for a common problem. It's just easier to follow object-oriented principles for common problems, which is one of the goals of the language, commonly referred to as the pit of success.
On the other hand, there are some tasks that wouldn't be solvable without that mechanism, e.g. working with run-time generate types (though, it is going to be much-much easier starting from .NET 4.0 with it's DLR and the "dynamic" variables in C# 4.0).

You are right that reflection can be opposed to any number of good design principles, but it can also be an essential building block that you can use to support good design principles - e.g. software that can be extended by plugins, inversion of control, etc.
If you're worried that it represents a capability that should be discouraged, you may have a point. But it's not as convenient to use as true language features, so it is easier to do things the right way.
If you think reflection ought to be impossible, you're dreaming!
In C++ there is no reflection as such. But there is an underlying "object model" that the compiler-generated machine code uses to access the structure of objects (and virtual functions). So a C++ programmer can break encapsulation in the same way.
class RealClass
{
private:
int m_secret;
};
class FakeClass
{
public:
int m_notSecret;
};
We can take a pointer to an object of type RealClass and simply cast it to FakeClass and access the "private" member.
Any restricted system has to be implemented on top of a more flexible system, so it is always possible to circumvent it. If reflection wasn't provided as a BCL feature, someone could add it with a library using unsafe code.
In some languages there are ways to encapsulate data so that it is not possible within the language to get at the data except in certain prescribed ways. But it will always be possible to cheat if you can find a way to escape out of the language. An extreme example would be scoped variables in JavaScript:
(function() {
var x = 5;
myGetter = function() { return x; };
mySetter = function(v) { x = v; };
})();
After that executes, the global namespace contains two functions, myGetter and mySetter, which are the only way to access the value of x. Javascript has no reflective ability to get at x any other way. But it has to run in some kind of host interpreter (e.g. in the browser), and so there is certainly some horrible way to manipulate x. A memory corruption bug in a plugin could do it by accident!

I suppose you could say that. You could also say that the CodeDom, Reflection Emit, and Expression Tree APIs break encapsulation by allowing you to write code on the fly. They are merely facilities provided by .NET. You don't have to use them.
Don't forget that you have to (a) be running in full trust, and (b) explicitly specify BindingFlags.NonPublic, in order to access private data. That should be enough of a safety net.

One can say that reflection breaks inheritance and polymorphism as well. When instead of supplying each object type with its own overloaded method version you have a single method that checks object type at run time and switches to a particular behavior for each.
Reflection is a nice tool nevertheless. Sometimes you need to instantiate an object with a private constructor because it's an optimal course of action and you cannot change the class implementation (closed library, can't get in touch with the author any longer). Or you wish to perform some auxiliary operation on a variety of objects in one place. Or maybe you wish to perform logging for certain types. These are the situation when reflection does help without causing any harm.

That's part of the functionality provided by Reflection, but not, I would say, the best use of it.
It only works under Full Trust.

The encapsulation principle is hold by your API, but reflection gives you a way to work around the API.
Whoever uses reflection knows that he is not using your API correctly!

No, it is not abnormal.
It is something reflection allows you to do, in some situation what you did might be of some use, in many other that simply will make your code a time bomb.
Reflection is a tool, it can be used or abused.
Regarding the question after your edit: the answer is simply no.
An API designer could do his best, putting a lot of effort into exposing a clean interface and see his API overly misused by using reflection.
An engineer can model a perfect washing machine that is secure, doesn't shock you with electricity and so on, but if you smash it with an hammer until you see the cables no one is going to blame the engineer if you get a shock :D

Reflection can help you to keep your code clean. E.g. when you use hibernate you can directly bind the private variables to DB values and you don't have to write unnecessary setter methods that you would not need otherwise. Seen from this point of view, reflection can help you to keep encapsulation.

Advice on C# Expression Trees

I'm working on a method that accepts an expression tree as a parameter, along with a type (or instance) of a class.
The basic idea is that this method will add certain things to a collection that will be used for validation.
public interface ITestInterface
{
//Specify stuff here.
}
private static void DoSomething<T>(Expression<Func<T, object>> expression, params IMyInterface[] rule)
{
// Stuff is done here.
}
The method is called as follows:
class TestClass
{
public int MyProperty { get; set; }
}
class OtherTestClass : ITestInterface
{
// Blah Blah Blah.
}
static void Main(string[] args)
{
DoSomething<TestClass>(t => t.MyProperty,
new OtherTestClass());
}
I'm doing it this way because I'd like for the property names that are passed in to be strong typed.
A couple of things I'm struggling with..
Within DoSomething, I'd like to get a PropertyInfo type (from the body passed in) of T and add it to a collection along with rule[]. Currently, I'm thinking about using expression.Body and removing [propertyname] from "Convert.([propertyname])" and using reflection to get what I need. This seems cumbersome and wrong. Is there a better way?
Is this a specific pattern I'm using?
Lastly, any suggestions or clarifications as to my misunderstanding of what I'm doing are appreciated and / or resources or good info on C# expression trees are appreciated as well.
Thanks!
Ian
Edit:
An example of what expression.Body.ToString() returns within the DoSomething method is a string that contains "Convert(t.MyProperty)" if called from the example above.
I do need it to be strongly typed, so it will not compile if I change a property name.
Thanks for the suggestions!

I rely heavily on expression trees to push a lot of what I want to do with my current application to compile-time, i.e. static type checking.
I traverse expression trees to translate them into something else which "makes sense".
One thing I've ended up doing a lot is that instead of URLs I rely on a MVC like approach where I declare lambda functions, and translates that... interpret, the compiler generated expression tree into an URL. When this URL is invoked, I do the opposite. This way, I have what I call compile-time checks for broken links and this works great with refactoring and overloads as well. I think it's cool to think about using expression trees in this way.
You might wanna check out the visitor pattern, it's a pain to get started with because it doesn't make much sense in the beginning but it ties everything together and it's a very formal way to solve type checking in compiler construction. You could do the same, but instead of type checking emit what ever you need.
Something which I'm currently pounding my head against is the ability to build a simple framework for translating (or actually I should say interpret) expression tress and emit JavaScript. The idea is that the compiler generated expression trees will translate into valid JavaScript which interfaces with some object model.
What's exciting about this is the way the compiler is always able to tell me when I go wrong and sure the end result is just a bunch of strings but the important part is how these strings got created. They went through some verification and that means something.
Once you get that going there is little you can't do with expression trees.
While working with the System.Reflection.Emit stuff I found myself using expression trees to create a light-weight framework for dynamic compilation, which at compile time could basically say if my dynamically created assemblies would compile as well, and this worked seamlessly with reflection and static type checking. It took this further and further and ended up with something which in the end saved a lot of time and proved to be very agile and robust.
So I love this kind of stuff, and this is what meta programming is all about, writing programs in your programs that do programs. I say keep it coming!

Collecting PropertyInfo objects from Expression.Body seems similar to my solution to another question.

I appreciate what you are trying to do with the property here. I have run into this conundrum. It always feels weird to write:
DoSomething("MyProperty", new OtherClass());
If the property ever changes name, or the text is mistyped in the call, then there will be a problem. What I have come to learn is that this is something you probably have to deal with via testing. Specifically, unit testing. I would write unit tests to enforce that the "DoSomething" calls work correctly.
The other thing you might try is to decorate your properties with attributes, and then reflect against your class when it is constructed looking for properties with the attribute, and load rules.
[DoSomething(typeof(OtherClass), typeof(OtherClass2))]
public int MyProperty
{
get;
set;
}
In this case the constructor (perhaps in a base class?) would dynamically create an OtherClass object and a OtherClass2 object, and load them into a collection along with the name of the property.