Develop Reference

Where function passed as UnmanagedFunctionPointer to C is executed?

In C, there is a function that accepts a pointer to a function to perform comparison:
[DllImport("mylibrary.dll", CallingConvention = CallingConvention.Cdecl)]
private static extern int set_compare(IntPtr id, MarshalAs(UnmanagedType.FunctionPtr)]CompareFunction cmp);
In C#, a delegate is passed to the C function:
[UnmanagedFunctionPointer(CallingConvention.Cdecl)]
delegate int CompareFunction(ref IntPtr left, ref IntPtr right);
Currently, I accept Func<T,T,int> comparer in a constructor of a generic class and convert it to the delegate. "mylibrary.dll" owns data, managed C# library knows how to convert pointers to T and then compare Ts.
//.in ctor
CompareFunction cmpFunc = (ref IntPtr left, ref IntPtr right) => {
var l = GenericFromPointer<T>(left);
var r = GenericFromPointer<T>(right);
return comparer(l, r);
};
I also have an option to write a CompareFunction in C for most important data types that are used in 90%+ cases, but I hope to avoid modifications to the native library.
The question is, when setting the compare function with P/Invoke, does every subsequent call to that function from C code incurs marshaling overheads, or the delegate is called from C as if it was initially written in C?
I imagine that, when compiled, the delegate is a sequence of machine instructions in memory, but do not understand if/why C code would need to ask .NET to make the actual comparison, instead of just executing these instructions in place?
I am mostly interested in better understanding how interop works. However, this delegate is used for binary search on big data sets, and if every subsequent call has some overheads as a single P/Invoke, rewriting comparers in native C could be a good option.

I imagine that, when compiled, the delegate is a sequence of machine instructions in memory, but do not understand if/why C code would need to ask .NET to make the actual comparison, instead of just executing these instructions in place?
I guess you're a bit confused about how .NET works. C doesn't ask .NET to execute code.
First, your lambda is turned into a compiler-generated class instance (because you're closing over the comparer variable), and then a delegate to a method of this class is used. And it's an instance method since your lambda is a closure.
A delegate is similar to a function pointer. So, like you say, it points to executable code. Whether this code is generated from a C source or a .NET source is irrelevant at this point.
It's in the interop case when this starts to matter. P/Invoke won't pass your delegate as-is as a function pointer to C code. It will pass a function pointer to a thunk which calls the delegate. Visual Studio will display this as a [Native to Managed Transition] stack frame. This is needed for different reasons such as marshaling or passing additional parameters (like the instance of the class backing your lambda for instance).
As to the performance considerations of this, here's what MSDN says, quite obviously:
Thunking. Regardless of the interoperability technique used, special transition sequences, which are known as thunks, are required each time a managed function calls an native function, and vice-versa. Because thunking contributes to the overall time that it takes to interoperate between managed code and native code, the accumulation of these transitions can negatively affect performance.
So, if your code requires a lot of transitions between managed and native code, you should get better performance by doing your comparisons on the C side if possible, so that you avoid the transitions.

What is the difference between a Function Template and a Delegate?

I have arrived at a point in my self-taught studies where I am not fully grasping what a delegate in C# is useful for. Additionally, on a whim, I decided to take a look at an intro to C++ site and I came across function templates.
Maybe I'm comparing apples and oranges here, but I understood a delegate to be a sort of template for a function that would be defined at run-time. Is this true? If so, how does that differ from a function template in C++?
Is it possible to see (realistic) examples of each in use?

A delegate is a way of taking a member function of some object, and creating a...thing that can be called independently.
In other words, if you have some object A, with some member function F, that you'd normally call as something like: A.F(1);, a delegate is a single entity that you can (for example) pass as a parameter, that acts as a proxy for that object/member function, so when the delegate is invoked, it's equivalent to invoking that member function of that object.
It's a way of taking existing code, and...packaging it to make it easier to use in a fairly specific way (though I feel obliged to add, that 'way' is quite versatile so delegates can be extremely useful).
A C++ function template is a way of generating functions. It specifies some set of actions to take, but does not specify the specific type of object on which those actions will happen. The specification is at a syntactic level, so (for example) I can specify adding two things together to get a third item that's their sum. If I apply that to numbers, it sums like you'd expect. If I do the same with strings, it'll typically concatenate the strings. This is because (syntactically) the template just specifies something like a+b, but + is defined to do addition of numbers, and concatenation of strings.
Looked at slightly differently, a function template just specifies the skeleton for some code. The rest of that code's body is "filled in" based on the type, when you instantiate the template over some specific type.

In C++ terms a C# delegate combines an object pointer and a member function pointer into one callable entity, which calls the member function on the pointed to object.
You can do about the same with std::bind and std::function.
Before C++11 there was a short flurry of articles on how to do very efficient delegates in C++. std::function is a very reasonable compromise and may even attain those levels of efficiency.
Example:
#include <iostream>
#include <functional>
using namespace std;
// Here `function<void()>` serves as a "delegate" type.
void callback_on( function<void()> const f )
{
f();
}
struct S
{
int x;
void foo() const { cout << x << endl; }
};
int main()
{
S o = {42};
callback_on( bind( &S::foo, &o ) );
}

Writing a managed wrapper for unmanaged (C++) code - custom types/structs

faacEncConfigurationPtr FAACAPI faacEncGetCurrentConfiguration(
faacEncHandle hEncoder);
I'm trying to come up with a simple wrapper for this C++ library; I've never done more than very simple p/invoke interop before - like one function call with primitive arguments.
So, given the above C++ function, for example, what should I do to deal with the return type, and parameter?
FAACAPI is defined as: #define FAACAPI __stdcall
faacEncConfigurationPtr is defined:
typedef struct faacEncConfiguration
{
int version;
char *name;
char *copyright;
unsigned int mpegVersion;
unsigned long bitRate;
unsigned int inputFormat;
int shortctl;
psymodellist_t *psymodellist;
int channel_map[64];
} faacEncConfiguration, *faacEncConfigurationPtr;
AFAIK this means that the return type of the function is a reference to this struct?
And faacEncHandle is:
typedef struct {
unsigned int numChannels;
unsigned long sampleRate;
...
SR_INFO *srInfo;
double *sampleBuff[MAX_CHANNELS];
...
double *freqBuff[MAX_CHANNELS];
double *overlapBuff[MAX_CHANNELS];
double *msSpectrum[MAX_CHANNELS];
CoderInfo coderInfo[MAX_CHANNELS];
ChannelInfo channelInfo[MAX_CHANNELS];
PsyInfo psyInfo[MAX_CHANNELS];
GlobalPsyInfo gpsyInfo;
faacEncConfiguration config;
psymodel_t *psymodel;
/* quantizer specific config */
AACQuantCfg aacquantCfg;
/* FFT Tables */
FFT_Tables fft_tables;
int bitDiff;
} faacEncStruct, *faacEncHandle;
So within that struct we see a lot of other types... hmm.
Essentially, I'm trying to figure out how to deal with these types in my managed wrapper?
Do I need to create versions of these types/structs, in C#? Something like this:
[StructLayout(LayoutKind.Sequential)]
struct faacEncConfiguration
{
uint useTns;
ulong bitRate;
...
}
If so then can the runtime automatically "map" these objects onto eachother?
And, would I have to create these "mapped" types for all the types in these return types/parameter type hierarchies, all the way down until I get to all primitives?
I know this is a broad topic, any advice on getting up-to-speed quickly on what I need to learn to make this happen would be very much appreciated! Thanks!

Your are on the right track with how you would need to create managed structures that represent unamanged structures for use with P/Invoke.
It is however not the best strategy for interop with unmanaged libraries, because using this API from C# would still feel like using a C API - create and initialise a structure, pass it to a function and get some other structure in return.
It is OK to use P/Invoke for an odd function call that is not otherwise exposed as .NET API, but for a full blown API wrapper I would highly recommend using managed C++(C++/CLI). It is absolutely unparalleled in creating unmanaged interop layers for .NET.
The biggest challenge would be to convert this essentially C interface into an object orientated interface where you call methods off objects, as opposed to calling global functions with structures of all-public members.
As you start writing elaborate structure graphs for P/Invoke you would yourself having to deal with quite a bit of "magic" that governs how managed primitive types convert to unmanaged types. Often, using incorrect types will cause a run-time error.
With managed C++ (IJW - It Just Works) you define managed structures, classes, interfaces in C++, which allows more natural use of the underlying library and a more native interface to your C# application.
This is a great overview of C++/CLI. Also MSDN has extensive documentation for all managed C++ features.

Yes, you would need to declare all these structures in c#. Be careful to declare members with correct sizes. For example, 'long' is 32-bit in C++, but 64-bit in C#. For a pointer or void* in C++, use IntPtr in C#, etc.

C++: Syntax for map where Data type is function?

In C#, what I want would look something like this:
IDictionary<string, action()> dict = new Dictionary<string, action()>();
How do I do this in C++? This gives compiler errors:
map<string, void()> exercises;

Use boost::function, a polymorphous wrapper for any object that can be called with your signature (including functions, function objects etc).
map<string, function<void()>> ...;
Note that the new C++ standard has already included function in <functional>.
To explain the backgrounds: The only builtin mechanism of this kind in C++ are old C-style function pointers (void (*)()). These are extremely low-level, basically just storing the memory address of a function, and therefore far from even coming close to the power of C#'s delegates.
You can't create anonymous functions, neither can you refer to a particular object's member functions or any variables or data (closures).
Thus, one often utilizes so called functors which are classes that mimic the behaviour of a function by overloading the operator (). In combination with templates, they are used whereever ordinary function pointers can be used.
The problem is that these functors often consist of peculiar or anonymous types that can't be referred to conveniently.
The function is the coolest way to address them all - Everything that behaves like a function, including cool new lambda-style expressions.
function<void()> f = []() { cout << "Hello, World"> };
f();

In C++ function pointers need to be used. Using your C++ example, you could inline it as:
map<string, void(*)()> exercises;
But it may be more useful to the casual observer to do it in two lines:
typedef void (*pointer_to_void_function)();
map<string, pointer_to_void_function> exercises;
(Choose names to suit.)
Note, that it's probably even easier to casual observers to add another typedef...
typedef void (*pointer_to_void_function)();
typedef map<string, pointer_to_void_function> ExerciseMapping;
ExerciseMapping exercises;
Then, when you want to use iterators or whatever, you just type ExerciseMapping::iterator instead of needing to type the map... stuff. (This also makes it easier to change your implementation later.)

Try using a function object. For example, you can use boost::function.

The Benefits of Using Function Pointers

I have been programming for a few years now and have used function pointers in certain cases. What I would like to know is when is it appropriate or not to use them for performance reasons and I mean in the context of games, not business software.
Function pointers are fast, John Carmack used them to the extent of abuse in the Quake and Doom source code and because he is a genius :)
I would like to use function pointers more but I want to use them where they are most appropriate.
These days what are the best and most practical uses of function pointers in modern c-style languages such as C, C++, C# and Java, etc?

There is nothing especially "fast" about function pointers. They allow you to call a function which is specified at runtime. But you have exactly the same overhead as you'd get from any other function call (plus the additional pointer indirection). Further, since the function to call is determined at runtime, the compiler can typically not inline the function call as it could anywhere else. As such, function pointers may in some cases add up to be significantly slower than a regular function call.
Function pointers have nothing to do with performance, and should never be used to gain performance.
Instead, they are a very slight nod to the functional programming paradigm, in that they allow you to pass a function around as parameter or return value in another function.
A simple example is a generic sorting function. It has to have some way to compare two elements in order to determine how they should be sorted. This could be a function pointer passed to the sort function, and in fact c++'s std::sort() can be used exactly like that. If you ask it to sort sequences of a type that does not define the less than operator, you have to pass in a function pointer it can call to perform the comparison.
And this leads us nicely to a superior alternative. In C++, you're not limited to function pointers. You often use functors instead - that is, classes that overload the operator (), so that they can be "called" as if they were functions. Functors have a couple of big advantages over function pointers:
They offer more flexibility: they're full-fledged classes, with constructor, destructor and member variables. They can maintain state, and they may expose other member functions that the surrounding code can call.
They are faster: unlike function pointers, whose type only encode the signature of the function (a variable of type void (*)(int) may be any function which takes an int and returns void. We can't know which one), a functor's type encodes the precise function that should be called (Since a functor is a class, call it C, we know that the function to call is, and will always be, C::operator()). And this means the compiler can inline the function call. That's the magic that makes the generic std::sort just as fast as your hand-coded sorting function designed specifically for your datatype. The compiler can eliminate all the overhead of calling a user-defined function.
They are safer: There's very little type safety in a function pointer. You have no guarantee that it points to a valid function. It could be NULL. And most of the problems with pointers apply to function pointers as well. They're dangerous and error-prone.
Function pointers (in C) or functors (in C++) or delegates (in C#) all solve the same problem, with different levels of elegance and flexibility: They allow you to treat functions as first-class values, passing them around as you would any other variable. You can pass a function to another function, and it will call your function at specified times (when a timer expires, when the window needs redrawing, or when it needs to compare two elements in your array)
As far as I know (and I could be wrong, because I haven't worked with Java for ages), Java doesn't have a direct equivalent. Instead, you have to create a class, which implements an interface, and defines a function (call it Execute(), for example). And then instead of calling the user-supplied function (in the shape of a function pointer, functor or delegate), you call foo.Execute(). Similar to the C++ implementation in principle, but without the generality of C++ templates, and without the function syntax that allows you to treat function pointers and functors the same way.
So that is where you use function pointers: When more sophisticated alternatives are not available (i.e. you are stuck in C), and you need to pass one function to another. The most common scenario is a callback. You define a function F that you want the system to call when X happens. So you create a function pointer pointing to F, and pass that to the system in question.
So really, forget about John Carmack and don't assume that anything you see in his code will magically make your code better if you copy it. He used function pointers because the games you mention were written in C, where superior alternatives are not available, and not because they are some magical ingredient whose mere existence makes code run faster.

They can be useful if you do not know the functionality supported by your target platform until run-time (e.g. CPU functionality, available memory). The obvious solution is to write functions like this:
int MyFunc()
{
if(SomeFunctionalityCheck())
{
...
}
else
{
...
}
}
If this function is called deep inside of important loops then its probably better to use a function pointer for MyFunc:
int (*MyFunc)() = MyFunc_Default;
int MyFunc_SomeFunctionality()
{
// if(SomeFunctionalityCheck())
..
}
int MyFunc_Default()
{
// else
...
}
int MyFuncInit()
{
if(SomeFunctionalityCheck()) MyFunc = MyFunc_SomeFunctionality;
}
There are other uses of course, like callback functions, executing byte code from memory or for creating an interpreted language.
To execute Intel compatible byte code on Windows, which might be useful for an interpreter. For example, here is an stdcall function returning 42 (0x2A) stored in an array which can be executed:
code = static_cast<unsigned char*>(VirtualAlloc(0, 6, MEM_COMMIT | MEM_RESERVE, PAGE_EXECUTE_READWRITE));
// mov eax, 42
code[0] = 0x8b;
code[1] = 0x2a;
code[2] = 0x00;
code[3] = 0x00;
code[4] = 0x00;
// ret
code[5] = 0xc3;
// this line executes the code in the byte array
reinterpret_cast<unsigned int (_stdcall *)()>(code)();
...
VirtualFree(code, 6, MEM_RELEASE);
);

As per my personal experience they can can help you save significant lines of code.
Consider the condition:
switch(sample_var)
{
case 0:
func1(<parameters>);
break;
case 1:
func2(<parameters>);
break;
up to case n:
funcn(<parameters>);
break;
}
where func1() ... funcn() are functions with same prototype.
What we could do is:
Declare an array of function pointers arrFuncPoint containing the addresses of functions
func1() to funcn()
Then the whole switch case would be replaced by
*arrFuncPoint[sample_var];

Any time you use a event handler or delegate in C#, you are effectively using a function pointer.
And no, they are not about speed. Function pointers are about convenience.
Jonathan

Function pointers are used as callbacks in many cases. One use is as a comparison function in sorting algorithms. So if you are trying to compare customized objects, you can provide a function pointer to the comparison function that knows how to handle that data.
That said, I'll provide a quote I got from a former professor of mine:
Treat a new C++ feature like you would treat a loaded automatic weapon in a crowded room: never use it just because it looks nifty. Wait until you understand the consequences, don't get cute, write what you know, and know what you write.

These days what are the best and most practical uses of integers in modern c-style languages?

In the dim, dark ages before C++, there was a common pattern I used in my code which was to define a struct with a set of function pointers that (typically) operated on that struct in some way and provided particular behaviors for it. In C++ terms, I was just building a vtable. The difference was that I could side-effect the struct at runtime to change behaviors of individual objects on the fly as needed. This offers a much richer model of inheritance at the cost of stability and ease of debugging. The greatest cost, however, was that there was exactly one person who could write this code effectively: me.
I used this heavily in a UI framework that let me change the way objects got painted, who was the target of commands, and so on, on the fly - something that very few UIs offered.
Having this process formalized in OO languages is better in every meaningful way.

Just speaking of C#, but function pointers are used all over C#. Delegates and Events (and Lambdas, etc) are all function pointers under the hood, so nearly any C# project is going to be riddled with function pointers. Basically every event handler, near every LINQ query, etc - will be using function pointers.

There are occasions when using function pointers can speed up processing. Simple dispatch tables can be used instead of long switch statements or if-then-else sequences.

Function pointers are a poor man's attempt to be functional. You could even make an argument that having function pointers makes a language functional, since you can write higher order functions with them.
Without closures and easy syntax, they're sorta gross. So you tend to use them far less than desireable. Mainly for "callback" functions.
Sometimes, OO design works around using functions by instead creating a whole interface type to pass in the function needed.
C# has closures, so function pointers (which actually store an object so it's not just a raw function, but typed state too) are vastly more usable there.
Edit
One of the comments said there should be a demonstration of higher order functions with function pointers. Any function taking a callback function is a higher order function. Like, say, EnumWindows:
BOOL EnumWindows(
WNDENUMPROC lpEnumFunc,
LPARAM lParam
);
First parameter is the function to pass in, easy enough. But since there are no closures in C, we get this lovely second parameter: "Specifies an application-defined value to be passed to the callback function." That app-defined value allows you to manually pass around untyped state to compensate for lack of closures.
The .NET framework is also filled with similar designs. For instance, IAsyncResult.AsyncState: "Gets a user-defined object that qualifies or contains information about an asynchronous operation." Since the IAR is all you get on your callback, without closures, you need a way to shove some data into the async op so you can cast it out later.

Function pointers are fast
In what context? Compared to?
It sounds like you just want to use function pointers for the sake of using them. That would be bad.
A pointer to a function is normally used as a callback or event handler.