Develop Reference

How do you access the Profiler API from pure managed C# code?

Background
I am developing a library called Harmony that currently uses detouring at the assembler level to monkey patch methods at runtime. This works fine and I got this to work on all combinations of hardware and .NET but it is sort of an ugly hack that does not work when methods are inlined.
Scope
I know roughly about the profiler API and that you can alter the IL before it reaches the JITer. My library already provides a high level way to get the modified IL body so all I need to do is to use the profiler API.
Asking around it was recommended to write small C/C++ modules accessing it but I rather would want to do this from my managed code. Reasons: my library should only be one final dll, I rather not want to deal with C/C++ code, different environments might make this hard.
Question
Is this approach possible? Is there an easier way? Is there a profiler library that hides this implementation and just gives me a high level replacement callback?
Note
Please do not question the motives of my library, I am fully aware that this falls outside of any “normal” C# programming. It’s mainly for patching and modding games and so far with 1500+ stars on GitHub a very successful project. I just want to lift it to an even more compatible level.

Bestpractice approaches for reverse engineering VB6 code with out knowledge of the domain

target state: Porting VB6 Code into C#, undertake the whole project with all conceivable processes that are involved.
What would be your approach if you do not have knowledge about the domain?
There is nearly any documentation, just legacy code (up to 100.000 - 300.000 lines of code and comments vb6 files that contain up to 14.000 lines of code) written in VB6.

Disclaimer: I work for Great Migrations
We rewrite large VB6/ASP/COM applications to .NET (primarily C#) for a living and we have developed a software analysis and reengineering tool to help us do it. This tool is essentially like a VB6/ASP/COM compiler and a decompiler that authors .NET codes. Of course since the VB6 platform is very different from .NET, a direct compile/decompile is not desirable or viable, so our tool has an "analyzer" that implements various code reengineering algorithms to deal with VB6-C# incompatibilities. There is also a programmable "author" that allows the migration team to prescribe rules for setting up .NET code files, restructuring the code, and doing things like replacing COM APIs and ActiveX controls with .NET classes -- depending on what the team needs or wants.
As a by product of compiling and analyzing the code our tool produces a model of the entire VB6/ASP/COM system being upgraded. This model can be used to produce extremely detailed reports about the internal structure of the system. These models can be used to help reverse-engineer the code -- if you know the right questions to ask and you would need to understand the problem domain to do a good job.
Of course once you have build-complete .NET, you can use the various analytics and code review tools that work off assemblies. Some versions of Visual Studio have these tools and there are open source tools such as FxCop, NDepends). There are also some fantastic dynamic analysis tools (EQUATEC Tracer) that I have used.
In the end though migration teams are going to very hard pressed to verify any unknown system. Even if you are staying on the same platform, you would unable to prove an application it is "correct" if you do not know how to run it and how to setup/enter expected inputs and find/verify expected outputs. We normally leave this to the customer!
If we are doing verification for the customer, we rely heavily on side-by-side testing to validate the new version of your system -- assuming we know how to run the legacy application we assume that given the same sets of inputs and usecases it should exhibit the same behaviors and produce the same results. I have heard this Approval Tests in unit testing circles.
I admit we also rely heavily on the knowledge that the VB6/COM code is a complete, detailed, formal and production tested description of the data structures and logic of the system and that we are putting this information through a tested and retested systematic transformation. We have been developing compilers since 1977 and we have worked very hard on this VB6/ASP compiler to make sure the .NET codes that we generate preserves the semantics of the original VB6. It is not 100% every time - but it is getting closer all the time. Then again doing things by hand does not guarantee 100% correct code on the first try either...

mark's answer about Great Migrations is excellent. Do be aware there are competitor automatic tools, which also have a very good reputation.
Artinsoft's VB Upgrade Companion
Francesco Balena's VB Migration Partner

2 basic but interesting questions about .NET

when I first saw C#, I thought this must be some joke. I was starting with programming in C. But in C# you could just drag and drop objects, and just write event code to them. It was so simple.
Now, I still like C the most, because I am very attracted to the basic low level operations, and C is just next level of assembler, with few basic routines, so I like it very much. Even more because I write little apps for micro-controllers.
But yesterday I wrote very simple control program for my micro-controller based LED cube in asm, and I needed some way to simply create animation sequences to the Cube. So, I remembered C#. I have practically NO C# skills, but still I created simple program to make animation sequences in about hour with GUI, just with help of google and help of the embedded function descriptions in C#.
So, to get to the point, is there some other reason then top speed, to use any other language than C#? I mean, it is so effective. I know that Java is a bit of similar, but I expect C# to be more Windows effective since its directly from Microsoft.
The second question is, what is the advantage of compiling into CIL, and than run by CLR, than directly compile it into machine code? I know that portability is one, but since C# is mainly for Windows, wouldn´t it be more powerful to just compile it directly? Thanks.

1 - diff languages have their pros and cons. There are families of languages (functional, dynamic, static, etc.) which are better for specific problem domains. You'd need to learn one in each family to know when to choose which one. e.g. to write a simple script, I'd pick Ruby over C#
2 - Compiling it to CIL: Portability may not be a big deal.. but to be precise Mono has an implementation of the CLR on Linux. So there. Also CIL helps you to mix-and-match across languages that run on the CLR. e.g. IronRuby can access standard framework libraries written in C#. It also enables the CLR to leverage the actual hardware (e.g. turn on optimizations, use specific instructions) on which the program is run. The CLR on 2 machines would produce the best native code from the same IL for the respective machine.

Language and platform choice are a function of project goal. It sounds like you enjoy system level programming, which is one of the strong points of using C/C++. So, keep writing systems level code if that's what you enjoy.
Writing in C# is strong in rapid business application development where the goals are inherently different. Writing good working code faster is worth money in both man-hours and time to market. Microsoft does us a huge favor with providing an expressive language and a solid framework of functionality that prevents us from having to write low level code or tooling for 95% of business needs.

One important advantage of IL is language independance. You can define modules in project which should be done in C++, some in C# and some in VB.net. All these projects when compiled give respective assemblies(.dll/.exe). This you can use the assembly for C++ project in the c# one and vice versa. This is possible because.. no matter which language (.net supported) you choose.. all compile to the same IL code.

I'm not sure that C# is more effective only because is a Microsoft product. If you use the Visual Studio, or other RAD, some of the code is auto-generated and sometimes is less efficient. Some years ago I was a dogmatic, thinking only C can response all our prayers :-P , but now I think virtual machines can help a lot in the way to optimize code before to execute it (like a RDBMS), storing in caché pieces of code to execute later, etc. Including the possibility to create "clusters" of virtual machines as Terracotta does. At least the benefits of having an extra abstraction layer are bigger that don't have it.

I agree with spoulson. C# is really good at solving business problems. You can very effective create a framework that models your business processes and solve many of those problems with object orientation and design patterns. In that respect it provides much of the nice object oriented capability that C++ has.
If you are concerned with speed, C is the route to go for the reasons that you stated.

Further on the second question: you can run NGEN to generate a native image of the assembly, which can improve performance. Not quite machine code, but since it bypasses the JIT (just-in-time compile) phase, the app will tend to run much faster.
http://msdn.microsoft.com/en-us/library/6t9t5wcf(VS.80).aspx
The Native Image Generator (Ngen.exe)
is a tool that improves the
performance of managed applications.
Ngen.exe creates native images, which
are files containing compiled
processor-specific machine code, and
installs them into the native image
cache on the local computer. The
runtime can use native images from the
cache instead of using the
just-in-time (JIT) compiler to compile
the original assembly.

"is there some other reason then top
speed, to use any other language than
C#?"
I can think of at least four, all somewhat related:
I have a a large current investment in 'language X', and I don't have the time or money to switch to something else. (Port an existing code base, buy/acquire/port libraries, re-develop team skills in C#, learn different tools.)
An anticipated need to port the code to a platform where C# is not supported.
I need to use tools that are not available in C#, or are not as well supported. (IDE's, alternate compilers, code generators, libraries, the list goes on and on...)
I've found a language that's even more productive. ;-)
"what is the advantage of compiling
into CIL, and than run by CLR, than
directly compile it into machine
code?"
It's all about giving the runtime environment more control over the way the code executes. If you compile to machien code, a lot becomes 'set in stone' at that time. Deferring compilation to machine code until you know more about the runtime environment lets you optimize in ways you might not be able to otherwise. Just a few off the top of my head:
Deferring compilation lets you select instructions that more closely match your host CPU. (To use 64-bit native instructions when you have them, or the latest SSE extensions.)
Deferring code lets you optimize in ways you might not be able to otherwise. (If you have only one class at runtime that's derived from a specific interface, you can start to inline even virtual methods, etc.)
Garbage collectors sometimes need to insert checkpoints into user code. Deferring compilation lets the GC have more control and flexibility over how that's done.

First answer: C# should be used by default for new projects. There are a few cases where it hasn't caught up yet to C++ (in terms of multi-paradign support), but it is heading in that direction.
Second answer: "portability" also includes x86 / x64 portability, which can be achieved by setting the platform to AnyCPU. Another (more theoretical at this point) advantage is that the JIT compiler can take advantage of the CPU-specific instruction set and thus optimize more effectively.

Recommended migration strategy for C++ project in Visual Studio 6

For a large application written in C++ using Visual Studio 6, what is the best way to move into the modern era?
I'd like to take an incremental approach where we slowly move portions of the code and write new features into C# for example and compile that into a library or dll that can be referenced from the legacy application.
Is this possible and what is the best way to do it?
Edit: At this point we are limited to the Express editions which I believe don't allow use of the MFC libraries which are heavily used in our current app. It's also quite a large app with a lot of hardware dependencies so I don't think a wholesale migration is in the cards.
Edit2: We've looked into writing COM-wrapped components in C# but having no COM experience this is scary and complicated. Is it possible to generate a C# dll with a straight-C interface with all the managed goodness hidden inside? Or is COM a necessary evil?

I'd like to take an incremental
approach where we slowly move portions
of the code
That's the only realistic way to do it.
First, what kind of version control do you use? (If you use branching version control that allows you to make experiments and see what works, while minimizing the risk of compromising your code; others are OK also, but you'll have to be really careful depending on what you are using).
Edit: I just saw you are using SVN. It may be worthwile to move to mercurial or git if you have the liberty to do that (the change provides a quantum leap in what you can do with the code-base).
and write new features into C# for
example and compile that into a
library or dll that can be referenced
from the legacy application.
That's ... not necessarily a good idea. C# code can expose COM interfaces that are accessible in C++. Writing client code in C++ for modules written in C# can be fun, but you may find it taxing (in terms of effort to benefits ratio); It is also slow and error-prone (compared to writing C# client code for modules written in C++).
Better consider creating an application framework in C# and using modules (already) written in C++ for the core functionality.
Is this possible and what is the best
way to do it?
Yes, it's possible.
How many people are involved in the project?
If there are many, the best way would be to have a few (two? four?) work on the new application framework and have the rest continue as usual.
If there are few, you can consider having either a person in charge of this, or more people working part-time on it.
The percentage of people/effort assigned on each (old code maintenance and new code development) should depend on the size of the team and your priorities (Is the transition a low priority issue? Is it necessary to be finished by a given date?)
The best way to do this would be to start adapting modules of the code to be usable in multiple scenarios (with both the old code and the new one) and continue development in parallel (again, this would be greatly eased by using a branching distributed version control system).
Here's how I would go about it (iterative development, with small steps and lots of validity checks in between):
Pick a functional module (something that is not GUI-related) in the old code-base.
Remove MFC code (and other libraries not available in VS2010 Express - like ATL) references from the module picked in step 1.
Do not attempt to rewrite MFC/ATL functionality with custom code, unless for small changes (that is, it is not feasible to decide to create your own GUI framework, but it is OK to decide to write your own COM interface pointer wrapper similar to ATL's CComPtr).
If the code is heavily dependent on a library, better separate it as much as possible, then mark it down to be rewritten at a future point using new technologies. Either way, for a library heavily-dependent on MFC you're better off rewriting the code using something else (C#?).
reduce coupling with the chosen module as much as possible (make sure the code is in a separate library, decide clearly what functionality the module exposes to client code) and access the delimited functionality only through the decided exposed interface (in the old code).
Make sure the old code base still works with the modified module (test - eventually automate the testing for this module) - this is critical if you need to still stay in the market until you can ship the new version.
While maintaining the current application, start a new project (C# based?) that implements the GUI and other parts you need to modernize (like the parts heavily-dependent on MFC). This should be a thin-layer application, preferably agnostic of the business logic (which should remain in the legacy code as much as possible).
Depending on what the old code does and the interfaces you define, it may make sense to use C++/CLI instead of C# for parts of the code (it can work with native C++ pointers and managed code, allowing you to make an easy transition when comunicating between managed .NET code and C++ native code).
Make the new application use the module picked in step 1.
Pick a new module, go back to step 2.
Advantages:
refactoring will be performed (necessary for the separation of modules)
at the end you should have a battery of tests for your functional modules (if you do not already).
you still have something to ship in between.
A few notes:
If you do not use a distributed branching version control system, you're better off working on one module at a time. If you use branching/distributed source control, you can distribute different modules to different team members, and centralize the changes every time something new has been ported.
It is very important that each step is clearly delimited (so that you can roll back your changes to the last stable version, try new things and so on). This is another issue that is difficult with SVN and easy with Mercurial / Git.
Before starting, change the names of all your project files to have a .2005.vcproj extension, and do the same for the solution file. When creating the new project file, do the same with .2010.vcxproj for the project files and solution (you should still do this if you convert the solutions/projects). The idea is that you should have both in parallel and open whichever you want at any point. You shouldn't have to make a source-tree update to a different label/tag/date in source control just to switch IDEs.
Edit2: We've looked into writing
COM-wrapped components in C# but
having no COM experience this is scary
and complicated.
You can still do it, by writing wrapper code (a small templated smart pointer class for COM interfaces wouldn't go amiss for example - similar to CComPtr in ATL). If you isolated the COM code behind some wrappers you could write client code (agnostic of COM) with (almost) no problems.
Is it possible to generate a C# dll
with a straight-C interface with all
the managed goodness hidden inside? Or
is COM a necessary evil?
Not that I know of. I think COM will be a necessary evil if you plan to use server code written in C# and client code in C++.
It is possible the other way around.

Faced with the same task, my strategy would be something like:
Identify what we hope to gain by moving to 2010 development - it could be
improved quality assurance: unit testing, mocking are part of modern development tools
slicker UI: WPF provides a modern look and feel.
productivity: in some areas, .NET development is more productive than C++ development
support: new tools are supported with improvements and bugfixes.
Identify which parts of the system will not gain from being moved to C#:
hardware access, low-level algorithmic code
pretty much most bespoke non-UI working code - no point throwing it out if it already works
Identify which parts of the system need to be migrated to c#. For these parts, ensure that the current implementation in C++ is decoupled and modular so that those parts can be swapped out. If the app is a monolith, then considerable work will be needed refactoring the app so that it can be broken up and select pieces reimplemented in c#. (It is possible to refactor nothing, instead just focus on implementing new application functionality in c#.)
Now that you've identified which parts will remain in C++ and which parts will be implemented in c#, (or just stipulate that new features are in c#) then focus turns to how to integrate c# and c++ into a single solution
use COM wrappers - if your existing C++ project makes good use of OO, this is often not as difficult as it may seem. With MSVC 6 you can use the ATL classes to expose your classes as COM components.
Integrate directly the native and c# code. Integrating "legacy" compiled code requires an intermediate DLL - see here for details.
Mixing the MFC UI and c# UI is probably not achieveable, and not adviseable either as it would produce a UI mix of two distinct styles (1990s grey and 2010 vibe). It is simpler to focus on achieving incremental migration, such as implementing new application code in c# and calling that from the native C++ code. This keeps the amount of migrated c# code small to begin with. As you get more into the 2010 development, you can then take the larger chunks that cannot be migrated incrementally, such as the UI.

First, your definition of modern era is controversial. There's no reason to assume C# is better in any sense than C++. A lot has been said on whether C# helps you better avoid memory management errors, but this is hardly so with modern facilities in C++, and, it's very easy to do mess with C# in terms of resource acquisition timing, that may be dependent on what other programs are doing.

If you move straight from 6 to 2010 you may end up with some messed up project settings. If this isn't a fairly large project, and it's one of few that you need to convert, then that should be fine. Just open it in 2010, and follow the conversion wizard. Make sure to back up your project first, and verify your project settings when you're done.
In my opinion though the best way is to convert it step by step through each iteration of Visual Studio. I had to modernize 1400 projects from 2003 to 2010, and the best way that I found was to convert everything to 2005, then to 2008, and then finally to 2010. This caused the least amount of issues to arise for me.
If you only have 6 and the newest Visual Studio you may end up just having to try and go straight to the new one using the wizard. Expect some manual cleanup before everything builds correctly for you again.
Also, one more time, BACK IT UP FIRST! :)

High-level C++ code calling low-level C# code doesn't look like a good idea. The areas where .NET languages are better, are user interface, database access, networking, XML files handling. Low-level stuff like calculations, hardware access etc. is better to keep as native C++ code.
Moving to .NET, in most cases it is better to rewrite UI completely, using WPF or Windows Forms technologies. Low-level stuff remains native, and different interoperability technologies are used to connect C# and native code: PInvoke, C++/CLI wrappers or COM interoperability. After some time, you may decide to rewrite low-level native components in C#, only if it is really necessary.
About compiling native C++ code in VS2010 - I don't see any problems. Just fix all compilation errors - new compilers have more strict type checking and syntax restrictions, and catch much more bugs at compilation time.

Not sure why so many folks are advocating for COM. If you haven't already got a lot of COM in there, learning how to do it on the C++ side is going to hurt, and then you're using the slowest possible interop from the managed side. Not my first choice.
Ideally you have refactored your UI from your business logic. You can then build a new UI (WPF, WinForms, ASP.NET, web services that support some other client, whatever) and call into your business logic through P/Invoke or by writing a C++/CLI wrapper. #mdma has good advice for you assuming that the refactoring is possible.
However if you were paying me to come in and help you my very first question would be why do you want to do this? Some clients say they don't want to pay C++ devs any more, so they want all the C++ code gone. This is a scary objective because we all hate to touch code that works. Some clients want to expose their logic to ASP.NET or Reporting Services or something, so for them we concentrate on the refactoring. And some say "it looks so 1999" and for them I show them what MFC looks like now. Colours, skinning/theming including office and win7 looks, ribbon, floating/docking panes and windows, Windows 7 taskbar integration ... if you just want to look different, take a look at MFC in VS 2010 and you might not have to adjust any code at all.
Finally to make non-Express versions of VS 2010 affordable look into the Microsoft Partner Program. If you have sold your software to at least 3 customers who still speak to you, and can get through the Windows 7 logo self test (I have got VB 6 apps through that in a day or two) then you can have 5-10 copies of everything (Windows, Office, VS) for $1900 or so a year, depending on where you live.

To start I'd try and keep as much code as possible to avoid a rewrite. I'd also remove all unused code before starting the conversion.
Since VC++ 6.0 Microsoft changed the MFC libraries and the C++ Standard Library.
I recommend to start building your DLLs with no dependencies, then looking at your third party libraries, and then rebuild one dependent DLL/EXE at a time.
Introduce unit tests to make sure the behaviour of code does not change.
If you have a mixed build, using different versions of VC++, you need to guard against passing resources (file handles) between DLLs that use different versions of the VC runtime.

If at all financially possible I would strongly consider just paying the money for the version of Visual Studio that you need because you could very well lose more money on the time you spend. I do not know enough about the express editions to give a good answer on them but when integrating some code from a subcontractor that was written in C++ I used C++ / CLI. You will probably be able to reuse most of your codebase and will be familiar with the language but you will also have access to managed code and libraries. Also if you want to start writing new code in C# you can do that. The biggest problem I had with it was that in VS 2010 there is no intellisense in C++ / CLI.

Visual Studio 6 is legendary for being buggy and slow. Moving into the modern era would best be done by getting a new compiler. What is probably the easiest thing to do is write the legacy app into a DLL, then write your exe into C# and use P/Invoke. Then you never have to touch the old code again- you can just write more and more in C# and use less and less of the old DLL.
If your old code is very heavily OO, you can use C++/CLI to write wrapper classes that allow .NET to call methods on C++ objects, and collect them too if you use a reference counted smart pointer.

You can use C# to write your new components with a COM or COM+ (System.EnterpriseServices) wrapper, which will be callable from your existing C++ code.

IronPython vs. C# for small-scale projects

I currently use Python for most of my programming projects (mainly rapid development of small programs and prototypes). I'd like to invest time in learning a language that gives me the flexibility to use various Microsoft tools and APIs whenever the opportunity arises. I'm trying to decide between IronPython and C#. Since Python is my favorite programming language (mainly because of its conciseness and clean syntax), IronPython sounds like the ideal option. Yet after reading about it a little bit I have several questions.
For those of you who have used IronPython, does it ever become unclear where classic Python ends and .NET begins? For example, there appears to be significant overlap in functionality between the .NET libraries and the Python standard library, so when I need to do string operations or parse XML, I'm unclear which library I'm supposed to use. Also, I'm unclear when I'm supposed to use Python versus .NET data types in my code. For example, which of the following would I be using in my code?
d = {}
or
d = System.Collections.Hashtable()
(By the way, it seems that if I do a lot of things like the latter I might lose some of the conciseness, which is why I favor Python in the first place.)
Another issue is that a number of Microsoft's developer tools, such as .NET CF and Xbox XNA, are not available in IronPython. Are there more situations where IronPython wouldn't give me the full reach of C#?

I've built a large-scale application in IronPython bound with C#.
It's almost completely seamless. The only things missing in IronPython from the true "python" feel are the C-based libraries (gotta use .NET for those) and IDLE.
The language interacts with other .NET languages like a dream... Specifically if you embed the interpreter and bind variables by reference.
By the way, a hash in IronPython is declared:
d = {}
Just be aware that it's actually an IronPython.Dict object, and not a C# dictionary. That said, the conversions often work invisibly if you pass it to a .NET class, and if you need to convert explicitly, there are built-ins that do it just fine.
All in all, an awesome language to use with .NET, if you have reason to.
Just a word of advice: Avoid the Visual Studio IronPython IDE like the plague. I found the automatic line completions screwed up on indentation, between spaces and tabs. Now -that- is a difficult-to-trace bug inserted into code.

I'd suggest taking a look at Boo [http://boo.codehaus.org/], a .NET-based language with a syntax inspired by Python, but which provides the full range of .NET 3.5 functionality.
IronPython is great for using .NET-centric libraries -- but it isn't well-suited to creating them due to underlying differences in how the languages do typing. As Boo does inference-based typing at compile time except where duck typing is explicitly requested (or a specific type is given by the user), it lets you build .NET-centric libraries easily usable from C# (and other languages') code, which IronPython isn't suitable for; also, as it has to do less introspection at runtime, Boo compiles to faster code.