writing a DSL for the .Net platform - c#

I am thinking of writing a DSL to be run in the .Net environment. I am most likely to implement this in C#, though I am flexible on this.
Is there a recommended online resources that show the main steps involved in writing a DSL for the .Net platform?
Ideally, I would like a resource that would provide at least an overview on the following:
'Spec'ing a DSL
How to map the specs to the .Net framework
Preferably a helloworld example of a trivial DSL implemented in a .Net language
[Edit]
Actually, I have just seen this article - but it is slightly dated. Does anyone have a view on whether the article is a good starting point or not (the .Net framework and C# seem to evolve at a very rapid pace)

If you are willing to buy a book on the topic, I highly recommend "DSLs in Boo: Domain Specific Languages in .NET" by Ayende Rahien. Very informative and gently takes you through the process of writing a DSL. The author uses a lightweight .NET language called Boo to serve as basis for the DSL's syntax.
Also you can look into VS2012 corner:
Microsoft Visual Studio 2012 Visualization & Modeling SDK
Microsoft Visual Studio 2010 Visualization & Modeling SDK

There's a bunch of different solutions you could use, including the article you linked, but some other examples from MS...
FsLex/FsYacc - Ports of the popular Lex and Yacc lexer/parsers for F#, but don't be turned off right away. If you've not used it before, F# has a feature called "pattern matching", which allows you to match very complex constructs (such as walking a tree), without an extensive amount of if/else/or blocks all over. This is perfectly suited to language compiling - because almost all DSL solutions you will find will work by parsing the language into an AST (Abstract syntax tree). In this F# solution, you get a strongly typed tree to work with. You can grab the F# Parsed Language Started to get you going. (There's plenty of existing grammars for Lex/Yacc that can help you out too).
SQL Server Modeling Tools (formerly "Oslo") - Contains a language called M, formerly broken into several parts, one being MGrammar. It's quite an advanced parser and can save you plenty of time over other grammars - code patterns (or generic grammar rules) and precedence are built in and easy to use. I would perhaps recommend this if you're starting out with parsing, because it comes with a visual tool - Intellipad, which has a 3-panel DSL mode, where you type in your language and some example code - and it'll show you the output AST as you type - it's quite productive to use. The generated AST is a valid M language constructor (MGraph), which can be used with services like SQL and XML. A downside to MGrammar IMO, is that walking the AST from C# or elsewhere is a tiresome process. There's nothing strongly typed, and you're working with objects and searching with strings - awkward and easy to make mistakes. There's some samples on msdn, and some vids on channel9 which can help you get started like this lengthy overview
The Visualization and Modeling SDK - An entire solution built into VS, which focuses largely on building your with Visual Studio's design tools over code. It comes with a minimum language starter template to help you. Haven't any experience with this to recommend it.
There's plenty of other non-MS solutions, like the one you've mentioned, C# targets for ANTLR etc. These are particularly useful if you're re-using existing grammars - because there's dozens already out there.

You could try JetBrains' MPS. It is a very rich and robust ecosystem for generating DSLs. I've never used it myself, so caveat emptor, but it's free so I guess it can't hurt (much) to give it a go.

Check out my open source project meta#. It sounds like what you are looking for.

Related

JSIL vs Script# vs SharpKit

I'm looking at Script#, JSIL and SharpKit as a tool to use to compile C# to Javascript, so I can program the client side functions of AJAX using C# in Visual Studio.
What are the pros and cons of each JSIL, Script# and SharpKit?
My project is a MVC4 project using razor engine and C#, if it matters.
If you're looking to integrate directly with an MVC project, something like Script# or SharpKit or something is probably your best bet - I know for a fact that Script# has stuff built in to make that sort of integration easier, so I would start there.
If you do want to try using JSIL, it probably has the core features you need, but things that you might want - like visual studio integration, automated deployment, etc - are not there. At present it is primarily targeted at cross-compilation of applications, so it does a good job of that but not as good a job of other use cases.
I'll try to give a summary of reasons why you might want to consider JSIL over those other alternatives - I can't really comment on the pros and cons of those alternatives in depth since I haven't used them:
JSIL has extremely wide support for the features available in C# 4. Notable ones (either because other tools don't support them, or they're complicated) include:
dynamic, yield, Structs, ref / out, Delegates, Generics, Nullables, Interfaces, and Enums.
Some of the above, of course, don't have complete support - to get an idea of things that absolutely will work, you can look at the test cases - each one is a small self-contained .cs file that is tested to ensure that JSIL and native C# produce the same output.
The reason for this extensive support is that my goal is for JSIL to enable you to translate a completely unmodified C# application to working JS. For all the demos up on the JSIL site, this is true, and I have a few nearly finished ports of larger real games in the wings for which this is also true.
Another reason is that JSIL makes it relatively straightforward for your C# and your JavaScript to talk.
All your C# types and methods are exposed via an interface that is as javascript-friendly as possible. The JS versions have basic overload resolution and dispatch so that native C# interfaces are callable from script code as if they were native JS in most cases. You don't have to take any steps to specifically tag methods you wish to expose to JS, or give them special names, or anything like that unless you want to.
When you want to call out from C# to JS, you can do it a few ways:
JSIL.Verbatim.Expression lets you insert raw javascript directly into the translated version of a function.
JSIL.Builtins.Global can be combined with dynamic and var to write JavaScript-like code directly in your C# function bodies.
The JSReplacement attribute can be used to replace invocations of a C# function with a parameterized JavaScript expression.
All of the above features can be combined with JSIL's mechanism for altering type information, called Proxies, to allow you to alter the type information of libraries you use, even if you don't have source code, in order to map their methods to JavaScript you've written.
And finally, C# methods that aren't translated to JS produce an empty method called an External that you can then replace with JavaScript at runtime to make it work again. Any External methods that you haven't replaced produce clear warning message at runtimes so you know what's missing.
JSIL makes aggressive use of type information, along with metadata you provide, to try and safely optimize the JavaScript it generates for you. In some cases this can produce better equivalent JavaScript than you would have written by hand - the main area where this is true at present is code that uses structs, but it also can apply in other cases.
For example, in this code snippet, JSIL is able to statically determine that despite the number of struct copies implied by the code, none of the copies are actually necessary for the code to behave correctly. The resulting JavaScript ends up not having any unnecessary copies, so it runs much faster than what you'd get if you naively translated the semantics of the original C#. This is a nice middle ground between writing the naive struct-based thing (Vector2s everywhere!) and going completely nuts with named return value optimization by hand, which, as I've described in the past, is pretty error-prone.
Okay, now for some downsides. Don't consider this list exhaustive:
Large portions of the .NET BCL don't have implementations provided for you by JSIL. In the future this may be addressed by translating the entire Mono mscorlib to JavaScript, but I don't have that working well enough to advocate it as an immediate solution. (This is fine for games so far, since they don't use much of the BCL.) This issue is primarily due to the IP problems related to translating Microsoft's mscorlib - if I could do that legally, I'd be doing it right now - it worked the last time I tested it.
As mentioned above, no visual studio integration. JSIL is pretty easy to use - you can feed it a .sln file to get a bunch of .js outputs automatically, and configure it automatically with a configuration file next to the project - but it's nowhere near as polished or integrated as say, Script#.
No vendor or support staff. If you want a bug fixed yesterday or you're having issues, I'm pretty much your only bet at present (though there are a few prolific contributors helping make things better, and more are always welcome!)
JavaScript performance is a goddamn labyrinth full of invisible land mines. If you just want apps to work, you probably won't have any issues here, but if like me you're trying to make real games run fast in browsers, JavaScript will make your life hell and in some cases JSIL will make it worse. The only good thing I can say here is that I'm working on it. :)
JavaScript minifiers and optimizers like Closure are explicitly not supported, because they require your code generator to jump through a bunch of hoops. I could see this being a real blocker depending on how you intend to use your code.
The static analyzer is still kind of fragile and there are still gaps in the language support. Each big application I port using JSIL usually reveals one or two bugs in JSIL - not huge game breakers, but ones that definitely break a feature or make things run slow.
Hope this information is helpful! Thanks for your interest.
Script# pros:
Free
Open source
Generates clean JavaScript
Script# cons:
Supports a subset of C# 2.0 language only
Can be compiled only in a separate project, cannot mix / re-use code between client and server
Low frequency of version updates
Does not offer support
Limited 3rd party library support, C# API is different than JavaScript API.
Not open source
Debugging in JavaScript only
SharpKit pros:
Commercial product
Supports full C# 4.0 language
High frequency of version updates
Support is available
Client / server code can be mixed and re-used within the same project
Extensive 3rd party library support, maintained as open-source - C# API matches exactly to JavaScript API
Supports basic C# debugging for Chrome browsers
Generates clean JavaScript
SharpKit cons:
Has a free version with no time limit, but limited to small / open-source projects
Not open source (only libraries are open-source)
JSIL pros:
Free
Open-source
JSIL cons:
Converts from IL (intermediate language), not from C#, which means a lower abstraction layer since code is already low-level.
Complex generated JavaScript code - almost like IL, hard to read and debug
Answers to feedbacks:
Kevin: JSIL output is not bad, it's simply generated to achieve full .NET behavior, much like SharpKit's CLR mode. On the other hand, SharpKit supports native code generation, in which any native JavaScript code can be generated from C#, exactly as it would have written by hand.
Sample of SharpKit's clean generated JavaScript code:
http://sharpkit.net/Wiki/Using_SharpKit.wiki
Developer can choose to create more complex code generation and gain more features, like support for compile-time method overloads. When specified, SharpKit generates method suffixes to overloaded methods.
Script# requires .NET 4 in order to run, but it does not support full C# 4.0 syntax, like Generics, ref and out parameters, namespace aliases, etc...
Another alternative is WootzJs. Full Disclosure, I am its author.
WootzJs is open-source and strives to be a fairly lightweight cross-compiler that allows for all the major C# language features.
Notable Language Features Supported:
yield statements (generated as an efficient state machine)
async/await methods (generated as a state machine like the C# compiler)
ref and out parameters
expression trees
lambdas and delegates (with proper capturing of this)
generics support in both the compiler and the runtime (invalidly casting to T will throw a cast exception)
C# semantics (as opposed to Javascript semantics) for closed varaibles
It is implemented using Roslyn, which means it will be first in line to take
advantage of future language improvements, since those will now be implemented via Roslyn itself. It provides a custom version of mscorlib so you know exactly what library functionality is actually available to you in your scripts.
What Are its Downsides?
The Javascript is not intended to look "pretty". It is clearly machine generated, though individual methods should be easy to reason about by looking at them.
Because of its extensive support for core libraries and reflection, the generated output is not the smallest on the block. Minification should produce an ~100k JS file, but minification is not yet supported.
WootzJs unabashedly pollutes native types with functions to encapsulate behavior for those types that would only be found in C#. For example, all the methods of System.String are added to the native Javascript String type.
Little support for binding to 3rd-party Javascript libraries presently exist. (Currently only jQuery)
Comparisons with Other Cross-Compilers:
Script# is very stable and has extensive integration with 3rd party Javascript libraries. Furthermore, it has excellent Visual Studio integration, and it provides a custom implementation of mscorlib. This means that you know precisely what functionality has actually been implemented at the tooling level. If, for example, Console.Write() is not implemented, that method will not be available in your editor.
However, due to its custom parser, it is still stuck in C# 2.0 (without even the generics found in that version of C#). This means that the modern C# developer is giving up an enormous set of language features that most of us depend on without reservation -- particularly the aforementioned generics in addition to lambdas and LINQ. This makes Script# essentially a non-starter for many developers.
JSIL is an extremely impressive work that cross-compiles IL into Javascript. It is so robust it can easily handle the cross-compilation of large 3d video games. The downside is that because of its completeness the resultant Javascript files are enormous. If you just want mscorlib.dll and System.dll, it's about a 50MB download. Furthermore, this project is really not designed to be used in the context of a web application, and the amount of effort required to get started is a bit daunting.
This toolkit too implements a custom mscorlib, again allowing you to know what capabilities are available to you. However, it has poor Visual Studio integration, forcing you to create all the custom build steps necessary to invoke the compiler and copy the output to the desired location.
SharpKit: this commercial product strives to provide support for most of the C# 4.0 language features. It generally
succeeds and there's a decent chance this product will meet your needs. It is lightweight (small .JS files), supports modern C# language features (generics, LINQ, etc.) and is usually reliable. It also has a large number of bindings for 3rd party Javascript librarires. However, there are a surprising number of edge cases that you will invariably encounter that are not supported.
For example, the type system is shallow and does not support representing generics or arrays (i.e. typeof(Foo[]) == typeof(Bar[]), typeof(List<string>) == typeof(List<int>)). The support for reflection is limited, with various member types incapable of supporting attributes. Expression tree support is non-existent, and the yield implementation is inefficient (no state machine). Also, a custom mscorlib is not available, and script C# files and normal C# files are intermingled in your projects, forcing you to decorate each and every script file with a [JsType] attribute to distinguish them from normally compiled classes.
We have SharpKit for two years and I must say that's upgraded the way we write code.
The pros as I see them:
The code is much more structured - we can now developed infrastrcture just like we did in C# without "banging our heads" with prototype.
It is very easy to refactor
We can use Code Snippets which results in better productivity and less development time
You can control the way the JS is rendered (you have several modes to choose from).
We can debug our C# code in the browser (Currently supported on Chrome only, but still :->)
Great support! If you send them a query you get a response very fast.
Support a large number of libraries & easily extensible
The cons:
The documentation is a bit poor, however once you get a hang of it you'll boost your development.
Glad if this could help!
For ScriptSharp, this stackoverflow link could be of help.
What advantages can ScriptSharp bring to my tool kit?
If you have any SVN tool, please download a sample from https://github.com/kevingadd/JSIL, this is a working source code and can help you go miles.

An alternative for .NET Script Editor

I am developing an API for some application. And I need to attach there a script engine to make it possible to invoke API from script.
It would be quite nice to have autocomplete, syntax highlight and debug in scripts.
I have found this solution: http://www.codeproject.com/Articles/27744/Net-Script-Editor-C-Vb-net-Mini-IDE
But there are a lot of bugs.
Does anybody have any ideas of alternative solution?
LinqPad has many of the features you are looking for - some of them cost a small extra fee, but it's probably worth the fee!
http://www.linqpad.net/
Snippet Compiler is a nice tool.
You don't mention which language your scripts are in, but based on your other question about debuggers I'll assume you are using IronPython.
The most complete (free/open source) option that I've been able to find is SharpDevelop. It includes all of the features you've mentioned for IronPython as well as a debugger. It is open source and includes many useful extensibility points, so it should provide a good starting point to fix bugs and add additional language support if needed (provided your use conforms to its license terms).
If your situation requires use beyond what is provided by the LGPL then the best choice (though not standalone or free) is to extend Visual Studio (there's already an extension for IronPython though I'm unsure if it supports debugging).
It doesn't sound like you're looking to do much, if any, development of your own for this but if I'm misreading your question then there are some editor controls available that provide the features listed (and then some) for .NET (not exhaustive):
Scintilla.Net
AvalonEdit (used by SharpDevelop)
An example of a project that uses Scintilla.Net is Peter.
You can try with SharpDevelop .

Classification Library

I need to test various famous classification methods like kNN, ID3 and ... on a huge data-set of a project, and choose one for future use.
I have no limitation on language but performance and readable code both in learning and classification phase are very important.
therefore, I'm looking for a good library with following features:
includes various classification methods
high performance
easily usable
any suggestions?
You shuold take a look at PyBrain, a great machine learning module for Python. Can't tell you much about it, because I never really used it (just read about it and looked at several projects solved with it), but it seems to be very good.
You may also want to take a look at this list of Python machine learning modules:
http://web.media.mit.edu/~stefie10/technical/pythonml.html
Or at this SO question:
Is there a recommended package for machine learning in Python?
Try scikit learn - it is written in python and has variety classifiers and also metods for testing them.
Take a look at RapidMiner which comes with a Java-API and graphical tools for data mining. The community edition is free, I think.
I used the predecessor of this tool/library as a student but do not have professional experience with it, though.

.NET/ASP.NET vs Java on ADVANCED Features

Most comparisons I can read on the internet are about the syntax whereas it doesn't matter much. I care more about advanced features. For example, I read on .NET about these features:
Partial Class .Net support not java java partial classes
Adaptive Control Behavior http://msdn.microsoft.com/en-us/library/67276kc5.aspx
Parallel Class http://msdn.microsoft.com/fr-fr/library/system.threading.tasks.parallel.aspx
Solver Foundation http://www.solverfoundation.com/
Reverse Enginiering with UML Sequence Diagram http://blogs.msdn.com/b/habibh/archive/2009/07/29/use-uml-sequence-diagrams-in-team-system-2010-to-reverse-engineer-your-code.aspx
MS-CAPI (Microsoft Crypto API) certified FIPS 140-1 compliant http://groovy.codehaus.org/Bridging+the+Gap+Between+Java+and+.NET+with+Groovy+and+Scriptom
These are features I would need for future projects maintenance. Currently we use Java and I can't really find any equivalence of them on java platform or can you point me towards them that are free and well integrated (not having to hack to make things work smoothly - like using another 3rd language to wrap calls - or pay 10000$ more to get the feature) ?
In terms of language "advanced" features and "power", I generally find: Java << C#3 (.NET) << Scala. (Although C#/CLR does have things not present in Scala.)
I'm not even sure why ASP.NET is in the title though. You also throw VS into the confusion. It's just a (popular ^^) C# IDE.
As far as "need"? Well, there are a million Java projects and libraries. They just don't always act the same way as the C#/CLR (or Scala) counterparts. It doesn't mean you can't do it in Java.

Is anybody using the Specter BDD Framework?

I was reading the example chapter from the book by Ayende and on the website of the Boo language I saw a reference to the Specter BDD Framework.
I am wondering if anybody is using it in their project, how that works out and if there are more examples and/or suggested readings.
Just in case you are wondering, I'm a C# developer and so I plan to use it in a C#/.NET environment.
A few year later visiting this question. I think we can safely assume Specflow and some others like NSpec became the tools we are using.
I'm not using it, but I've seen demos of it. It's very nice.
Boo has a lot of interesting extensibility points in parsing and interpreting the language itself that make it ideal for writing frameworks like Specter. The end result is much nicer looking than you'd be able to get with languages like C#.
Unfortunately, the fact that Boo isn't "in the box" and can't simply be something you check into your source tree and use really holds it back here. It's a much heavier adoption cost than just picking a framework like NSpec.
I have used it a little, I'm starting a new project right now and I plan on using specter. I'm really enjoying it.

Categories

Resources