Develop Reference

I'm creating a dll and looking for current SHA-3 C# code

Need Current SHA-3 C# code?
Greetings,
Does anybody know of good SHA-3 C# code? I'm trying to create my own little dll file in C#. So I'm not really looking for libraries like Bouncy Castle.
I've been running into numerous code examples that say they are SHA-3, but in fact are actually SHA-3 (Keccak) using old code. It's my understanding that although Keccak was the winner, there his been a slight padding change of one bit and the name officially became SHA-3. see FIPS 202.
So I'm looking for SHA3–256 coding that produces an empty string hash of "a7ffc6f8bf1ed76651c14756a061d662f580ff4de43b49fa82d80a4b80f8434a". The older version (Keccak-256) produces "c5d2460186f7233c927e7db2dcc703c0e500b653ca82273b7bfad8045d85a470" from an empty string.
Another option would be adding the padding change to the current implementation. However, that's something I don't know how to do.
Thank you in advance.

My original SHA3 port is available:
https://www.nuget.org/packages/SHA3/1.0.0-rc
https://github.com/jdluzen/sha3/tree/netstandard
Use the property UseKeccakPadding to control which padding you need.
This is a self contained port on top of System.Security.Cryptography.HashAlgorithm. It's very small, and can be used directly as is.

Best approach to render MediaWiki in C#?

Question:
I want to render MediaWiki syntax (and I mean MediaWiki syntax as used by WikiPedia, not some other wiki format from some other engine such as WikiPlex), and that in C#.
Input: MediaWiki Markup string
Output: HTML string
There are some alternative mediawiki parsers, but nothing in C#, and additionally pinvoking C/C++ looks bleak, because of the structure of those libaries.
As syntax guidance, I use
http://en.wikipedia.org/wiki/Wikipedia:Cheatsheet
My first goal is to render that page's markup correctly.
Markup can be seen here:
http://en.wikipedia.org/w/index.php?title=Wikipedia:Cheatsheet&action=edit
Now, if I use Regex, it's not of much use, because one can't exactly say which tag ends which starting ones, especially when some elements, such as italic, become an attribute of the parent element.
On the other hand, parsing character by character is not a good approach either, because
for example ''' means bold, '' means italic, and ''''' means bold and italic...
I looked into porting some of the other parsers' code, but the java implementations are obscure, and the Python implementations have have a very different regex syntax.
The best approach I see so far would be to port mwlib to IronPython
http://www.mediawiki.org/wiki/Alternative_parsers
But frankly, I'm not looking forward to having the IronPython runtime added as a dependency to my application, and even if I would want to, the documentation is bad at best.

Update per 2017:
You can use ParseoidSharp to get a fully compatible MediaWiki-renderer.
It uses the official Wikipedia Parsoid library via NodeServices.
(NetStandard 2.0)
Since Parsoid is GPL 2.0, and and the GPL-code is invoked in nodejs in a separate process via network, you can even use any license you like ;)
Pre-2017
Problem solved.
As originally assumed, the solution lies in using one of the existing alternative parsers in C#.
WikiModel (Java) works well for that purpose.
First attempt was pinvoke kiwi.
It worked, but but failed because:
kiwi uses char* (fails on anything non-English/ASCII)
not thread safe.
bad because of the need have a native dll in the code for every architecture
(did add x86 and amd64, then it went kaboom on my ARM processor)
Second attempt was mwlib.
That failed because somehow IronPython doesn't work as it should.
Third attempt was Swebele, which essentially turned out to be academic vapoware.
The fourth attempt was using the original mediawiki renderer, using Phalanger. That failed because the MediaWiki renderer is not really modular.
The fifth attempt was using Wiky.php via Phalanger, which worked, but was slow and Wiky.php doesn't very completely implement MediaWiki.
The sixth attempt was using bliki via ikvmc, which failed because of the excessive use of 3rd party libraries ==> it compiles, but yields null-reference exceptions only
The seventh attempt was using JavaScript in C#, which worked but was very slow, plus the MediaWiki functionality implemented was very incomplete.
The 8th attempt was writing an own "parser" via Regex.
But the time required to make it work is just excessive, so I stopped.
The 9th attempt was successful.
Using ikvmc on WikiModel yields a useful dll.
The problem there was the example-code was hoplessly out of date.
But using google and the WikiModel sourcecode, I was able to piece it together.
The end-result can be found here:
https://github.com/ststeiger/MultiWikiParser

Why shouldn't this be possible with regular expressions?
inputString = Regex.Replace(inputString, #"(?:'''''')(.*?)(?:'''''')", #"<strong><em>$1</em></strong>");
inputString = Regex.Replace(inputString, #"(?:''')(.*?)(?:''')", #"<strong>$1</strong>");
inputString = Regex.Replace(inputString, #"(?:'')(.*?)(?:'')", #"<em>$1</em>");
This will, as far as I can see, render all 'Bold and italic', 'Bold' and 'Italic' text.

Here is how I once implemented a solution:
define your regular expressions for Markup->HTML conversion
regular expressions must be non greedy
collect the regular expressions in a Dictionary<char, List<RegEx>>
The char is the first (Markup) character in each RegEx, and RegEx's must be sorted by Markup keyword length desc, e.g. === before ==.
Iterate through the characters of the input string, and check if Dictionary.ContainsKey(char). If it does, search the List for matching RegEx. First matching RegEx wins.
As MediaWiki allows recursive markup (except for <pre> and others), the string inside the markup must also be processed in this fashion recursively.
If there is a match, skip ahead the number of characters matching the RegEx in input string. Otherwise proceed to next character.

Kiwi (https://github.com/aboutus/kiwi, mentioned on http://mediawiki.org/wiki/Alternative_parsers) may be a solution. Since it is C based, and I/O is simply done by stdin/stdout, it should not be too hard to create a "PInvoke"-able DLL from it.

As with the accepted solution I found parsoid is the best way forward as it's the official library - and has the greatest support for the wikimedia markup; that said I found ParseoidSharp to be using obsolete methods such as Microsoft.AspNetCore.NodeServices and really it's just a wrapper for a fairly old version of pasoid's npm package.
Since there is a fairly current version of parsoid in node.js you can use Jering.Javascript.NodeJS to do the same thing as ParseoidSharp, the steps are fairly similar also.
Install nodeJS (
download parsoid https://www.npmjs.com/package/parsoid place the required files in your project.
in powershell cd to your project
npm install
Then it's as simple as
output = StaticNodeJSService.InvokeFromFileAsync(Of String)(HttpContext.Current.Request.PhysicalApplicationPath & "./NodeScripts/parsee.js", args:=New Object() {Markup})
Bonus it's now much easier than ParseoidSharp's method to add the options required, e.g. you'll probably want to set the domain to your own domain.

block TEA in c#

Using block tea (XXTEA), how can I convert encrypted text into human readable form?
string encryptedText = "ASDFSDAFSADFSDFSDFSDAF"; (assume it is in correct format)
I have the key/pwd also.

There is a Visual Basic implementation of the block tea algorithm here:
http://www.blog-dotnet.com/post/Tiny-Encryption-Algorithm-(TEA)-in-Visual-BasicNET.aspx
Block tea is meant to be easily implementable, so you shouldn't have a hard time converting that to C# (or compiling it into a .NET assembly with VB and then referencing it from .NET).
You should use the above as a base and then modify it as per the XXTEA algorithm, found here:
http://en.wikipedia.org/wiki/XXTEA
Again, it's meant as being easy to implement, so you shouldn't be too hard. If there are specific parts you are having issues with, please make sure to follow up.

Edit1: Core XTEA algorithm in C# with other parts in VB.NET at CodeProject
Tiny Encryption Algorithm and XTEA for the Compact Framework at CodeProject
Here is a JavaScript Implementation of the XXTEA algorithm and here.
Here is a Pascal implementation of all of them.
With all the other answers you should be able to put an implementation together. Once you do it, make a blog or report back here so it is recorded for the next guy!

I found this https://github.com/WooCode/xxTea which is a C# implementation, with unit tests provided. I haven't verified it's accuracy.

C# Mersenne Twister random integer generator implementation (SFMT) monte carlo simulation

So far I've been using the C# Mersenne Twister found here to generate random numbers:
http://www.centerspace.net/resources.php
I just discovered SFMT which is supposed to be twice as fast here:
http://www.math.sci.hiroshima-u.ac.jp/~m-mat/MT/SFMT/
Can anyone point me at a C# implementation of SFMT?
My requirements are to generate an integer between (and including) 0 and 2^20 (1048576).
I need to do this trillions of times everyday for a simulation running on a 24 hour clock so I am prepared to spend days tweaking this to perfection.
Currently I've tweaked the Center Space Mersenne Twister by adding a new method to fit my requirements:
public uint Next20()
{
return (uint)(genrand_int32() >> 12);
}
Using the method genrand_int32() I'd like to produce my own version, genrand_int20(), that generates an integer between (and including) 0 and 2^20 to save on the cast above and shift but I don't understand the mathematics. Exactly how can I do this?
Also is using an uint going to be faster that int, or is just a matter of addressable numbers? Because I only need up to 1048576, I am only concerned with speed.
Also this will be running on a Windows Server 2003 R2 SP2 (32bit) box with .NET 2. Processor is AMD Opteron 275 (4 core).

What you can do is download the source from the link you discovered on Code Project. Unzip it, load the solution in Visual Studio and compile it. This will give you source, an unmanaged c dll and a .lib file.
You can P/Invoke the functions in this dll, (there are only 5 simple functions exported, of which you need only two) or you can use this dll, lib, and the SFMT header file to create a managed wrapper dll you can use in C# without P/Invoke. I just tried this method and it was very simple to do. There was no explicit marshalling involved.
Here's how. Once you have downloaded and compiled the source (you need the header and the lib file that is created in addition to the dll) create a new C++ CLR Class Library project. Call it WrapSFMT or something. Go the project properties. Under C++/Precompiled Headers, change to "Not using precompiled headers." Under the Linker/General/Additional Library Directories, enter the path to the SFMT.lib. Under Linker/Input/Additional Dependencies, add SFMT.lib. Close the property pages. Copy SFMT.h to your project folder and include it in the project.
Edit WrapSFMT.h to read as follows:
#pragma once
#include "SFMT.H"
using namespace System;
namespace WrapSFMT {
public ref class SRandom
{
public:SRandom(UInt32);
public:UInt32 Rand32(void);
};
}
These declare the methods that will be in your class. Now edit WrapSFMT.cpp to read:
#include "WrapSFMT.h"
namespace WrapSFMT {
SRandom::SRandom(UInt32 seed)
{
init_gen_rand(seed);
}
UInt32 SRandom::Rand32()
{
return gen_rand32();
}
}
These implement the methods you declared in the header file. All you are doing is calling functions from the SFMT.dll, and C++/CLI is automatically handling the conversion from unmanaged to managed. Now you should be able to build the WrapSFMT.dll and reference it in your C# project. Make sure the SFMT.dll is in the path, and you should have no problems.

You can find a C# implementation of SFMT (plus other RNG algorithms) at...
http://rei.to/random.html
The page and source code comments are in Japanese but you should be able to figure it out.
You can also find a Google-translated version (to English) of the page at...
http://translate.google.com/translate?hl=en&sl=ja&u=http://rei.to/random.html

I don't really see your problem with speed here. On my machine (Core 2 Duo T7200 # 2 GHz) generating a random integer with MT19937 or MT19937-64 takes around 20 ns (on average, when drawing 50000 numbers). So that'd be around 4,32 × 1012 (so around 4 trillion numbers) a day. And that's for one core. With Java. So I think you can expect the performance to be more than adequate for your needs.
To actually answer your question: I don't know of a C# implementation of SFMT, but conversion of the C code to C# should be fairly straightforward. However, you're not gaining much, as SFMT is optimized for SIMD and C# currently doesn't support this directly.

Is there a reason you can't compile the C implementation into a DLL and call this from your C# code?
EDIT:
I'm sorry, but I have only a very limited knowledge of C (and indeed C#), but the "How to create a C dll" may be answered here: http://www.kapilik.com/2007/09/17/how-to-create-a-simple-win32-dll-using-visual-c-2005/ and the how fast can be checked by profiling the code.

Maybe this is what you're looking for?
There is a list of several implementations.
Specifically, this one (by Cory Nelson) might be useful.

Building an assembler

I need to build an assembler for a CPU architecture that I've built. The architecture is similar to MIPS, but this is of no importance.
I started using C#, although C++ would be more appropriate. (C# means faster development time for me).
My only problem is that I can't come with a good design for this application. I am building a 2 pass assembler. I know what I need to do in each pass.\
I've implemented the first pass and I realised that if I have to lines assembly code on the same line ...no error is thrown.This means only one thing poor parsing techniques.
So almighty programmers, fathers of assembler enlighten me how should I proceed.
I just need to support symbols and data declaration. Instructions have fixed size.
Please let me know if you need more information.

I've written three or four simple assemblers. Without using a parser generator, what I did was model the S-C assembler that I knew best for 6502.
To do this, I used a simple syntax - a line was one of the following:
nothing
[label] [instruction] [comment]
[label] [directive] [comment]
A label was one letter followed by any number of letters or numbers.
An instruction was <whitespace><mnemonic> [operands]
A directive was <whitespace>.XX [operands]
A comment was a * up to end of line.
Operands depended on the instruction and the directive.
Directives included
.EQ equate for defining constants
.OR set origin address of code
.HS hex string of bytes
.AS ascii string of bytes - any delimiter except white space - whatever started it ended it
.TF target file for output
.BS n reserve block storage of n bytes
When I wrote it, I wrote simple parsers for each component. Whenever I encountered a label, I put it in a table with its target address. Whenever I encountered a label I didn't know, I marked the instruction as incomplete and put the unknown label with a reference to the instruction that needed fixing.
After all source lines had passed, I looked through the "to fix" table and tried to find an entry in the symbol table, if I did, I patched the instructions. If not, then it was an error.
I kept a table of instruction names and all the valid addressing modes for operands. When I got an instruction, I tried to parse each addressing mode in turn until something worked.
Given this structure, it should take a day maybe two to do the whole thing.

Look at this Assembler Development Kit from Randy Hyde's author of the famous "The Art of Assembly Language":
The Assembler Developer's Kit

The first pass of a two-pass assembler assembles the code and puts placeholders for the symbols (as you don't know how big everything is until you've run the assembler). The second pass fills in the addresses. If the assembled code subsequently needs to be linked to external references, this is the job of the eponymous linker.

If you are to write an assembler that just works, and spits out a hex file to be loaded on a microcontroller, it can be simple and easy. Part of my ciforth library is a full Pentium assembler to add inline definitions, of about 150 lines. There is an assembler for the 8080 of a couple dozen lines.
The principle is explained http://home.hccnet.nl/a.w.m.van.der.horst/postitfixup.html .
It amounts to applying the blackboard design pattern to the problem. You start with laying down the instruction, leaving holes for any and all operands. Then you fill in the holes, when you encounter the parameters.
There is a strict separation between the generic tool and the instruction set.
In case the assembler you need is just for yourself, and there are no requirements than usability (not a homework assignment), you can have an example implementation in http://home.hccnet.nl/a.w.m.van.der.horst/forthassembler.html. If you dislike Forth, there is also an example implementation in Perl. If the Pentium instruction set is too much too chew, then still you must be able to understand the principle and the generic part.
You're advised to have a look at the asi8080.frt file first. This is 389 WOC (Words Of Code, not Lines Of Code). An experienced Forther familiar with the instruction set can crank out an assembler like that in an evening. The Pentium is a bitch.