Below is a Win32 Console App procedure that demonstrates the dependence of various pointers on an array. A change to the values in the original array (model) by for example uncommenting the lines marked '// uncomment ...' results in a change to the output. My question is how do I get or mimic this behaviour in a C# managed code environment (i.e. without using unsafe and pointers)?
#include "stdafx.h"
#include <iostream>
using namespace std;
int _tmain(int argc, _TCHAR* argv[])
{
float model[100];
for(int i = 0; i < 100; i++) { model[i] = i; }
// uncomment these to alter the results
//model[5] = 5000;
//model[20] = 20000;
//model[38] = 38000;
static const int componentCount = 5;
float* coefs = model; // coefs points to model[0]
float* mean = coefs + componentCount; // mean points to model[0 + componentCount] == model[5]
float* cov = mean + 3*componentCount; // cov points to model[0 + componentCount + 3*componentCount] == model[20]
int ci = 2;
float* c = cov + 9*ci; // c points to model[0 + componentCount + 3*componentCount + 9*ci] == model[38]
int i = 0;
cout <<"model : "<< model[i] << endl; // 0
cout <<"coefs : "<< coefs[i] << endl; // 0
cout <<"mean : "<< mean[i] << endl; // 5 (or 5000)
cout <<"cov : "<< cov[i] << endl; // 20 (or 20000)
cout <<"ci : "<< ci << endl; // 2
cout <<"c : "<< c[i] << endl; // 38 (or 38000)
cin.get(); }
You can do the same thing in C# without unsafe code:
struct ArrayPointer<T>
{
private T[] array;
private int offset;
public ArrayPointer(T[] array) : this(array, 0)
{
}
private ArrayPointer(T[] array, int offset)
{
Debug.Assert(array != null);
Debug.Assert(offset >= 0);
Debug.Assert(offset < array.Length);
this.array = array;
this.offset = offset;
}
public static ArrayPointer<T> operator+(ArrayPointer<T> p1, int p2)
{
return new ArrayPointer<T>(p1.array, p1.offset + p2);
}
And so on. Define operators for addition, subtraction, increment, decrement, comparison, indexing, conversion from arrays, and so on. Then you can say:
int[] arr = whatever;
ArrayPointer<int> pointer = arr;
pointer+=2;
pointer--;
int x = pointer[3];
and so on.
This approach has a lot of nice properties. For example, you can do a debug assert if you ever compare p1 > p2 when p1 and p2 are pointers to the interiors of different arrays. That is almost always a bug in C, but a hard one to catch.
You could write a class that represents an array with some offset, similar to the one below. Additionaly, you might want it to implement ICollection<T> or at least IEnumerable<T>.
class ArrayWithOffset<T>
{
T[] m_array;
int m_offset;
public ArrayWithOffset(T[] array, int offset)
{
m_array = array;
m_offset = offset;
}
public T this[int i]
{
return m_array[offset + i]
}
}
Instead of one parameter, pointer to array item, use pair of parameters (array, offset).
Related
This question already has answers here:
How to return an array from a function?
(5 answers)
Closed 3 years ago.
I have my C# code that returns uint array but I want to do it in C++. I looked other posts; they use uint pointer array where my array is not. Does anyone know how to return uint16_t array properly?
This is C# code works fine
public static UInt16[] GetIntArrayFromByteArray(byte[] byteArray)
{
if ((byteArray.Length % 2) == 1)
Array.Resize(ref byteArray, byteArray.Length + 1);
UInt16[] intArray = new UInt16[byteArray.Length / 2];
for (int i = 0; i < byteArray.Length; i += 2)
intArray[i / 2] = (UInt16)((byteArray[i] << 8) | byteArray[i + 1]);
return intArray;
}
This is C++ code that creates syntax error
uint16_t[] GetIntArrayFromByteArray(byte[] byteArray)
{
//if ((byteArray.Length % 2) == 1)
//Array.Resize(ref byteArray, byteArray.Length + 1);
uint16_t[] intArray = new uint16_t[10];
for (int i = 0; i < 10; i += 2)
intArray[i / 2] = (uint16_t)((byteArray[i] << 8) | byteArray[i + 1]);
return intArray;
}
Do not use Type[] ever. Use std::vector:
std::vector<uint16_t> GetIntArrayFromByteArray(std::vector<byte> byteArray)
{
// If the number of bytes is not even, put a zero at the end
if ((byteArray.size() % 2) == 1)
byteArray.push_back(0);
std::vector<uint16_t> intArray;
for (int i = 0; i < byteArray.size(); i += 2)
intArray.push_back((uint16_t)((byteArray[i] << 8) | byteArray[i + 1]));
return intArray;
}
You can also use std::array<Type, Size> if the array would be fixed size.
More optimal version (thanks to #Aconcagua) (demo)
Here is a full code with more optimal version that doesn't copy or alter the input. This is better if you'll have long input arrays. It's possible to write it shorter, but I wanted to keep it verbose and beginner-friendly.
#include <iostream>
#include <vector>
using byte = unsigned char;
std::vector<uint16_t> GetIntArrayFromByteArray(const std::vector<byte>& byteArray)
{
const int inputSize = byteArray.size();
const bool inputIsOddCount = inputSize % 2 != 0;
const int finalSize = (int)(inputSize/2.0 + 0.5);
// Ignore the last odd item in loop and handle it later
const int loopLength = inputIsOddCount ? inputSize - 1 : inputSize;
std::vector<uint16_t> intArray;
// Reserve space for all items
intArray.reserve(finalSize);
for (int i = 0; i < loopLength; i += 2)
{
intArray.push_back((uint16_t)((byteArray[i] << 8) | byteArray[i + 1]));
}
// If the input was odd-count, we still have one byte to add, along with a zero
if(inputIsOddCount)
{
// The zero in this expression is redundant but illustrative
intArray.push_back((uint16_t)((byteArray[inputSize-1] << 8) | 0));
}
return intArray;
}
int main() {
const std::vector<byte> numbers{2,0,0,0,1,0,0,1};
const std::vector<uint16_t> result(GetIntArrayFromByteArray(numbers));
for(uint16_t num: result) {
std::cout << num << "\n";
}
return 0;
}
Is there any documentation on what RNG algorithm PowerShell's Get-Random cmdlet officially uses in PowerShell 5.1?
I did some investigating (via decompiling), and it seems Get-Random is just a wrapper for the native .NET Random class. I can confirm this by getting the same values on PowerShell 2.0 (Windows 7) vs C# (targeting .NET 4.5.2). However, Powershell 5.1 (Windows 10) seems to output different numbers.
PowerShell 2.0:
Get-Random -SetSeed 0 -Minimum 0 -Maximum 2147483647
# Produces 1559595546
PowerShell 5.1:
Get-Random -SetSeed 0 -Minimum 0 -Maximum 2147483647
# Produces: 1866861594
C#:
new Random(0).Next(0, 2147483647);
# Produces 1559595546
I did read that after PowerShell 2.0, Get-Random is supposed to support 64-bit numbers, but I set the minimum and maximum above to the 32-bit range for proper testing. Even different seeds, or altering the ranges to something like [0, 100] still yields different results on PowerShell 5.1.
My end goal is basically trying to reproduce random numbers produced in PowerShell 5.1 in either C++ or C# for sake of performance. I already have the C# Random class translated to C++.
You can view Power Shell's Get Random implementation on GitHub.
Comments in the source code show it is using its own generator which have comments indicating it has some deviations from the .net / CRL implementation.
In particular, it has its own PolymorphicRandomNumberGenerator class that provides a "re-implementation" of methods using the NextBytes() primitive based on the CLR implementation:
/// <summary>
/// Provides an adapter API for random numbers that may be either cryptographically random, or
/// generated with the regular pseudo-random number generator. Re-implementations of
/// methods using the NextBytes() primitive based on the CLR implementation:
/// http://referencesource.microsoft.com/#mscorlib/system/random.cs
/// </summary>
internal class PolymorphicRandomNumberGenerator
For example:
/// <summary>
/// Generates a non-negative random integer.
/// </summary>
/// <returns>A non-negative random integer.</returns>
internal int Next()
{
int result;
// The CLR implementation just fudges
// Int32.MaxValue down to (Int32.MaxValue - 1). This implementation
// errs on the side of correctness.
do
{
result = InternalSample();
}
while (result == Int32.MaxValue);
if (result < 0)
{
result += Int32.MaxValue;
}
return result;
}
The powershell implementation, while using the same underlying System.Random, will use different methods to generate the random values depending on the input. With your issue the power shell implementation does this:
var rnd = new Random(0);
int result;
byte[] data = new byte[sizeof(int)];
rnd.NextBytes(data);
result = BitConverter.ToInt32(data, 0);
console.log("result = {0}", result);
// result = 1866861594
Where which does not match the output of:
var rresult = new Random(0).Next(0, int.MaxValue);
console.log("result = {0}", result);
// result = 1559595546
Here's my ported C++ code for the PowerShell 5.0 PRNG, if it's of any use to anyone else searching. Confirmed it produces the same numbers as PowerShell 5.1 on Windows 10.
It utilizes my Random class that is a ported version of the .NET RNG, which I separated a bit to make both inherit from a common interface (Random.h) and renamed to RandomDotNet: https://stackoverflow.com/a/39338606/1301139
Random.h
#include <limits>
#include <Windows.h>
#pragma once
class Random
{
public:
virtual ~Random() {}
virtual int Next() = 0;
virtual int Next(int minValue, int maxValue) = 0;
virtual int Next(int maxValue) = 0;
virtual void NextBytes(BYTE *buffer, int bufferLen) {};
virtual double NextDouble() = 0;
};
RandomPS5.h
#include <limits>
#include <Windows.h>
#include "Random.h"
#pragma once
class RandomPS5 : public Random
{
protected:
double InternalSampleLargeRange();
int InternalSample();
int BytesToInt(BYTE *dword);
Random *pseudoGenerator;
public:
RandomPS5(int seed);
~RandomPS5();
int Next();
int Next(int minValue, int maxValue);
int Next(int maxValue);
double NextDouble();
void NextBytes(BYTE *buffer, int bufferLen);
};
RandomPS5.cpp
#include "stdafx.h"
#include "RandomPS5.h"
#include "RandomDotNet.h"
#include <limits.h>
#include <math.h>
#include <stdexcept>
#include <string>
// Naive conversion of BitConverter.ToInt32
int RandomPS5::BytesToInt(BYTE *b) {
int Int32 = 0;
Int32 = (Int32 << 8) + b[3];
Int32 = (Int32 << 8) + b[2];
Int32 = (Int32 << 8) + b[1];
Int32 = (Int32 << 8) + b[0];
return Int32;
}
RandomPS5::RandomPS5(int seed) {
pseudoGenerator = new RandomDotNet(seed);
}
RandomPS5::~RandomPS5(){
delete pseudoGenerator;
}
double RandomPS5::NextDouble() {
return Next() * (1.0 / 0x7FFFFFFF);
}
int RandomPS5::Next() {
int result;
do {
result = InternalSample();
} while (result == 0x7FFFFFFF);
if (result < 0) {
result += 0x7FFFFFFF;
}
return result;
}
int RandomPS5::Next(int maxValue) {
if (maxValue<0) {
throw std::invalid_argument("maxValue must be positive");
}
return Next(0, maxValue);
}
int RandomPS5::Next(int minValue, int maxValue) {
if (minValue > maxValue)
{
throw std::invalid_argument("minValue is larger than maxValue");
}
long range = (long)maxValue - (long)minValue;
if (range <= 0x7FFFFFFF)
{
return ((int)(NextDouble() * range) + minValue);
}
else
{
double largeSample = this->InternalSampleLargeRange() * (1.0 / (2 * 0x7FFFFFFF));
int result = (int)((long)(largeSample * range) + minValue);
return result;
}
}
int RandomPS5::InternalSample() {
BYTE *data = (BYTE*)malloc(sizeof(int));
this->NextBytes(data, sizeof(int));
int result = BytesToInt(data);
free(data);
return result;
}
double RandomPS5::InternalSampleLargeRange() {
double result;
do{
result = this->InternalSample();
} while (result == 0x7FFFFFFF);
result += 0x7FFFFFFF;
return result;
}
void RandomPS5::NextBytes(BYTE *buffer, int bufferLen) {
this->pseudoGenerator->NextBytes(buffer, bufferLen);
}
Main.cpp
#include "RandomDotNet.h"
#include "RandomPS5.h"
#include <Windows.h>
// Length of charset string
#define CHARSETLEN 62
// Random charset
const char charset[CHARSETLEN + 1] = "0123456789ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz";
// Function that processes a record like PowerShell does for -ObjectList
void processRecord(CHAR *string, int len, Random *r) {
// Processed characters
int processed = 0;
int i, indexToReplace;
CHAR temp;
// Iterate the charset
for (i = 0; i < CHARSETLEN; ++i) {
if (processed < len) {
string[processed] = charset[i];
}
else if (r->Next(processed + 1) < len) {
string[r->Next(len)] = charset[i];
}
++processed;
}
// Iterate selected items to return them in "random" order
for (i = 0; i < len; ++i) {
// Get random index
indexToReplace = r->Next(i, len);
if (i != indexToReplace) {
// Swap
temp = string[i];
string[i] = string[indexToReplace];
string[indexToReplace] = temp;
}
}
// Terminate the string
string[len] = '\0';
}
int main(int argc, char* argv[]){
// Example usage with a given seed
Random *r = new RandomPS5(1000);
// Length of random string
int len = 49;
// Random string buffer
CHAR *buffer = (CHAR*)malloc(len + 1);
// ([char[]](Get-Random -Input $(48..57 + 65..90 + 97..122) -Count 49 -SetSeed 1000)) -Join ""
processRecord(buffer, len, r);
// Produces: y6FLfcKrpINqgP25GXS7Z0dVBmJOzntlQ3hjbHMAU1ExkewWY
printf("Random string: %s", buffer);
delete r;
return 0;
}
Could anyone help me optimize this piece of code? Its currently a large bottleneck as it gets called very often. Even a 25% speed improvement would be significant.
public int ReadInt(int length)
{
if (Position + length > Length)
throw new BitBufferException("Not enough bits remaining.");
int result = 0;
while (length > 0)
{
int off = Position & 7;
int count = 8 - off;
if (count > length)
count = length;
int mask = (1 << count) - 1;
int bits = (Data[Position >> 3] >> off);
result |= (bits & mask) << (length - count);
length -= count;
Position += count;
}
return result;
}
Best answer would go to fastest solution. Benchmarks done with dottrace. Currently this block of code takes up about 15% of the total cpu time. Lowest number wins best answer.
EDIT: Sample usage:
public class Auth : Packet
{
int Field0;
int ProtocolHash;
int Field1;
public override void Parse(buffer)
{
Field0 = buffer.ReadInt(9);
ProtocolHash = buffer.ReadInt(32);
Field1 = buffer.ReadInt(8);
}
}
Size of Data is variable but in most cases 512 bytes;
How about using pointers and unsafe context? You didn't say anything about your input data, method context, etc. so I tried to deduct all of these by myself.
public class BitTest
{
private int[] _data;
public BitTest(int[] data)
{
Length = data.Length * 4 * 8;
// +2, because we use byte* and long* later
// and don't want to read outside the array memory
_data = new int[data.Length + 2];
Array.Copy(data, _data, data.Length);
}
public int Position { get; private set; }
public int Length { get; private set; }
and ReadInt method. Hope comments give a little light on the solution:
public unsafe int ReadInt(int length)
{
if (Position + length > Length)
throw new ArgumentException("Not enough bits remaining.");
// method returns int, so getting more then 32 bits is pointless
if (length > 4 * 8)
throw new ArgumentException();
//
int bytePosition = Position / 8;
int bitPosition = Position % 8;
Position += length;
// get int* on array to start with
fixed (int* array = _data)
{
// change pointer to byte*
byte* bt = (byte*)array;
// skip already read bytes and change pointer type to long*
long* ptr = (long*)(bt + bytePosition);
// read value from current pointer position
long value = *ptr;
// take only necessary bits
value &= (1L << (length + bitPosition)) - 1;
value >>= bitPosition;
// cast value to int before returning
return (int)value;
}
}
}
I didn't test the method, but would bet it's much faster then your approach.
My simple test code:
var data = new[] { 1 | (1 << 8 + 1) | (1 << 16 + 2) | (1 << 24 + 3) };
var test = new BitTest(data);
var bytes = Enumerable.Range(0, 4)
.Select(x => test.ReadInt(8))
.ToArray();
bytes contains { 1, 2, 4, 8}, as expected.
I Don't know if this give you a significant improvements but it should give you some numbers.
Instead of creating new int variables inside the loop (this requires a time to create) let reserved those variables before entering the loop.
public int ReadInt(int length)
{
if (Position + length > Length)
throw new BitBufferException("Not enough bits remaining.");
int result = 0;
int off = 0;
int count = 0;
int mask = 0;
int bits = 0
while (length > 0)
{
off = Position & 7;
count = 8 - off;
if (count > length)
count = length;
mask = (1 << count) - 1;
bits = (Data[Position >> 3] >> off);
result |= (bits & mask) << (length - count);
length -= count;
Position += count;
}
return result;
}
HOPE THIS increase your performance even a bit
Not so sure how to ask this question, but I have 2 ways (so far) for a lookup array
Option 1 is:
bool[][][] myJaggegArray;
myJaggegArray = new bool[120][][];
for (int i = 0; i < 120; ++i)
{
if ((i & 0x88) == 0)
{
//only 64 will be set
myJaggegArray[i] = new bool[120][];
for (int j = 0; j < 120; ++j)
{
if ((j & 0x88) == 0)
{
//only 64 will be set
myJaggegArray[i][j] = new bool[60];
}
}
}
}
Option 2 is:
bool[] myArray;
// [998520]
myArray = new bool[(120 | (120 << 7) | (60 << 14))];
Both ways work nicely, but is there another (better) way of doing a fast lookup and which one would you take if speed / performance is what matter?
This would be used in a chessboard implementation (0x88) and mostly is
[from][to][dataX] for option 1
[(from | (to << 7) | (dataX << 14))] for option 2
I would suggest using one large array, because of the advantages of having one large memory block, but I would also encourage writing a special accessor to that array.
class MyCustomDataStore
{
bool[] array;
int sizex, sizey, sizez;
MyCustomDataStore(int x, int y, int z) {
array=new bool[x*y*z];
this.sizex = x;
this.sizey = y;
this.sizez = z;
}
bool get(int px, int py, int pz) {
// change the order in whatever way you iterate
return array [ px*sizex*sizey + py*sizey + pz ];
}
}
I just update dariusz's solution with an array of longs for z-size <= 64
edit2: updated to '<<' version, size fixed to 128x128x64
class MyCustomDataStore
{
long[] array;
MyCustomDataStore()
{
array = new long[128 | 128 << 7];
}
bool get(int px, int py, int pz)
{
return (array[px | (py << 7)] & (1 << pz)) == 0;
}
void set(int px, int py, int pz, bool val)
{
long mask = (1 << pz);
int index = px | (py << 7);
if (val)
{
array[index] |= mask;
}
else
{
array[index] &= ~mask;
}
}
}
edit: performance test:
used 100 times 128x128x64 fill and read
long: 9885ms, 132096B
bool: 9740ms, 1065088B
In C# or C++ how can I implement a branch-free sort of three (integer) numbers?
Is this possible?
No conditionals. Only a cast to uint. Perfect solution.
int abs (int a)
{
int b = a;
b = (b >> (sizeof(int)*CHAR_BIT-1) & 1);
return 2 * b * (a) + a;
}
int max (int a, int b) { return (a + b + abs(a - b)) / 2; }
int min (int a, int b) { return (a + b - abs(a - b)) / 2; }
void sort (int & a, int & b, int & c)
{
int maxnum = max(max(a,b), c);
int minnum = min(min(a,b), c);
int middlenum = a + b + c - maxnum - minnum;
a = maxnum;
b = middlenum;
c = minnum;
}
You can write max, min and swap branch-free functions. Once you have these functions, you can use them to write sort function as:
void sort(int &a, int &b, int &c)
{
int m1 = max(a,b,c);
int m2 = min(a,b,c);
b = a + b + c - m1 - m2;
swap(m1, a);
swap(m2, c);
}
And here are the helper functions:
void swap(int &a, int &b)
{
int tmp = a; a = b; b = tmp;
}
int max( int a, int b, int c ) {
int l1[] = { a, b };
int l2[] = { l1[ a<b ], c };
return l2[ l2[0] < c ];
}
int min( int a, int b, int c ) {
int l1[] = { a, b };
int l2[] = { l1[ a>b ], c };
return l2[ l2[0] > c ];
}
Test code:
int main() {
int a,b,c;
std::cin >> a >> b >> c;
sort(a,b,c);
std::cout << a <<"," << b << "," << c << std::endl;
return 0;
}
Input:
21 242 434
Output (descending order):
434, 242, 21
Demo : http://ideone.com/3ZOzc
I have taken the implementation of max from #David's answer from here, and implemented min with little twist.
You can do this in C++ with:
#include <iostream>
void sort(int *in) {
const int sum = in[0]+in[1];
const int diff = abs(in[1]-in[0]);
in[0] = (sum + diff) / 2;
in[1] = (sum - diff) / 2;
}
int main() {
int a[] = {3,4,1};
sort(a);
sort(a+1);
sort(a);
std::cout << a[0] << "," << a[1] << "," << a[2] << std::endl;
int b[] = {1,2,3};
sort(b);
sort(b+1);
sort(b);
std::cout << b[0] << "," << b[1] << "," << b[2] << std::endl;
}
The trick is in expressing the min/max elements as arithmetic operations, not branching and then calling sort on pairs enough times to "bubble sort" them.
I've made a totally generic version, using template meta-programming to call sort the right number of times. It all gets inlined exactly as you'd hope with gcc 4.7.0 on my x86 box (although call is unconditional on x86 anyway). I've also implemented an abs function that avoids branches on x86 (it makes a few assumptions about integers that make it less portable, it's based on gcc's __builtin_abs implementation for x86 though):
#include <iostream>
#include <limits.h>
void myabs(int& in) {
const int tmp = in >> ((sizeof(int) * CHAR_BIT) - 1);
in ^= tmp;
in = tmp - in;
}
template <int N, int I=1, bool C=false>
struct sorter {
static void sort(int *in) {
const int sum = in[I-0]+in[I-1];
int diff = in[I-1]-in[I-0];
myabs(diff);
in[I-0] = (sum + diff) / 2;
in[I-1] = (sum - diff) / 2;
sorter<N, I+1, I+1>=N>::sort(in);
}
};
template <int N,int I>
struct sorter<N,I,true> {
static void sort(int *in) {
sorter<N-1>::sort(in);
}
};
template <int I, bool C>
struct sorter<0,I,C> {
static void sort(int *) {
}
};
int main() {
int a[] = {3,4,1};
sorter<3>::sort(a);
std::cout << a[0] << "," << a[1] << "," << a[2] << std::endl;
}