Develop Reference

how to manually calculate the memory been used

is there any way of calculate manually the memory that an array is goin to consume.
i am using for languaje C# in a 64 bit OS
let say i have the next array:
int number[][]= new int[2][2];
number[0][0]=25;
number[0][1]=60;
....
...
so my fist question is, each dimension of the array has the same bit asignation? lets say number[0][0] has a 12 bit asing (i dont now if 12 bits is the right answer) so this will make the first line a 24 bit of memory asing?
how much fisical and virtual memory does each dimension takes?
if i use int, double or string for the array is there any diference of memory to been used?
at the end if i used GC.GetTotalMemory will i recibe the same result of the total of memory been used by array?

You need to use the sizeof function to get how many bytes are allocated to your Type.
int number[][] = new int[2][];
for (int i = 0; i < number.Length; i++)
{
number[i] = new int[2];
}
int size = sizeof(int) * number.Length * number[0].Length;

What is the Fastest way to convert byte[] to float[] and vice versa?

Which is the fastest way to convert a byte[] to float[] and vice versa (without a loop of course).
I'm using BlockCopy now, but then I need the double memory. I would like some kind of cast.
I need to do this conversion just to send the data through a socket and reconstruct the array in the other end.

Surely msarchet's proposal makes copies too. You are talking about just changing the way .NET thinks about a memory area, if you dont' want to copy.
But, I don't think what you want is possible, as bytes and floats are represented totally different in memory. A byte uses exactly a byte in memory, but a float uses 4 bytes (32 bits).
If you don't have the memory requirements to store your data, just represent the data as the data type you will be using the most in memory, and convert the values you actually use, when you use them.
How do you want to convert a float (which can represent a value between ±1.5 × 10−45 and±3.4 × 10^38) into a byte (which can represent a value between 0 and 255) anyway?
(see more info her about:
byte: http://msdn.microsoft.com/en-us/library/5bdb6693(v=VS.100).aspx
float: http://msdn.microsoft.com/en-us/library/b1e65aza.aspx
More about floating types in .NET here: http://csharpindepth.com/Articles/General/FloatingPoint.aspx

You can use StructLayout to achieve this (from Stack Overflow question C# unsafe value type array to byte array conversions):
[StructLayout(LayoutKind.Explicit)]
struct UnionArray
{
[FieldOffset(0)]
public Byte[] Bytes;
[FieldOffset(0)]
public float[] Floats;
}
static void Main(string[] args)
{
// From bytes to floats - works
byte[] bytes = { 0, 1, 2, 4, 8, 16, 32, 64 };
UnionArray arry = new UnionArray { Bytes = bytes };
for (int i = 0; i < arry.Bytes.Length / 4; i++)
Console.WriteLine(arry.Floats[i]);
}

IEnumerable<float> ToFloats(byte[] bytes)
{
for(int i = 0; i < bytes.Length; i+=4)
yield return BitConverter.ToSingle(bytes, i);
}

Two ways if you have access to LINQ:
var floatarray = ByteArry.AsEnumerable.Cast<float>().ToArray();
or just using Array Functions
var floatarray = Array.ConvertAll(ByteArray, item => (float)item);

Converting binary reading function from C# to C

I am honestly really confused on reading binary files in C.
My data is in a format like:
int header
Cell[][] cells, 8x8 matrix
Each cell is just a short id and a sbyte z.
However, something is clearly messing up and none of the values are accurate.
int i, j;
Block block;
// read 32 bits
fread( &(block.header), sizeof( int ), 1, mapFile );
// loop through to fill the 8x8 matrix
for( i = 0; i < 8; i++ )
{
for( j = 0; j < 8; j++ )
{
// read 16 bits
fread( &(block.cells[i][j].tileId), sizeof( short ), 1, mapFile );
// read 8 bits
fread( &(block.cells[i][j].z), sizeof( char ), 1, mapFile );
printf( "[%i][%i]: %x %i", i, j, block.cells[i][j].tileId, block.cells[i][j].z );
}
}
printf( "header: %i", block.header );
Output of above shows a bunch of lines with [n][n]: ffffa800 251.
My C# version works fine though:
Block block;
block.header = reader.ReadInt32();
block.cells = new Cell[8,8];
for( int i = 0; i < 8; i++ )
{
for ( int j = 0; j < 8; j++ )
{
block.cells[i, j].tileId = reader.ReadInt16();
block.cells[i, j].z = reader.ReadSByte();
}
}
reader.Close();
Output from that (correctly) shows [n][n]: a8 -5.

You should use more strictly defined types in C. For example, int32_t (stdint.h). It's possible that your short is 32-bit (or even your int is 16-bit), albeit unlikely.
It's probably also worthwhile to show us how the Block struct is defined.
For reference:
SByte = int8_t
Int16 = int16_t
Int32 = int32_t

This might very well have to do with the fact that a short is not always guaranteed to be two bytes. Maybe it is better to read the specified number of bytes and then convert it into the desired datatype? this also goes for int. By the way: fread returns the number of bytes read, so you are able to check if the desired number of bytes is actually read.
Regards, Perry

I believe you may be using C# to write the file (BinaryWriter) and C to read it back. Thus the file may have some unexpected value at the beginning (some header, length, a BOM whatever). Use a hex editor to be sure.

In C variables are initialized by default to an undefined state, so you cannot rely on their contents until you have explicitly initialized them. Remember to allocate all the memory that you will need and to zero out the contents (initializing them) before they are used.
For example: memset(&block, '\0', sizeof block);

Fast casting in C# using BitConverter, can it be any faster?

In our application, we have a very large byte-array and we have to convert these bytes into different types. Currently, we use BitConverter.ToXXXX() for this purpose. Our heavy hitters are, ToInt16 and ToUInt64.
For UInt64, our problem is that the data stream has actually 6-bytes of data to represent a large integer. Since there is no native function to convert 6-bytes of data to UInt64, we do:
UInt64 value = BitConverter.ToUInt64() & 0x0000ffffffffffff;
Our use of ToInt16 is simpler, do don't have to do any bit manipulation.
We do so many of these 2 operations that I wanted to ask the SO community whether there's a faster way to do these conversions. Right now, approximately 20% of our entire CPU cycles is consumed by these two functions.

Have you thought about using memory pointers directly. I can't vouch for its performance but it is a common trick in C++\C...
byte[] arr = { 1, 2, 3, 4, 5, 6, 7, 8 ,9,10,11,12,13,14,15,16};
fixed (byte* a2rr = &arr[0])
{
UInt64* uint64ptr = (UInt64*) a2rr;
Console.WriteLine("The value is {0:X2}", (*uint64ptr & 0x0000FFFFFFFFFFFF));
uint64ptr = (UInt64*) ((byte*) uint64ptr+6);
Console.WriteLine("The value is {0:X2}", (*uint64ptr & 0x0000FFFFFFFFFFFF));
}
You'll need to make your assembly "unsafe" in the build settings as well as mark the method in which you'd be doing this unsafe aswell. You are also tied to little endian with this approach.

You can use the System.Buffer class to copy a whole array over to another array of a different type as a fast, 'block copy' operation:
The BlockCopy method accesses the bytes in the src parameter array using offsets into memory, not programming constructs such as indexes or upper and lower array bounds.
The array types must be of 'primitive' types, they must align, and the copy operation is endian-sensitive. In your case of 6-bytes integers, it can't align with any of .NET's 'primitive' types, unless you can obtain the source array with two bytes of padding for each six, which will then align to Int64. But this method will work for arrays of Int16, which may speed up some of your operations.

Why not:
UInt16 valLow = BitConverter.ToUInt16();
UInt64 valHigh = (UInt64)BitConverter.ToUInt32();
UInt64 Value = (valHigh << 16) | valLow;
You can make that a single statement, although the JIT compiler will probably do that for you automatically.
That will prevent you from reading those extra two bytes that you end up throwing away.
If that doesn't reduce CPU, then you'll probably want to write your own converter that reads the bytes directly from the buffer. You can either use array indexing or, if you think it's necessary, unsafe code with pointers.
Note that, as a commenter pointed out, if you use any of these suggestions, then either you're limited to a particular "endian-ness", or you'll have to write your code to detect little/big endian and react accordingly. The code sample I showed above works for little endian (x86).

See my answer for a similar question here.
It's the same unsafe memory manipulation as in Jimmy's answer, but in a more "friendly" way for consumers. It'll allow you to view your byte array as UInt64 array.

For anyone else who stumbles across this if you only need little endian and do not need to auto detect big endian and convert from that. Then I've written an extended version of bitconverter with a number of additions to handle Span as well as converting arrays of type T for example int[] or timestamp[]
Also extended the types supported to include timestamp, decimal and datetime.
https://github.com/tcwicks/ChillX/blob/master/src/ChillX.Serialization/BitConverterExtended.cs
Example usage:
Random rnd = new Random();
RentedBuffer<byte> buffer = RentedBuffer<byte>.Shared.Rent(BitConverterExtended.SizeOfUInt64
+ (20 * BitConverterExtended.SizeOfUInt16)
+ (20 * BitConverterExtended.SizeOfTimeSpan)
+ (10 * BitConverterExtended.SizeOfSingle);
UInt64 exampleLong = long.MaxValue;
int startIndex = 0;
startIndex += BitConverterExtended.GetBytes(exampleLong, buffer.BufferSpan, startIndex);
UInt16[] shortArray = new UInt16[20];
for (int I = 0; I < shortArray.Length; I++) { shortArray[I] = (ushort)rnd.Next(0, UInt16.MaxValue); }
//When using reflection / expression trees CLR cannot distinguish between UInt16 and Int16 or Uint64 and Int64 etc...
//Therefore Uint methods are renamed.
startIndex += BitConverterExtended.GetBytesUShortArray(shortArray, buffer.BufferSpan, startIndex);
TimeSpan[] timespanArray = new TimeSpan[20];
for (int I = 0; I < timespanArray.Length; I++) { timespanArray[I] = TimeSpan.FromSeconds(rnd.Next(0, int.MaxValue)); }
startIndex += BitConverterExtended.GetBytes(timespanArray, buffer.BufferSpan, startIndex);
float[] floatArray = new float[10];
for (int I = 0; I < floatArray.Length; I++) { floatArray[I] = MathF.PI * rnd.Next(short.MinValue, short.MaxValue); }
startIndex += BitConverterExtended.GetBytes(floatArray, buffer.BufferSpan, startIndex);
//Do stuff with buffer and then
buffer.Return(); //always better to return it as soon as possible
//Or in case you forget
buffer = null;
//and let RentedBufferContract do this automatically
it supports reading from and writing to both byte[] or RentedBuffer however using the RentedBuffer class greatly reduces GC collection overheads.
RentedBufferContract class internally handles returning buffers to the pool to prevent memory leaks.
Also includes a serializer which is similar to messagepack.
Note: MessagePack is a faster serializer with more features however this serializer reduces GC collection overheads by reading from and writing to rented byte buffers.
https://github.com/tcwicks/ChillX/blob/master/src/ChillX.Serialization/ChillXSerializer.cs

How do I parse a polyline metafile record out of a byte array?

I need a little help in defining the following Windows GDI type in C#. I have the data in the form of a byte[] in C#, and I need to somehow marshal or cast it as the following in C#. I suppose I need to define the proper struct? This is the type:
NAME
META_POLYLINE
NEAREST API CALL
#include <windows.h>
BOOL32 Polyline
(
HDC32 hdc,
const POINT32 *pt,
INT32 count
);
DESCRIPTION
U16 array no Value
--------------------------- --------------
0 no of points
1 each odd until the end x of the point
2 each even until the end y of the point
A polyline is a list of points. Unlike a polygon, a polyline is always unfilled, and can be open.

byte[] buffer;
fixed (byte* b = buffer)
{
ushort* ptr = (ushort*)b;
int count = (int)*ptr;
var points = new Point[count];
for (int i = 0; i < count; i++)
{
int x = (int)*(++ptr);
int y = (int)*(++ptr);
points[i] = new Point(x, y);
}
}
(Untested)

Have you looked at the Polyline entry on PInvoke.net yet?

Okay, a metafile record for a polyline... You might want to try doing a Buffer.BlockCopy from the byte array to a UInt16 array.