I am honestly really confused on reading binary files in C.
My data is in a format like:
int header
Cell[][] cells, 8x8 matrix
Each cell is just a short id and a sbyte z.
However, something is clearly messing up and none of the values are accurate.
int i, j;
Block block;
// read 32 bits
fread( &(block.header), sizeof( int ), 1, mapFile );
// loop through to fill the 8x8 matrix
for( i = 0; i < 8; i++ )
{
for( j = 0; j < 8; j++ )
{
// read 16 bits
fread( &(block.cells[i][j].tileId), sizeof( short ), 1, mapFile );
// read 8 bits
fread( &(block.cells[i][j].z), sizeof( char ), 1, mapFile );
printf( "[%i][%i]: %x %i", i, j, block.cells[i][j].tileId, block.cells[i][j].z );
}
}
printf( "header: %i", block.header );
Output of above shows a bunch of lines with [n][n]: ffffa800 251.
My C# version works fine though:
Block block;
block.header = reader.ReadInt32();
block.cells = new Cell[8,8];
for( int i = 0; i < 8; i++ )
{
for ( int j = 0; j < 8; j++ )
{
block.cells[i, j].tileId = reader.ReadInt16();
block.cells[i, j].z = reader.ReadSByte();
}
}
reader.Close();
Output from that (correctly) shows [n][n]: a8 -5.
You should use more strictly defined types in C. For example, int32_t (stdint.h). It's possible that your short is 32-bit (or even your int is 16-bit), albeit unlikely.
It's probably also worthwhile to show us how the Block struct is defined.
For reference:
SByte = int8_t
Int16 = int16_t
Int32 = int32_t
This might very well have to do with the fact that a short is not always guaranteed to be two bytes. Maybe it is better to read the specified number of bytes and then convert it into the desired datatype? this also goes for int. By the way: fread returns the number of bytes read, so you are able to check if the desired number of bytes is actually read.
Regards, Perry
I believe you may be using C# to write the file (BinaryWriter) and C to read it back. Thus the file may have some unexpected value at the beginning (some header, length, a BOM whatever). Use a hex editor to be sure.
In C variables are initialized by default to an undefined state, so you cannot rely on their contents until you have explicitly initialized them. Remember to allocate all the memory that you will need and to zero out the contents (initializing them) before they are used.
For example: memset(&block, '\0', sizeof block);
Related
I'm trying to convert array type of ushort[4k*4k] of values 0-65k to similiar array type of int[] of same values.
It seems to mee that Buffer.BlockCopy is the fastest way to do that.
I'm trying the following code:
ushort[] uPixels = MakeRandomShort(0, 65000, 4000 * 4000);// creates ushort[] array
int[] iPixels = new int[4000 * 4000];
int size = sizeof(ushort);
int length = uPixels.Length * size;
System.Buffer.BlockCopy(uPixels, 0, iPixels, 0, length);
But iPixels stores some strange values in very strange range +-1411814783, +- 2078052064, etc.
What is wrong, and what I need to do to make it work properly?
thanks!
There is a related discussion on GitHub.
To copy an ushort[] to an int[] array does not work with a routine tuned for contiguous memory ranges.
Basically, you have to clear the upper halves of the target int cells.
Then, some sort of (parallelized?) loop is needed to copy the actual data.
It could be possible to use unsafe code with pointers advanced in steps of two bytes. The implementation of Buffer.BlockCopy is not visible in the Microsoft source repository. It might make sense to hunt for the source and modify it.
Update
I implemented two C++ functions and did a rough measurement of the resulting performance compared to the C# loop copy.
C# implementation
const int LEN = 4000 * 4000;
for (int i = 0; i < LEN; i++)
{
iPixels[i] = uPixels[i];
}
C++ implementation SpeedCopy1
// Copy loop with casting from unsigned short to int
__declspec(dllexport) void SpeedCopy1(unsigned short *uArray, int * iArray, int len)
{
for (int i = 0; i < len; i++)
{
*iArray++ = *uArray++;
}
}
C++ implementation SpeedCopy2
/// Copy loop with unsigned shorts
/// Clear upper half of int array elements in advance
__declspec(dllexport) void SpeedCopy2(unsigned short* uArray, int* iArray, int len)
{
unsigned short* up = (unsigned short*)iArray;
memset(iArray, 0, sizeof(int) * len);
for (int i = 0; i < len; i++)
{
*up = *uArray++;
up += 2;
}
}
Resulting times:
C# loop copy 27 ms
SpeedCopy1 9 ms
SpeedCopy2 18 ms
Compared to the C# loop, the external C++ function can reduce the copy time down a third.
It remains to be shown, what effect could be gained by multi-threading.
I've tried several ways, but haven't found what I need.
My goal: run a function with each possible value of an UInt32 and record the result.
The order of uints shouldn't matter, but I would prefer non-sequential.
Any ideas?
These are some that I tried:
for ( var u = UInt32.MinValue; u < UInt32.MaxValue; u++ ) { Debug.WriteLine( u ); } //works, but sequential
var list = Enumerable.Range( Int32.MinValue, Int32.MaxValue ).Select( i => ( UInt32 ) i ); //works, but sequential
var list = Enumerable.Range( Int32.MinValue, Int32.MaxValue ).OrderBy( o => rnd.Next() ).Select( i => ( UInt32 ) i ); //OutOfMemoryException because of the OrderBy
I also tried allocating an array and swapping all the elements, but I don't have that code handy. But it also threw a OutOfMemoryException.
The next tactic I'm going to look at it chunking up the UInt32 range into manageable chunks.
The next tactic after that is populating a sql table and seeing how long a random sort would take.
I'm hoping someone has a helpful hint.
EDIT: Ilya Bursov's idea might work.. Thank you!
Maybe something like this, not random, but at least non sequential and covers full range:
UInt32 i = 0;
do
{
UInt32 newIndex = ((i & 0xffff) << 16) | (i >> 16); // swap 2 high bytes with 2 low bytes
// do something with element in array at index newIndex
i++;
} while (i != 0);
This does it in sequential order, but this would work:
for (uint i = UInt32.MinValue; i < UInt32.MaxValue; ++i)
{
...
}
is there any way of calculate manually the memory that an array is goin to consume.
i am using for languaje C# in a 64 bit OS
let say i have the next array:
int number[][]= new int[2][2];
number[0][0]=25;
number[0][1]=60;
....
...
so my fist question is, each dimension of the array has the same bit asignation? lets say number[0][0] has a 12 bit asing (i dont now if 12 bits is the right answer) so this will make the first line a 24 bit of memory asing?
how much fisical and virtual memory does each dimension takes?
if i use int, double or string for the array is there any diference of memory to been used?
at the end if i used GC.GetTotalMemory will i recibe the same result of the total of memory been used by array?
You need to use the sizeof function to get how many bytes are allocated to your Type.
int number[][] = new int[2][];
for (int i = 0; i < number.Length; i++)
{
number[i] = new int[2];
}
int size = sizeof(int) * number.Length * number[0].Length;
I have a byte array in c#. I need to pull out a certain number of bytes starting at a bit position that may not lie on a byte boundary.
Write a little helper method which uses the shift operators to get a byte out
e.g.
byte[] x = new[] {0x0F, 0xF0}
result = x[0] << 4 | x[1] >> 4;
returns 8 bits from the 5th bit position 0xFF
You could easily vary the position using the modulo operator %
a byte is the minimal alignment you can read with the standard stream readers in .NET
If you want to read bits, you need to use bitwise operators and masks to determine if a bit is on (1) or off (0).
But, this means you could use boolean true/false to tell what the contents of a byte are. One way is to read the bits into a boolean enumeration. Something like this extension method could work:
public static IEnumerable<bool> BitsToBools(IEnumerable<byte> input)
{
int readByte;
while((readByte = input.MoveNext()) >= 0)
{
for(int i = 7; i >= 0; i--) // read left to right
yield return ((readByte >> i) & 1) == 1;
}
}
You could add a startIndex and a count to the extension method if you want, or pass in the range from the calling method.
I need a little help in defining the following Windows GDI type in C#. I have the data in the form of a byte[] in C#, and I need to somehow marshal or cast it as the following in C#. I suppose I need to define the proper struct? This is the type:
NAME
META_POLYLINE
NEAREST API CALL
#include <windows.h>
BOOL32 Polyline
(
HDC32 hdc,
const POINT32 *pt,
INT32 count
);
DESCRIPTION
U16 array no Value
--------------------------- --------------
0 no of points
1 each odd until the end x of the point
2 each even until the end y of the point
A polyline is a list of points. Unlike a polygon, a polyline is always unfilled, and can be open.
byte[] buffer;
fixed (byte* b = buffer)
{
ushort* ptr = (ushort*)b;
int count = (int)*ptr;
var points = new Point[count];
for (int i = 0; i < count; i++)
{
int x = (int)*(++ptr);
int y = (int)*(++ptr);
points[i] = new Point(x, y);
}
}
(Untested)
Have you looked at the Polyline entry on PInvoke.net yet?
Okay, a metafile record for a polyline... You might want to try doing a Buffer.BlockCopy from the byte array to a UInt16 array.