The application is a windows-based C# user interface for an embedded control and monitoring system.
The embedded system (written in C) maintains a table of faults which it sends to the c# application. The fault table contains one bit for each fault, stored in structure. Now from the point of view of the user interface the table is somewhat sparse - There are only a few of the fault bits that we are interested in, so the structure can be represented as below:
typedef struct
{
BYTE FaultsIAmNotInterestedIn0[14];
BYTE PowerSupplyFaults[4];
BYTE FaultsIAmNotInterestedIn1[5];
BYTE MachineryFaults[2];
BYTE FaultsIAmNotInterestedIn2[5];
BYTE CommunicationFaults[4];
}FAULT_TABLE;
Now, I want to be able to index each fault bit that I am interested in. In C I would use an enumeration to do this:
typedef enum
{
FF_PSU1 = offsetof(FAULT_TABLE,PowerSupplyFaults)*8,
FF_PSU2,
FF_PSU3,
FF_PSU4,
FF_PSU5,
FF_PSU6,
FF_PSU7,
FF_PSU8,
FF_PSU9,
FF_PSU10,
FF_PSU11,
FF_PSU12,
FF_PSU13,
FF_MACHINERY1 = offsetof(FAULT_TABLE,MachineryFaults)*8,
FF_MACHINERY2,
FF_MACHINERY3,
FF_MACHINERY4,
FF_MACHINERY5,
FF_MACHINERY6,
FF_MACHINERY7,
FF_MACHINERY8,
FF_MACHINERY9,
FF_MACHINERY10,
FF_COMMS1 = offsetof(FAULT_TABLE,CommunicationFaults)*8,
FF_COMMS2,
FF_COMMS3,
FF_COMMS4,
FF_COMMS5,
FF_COMMS6,
FF_COMMS7,
FF_COMMS8,
FF_COMMS9,
FF_COMMS10,
FF_COMMS11,
FF_COMMS12,
FF_COMMS13
}FAULT_FLAGS;
Is there a way that I can create a similar enumeration, based on a data structure in C#?
Add a Flags attribute to your enum. See http://msdn.microsoft.com/query/dev10.query?appId=Dev10IDEF1&l=EN-US&k=k(SYSTEM.FLAGSATTRIBUTE);k(TargetFrameworkMoniker-%22.NETFRAMEWORK%2cVERSION%3dV4.0%22);k(DevLang-CSHARP)&rd=true
You can get the offset of an element of a struct with Marshal.OffsetOf
http://msdn.microsoft.com/en-us/library/system.runtime.interopservices.marshal.offsetof.aspx
Your struct can be created in pretty much the same way although you need to ensure you specify the layout (although I suppose this depends on how the struct is instantiated).
http://msdn.microsoft.com/en-us/library/system.runtime.interopservices.structlayoutattribute(v=vs.71).aspx
However... the problem will come with the Enum. Enum values need to be compile time constants and Marshal.OffsetOf isn't. So you may have to store the offsets as static members of a class instead.
I also think you'll probably have to stray into unsafe code to make any real use of these offsets once you have them - and there is probably a better way to approach whatever you want to do in managed code.
You can create an enum to map the bits/flags, but you cannot map more than 32 bits.
In your example, I would create several const of long (64 bits). That won't solve the bit limitation when your array is very long (e.g. the first one).
Another trick is creating a static function that gives you the value of the bit (as a bool), upon the bit position in the array. In this case, the position (being a constant) is surely within the int range.
public static bool GetBit(byte[] buffer, int pos)
{
return (buffer[pos >> 3] & (1 << (pos & 7)) != 0);
}
Hope it helps.
Create an enum that specifies the bit-offset within each set of bytes:
enum PowerSupplyFaults
{
PowerSupplyFault1, PowerSupplyFault2, PowerSupplyFault3
}
PowerSupplyFault1 will be auto-assigned 0, Fault2 as 1, etc.
Assuming you receive a struct that mirrors the C version:
struct FaultTable
{
// ...
public byte[] PowerSupplyFaults;
public byte[] MachineFaults;
// ...
}
You can test for a fault by feeding the bytes into a BitArray and testing by index:
FaultTable faults = GetFaultTable();
BitArray psu_faults = new BitArray( fault_table.PowerSupplyFaults );
if ( psu_faults[ (int)PowerSupplyFaults.PowerSupplyFault3 ] )
{
// ...
}
Alternatively, you can walk all the bits and see which are set:
for ( int i = 0; i < psu_faults.Length; i++ )
{
if ( psu_faults[ i ] )
{
Console.WriteLine( "PSU fault: {0}", (PowerSupplyFaults)i );
}
}
Related
I'm working with a .NET application that stores a long list of true/false values into BitArray which gets stored in a SQL Server as a binary(32) value. The data comes back from the database as a byte[].
I'm trying to migrate the project over to Kotlin using Spring. After a lot of testing and messing around, I was finally able to get the same array in Kotlin that I do from the BitArray in C#. However, this solution seems like quite a kludge just to get an array of true/false values from a ByteArray.
I'm sure there's a cleaner way of going about this, but I'm still learning Kotlin and this is what I've come up with:
fun convertByteArrayToBitArray(array: ByteArray): List<Boolean>{
val stringBuilder = StringBuilder()
array.forEach {
// Convert to Integer to convert to binaryString
var int = it.toInt()
// Because the array has signed values, convert to unsigned value. Either this or add
// '.takeLast(8)' to the end of the String.format call
if(int < 0) int += 256
// Convert to binary string padding with leading 0s
val binary = String.format("%8s", Integer.toBinaryString(int)).replace(" ", "0")
// Through testing, this binary value needs to be reversed to give the right values
// at the right index
stringBuilder.append(binary.reversed())
}
// Convert stringBuilder to CharArray then to List of true/false values
return stringBuilder.toString().toCharArray().map {
Integer.parseInt(it.toString()) == 1
}
}
If you don't need multiplatform code, then you can use BitSet from the Java stdlib:
val bytes = byteArrayOf(0x11, 0x11)
val bits = BitSet.valueOf(bytes)
println(bits[0]) // true
println(bits[1]) // false
println(bits[4]) // true
println(bits[5]) // false
println(bits[8]) // true
println(bits[9]) // false
val bytes2 = bits.toByteArray()
It stores the data in little endian. If I read your code correctly, it also uses LE.
If you need List<Boolean> specifically, for example because some of your code already depends on it, then you can convert between BitSet and List<Boolean> like this:
fun BitSet.toBooleanList() = List(length()) { this[it] }
fun List<Boolean>.toBitSet() = BitSet(size).also { bitSet ->
forEachIndexed { i, item ->
bitSet.set(i, item)
}
}
Just note List<Boolean> is very memory-inefficient.
I am reverse engineering an old game format. For their textures, they store display type information in a single integer. I initially though the different bits were boolean flags. For example, if bit 1 was set, the texture is transparent. The more I have worked through the different display types, the less I think that is the case. So now I am handling them as just whole integers.
Here is some sample data:
1. When the texture is invisible, the flag integer is 0.
2. Diffuse - flag integer: -2147483647. Hex: 0x80000001
3. Semi Transparent - flag integer: -2147483643. Hex: 0x80000005
4. Masked - flag integer: -2147483629, Hex: 0x80000013
There are quite a few more types and they have their own unique integer value as well.
What is the easiest way to handle this data? An if/else statement with the huge negative integers is super ugly and hard to read. It looks like it would be easier to analyze them as hex values but each if/else statement requires an int.Parse statement like so:
if (parameters == int.Parse("0x80000001", NumberStyles.HexNumber))
{
// Handle this case
}
Is there any other suggested clean way of doing this and analyzing the data? Is it possible to just analyze the end of the hex value somehow?
There are quite a few more types and they have their own unique integer value as well...What is the easiest way to handle this data?
What you are describing is known as magic numbers. i.e. whenever your code needs to deal with arbitrary almost random numbers and it is not clear as to their meaning.
The best way to deal with them is to define them as constants once with comments and then to use the constants in code rather than repeating the “ugly” numbers throughout the code base.
Perhaps something like:
public static class TextureConstants
{
// TODO comments
public static int Diffuse = -2147483647;
public static int SemiTransparent = -2147483643;
public static int Masked= -2147483629;
}
Also their is no reason why the above could not have been done with an enum but if you are going to do lots of int comparison, comparing int with an enum requires pesky casting.
An if/else statement with the huge negative integers is super ugly and hard to read
Later you can use it in if or switch statements as so:
switch (someNumber)
{
case TextureConstants.Diffuse:
// do something
break;
case TextureConstants.SemiTransparent:
// do something
break;
case TextureConstants.Masked:
// do something
break;
}
Dictionary
If you don't like switch statements you can use a Dictionary<>:
public enum Textures
{
Diffuse,
SemiTransparent,
Masked
}
// initialise the dictionary
Dictionary<int, Textures> dict;
dict[TextureConstants.Diffuse] = Textures.Diffuse;
dict[TextureConstants.SemiTransparent] = Textures.SemiTransparent;
dict[TextureConstants.Masked] = Textures.Masked;
int someNumber = // ...
if (dict.TryGetValue (someNumber, out var texture))
{
// got one
if (texture == Textures.Diffuse) // ...
}
I have a managed array provided by a third party library. The type of the array does not directly reflect the data stored in the array, but instead it's the data interpreted as integers.
So int[] data = Lib.GetData(); gives me an integer array, that I would like to cast into an array of DataStructure, that could look like this.
struct DataStructure {
public int Id;
public double Value;
}
Current I use Marshal.Copy (an implementation can been seen here), but it seems a bit excessive to copy the entire thing.
Does something like this exists:
int[] data = Lib.GetData();
DataStructure[] dataStructs = InterpretAs<DataStructure>(data);
where no copying is required, but access to dataStruct elements can be done like dataStruct[1].Id?
EDIT 2:
If I just want a single DataStructure, I can use
public static T ToStruct<T>(byte[] bytes) where T : struct
{
GCHandle handle = GCHandle.Alloc(bytes, GCHandleType.Pinned);
T something = Marshal.PtrToStructure<T>(handle.AddrOfPinnedObject());
handle.Free();
return something;
}
where no copying is required.
EDIT:
The answers possible duplicate are currently about 7 years old, and they consists of either copying the data or implementing a hack.
There is actually a cheat, but it is an ugly unsafe totally unsafe cheat:
[StructLayout(LayoutKind.Sequential)]
//[StructLayout(LayoutKind.Sequential, Pack = 4)]
public struct DataStructure
{
public int Id;
public double Value;
}
[StructLayout(LayoutKind.Explicit)]
public struct DataStructureConverter
{
[FieldOffset(0)]
public int[] IntArray;
[FieldOffset(0)]
public DataStructure[] DataStructureArray;
}
and then you can convert it without problems:
var myarray = new int[8];
myarray[0] = 1;
myarray[3] = 2;
//myarray[4] = 2;
DataStructure[] ds = new DataStructureConverter { IntArray = myarray }.DataStructureArray;
int i1 = ds[0].Id;
int i2 = ds[1].Id;
Note that depending on the size of DataStructure (if it is 16 bytes or 12 bytes), you have to use Pack = 4 (if it is 12 bytes) or you don't need anything (see explanation (1) later)
I'll add that this technique is undocumented and totally unsafe. It even has a problem: ds.Length isn't the length of the DataStructure[] but is the length of the int[] (so in the example given it is 8, not 2)
The "technique" is the same I described here and originally described here.
explanation (1)
The sizeof(double) is 8 bytes, so Value is normally aligned on the 8 bytes boundary, so normally there is a "gap" between Id (that has sizeof(int) == 4) and Value of 4 bytes. So normally sizeof(DataStructure) == 16. Depending on how the DataStructure is built, there could not be this gap, so the Pack = 4 that forces alignment on the 4 byte boundary.
I know in C# we can always get the sub-array of a given array by using Array.Copy() method. However, this will consume more memory and processing time which is unnecessary in read-only situation. For example, I'm writing a heavy load network program which exchanges messages with other nodes in the cluster very frequently. The first 20 bytes of every message is the message header while the rest bytes make up the message body. Therefore, I will divide the received raw message into header byte array and body byte array in order to process them separately. However, this will obviously consume double memory and extra time. In C, we can easily use a pointer and assign offset to it to access different parts of the array.
For instance, in C language, if we have a char a[] = "ABCDEFGHIJKLMN", we can declare a char* ptr = a + 3 to represent the array DEFGHIJKLMN.
Is there a way to accomplish this in C#?
You might be interested in ArraySegments or unsafe.
ArraySegments delimits a section of a one-dimensional array.
Check ArraySegments in action
ArraySegments usage example:
int[] array = { 10, 20, 30 };
ArraySegment<int> segment = new ArraySegment<int>(array, 1, 2);
// The segment contains offset = 1, count = 2 and range = { 20, 30 }
Unsafe define an unsafe context in which pointers can be used.
Unsafe usage example:
int[] a = { 4, 5, 6, 7, 8 };
unsafe
{
fixed (int* c = a)
{
// use the pointer
}
}
First of all you must consider this as a premature optimization.
But you may use several ways to reduce memory consumption, if you sure you really need it:
1) You may use Flyweight pattern https://en.wikipedia.org/wiki/Flyweight_pattern to pool duplicated resources.
2) You may try to use unsafe directive and manual pointer management.
3) You may just switch to C for this functionality and just call native code from your C# program.
From my experience memory consumption for short-lived objects is not a big problem and I'd just write code with flyweight pattern and profile application afterwards.
Assuming you have a Message wrapper class in C#? Why not just add a property on it called header that returns the top 20 bytes.
You can easily accomplish this using skip and take suggested by Jonathon Reinhart above if you have the entire initial array in a memory array, but it sounds like you may have it in a network stream, which means the property might be a little more involved by doing a read of the initial 20 bytes from the the stream.
Something along the lines of:
class Message
{
private readonly Stream _stream;
private byte[] _inMemoryBytes;
public Message(Stream stream)
{
_stream = stream;
}
public IEnumerable<byte> Header
{
get
{
if (_inMemoryBytes.Length >= 20)
return _inMemoryBytes.Take(20);
_stream.Read(_inMemoryBytes, 0, 20);
return _inMemoryBytes.Take(20);
}
}
public IEnumerable<byte> FullMessage
{
get
{
// Read and return the whole message. You might want amend to data already read.
}
}
}
I'm unsure about whether to use a mutable struct or a mutable class.
My program stores an array with a lot of objects.
I've noticed that using a class doubles the amount of memory needed. However, I want the objects to be mutable, and I've been told that using mutable structs is evil.
This is what my type looks like:
struct /* or class */ Block
{
public byte ID;
public bool HasMetaData; // not sure whether HasMetaData == false or
// MetaData == null is faster, might remove this
public BlockMetaData MetaData; // BlockMetaData is always a class reference
}
Allocating a large amount of objects like this (notice that both codes below are run 81 times):
// struct
Block[,,] blocks = new Block[16, 256, 16];
uses about 35 MiB of memory, whilst doing it like this:
// class
Block[,,] blocks = new Block[16, 256, 16];
for (int z = 0; z < 16; z++)
for (int y = 0; y < 256; y++)
for (int x = 0; x < 16; x++)
blocks[x, y, z] = new Block();
uses about 100 MiB of ram.
So to conclude, my question is as follows:
Should I use a struct or a class for my Block type? Instances should be mutable and store a few values plus one object reference.
First off, if you really want to save memory then don't be using a struct or a class.
byte[,,] blockTypes = new byte[16, 256, 16];
BlockMetaData[,,] blockMetadata = new BlockMetaData[16, 256, 16];
You want to tightly pack similar things together in memory. You never want to put a byte next to a reference in a struct if you can possibly avoid it; such a struct will waste three to seven bytes automatically. References have to be word-aligned in .NET.
Second, I'm assuming that you're building a voxel system here. There might be a better way to represent the voxels than a 3-d array, depending on their distribution. If you are going to be making a truly enormous number of these things then store them in an immutable octree. By using the persistence properties of the immutable octree you can make cubic structures with quadrillions of voxels in them so long as the universe you are representing is "clumpy". That is, there are large regions of similarity throughout the world. You trade somewhat larger O(lg n) time for accessing and changing elements, but you get to have way, way more elements to work with.
Third, "ID" is a really bad way to represent the concept of "type". When I see "ID" I assume that the number uniquely identifies the element, not describes it. Consider changing the name to something less confusing.
Fourth, how many of the elements have metadata? You can probably do far better than an array of references if the number of elements with metadata is small compared to the total number of elements. Consider a sparse array approach; sparse arrays are much more space efficient.
Do they really have to be mutable? You could always make it an immutable struct with methods to create a new value with one field different:
struct Block
{
// I'd definitely get rid of the HasMetaData
private readonly byte id;
private readonly BlockMetaData metaData;
public Block(byte id, BlockMetaData metaData)
{
this.id = id;
this.metaData = metaData;
}
public byte Id { get { return id; } }
public BlockMetaData MetaData { get { return metaData; } }
public Block WithId(byte newId)
{
return new Block(newId, metaData);
}
public Block WithMetaData(BlockMetaData newMetaData)
{
return new Block(id, newMetaData);
}
}
I'm still not sure whether I'd make it a struct, to be honest - but I'd try to make it immutable either way, I suspect.
What are your performance requirements in terms of both memory and speed? How close does an immutable class come to those requirements?
An array of structs will offer better storage efficiency than an array of immutable references to distinct class instances having the same fields, because the latter will require all of the memory required by the former, in addition to memory to manage the class instances and memory required to hold the references. All that having been said, your struct as designed has a very inefficient layout. If you're really concerned about space, and every item in fact needs to independently store a byte, a Boolean, and a class reference, your best bet may be to either have two arrays of byte (a byte is actually smaller than a Boolean) and an array of class references, or else have an array of bytes, an array with 1/32 as many elements of something like BitVector32, and an array of class references.