Develop Reference

Optimal solution for "Bitwise AND" problem in C#

Problem statement:
Given an array of non-negative integers, count the number of unordered pairs of array elements, such that their bitwise AND is a power of 2.
Example:
arr = [10, 7, 2, 8, 3]
Answer: 6 (10&7, 10&2, 10&8, 10&3, 7&2, 2&3)
Constraints:
1 <= arr.Count <= 2*10^5
0 <= arr[i] <= 2^12
Here's my brute-force solution that I've come up with:
private static Dictionary<int, bool> _dictionary = new Dictionary<int, bool>();
public static long CountPairs(List<int> arr)
{
long result = 0;
for (var i = 0; i < arr.Count - 1; ++i)
{
for (var j = i + 1; j < arr.Count; ++j)
{
if (IsPowerOfTwo(arr[i] & arr[j])) ++result;
}
}
return result;
}
public static bool IsPowerOfTwo(int number)
{
if (_dictionary.TryGetValue(number, out bool value)) return value;
var result = (number != 0) && ((number & (number - 1)) == 0);
_dictionary[number] = result;
return result;
}
For small inputs this works fine, but for big inputs this works slow.
My question is: what is the optimal (or at least more optimal) solution for the problem? Please provide a graceful solution in C#. 😊

One way to accelerate your approach is to compute the histogram of your data values before counting.
This will reduce the number of computations for long arrays because there are fewer options for value (4096) than the length of your array (200000).
Be careful when counting bins that are powers of 2 to make sure you do not overcount the number of pairs by including cases when you are comparing a number with itself.

We can adapt the bit-subset dynamic programming idea to have a solution with O(2^N * N^2 + n * N) complexity, where N is the number of bits in the range, and n is the number of elements in the list. (So if the integers were restricted to [1, 4096] or 2^12, with n at 100,000, we would have on the order of 2^12 * 12^2 + 100000*12 = 1,789,824 iterations.)
The idea is that we want to count instances for which we have overlapping bit subsets, with the twist of adding a fixed set bit. Given Ai -- for simplicity, take 6 = b110 -- if we were to find all partners that AND to zero, we'd take Ai's negation,
110 -> ~110 -> 001
Now we can build a dynamic program that takes a diminishing mask, starting with the full number and diminishing the mask towards the left
001
^^^
001
^^
001
^
Each set bit on the negation of Ai represents a zero, which can be ANDed with either 1 or 0 to the same effect. Each unset bit on the negation of Ai represents a set bit in Ai, which we'd like to pair only with zeros, except for a single set bit.
We construct this set bit by examining each possibility separately. So where to count pairs that would AND with Ai to zero, we'd do something like
001 ->
001
000
we now want to enumerate
011 ->
011
010
101 ->
101
100
fixing a single bit each time.
We can achieve this by adding a dimension to the inner iteration. When the mask does have a set bit at the end, we "fix" the relevant bit by counting only the result for the previous DP cell that would have the bit set, and not the usual union of subsets that could either have that bit set or not.
Here is some JavaScript code (sorry, I do not know C#) to demonstrate with testing at the end comparing to the brute-force solution.
var debug = 0;
function bruteForce(a){
let answer = 0;
for (let i = 0; i < a.length; i++) {
for (let j = i + 1; j < a.length; j++) {
let and = a[i] & a[j];
if ((and & (and - 1)) == 0 && and != 0){
answer++;
if (debug)
console.log(a[i], a[j], a[i].toString(2), a[j].toString(2))
}
}
}
return answer;
}
function f(A, N){
const n = A.length;
const hash = {};
const dp = new Array(1 << N);
for (let i=0; i<1<<N; i++){
dp[i] = new Array(N + 1);
for (let j=0; j<N+1; j++)
dp[i][j] = new Array(N + 1).fill(0);
}
for (let i=0; i<n; i++){
if (hash.hasOwnProperty(A[i]))
hash[A[i]] = hash[A[i]] + 1;
else
hash[A[i]] = 1;
}
for (let mask=0; mask<1<<N; mask++){
// j is an index where we fix a 1
for (let j=0; j<=N; j++){
if (mask & 1){
if (j == 0)
dp[mask][j][0] = hash[mask] || 0;
else
dp[mask][j][0] = (hash[mask] || 0) + (hash[mask ^ 1] || 0);
} else {
dp[mask][j][0] = hash[mask] || 0;
}
for (let i=1; i<=N; i++){
if (mask & (1 << i)){
if (j == i)
dp[mask][j][i] = dp[mask][j][i-1];
else
dp[mask][j][i] = dp[mask][j][i-1] + dp[mask ^ (1 << i)][j][i - 1];
} else {
dp[mask][j][i] = dp[mask][j][i-1];
}
}
}
}
let answer = 0;
for (let i=0; i<n; i++){
for (let j=0; j<N; j++)
if (A[i] & (1 << j))
answer += dp[((1 << N) - 1) ^ A[i] | (1 << j)][j][N];
}
for (let i=0; i<N + 1; i++)
if (hash[1 << i])
answer = answer - hash[1 << i];
return answer / 2;
}
var As = [
[10, 7, 2, 8, 3] // 6
];
for (let A of As){
console.log(JSON.stringify(A));
console.log(`DP, brute force: ${ f(A, 4) }, ${ bruteForce(A) }`);
console.log('');
}
var numTests = 1000;
for (let i=0; i<numTests; i++){
const N = 6;
const A = [];
const n = 10;
for (let j=0; j<n; j++){
const num = Math.floor(Math.random() * (1 << N));
A.push(num);
}
const fA = f(A, N);
const brute = bruteForce(A);
if (fA != brute){
console.log('Mismatch:');
console.log(A);
console.log(fA, brute);
console.log('');
}
}
console.log("Done testing.");

int[] numbers = new[] { 10, 7, 2, 8, 3 };
static bool IsPowerOfTwo(int n) => (n != 0) && ((n & (n - 1)) == 0);
long result = numbers.AsParallel()
.Select((a, i) => numbers
.Skip(i + 1)
.Select(b => a & b)
.Count(IsPowerOfTwo))
.Sum();
If I understand the problem correctly, this should work and should be faster.
First, for each number in the array we grab all elements in the array after it to get a collection of numbers to pair with.
Then we transform each pair number with a bitwise AND, then counting the number that satisfy our 'IsPowerOfTwo;' predicate (implementation here).
Finally we simply get the sum of all the counts - our output from this case is 6.
I think this should be more performant than your dictionary based solution - it avoids having to perform a lookup each time you wish to check power of 2.
I think also given the numerical constraints of your inputs it is fine to use int data types.

Difference in C# and C++ Byte array conversion for object

I am trying to convert some C++ code to C# for an application. The function I am trying to convert calculates the Checksum of an object which comprises of MAC address among other details. The Checksum function in C++ is defined as :
unsigned short CalculateCheckSum(unsigned char* p, int n)
{
unsigned short x, checksum = 0;
for (unsigned long i = 0; i < n; ++i)
{
x = p[i];
x <<= i % 8;
checksum += x;
}
return checksum != 0 ? checksum : 51;
I have written the same function defined in C# is :
public static ushort CalculateCheckSum(byte[] p, int n)
{
ushort x, checksum = 0;
for (int i = 0; i < n; ++i)
{
x = p[i];
x <<= i % 8;
checksum += x;
}
return (ushort)(checksum != 0 ? checksum : 51);
}
Here is the code that calculates the checksum in C++ :
PCInfoClass pcInfo;
char nicIDStr[1024];
strcpy_s(nicIDStr, "34-29-8f-93-16-61");
NICAddressStrToBinary(nicIDStr, pcInfo.nicID);
char outbuf[1000];
pcInfo.timeStamp = 1234;
pcInfo.expDate = 0;
I32 pcInfoSz = 20;
pcInfo.checksum = 0;
unsigned char* byteStr;
byteStr = (unsigned char*)&pcInfo;
pcInfo.checksum = CalculateCheckSum(byteStr, pcInfoSz);
Since the CalculateCheckSum method, takes a Byte array as an argument, I have used the BinaryFormatter class which comes with System.Runtime. I have tried to replicate the same functionality in C# with the following lines :
PCInfoClass pcInfo = new PCInfoClass();
char[] nicIDStr = new char[1024];
string str = "34-29-8f-93-16-61";
for (int i = 0; i < str.Length; i++)
{
nicIDStr[i] = str[i];
}
NICAddressStrToBinary(nicIDStr, pcInfo.nicID);
pcInfo.timeStamp = 1234;
pcInfo.expDate = 0;
int pcInfoSz = 20;
pcInfo.checksum = 0;
pcInfo.checksum = CalculateCheckSum(ObjectToByteArray1(pcInfo), pcInfoSz);
public static byte[] ObjectToByteArray1(Object obj)
{
if (obj == null)
return null;
BinaryFormatter bf = new BinaryFormatter();
MemoryStream ms = new MemoryStream();
bf.Serialize(ms, obj);
return ms.ToArray();
}
Unfortunately the value of checksum comes out to be different for both approaches so the conversion is stuck at this point.
The other method used in this code is NICAddressStrToBinary, in C++ its defined as :
bool NICAddressStrToBinary(const char* nicIDStr, unsigned char* outbuf)
{
int c, i, dgt;
if (nicIDStr == NULL) return false;
//converted char to integer as ascii number.
for (dgt = 0, i = 0; (c = nicIDStr[i]) != '\0'; ++i)
{
//if it is 45 '-' then the loop will continue;
if (c == '-') continue;
//if the ascii value is between 48 to 57 then we will decrrease with 48 of given integer
if ('0' <= c && c <= '9')
{
c -= '0';
}
else
if ('a' <= c && c <= 'f')
{
c -= 'a' - 10;
}
else
if ('A' <= c && c <= 'F')
{
c -= 'A' - 10;
}
else
{
return false;
}
if (dgt >= 6 * 2)
{
return false;
}
if (outbuf != NULL)
{
if ((dgt & 1) == 0)
{
//// it means c<<4 is c*2power4
outbuf[dgt / 2] = c << 4;
}
else
{
outbuf[dgt / 2] |= c;
}
}
dgt++;
}
if (dgt < 6 * 2)
{
return false;
}
return true;
}
In C# its been rewritten as :
public static void NICAddressStrToBinary(char[] nicIDStr, byte[] outbuf)
{
int c, i, dgt;
if (nicIDStr == null) return ;
for (dgt = 0, i = 0; i<=nicIDStr.Length-1; ++i)
{
c = nicIDStr[i];
if (c == '-') continue;
if ('0' <= c && c <= '9')
{
c -= '0';
}
else if ('a' <= c && c <= 'f')
{
c -= 'a' - 10;
}
else if ('A' <= c && c <= 'F')
{
c -= 'A' - 10;
}
else
{
return;
}
/* make sure there aren't too many digits
*/
if (dgt >= 6 * 2)
{
return ;
}
/* accumulate the binary NIC ID
* remembering that we're starting
* with the most significant digits first
*/
if (outbuf != null)
{
if ((dgt & 1) == 0)
{
//// it means c<<4 is c*2power4
outbuf[dgt / 2] = (byte)(c << 4);
}
else
{
outbuf[dgt / 2] |= (byte)c;
}
}
/* advance the digit index
*/
dgt++;
}
/* make sure I have enough digits
*/
if (dgt < 6 * 2)
{
return ;
}
return ;
}
Can someone please tell me what could be the cause of different values being calculated in C++ and C#?

Since the CalculateCheckSum method, takes a Byte array as an argument, I have used the BinaryFormatter class which comes with System.Runtime.
That would have been a reasonable choice in isolation, but BinaryFormatter uses a complicated format that isn't "just the bytes of the object". Those bytes are probably in there somewhere, but a lot of other stuff is too. So in this case it doesn't work out.
Even in C# there are ways to get the raw bytes of a given object, but you would have to specifically design PCInfoClass for that purpose: make it a struct, use a fixed-size array for nicID (references are a no-go). Then you can use some tricks (which trick you can use depends on the version of .NET you're targeting) to get the raw bytes of that struct.
My recommendation would be to use a BinaryWriter to manually write each field to a MemoryStream, then use ToArray as you did. Be very careful to call the right overloads of Write, and explicitly write padding bytes as well. I cannot write that code for you without knowing what the class definition looked like in C++.

How to get last set bit in BitArray?

What is effective(fast) way to get last set bit in BitArray. (LINQ or simple backward for loop isn't very fast for large bitmaps. And I need fast) BitArray
I see next algorithm: go back through BitArray internal int array data and use some compiler Intrinsic Like C++ _BitScanReverse( don't know analog in C#).

The "normal" solution:
static long FindLastSetBit(BitArray array)
{
for (int i = array.Length - 1; i >= 0; i--)
{
if (array[i])
{
return i;
}
}
return -1;
}
The reflection solution (note - relies on implementation of BitArray):
static long FindLastSetBitReflection(BitArray array)
{
int[] intArray = (int[])array.GetType().GetField("m_array", System.Reflection.BindingFlags.Instance | System.Reflection.BindingFlags.NonPublic).GetValue(array);
for (var i = intArray.Length - 1; i >= 0; i--)
{
var b = intArray[i];
if (b != 0)
{
var pos = (i << 5) + 31;
for (int bit = 31; bit >= 0; bit--)
{
if ((b & (1 << bit)) != 0)
return pos;
pos--;
}
return pos;
}
}
return -1;
}
The reflection solution is 50-100x faster for me on large BitArrays (on very small ones the overhead of reflection will start to appear). It takes about 0.2 ms per megabyte on my machine.
The main thing is that if (b != 0) checks 32 bits at once. The inner loop which checks specific bits only runs once, when the correct word is found.
Edited: unsafe code removed because I realized almost nothing is gained by it, it only avoids the array boundary check and as the code is so fast already it doesn't matter that much. For the record, unsafe solution (~30% faster for me):
static unsafe long FindLastSetBitUnsafe(BitArray array)
{
int[] intArray = (int[])array.GetType().GetField("m_array", System.Reflection.BindingFlags.Instance | System.Reflection.BindingFlags.NonPublic).GetValue(array);
fixed (int* buffer = intArray)
{
for (var i = intArray.Length - 1; i >= 0; i--)
{
var b = buffer[i];
if (b != 0)
{
var pos = (i << 5) + 31;
for (int bit = 31; bit >= 0; bit--)
{
if ((b & (1 << bit)) != 0)
return pos;
pos--;
}
return pos;
}
}
}
return -1;
}

If you want the index of that last set bit you can do this in C# 6.
int? index = array.Select((b,i)=>{Index = i, Value = b})
.LastOrDefault(x => x.Value)
?.Index;
Otherwise you have to do something like this
var last = array.Select((b,i)=>{Index = i, Value = b})
.LastOrDefault(x => x.Value);
int? index = last == null ? (int?)null : last.Index;
Either way the index will be null if all the bits are zero.

I don't believe there is anything it can be done, other than iterate from last to first bit, and ask for each one if it is set. It could be done with something like:
BitArray bits = ...;
int lastSet = Enumerable.Range(1, bits.Length)
.Select(i => bits.Length - i)
.Where(i => bits[i])
.DefaultIfEmpty(-1)
.First();
That should return the last bit set, or -1 if none is. Haven't tested it myself, so it may need some adjustment.
Hope it helps.

Cannot assign XOR result to a BitArray type variable. How to fix this code?

I am trying to XOR the two LSBs of a blue byte of a pixel and store the result in an index of a BitArray variable, but C# gives me casting error of
Cannot implicitly convert type 'int' to 'bool'
I know C# treats XOR operation as int but I can't get this code fixed. Any idea how to fix this issue?
Bitmap img = new Bitmap(fname);
BitArray cipherBits = new BitArray(cipherStringSize * sizeof(Char) * 8);
int cipherBitCounter = 0;
Color pixel;
byte[] cipherByte = new byte[cipherBits.Length/8];
for (int i = 0; i < img.Height; i++)
{
for (int j = 0; j < img.Width; j++)
{
if (cipherBitCounter <= cipherBits.Length)
{
pixel = img.GetPixel(i, j);
cipherBits[cipherBitCounter] = ((pixel.B >> 1) & 1) ^ ((pixel.B) & 1); //error here
cipherBitCounter++;
}
else
{
//do something else
}
}

You could probably compare to 1:
cipherBits[cipherBitCounter] =( ((pixel.B >> 1) & 1) ^ ((pixel.B) & 1) ) == 1; //error here
But it is probably better to use Convert.ToBoolean:
https://msdn.microsoft.com/en-us/library/system.convert.toboolean%28v=vs.110%29.aspx
cipherBits[cipherBitCounter] = Convert.ToBoolean( ((pixel.B >> 1) & 1) ^ ((pixel.B) & 1) ); //error here

Operator '&' cannot be applied to operands of type 'ulong' and 'ulong*'

Operator '&' cannot be applied to operands of type 'ulong' and 'ulong*'
What am I doing wrong? I'm trying to find which masks a integer consists of, if that makes sense.
e.g.
63 = 1+2+4+8+16+32
unsafe
{
UInt64 n = Convert.ToUInt64(textAttributes.Text);
UInt64* p = &n;
for(UInt64 i = 1; i <= n; i <<= 1)
{
if (i & p)
{
switch(i)
{
default:
break;
}
}
}
}

You dont need unsafe code for this.
The compiler error is legitimate since you apply the operator & between a pointer and an integer
You probably want :
UInt64 n = 63;
for(int i = 0; i < 64; i++)
{
UInt64 j = ((UInt64) 1) << i;
if ((j & n) != 0)
{
Console.WriteLine(1 << i);
}
}

What you're trying to do there is a bitwise AND against a memory address. You need to dereference that pointer if you want to do anything with it:
if ((i & *p) != 0)
// ^^ dereference
Dereferencing, via an asterisk prefix, will retrieve the value at that memory address. Without it.. its the memory address itself 1
1. In C# it is a compiler error. But that is what it is.

Well you do not need unsafe context for such operation
Try this :
static void Main(string[] args)
{
UInt64 n = Convert.ToUInt64(63);
int size = Marshal.SizeOf(n) * 8;
for (int i = size - 1; i >= 0; i--)
{
Console.Write((n >> i) & 1);
}
}
This will print 0000000000000000000000000000000000000000000000000000000000111111 so you will know which bit are set!