Related
Problem statement:
Given an array of non-negative integers, count the number of unordered pairs of array elements, such that their bitwise AND is a power of 2.
Example:
arr = [10, 7, 2, 8, 3]
Answer: 6 (10&7, 10&2, 10&8, 10&3, 7&2, 2&3)
Constraints:
1 <= arr.Count <= 2*10^5
0 <= arr[i] <= 2^12
Here's my brute-force solution that I've come up with:
private static Dictionary<int, bool> _dictionary = new Dictionary<int, bool>();
public static long CountPairs(List<int> arr)
{
long result = 0;
for (var i = 0; i < arr.Count - 1; ++i)
{
for (var j = i + 1; j < arr.Count; ++j)
{
if (IsPowerOfTwo(arr[i] & arr[j])) ++result;
}
}
return result;
}
public static bool IsPowerOfTwo(int number)
{
if (_dictionary.TryGetValue(number, out bool value)) return value;
var result = (number != 0) && ((number & (number - 1)) == 0);
_dictionary[number] = result;
return result;
}
For small inputs this works fine, but for big inputs this works slow.
My question is: what is the optimal (or at least more optimal) solution for the problem? Please provide a graceful solution in C#. 😊
One way to accelerate your approach is to compute the histogram of your data values before counting.
This will reduce the number of computations for long arrays because there are fewer options for value (4096) than the length of your array (200000).
Be careful when counting bins that are powers of 2 to make sure you do not overcount the number of pairs by including cases when you are comparing a number with itself.
We can adapt the bit-subset dynamic programming idea to have a solution with O(2^N * N^2 + n * N) complexity, where N is the number of bits in the range, and n is the number of elements in the list. (So if the integers were restricted to [1, 4096] or 2^12, with n at 100,000, we would have on the order of 2^12 * 12^2 + 100000*12 = 1,789,824 iterations.)
The idea is that we want to count instances for which we have overlapping bit subsets, with the twist of adding a fixed set bit. Given Ai -- for simplicity, take 6 = b110 -- if we were to find all partners that AND to zero, we'd take Ai's negation,
110 -> ~110 -> 001
Now we can build a dynamic program that takes a diminishing mask, starting with the full number and diminishing the mask towards the left
001
^^^
001
^^
001
^
Each set bit on the negation of Ai represents a zero, which can be ANDed with either 1 or 0 to the same effect. Each unset bit on the negation of Ai represents a set bit in Ai, which we'd like to pair only with zeros, except for a single set bit.
We construct this set bit by examining each possibility separately. So where to count pairs that would AND with Ai to zero, we'd do something like
001 ->
001
000
we now want to enumerate
011 ->
011
010
101 ->
101
100
fixing a single bit each time.
We can achieve this by adding a dimension to the inner iteration. When the mask does have a set bit at the end, we "fix" the relevant bit by counting only the result for the previous DP cell that would have the bit set, and not the usual union of subsets that could either have that bit set or not.
Here is some JavaScript code (sorry, I do not know C#) to demonstrate with testing at the end comparing to the brute-force solution.
var debug = 0;
function bruteForce(a){
let answer = 0;
for (let i = 0; i < a.length; i++) {
for (let j = i + 1; j < a.length; j++) {
let and = a[i] & a[j];
if ((and & (and - 1)) == 0 && and != 0){
answer++;
if (debug)
console.log(a[i], a[j], a[i].toString(2), a[j].toString(2))
}
}
}
return answer;
}
function f(A, N){
const n = A.length;
const hash = {};
const dp = new Array(1 << N);
for (let i=0; i<1<<N; i++){
dp[i] = new Array(N + 1);
for (let j=0; j<N+1; j++)
dp[i][j] = new Array(N + 1).fill(0);
}
for (let i=0; i<n; i++){
if (hash.hasOwnProperty(A[i]))
hash[A[i]] = hash[A[i]] + 1;
else
hash[A[i]] = 1;
}
for (let mask=0; mask<1<<N; mask++){
// j is an index where we fix a 1
for (let j=0; j<=N; j++){
if (mask & 1){
if (j == 0)
dp[mask][j][0] = hash[mask] || 0;
else
dp[mask][j][0] = (hash[mask] || 0) + (hash[mask ^ 1] || 0);
} else {
dp[mask][j][0] = hash[mask] || 0;
}
for (let i=1; i<=N; i++){
if (mask & (1 << i)){
if (j == i)
dp[mask][j][i] = dp[mask][j][i-1];
else
dp[mask][j][i] = dp[mask][j][i-1] + dp[mask ^ (1 << i)][j][i - 1];
} else {
dp[mask][j][i] = dp[mask][j][i-1];
}
}
}
}
let answer = 0;
for (let i=0; i<n; i++){
for (let j=0; j<N; j++)
if (A[i] & (1 << j))
answer += dp[((1 << N) - 1) ^ A[i] | (1 << j)][j][N];
}
for (let i=0; i<N + 1; i++)
if (hash[1 << i])
answer = answer - hash[1 << i];
return answer / 2;
}
var As = [
[10, 7, 2, 8, 3] // 6
];
for (let A of As){
console.log(JSON.stringify(A));
console.log(`DP, brute force: ${ f(A, 4) }, ${ bruteForce(A) }`);
console.log('');
}
var numTests = 1000;
for (let i=0; i<numTests; i++){
const N = 6;
const A = [];
const n = 10;
for (let j=0; j<n; j++){
const num = Math.floor(Math.random() * (1 << N));
A.push(num);
}
const fA = f(A, N);
const brute = bruteForce(A);
if (fA != brute){
console.log('Mismatch:');
console.log(A);
console.log(fA, brute);
console.log('');
}
}
console.log("Done testing.");
int[] numbers = new[] { 10, 7, 2, 8, 3 };
static bool IsPowerOfTwo(int n) => (n != 0) && ((n & (n - 1)) == 0);
long result = numbers.AsParallel()
.Select((a, i) => numbers
.Skip(i + 1)
.Select(b => a & b)
.Count(IsPowerOfTwo))
.Sum();
If I understand the problem correctly, this should work and should be faster.
First, for each number in the array we grab all elements in the array after it to get a collection of numbers to pair with.
Then we transform each pair number with a bitwise AND, then counting the number that satisfy our 'IsPowerOfTwo;' predicate (implementation here).
Finally we simply get the sum of all the counts - our output from this case is 6.
I think this should be more performant than your dictionary based solution - it avoids having to perform a lookup each time you wish to check power of 2.
I think also given the numerical constraints of your inputs it is fine to use int data types.
I solved a task on Hackerrank.com, where the problem was like this:
You have an Array. This Array contains numbers.
Now you enter two numbers:
The first one describes a sum
The second one describes the amount of indexes (sequence length) you add together
In the end you get the amount of sequences whose sum is your defined number
For example:
Your array is [ 1, 2, 3, 4], your sum is 3 and your sequence length is 2.
Now you take the first two indexes and output the sum: [1, 2] = 3.
This is equal to your sum, so now you have found one sequence.
The next sequence is [ 2, 3 ] = 5. This is not equal to 3, so your sequence counter stays 1.
The last sequence is [3, 4] = 7. This is also not equal to 3 and in the end, you found one sequence.
I wrote this code for that:
static int GetSequences(List<int> s, int d, int m)
{
//m = segment-length
//d = sum
int count = 0;
int j = 0;
int k = 0;
do
{
try
{
List<int> temp = new List<int>();
for (int i = 0; i < m; i++)
{
temp.Add(s[i + k]);
}
if (temp.Sum() == d)
{
count++;
}
j++;
k++;
}
catch (ArgumentOutOfRangeException)
{
break;
}
} while (true);
return count;
}
As I didn't know how often I have to count
(For example a 6-Length-Array with a sequence-length of 3 has 4 sequences (1,2,3 | 2,3,4 | 3,4,5 | 4,5,6)),
I am stopping the while loop when the index is out of range. but I'm not sure if this solution is okay. Not just with program speed, but also with code cleanliness. Is this code acceptable, or is it better to use a for loop, which loops for example exactly 4 times for a 6-length array with 3-Length sequences?
It's not recommended, no. Exceptions should be reserved for stuff that isn't supposed to happen, not flow control or validation.
What you want is to use conditional logic (if statements) and the break keyword.
Also, codereview.stackexchange.com is better suited for these kinds of questions.
It would be better to fix your code so that it doesn't routinely throw exceptions:
You sum each of these segments:
0 1 2 3 start = 0
| | summing indexes: 0, 1
+--+
0 1 2 3 start = 1
| | summing indexes: 1, 2
+--+
0 1 2 3 start = 2
| | summing indexes: 2, 3
+--+
The bracket starts at the index start, and has a size of m. The length of s is given by s.Count. Therefore we want to keep going until start + m == s.Count.
(I always find it's useful to draw these things out, and put sample numbers in, in order to make sure you've got the maths right. In the sample above, you can see that we stop when start (2) + m (2) == the array size (4))
static int GetSequences(List<int> s, int d, int m)
{
//m = segment-length
//d = sum
int count = 0;
for (int start = 0; start + m <= s.Count; start++)
{
List<int> temp = new List<int>();
for (int i = 0; i < m; i++)
{
temp.Add(s[start + i]);
}
if (temp.Sum() == d)
{
count++;
}
}
return count;
}
However, you can improve your code a bit:
Use meaningful variable names
Don't create a new temporary list each time, just to sum it
Check your inputs
static int GetSequences(List<int> numbers, int targetSum, int segmentLength)
{
if (numbers == null)
throw new ArgumentNullException(nameof(numbers));
if (segmentLength > numbers.Count)
throw new ArgumentException("segmentLength must be <= numbers.Count");
int count = 0;
for (int start = 0; start + segmentLength <= numbers.Count; start++)
{
int sum = 0;
for (int i = 0; i < segmentLength; i++)
{
sum += numbers[start + i];
}
if (sum == targetSum)
{
count++;
}
}
}
Usually except for switch/case there is often no real reason to use break.
Also an exception MUST be as the name says exceptional, so it MUST NOT be a part of your logic.
As said Jeppe you can use the methods and attributes the framework provides you to do as you like.
Here s.Count seems to be the way to go.
int[] arr = new[] { 1, 2, 1, 2 };
// Sum and len are given by the task.
// 'last' is the last index where we should stop iterating.
int sum = 3, len = 2, last = arr.Length - len;
// One of the overloads of Where accepts index, i.e. the position of element.
// 1) We check that we don't go after our stop-index (last).
// 2) Avoid exception by using '&&'.
// 3) We use C# 8 Range (..) to get the slice of the numbers we need:
// we start from the current position (index) till then index,
// calculated as current index + length given by the task.
// 4) Sum all the numbers in the slice (Sum()) and compare it with the target sum,
// given by the task (== sum).
// 5) The count of all matches (Count()) is the sought amount of sequences.
int count = arr.Where((z, index) => index <= last && arr[index..(index+len)].Sum() == sum).Count();
I had a go at a project euler coding challenge below, the answer given by the code is correct but I do not understand why its taking nearly a minute to run. It was finishing with similar times prior to using a sieve. Others users are reporting times as low as milliseconds.
I assume I am making a basic error somewhere...
// The sum of the primes below 10 is 2 + 3 + 5 + 7 = 17.
// Find the sum of all the primes below two million.
public static long Ex010()
{
var sum = 0L;
var sieve = new bool[2000000];
var primes = new List<int>(10000);
for (int i = 2; i < sieve.Length; i++)
{
if (sieve[i-1])
continue;
var isPrime = true;
foreach (var prime in primes)
{
if (i % prime == 0) {
isPrime = false;
break;
}
}
if (isPrime) {
primes.Add(i);
sum += i;
for (var x = i * 2; x < sieve.Length; x += i) {
sieve[x-1] = true;
}
}
}
return sum;
}
EDIT:
The only thing that seemed missing was this optimization :
if (prime > Math.Sqrt(i))
break;
It brings the time down to 160 ms.
EDIT 2:
Finally clicked, took out the foreach as was suggested many times. Its now 12 ms. Final solution :
public static long Ex010()
{
var sum = 0L;
var sieve = new bool[2000000];
for (int i = 2; i < sieve.Length; i++)
{
if (sieve[i-1])
continue;
sum += i;
for (var x = i * 2; x < sieve.Length; x += i) {
sieve[x-1] = true;
}
}
return sum;
}
You are doing trial division in addition to a sieve.
The boolean array will already tell you if a number is prime, so you don't need the List of primes at all.
You can also speed it up by only sieving up to the square root of the limit.
If you want to save some memory also, you can use a BitArray instead of a boolean array.
public static long Ex010()
{
const int Limit = 2000000;
int sqrt = (int)Math.Sqrt(Limit);
var sum = 0L;
var isComposite = new bool[Limit];
for (int i = 2; i < sqrt; i++) {
if (isComposite[i - 2])
continue;//This number is not prime, skip
sum += i;
for (var x = i * i; x < isComposite.Length; x += i) {
isComposite[x - 2] = true;
}
}
//Add the remaining prime numbers
for (int i = sqrt; i < Limit; i++) {
if (!isComposite[i - 2]) {
sum += i;
}
}
return sum;
}
(tl;dr: 2 million in 0.8 ms, 2 billion in 1.25 s; segmented odds-only SoE, presieving, wheeled striding)
As always, the limit of Euler task #10 seems designed to pose a mild challenge on a ZX81, Apple ][ or C64 but on modern hardware you generally have to multiply the limits by 1000 to make things even remotely interesting. Or set a time limit like 5 seconds and try to see by how many orders of magnitude the Euler limit can be exceeded...
Dennis_E's solution is simple and efficient, but I'd recommend applying two small improvements that give a marked performance boost without any effort at all.
Represent only odd numbers in the sieve
All even numbers except for the number 2 are composite. If you pull the number 2 out of thin air when needed then you can drop all even numbers from the sieve. This halves workload and memory footprint, for a doubling of performance at the marginal cost of writing << or >> in a few places (to convert between the realm of numbers and the realm of bit indices). This is usually known as an 'odds-only sieve' or 'wheeled representation mod 2'; it has the added advantage that it largely removes the need for guarding against index overflow.
Skip a few extra small primes during decoding
Skipping a few small primes ('applying a wheel') is much easier when going through a range of numbers incrementally, compared to hopping wildly about with different strides as during sieving. This skipping only involves applying a cyclic sequence of differences between consecutive numbers that are not multiples of the primes in question, like 4,2,4,2... for skipping multiples of 2 and 3 (the mod 6 wheel) or 6,4,2,4,2,4,6,2... for skipping multiples of 2, 3 and 5.
The mod 6 wheel sequence alternates between only two numbers, which can easily be achieved by XORing with a suitable value. On top of the odds-only sieve the distances are halved, so that the sequence becomes 2,1,2,1... This skipping reduces the work during decoding by 1/3rd (for stepping mod 3), and the skipped primes can also be ignored during the sieving. The latter can have a marked effect on sieve times, since the smallest primes do the greatest number of crossings-off during the sieving.
Here's a simple Sieve of Eratosthenes, with both suggestions applied. Note: here and in the following I generally go with the flow of C#/.Net and use signed integers where I would normally use unsigned integers in any sane language. That's because I don't have the time to vet the code for the performance implications (penalties) resulting from the use of unsigned types, like that the compiler suddenly forgets how to replace division by a constant with multiplication of the inverse and so on.
static long sum_small_primes_up_to (int n)
{
if (n < 7)
return (0xAA55200 >> (n << 2)) & 0xF;
int sqrt_n_halved = (int)Math.Sqrt(n) >> 1;
int max_bit = (int)(n - 1) >> 1;
var odd_composite = new bool[max_bit + 1];
for (int i = 5 >> 1; i <= sqrt_n_halved; ++i)
if (!odd_composite[i])
for (int p = (i << 1) + 1, j = p * p >> 1; j <= max_bit; j += p)
odd_composite[j] = true;
long sum = 2 + 3;
for (int i = 5 >> 1, d = 1; i <= max_bit; i += d, d ^= 3)
if (!odd_composite[i])
sum += (i << 1) + 1;
return sum;
}
The first if statement handles the small fry (n in 0..6) by returning a suitable element of a precomputed list of numbers, and it serves to get all the special cases out of the way in one fell swoop. All other occurrences of shift operators are for converting between the realm of numbers and the realm of indices into the odds-only sieve.
This is pretty much the same code that I normally use for sieving small primes, up to 64K or so (the potential least factors for numbers up to 32 bits). It does Euler's piddling 2 million in 4.5 milliseconds but throwing bigger numbers at it shows its Achilles heel: it does a lot of striding over large distances, which interacts badly with modern memory subsystems where decent access speed can only be got from caches. The performance drops markedly when the capacity of the level 1 cache (typically 32 KiByte) is exceeded significantly, and it goes down even further when exceeding the L2 and L3 capacities (typically several megabytes). How sharp the drop is depends on the quality (price tag) of the computer, of course...
Here are some timings taken on my laptop:
# benchmark: small_v0 ...
sum up to 2 * 10^4: 21171191 in 0,03 ms
sum up to 2 * 10^5: 1709600813 in 0,35 ms // 11,0 times
sum up to 2 * 10^6: 142913828922 in 4,11 ms // 11,7 times
sum up to 2 * 10^7: 12272577818052 in 59,36 ms // 14,4 times
sum up to 2 * 10^8: 1075207199997334 in 1.225,19 ms // 20,6 times
sum up to 2 * 10^9: 95673602693282040 in 14.381,29 ms // 11,7 times
In the middlish ranges there are time increases that go well beyond the expected factor of about 11, and then things stabilise again.
And here comes how to speed up the beast an order of magnitude...
Process the range in cache-sized segments
The remedy is easy enough: instead of striding each prime all the way from one end of the range to the other - and hence all across the memory space - we sieve the range in cache-sized strips, memorising the final positions for each prime so that the next round can continue right where the previous round left off. If we don't need a big bad sieve full of bits at the end then we can process a strip (extract its primes) after it has been sieved and then discard its data, reusing the sieve buffer for the next strip. Both are variations on the theme of segmented sieving but the offsets are treated differently during the processing; when the distinction matters then the first approach (big bad sieve for the whole range) is usually called a segmented sieve and the latter an iterated sieve. The terms 'moving' or 'sliding' sieve might fit the latter somewhat but should be avoided because they normally refer to a totally different class of sieves (also known as deque sieves) that are deceptively simple but whose performance is worse by at least an order of magnitude.
Here's an example of an iterated sieve, a slightly modified version of a function that I normally use for sieving primes in given ranges [m, n], just like in SPOJ's PRIMES1 and PRINT. Here the parameter m is implicitly 0, so it doesn't need to be passed.
Normally the function takes an interface that is responsible for processing the raw sieve (and any lose primes it may get passed), and which can get queried for the number of primes that the processor skips ('decoder order') so that the sieve can ignore those during the sieving. For this exposition I've replaced this with a delegate for simplicity.
The factor primes get sieved by a stock function that may look somewhat familiar, and I've changed the logic of the sieve from 'is_composite' to 'not_composite' (and to a base type that can participate in arithmetic) for reasons that will be explained later. decoder_order is the number of additional primes skipped by the decoder (which would be 1 for the function shown earlier, because it skips multiples of the prime 3 during the prime extraction/summing, over and above the wheel prime 2).
const int SIEVE_BITS = 1 << 15; // L1 cache size, 1 usable bit per byte
delegate long sieve_sum_func (byte[] sieve, int window_base, int window_bits);
static long sum_primes_up_to (int n, sieve_sum_func sum_func, int decoder_order)
{
if (n < 7)
return 0xF & (0xAA55200 >> (n << 2));
n -= ~n & 1; // make odd (n can't be 0 here)
int sqrt_n = (int)Math.Sqrt(n);
var factor_primes = small_primes_up_to(sqrt_n).ToArray();
int first_sieve_prime_index = 1 + decoder_order; // skip wheel primes + decoder primes
int m = 7; // this would normally be factor_primes[first_sieve_prime_index] + 2
int bits_to_sieve = ((n - m) >> 1) + 1;
int sieve_bits = Math.Min(bits_to_sieve, SIEVE_BITS);
var sieve = new byte[sieve_bits];
var offsets = new int[factor_primes.Length];
int sieve_primes_end = first_sieve_prime_index;
long sum = 2 + 3 + 5; // wheel primes + decoder primes
for (int window_base = m; ; )
{
int window_bits = Math.Min(bits_to_sieve, sieve_bits);
int last_number_in_window = window_base - 1 + (window_bits << 1);
while (sieve_primes_end < factor_primes.Length)
{
int prime = factor_primes[sieve_primes_end];
int start = prime * prime, stride = prime << 1;
if (start > last_number_in_window)
break;
if (start < window_base)
start = (stride - 1) - (window_base - start - 1) % stride;
else
start -= window_base;
offsets[sieve_primes_end++] = start >> 1;
}
fill(sieve, window_bits, (byte)1);
for (int i = first_sieve_prime_index; i < sieve_primes_end; ++i)
{
int prime = factor_primes[i], j = offsets[i];
for ( ; j < window_bits; j += prime)
sieve[j] = 0;
offsets[i] = j - window_bits;
}
sum += sum_func(sieve, window_base, window_bits);
if ((bits_to_sieve -= window_bits) == 0)
break;
window_base += window_bits << 1;
}
return sum;
}
static List<int> small_primes_up_to (int n)
{
int upper_bound_on_pi = 32 + (n < 137 ? 0 : (int)(n / (Math.Log(n) - 1.083513)));
var result = new List<int>(upper_bound_on_pi);
if (n < 2)
return result;
result.Add(2); // needs to be pulled out of thin air because of the mod 2 wheel
if (n < 3)
return result;
result.Add(3); // needs to be pulled out of thin air because of the mod 3 decoder
int sqrt_n_halved = (int)Math.Sqrt(n) >> 1;
int max_bit = (n - 1) >> 1;
var odd_composite = new bool[max_bit + 1];
for (int i = 5 >> 1; i <= sqrt_n_halved; ++i)
if (!odd_composite[i])
for (int p = (i << 1) + 1, j = p * p >> 1; j <= max_bit; j += p)
odd_composite[j] = true;
for (int i = 5 >> 1, d = 1; i <= max_bit; i += d, d ^= 3)
if (!odd_composite[i])
result.Add((i << 1) + 1);
return result;
}
static void fill<T> (T[] array, int count, T value, int threshold = 16)
{
Trace.Assert(count <= array.Length);
int current_size = Math.Min(threshold, count);
for (int i = 0; i < current_size; ++i)
array[i] = value;
for (int half = count >> 1; current_size <= half; current_size <<= 1)
Buffer.BlockCopy(array, 0, array, current_size, current_size);
Buffer.BlockCopy(array, 0, array, current_size, count - current_size);
}
Here's sieve processor that is equivalent to the logic used in the function shown at the beginning, and a dummy function that can be used to measure the sieve time sans any decoding, for comparison:
static long prime_sum_null (byte[] sieve, int window_base, int window_bits)
{
return 0;
}
static long prime_sum_v0 (byte[] sieve, int window_base, int window_bits)
{
long sum = 0;
int i = window_base % 3 == 0 ? 1 : 0;
int d = 3 - (window_base + 2 * i) % 3;
for ( ; i < window_bits; i += d, d ^= 3)
if (sieve[i] == 1)
sum += window_base + (i << 1);
return sum;
}
This function needs to perform a bit of modulo magic to synchronise itself with the mod 3 sequence over the mod 2 sieve; the earlier function did not need to do this because its starting point was fixed, not a parameter. Here are the timings:
# benchmark: iter_v0 ...
sum up to 2 * 10^4: 21171191 in 0,04 ms
sum up to 2 * 10^5: 1709600813 in 0,28 ms // 7,0 times
sum up to 2 * 10^6: 142913828922 in 2,42 ms // 8,7 times
sum up to 2 * 10^7: 12272577818052 in 22,11 ms // 9,1 times
sum up to 2 * 10^8: 1075207199997334 in 223,67 ms // 10,1 times
sum up to 2 * 10^9: 95673602693282040 in 2.408,06 ms // 10,8 times
Quite a difference, n'est-ce pas? But we're not done yet.
Replace conditional branching with arithmetic
Modern processors like things to be simple and predictable; if branches are not predicted correctly then the CPU levies a heavy fine in extra cycles for flushing and refilling the instruction pipeline. Unfortunately, the decoding loop isn't very predictable because primes are fairly dense in the low number ranges we're talking about here:
if (!odd_composite[i])
++count;
If the average number of non-primes between primes times the cost of an addition is less than the penalty for a mispredicted branch then the following statement should be faster:
count += sieve[i];
This explains why I inverted the logic of the sieve compared to normal, because with 'is_composite' semantics I'd have to do
count += 1 ^ odd_composite[i];
And the rule is to pull everything out of inner loops that can be pulled out, so that I simply applied 1 ^ x to the whole array before even starting.
However, Euler wants us to sum the primes instead of counting them. This can be done in a similar fashion, by turning the value 1 into a mask of all 1 bits (which preserves everything when ANDing) and 0 zeroises any value. This is similar to the CMOV instruction, except that it works even on the oldest of CPUs and does not require a reasonably decent compiler:
static long prime_sum_v1 (byte[] sieve, int window_base, int window_bits)
{
long sum = 0;
int i = window_base % 3 == 0 ? 1 : 0;
int d = 3 - (window_base + 2 * i) % 3;
for ( ; i < window_bits; i += d, d ^= 3)
sum += (0 - sieve[i]) & (window_base + (i << 1));
return sum;
}
Result:
# benchmark: iter_v1 ...
sum up to 2 * 10^4: 21171191 in 0,10 ms
sum up to 2 * 10^5: 1709600813 in 0,36 ms // 3,6 times
sum up to 2 * 10^6: 142913828922 in 1,88 ms // 5,3 times
sum up to 2 * 10^7: 12272577818052 in 13,80 ms // 7,3 times
sum up to 2 * 10^8: 1075207199997334 in 157,39 ms // 11,4 times
sum up to 2 * 10^9: 95673602693282040 in 1.819,05 ms // 11,6 times
Unrolling and strength reduction
Now, a bit of overkill: a decoder with a fully unrolled wheel mod 15 (the unrolling can unlock some reserves of instruction-level parallelism).
static long prime_sum_v5 (byte[] sieve, int window_base, int window_bits)
{
Trace.Assert(window_base % 2 == 1);
int count = 0, sum = 0;
int residue = window_base % 30;
int phase = UpperIndex[residue];
int i = (SpokeValue[phase] - residue) >> 1;
// get into phase for the unrolled code (which is based on phase 0)
for ( ; phase != 0 && i < window_bits; i += DeltaDiv2[phase], phase = (phase + 1) & 7)
{
int b = sieve[i]; count += b; sum += (0 - b) & i;
}
// process full revolutions of the wheel (anchored at phase 0 == residue 1)
for (int e = window_bits - (29 >> 1); i < e; i += (30 >> 1))
{
int i0 = i + ( 1 >> 1), b0 = sieve[i0]; count += b0; sum += (0 - b0) & i0;
int i1 = i + ( 7 >> 1), b1 = sieve[i1]; count += b1; sum += (0 - b1) & i1;
int i2 = i + (11 >> 1), b2 = sieve[i2]; count += b2; sum += (0 - b2) & i2;
int i3 = i + (13 >> 1), b3 = sieve[i3]; count += b3; sum += (0 - b3) & i3;
int i4 = i + (17 >> 1), b4 = sieve[i4]; count += b4; sum += (0 - b4) & i4;
int i5 = i + (19 >> 1), b5 = sieve[i5]; count += b5; sum += (0 - b5) & i5;
int i6 = i + (23 >> 1), b6 = sieve[i6]; count += b6; sum += (0 - b6) & i6;
int i7 = i + (29 >> 1), b7 = sieve[i7]; count += b7; sum += (0 - b7) & i7;
}
// clean up leftovers
for ( ; i < window_bits; i += DeltaDiv2[phase], phase = (phase + 1) & 7)
{
int b = sieve[i]; count += b; sum += (0 - b) & i;
}
return (long)window_base * count + ((long)sum << 1);
}
As you can see, I performed a bit of strength reduction in order to make things easier for the compiler. Instead of summing window_base + (i << 1), I sum i and 1 separately and perform the rest of the calculation only once, at the end of the function.
Timings:
# benchmark: iter_v5(1) ...
sum up to 2 * 10^4: 21171191 in 0,01 ms
sum up to 2 * 10^5: 1709600813 in 0,11 ms // 9,0 times
sum up to 2 * 10^6: 142913828922 in 1,01 ms // 9,2 times
sum up to 2 * 10^7: 12272577818052 in 11,52 ms // 11,4 times
sum up to 2 * 10^8: 1075207199997334 in 130,43 ms // 11,3 times
sum up to 2 * 10^9: 95673602693282040 in 1.563,10 ms // 12,0 times
# benchmark: iter_v5(2) ...
sum up to 2 * 10^4: 21171191 in 0,01 ms
sum up to 2 * 10^5: 1709600813 in 0,09 ms // 8,7 times
sum up to 2 * 10^6: 142913828922 in 1,03 ms // 11,3 times
sum up to 2 * 10^7: 12272577818052 in 10,34 ms // 10,0 times
sum up to 2 * 10^8: 1075207199997334 in 121,08 ms // 11,7 times
sum up to 2 * 10^9: 95673602693282040 in 1.468,28 ms // 12,1 times
The first set of timings is for decoder_order == 1 (i.e. not telling the sieve about the extra skipped prime), for direct comparison to the other decoder versions. The second set is for decoder_order == 2, which means the sieve could skip the crossings-off for the prime 5 as well. Here are the null timings (essentially the sieve time without the decode time), to put things a bit into perspective:
# benchmark: iter_null(1) ...
sum up to 2 * 10^8: 10 in 94,74 ms // 11,4 times
sum up to 2 * 10^9: 10 in 1.194,18 ms // 12,6 times
# benchmark: iter_null(2) ...
sum up to 2 * 10^8: 10 in 86,05 ms // 11,9 times
sum up to 2 * 10^9: 10 in 1.109,32 ms // 12,9 times
This shows that the work on the decoder has decreased decode time for 2 billion from 1.21 s to 0.35 s, which is nothing to sneeze at. Similar speedups can be realised for the sieving as well, but that is nowhere near as easy as it was for the decoding.
Low-hanging fruit: presieving
Lastly, a technique that can sometimes offer dramatic speedups (especially for packed bitmaps and/or higher-order wheels) is blasting a canned bit pattern over the sieve before comencing a round of sieving, such that the sieve looks as if it had already been sieved by a handful of small primes. This is usually known as presieving. In the current case the speedup is marginal (not even 20%) but I'm showing it because it is a useful technique to have in one's toolchest.
Note: I've ripped the presieving logic from another Euler project, so it doesn't fit organically into the code I wrote for this article. But it should demonstrate the technique well enough.
const byte CROSSED_OFF = 0; // i.e. composite
const byte NOT_CROSSED = 1 ^ CROSSED_OFF; // i.e. not composite
const int SIEVE_BYTES = SIEVE_BITS; // i.e. 1 usable bit per byte
internal readonly static byte[] TinyPrimes = { 2, 3, 5, 7, 11, 13, 17, 19, 23, 29, 31 };
internal readonly static int m_wheel_order = 3; // == number of wheel primes
internal static int m_presieve_level = 0; // == number of presieve primes
internal static int m_presieve_modulus = 0;
internal static byte[] m_presieve_pattern;
internal static void set_presieve_level (int presieve_primes)
{
m_presieve_level = Math.Max(0, presieve_primes);
m_presieve_modulus = 1;
for (int i = m_wheel_order; i < m_wheel_order + presieve_primes; ++i)
m_presieve_modulus *= TinyPrimes[i];
// the pattern needs to provide SIEVE_BYTES bytes for every residue of the modulus
m_presieve_pattern = new byte[m_presieve_modulus + SIEVE_BYTES - 1];
var pattern = m_presieve_pattern;
int current_size = 1;
pattern[0] = NOT_CROSSED;
for (int i = m_wheel_order; i < m_wheel_order + presieve_primes; ++i)
{
int current_prime = TinyPrimes[i];
int new_size = current_size * current_prime;
// keep doubling while possible
for ( ; current_size * 2 <= new_size; current_size *= 2)
Buffer.BlockCopy(pattern, 0, pattern, current_size, current_size);
// copy rest, if any
Buffer.BlockCopy(pattern, 0, pattern, current_size, new_size - current_size);
current_size = new_size;
// mark multiples of the current prime
for (int j = current_prime >> 1; j < current_size; j += current_prime)
pattern[j] = CROSSED_OFF;
}
for (current_size = m_presieve_modulus; current_size * 2 <= pattern.Length; current_size *= 2)
Buffer.BlockCopy(pattern, 0, pattern, current_size, current_size);
Buffer.BlockCopy(pattern, 0, pattern, current_size, pattern.Length - current_size);
}
For a quick test you can hack the presieving into the sieve function as follows:
- int first_sieve_prime_index = 1 + decoder_order; // skip wheel primes + decoder primes
+ int first_sieve_prime_index = 1 + decoder_order + m_presieve_level; // skip wheel primes + decoder primes
plus
- long sum = 2 + 3 + 5; // wheel primes + decoder primes
+ long sum = 2 + 3 + 5; // wheel primes + decoder primes
+
+ for (int i = 0; i < m_presieve_level; ++i)
+ sum += TinyPrimes[m_wheel_order + i];
plus
- fill(sieve, window_bits, (byte)1);
+ if (m_presieve_level == 0)
+ fill(sieve, window_bits, (byte)1);
+ else
+ Buffer.BlockCopy(m_presieve_pattern, (window_base >> 1) % m_presieve_modulus, sieve, 0, window_bits);
and
set_presieve_level(4) // 4 and 5 work well
in the static constructor or Main().
This way you can use m_presieve_level for turning presieving on and off. The BlockCopy also works correctly after calling set_presieve_level(0), though, because then the modulus is 1. m_wheel_order should reflect the actual wheel order (= 1) plus the decoder order; it's currently set to 3, so it'll work only with the v5 decoder at level 2.
Timings:
# benchmark: iter_v5(2) pre(7) ...
sum up to 2 * 10^4: 21171191 in 0,02 ms
sum up to 2 * 10^5: 1709600813 in 0,08 ms // 4,0 times
sum up to 2 * 10^6: 142913828922 in 0,78 ms // 9,6 times
sum up to 2 * 10^7: 12272577818052 in 8,78 ms // 11,2 times
sum up to 2 * 10^8: 1075207199997334 in 98,89 ms // 11,3 times
sum up to 2 * 10^9: 95673602693282040 in 1.245,19 ms // 12,6 times
sum up to 2^31 - 1: 109930816131860852 in 1.351,97 ms
EDIT
Just was pointed that the requirements state peaks cannot be ends of Arrays.
So I ran across this site
http://codility.com/
Which gives you programming problems and gives you certificates if you can solve them in 2 hours. The very first question is one I have seen before, typically called the Peaks and Flags question. If you are not familiar
A non-empty zero-indexed array A consisting of N integers is given. A peak is an array element which is larger than its neighbours. More precisely, it is an index P such that
0 < P < N − 1 and A[P − 1] < A[P] > A[P + 1]
.
For example, the following array A:
A[0] = 1
A[1] = 5
A[2] = 3
A[3] = 4
A[4] = 3
A[5] = 4
A[6] = 1
A[7] = 2
A[8] = 3
A[9] = 4
A[10] = 6
A[11] = 2
has exactly four peaks: elements 1, 3, 5 and 10.
You are going on a trip to a range of mountains whose relative heights are represented by array A. You have to choose how many flags you should take with you. The goal is to set the maximum number of flags on the peaks, according to certain rules.
Flags can only be set on peaks. What's more, if you take K flags, then the distance between any two flags should be greater than or equal to K. The distance between indices P and Q is the absolute value |P − Q|.
For example, given the mountain range represented by array A, above, with N = 12, if you take:
two flags, you can set them on peaks 1 and 5;
three flags, you can set them on peaks 1, 5 and 10;
four flags, you can set only three flags, on peaks 1, 5 and 10.
You can therefore set a maximum of three flags in this case.
Write a function that, given a non-empty zero-indexed array A of N integers, returns the maximum number of flags that can be set on the peaks of the array.
For example, given the array above
the function should return 3, as explained above.
Assume that:
N is an integer within the range [1..100,000];
each element of array A is an integer within the range [0..1,000,000,000].
Complexity:
expected worst-case time complexity is O(N);
expected worst-case space complexity is O(N), beyond input storage (not counting the
storage required for input arguments).
Elements of input arrays can be modified.
So this makes sense, but I failed it using this code
public int GetFlags(int[] A)
{
List<int> peakList = new List<int>();
for (int i = 0; i <= A.Length - 1; i++)
{
if ((A[i] > A[i + 1] && A[i] > A[i - 1]))
{
peakList.Add(i);
}
}
List<int> flagList = new List<int>();
int distance = peakList.Count;
flagList.Add(peakList[0]);
for (int i = 1, j = 0, max = peakList.Count; i < max; i++)
{
if (Math.Abs(Convert.ToDecimal(peakList[j]) - Convert.ToDecimal(peakList[i])) >= distance)
{
flagList.Add(peakList[i]);
j = i;
}
}
return flagList.Count;
}
EDIT
int[] A = new int[] { 7, 10, 4, 5, 7, 4, 6, 1, 4, 3, 3, 7 };
The correct answer is 3, but my application says 2
This I do not get, since there are 4 peaks (indices 1,4,6,8) and from that, you should be able to place a flag at 2 of the peaks (1 and 6)
Am I missing something here? Obviously my assumption is that the beginning or end of an Array can be a peak, is this not the case?
If this needs to go in Stack Exchange Programmers, I will move it, but thought dialog here would be helpful.
EDIT
Obviously my assumption is that the beginning or end of an Array can
be a peak, is this not the case?
Your assumption is wrong since peak is defined as:
0 < P < N − 1
When it comes to your second example you can set 3 flags on 1, 4, 8.
Here is a hint: If it is possible to set m flags, then there must be at least m * (m - 1) + 1 array elements. Given that N < 100,000, turning the above around should give you confidence that the problem can be efficiently brute-forced.
Here is a hint: If it is possible to set m flags, then there must be
at least m * (m - 1) + 1 array elements. Given that N < 100,000,
turning the above around should give you confidence that the problem
can be efficiently brute-forced.
No, that is wrong. Codility puts custom solutions through a series of tests, and brute forcing can easily fail on time.
I give here my solution of the task that makes 100% score (correctness and performance) in codility, implemented in C++. To understand the solution you must realize for a given distance of indexes (for example, when first peak starts at index 2 and the last peak at index 58 the distance is 56), that contains n peaks there is an upper limit for the maximal number of peaks that can hold flags according to condition described in the task.
#include <vector>
#include <math.h>
typedef unsigned int uint;
void flagPeaks(const std::vector<uint> & peaks,
std::vector<uint> & flaggedPeaks,
const uint & minDist)
{
flaggedPeaks.clear();
uint dist = peaks[peaks.size() - 1] - peaks[0];
if (minDist > dist / 2)
return;
flaggedPeaks.push_back(peaks[0]);
for (uint i = 0; i < peaks.size(); ) {
uint j = i + 1;
while (j < (peaks.size()) && ((peaks[j] - peaks[i]) < minDist))
++j;
if (j < (peaks.size()) && ((peaks[j] - peaks[i]) >= minDist))
flaggedPeaks.push_back(peaks[j]);
i = j;
}
}
int solution(std::vector<int> & A)
{
std::vector<uint> peaks;
uint min = A.size();
for (uint i = 1; i < A.size() - 1; i++) {
if ((A[i] > A[i - 1]) && (A[i] > A[i + 1])) {
peaks.push_back(i);
if (peaks.size() > 1) {
if (peaks[peaks.size() - 1] - peaks[peaks.size() - 2] < min)
min = peaks[peaks.size() - 1] - peaks[peaks.size() - 2];
}
}
}
// minimal distance between 2 peaks is 2
// so when we have less than 3 peaks we are done
if (peaks.size() < 3 || min >= peaks.size())
return peaks.size();
const uint distance = peaks[peaks.size() - 1] - peaks[0];
// parts are the number of pieces between peaks
// given n + 1 peaks we always have n parts
uint parts = peaks.size() - 1;
// calculate maximal possible number of parts
// for the given distance and number of peaks
double avgOptimal = static_cast<double>(distance) / static_cast<double> (parts);
while (parts > 1 && avgOptimal < static_cast<double>(parts + 1)) {
parts--;
avgOptimal = static_cast<double>(distance) / static_cast<double>(parts);
}
std::vector<uint> flaggedPeaks;
// check how many peaks we can flag for the
// minimal possible distance between two flags
flagPeaks(peaks, flaggedPeaks, parts + 1);
uint flags = flaggedPeaks.size();
if (flags >= parts + 1)
return parts + 1;
// reduce the minimal distance between flags
// until the condition fulfilled
while ((parts > 0) && (flags < parts + 1)) {
--parts;
flagPeaks(peaks, flaggedPeaks, parts + 1);
flags = flaggedPeaks.size();
}
// return the maximal possible number of flags
return parts + 1;
}
I have a table with different codes. And their Id's are powers of 2. (20, 21, 22, 23...).
Based on different conditions my application will assign a value to the "Status" variable.
for ex :
Status = 272 ( which is 28+ 24)
Status = 21 ( Which is 24+ 22+20)
If Status = 21 then my method (C#) should tell me that 21 is sum of 16 + 4 + 1.
You can test all bits in the input value if they are checked:
int value = 21;
for (int i = 0; i < 32; i++)
{
int mask = 1 << i;
if ((value & mask) != 0)
{
Console.WriteLine(mask);
}
}
Output:
1
4
16
for (uint currentPow = 1; currentPow != 0; currentPow <<= 1)
{
if ((currentPow & QStatus) != 0)
Console.WriteLine(currentPow); //or save or print some other way
}
for QStatus == 21 it will give
1
4
16
Explanation:
A power of 2 has exactly one 1 in its binary representation. We take that one to be the rightmost one(least significant) and iteratively push it leftwards(towards more significant) until the number overflows and becomes 0. Each time we check that currentPow & QStatus is not 0.
This can probably be done much cleaner with an enum with the [Flags] attribute set.
This is basically binary (because binary is also base 2). You can bitshift values around !
uint i = 87;
uint mask;
for (short j = 0; j < sizeof(uint); j++)
{
mask = 1 << j;
if (i & mask == 1)
// 2^j is a factor
}
You can use bitwise operators for this (assuming that you have few enough codes that the values stay in an integer variable).
a & (a - 1) gives you back a after unsetting the last set bit. You can use that to get the value of the corresponding flag, like:
while (QStatus) {
uint nxtStatus = QStatus & (QStatus - 1);
processFlag(QStatus ^ nxtStatus);
QStatus = nxtStatus;
}
processFlag will be called with the set values in increasing order (e.g. 1, 4, 16 if QStatus is originally 21).