Develop Reference

Speed up processing 32 bit numbers in combinations (k from n)

I have a list of 128 32 bit numbers, and I want to know, is there any combination of 12 numbers, so that all numbers XORed give the 32 bit number with all bits set to 1.
So I have started with naive approach and took combinations generator like that:
private static IEnumerable<int[]> Combinations(int k, int n)
{
var state = new int[k];
var stack = new Stack<int>();
stack.Push(0);
while (stack.Count > 0)
{
var index = stack.Count - 1;
var value = stack.Pop();
while (value < n)
{
state[index++] = value++;
if (value < n)
{
stack.Push(value);
}
if (index == k)
{
yield return state;
break;
}
}
}
}
and used it like that (data32 is an array of given 32bit numbers)
foreach (var probe in Combinations(12, 128))
{
int p = 0;
foreach (var index in probe)
{
p = p ^ data32[index];
}
if (p == -1)
{
//print out found combination
}
}
Of course it takes forever to check all 23726045489546400 combinations...
So my question(s) are - am I missing something in options how to speedup the check process?
Even if I do the calculation of combinations in partitions (e.g. I could start like 8 threads each will check combination started with numbers 0..8), or speed up the XORing by storing the perviously calculated combination - it is still slow.
P.S. I'd like it to run in reasonable time - minutes, hours not years.
Adding a list of numbers as was requested in one of the comments:
1571089837
2107702069
466053875
226802789
506212087
484103496
1826565655
944897655
1370004928
748118360
1000006005
952591039
2072497930
2115635395
966264796
1229014633
827262231
1276114545
1480412665
2041893083
512565106
1737382276
1045554806
172937528
1746275907
1376570954
1122801782
2013209036
1650561071
1595622894
425898265
770953281
422056706
477352958
1295095933
1783223223
842809023
1939751129
1444043041
1560819338
1810926532
353960897
1128003064
1933682525
1979092040
1987208467
1523445101
174223141
79066913
985640026
798869234
151300097
770795939
1489060367
823126463
1240588773
490645418
832012849
188524191
1034384571
1802169877
150139833
1762370591
1425112310
2121257460
205136626
706737928
265841960
517939268
2070634717
1703052170
1536225470
1511643524
1220003866
714424500
49991283
688093717
1815765740
41049469
529293552
1432086255
1001031015
1792304327
1533146564
399287468
1520421007
153855202
1969342940
742525121
1326187406
1268489176
729430821
1785462100
1180954683
422085275
1578687761
2096405952
1267903266
2105330329
471048135
764314242
459028205
1313062337
1995689086
1786352917
2072560816
282249055
1711434199
1463257872
1497178274
472287065
246628231
1928555152
1908869676
1629894534
885445498
1710706530
1250732374
107768432
524848610
2791827620
1607140095
1820646148
774737399
1808462165
194589252
1051374116
1802033814

I don't know C#, I did something in Python, maybe interesting anyway. Takes about 0.8 seconds to find a solution for your sample set:
solution = {422056706, 2791827620, 506212087, 1571089837, 827262231, 1650561071, 1595622894, 512565106, 205136626, 944897655, 966264796, 477352958}
len(solution) = 12
solution.issubset(nums) = True
hex(xor(solution)) = '0xffffffff'
There are 128C12 combinations, that's 5.5 million times as many as the 232 possible XOR values. So I tried being optimistic and only tried a subset of the possible combinations. I split the 128 numbers into two blocks of 28 and 100 numbers and try combinations with six numbers from each of the two blocks. I put all possible XORs of the first block into a hash set A, then go through all XORs of the second block to find one whose bitwise inversion is in that set. Then I reconstruct the individual numbers.
This way I cover (28C6)2 × (100C6)2 = 4.5e14 combinations, still over 100000 times as many as there are possible XOR values. So probably still a very good chance to find a valid combination.
Code (Try it online!):
from itertools import combinations
from functools import reduce
from operator import xor as xor_
nums = list(map(int, '1571089837 2107702069 466053875 226802789 506212087 484103496 1826565655 944897655 1370004928 748118360 1000006005 952591039 2072497930 2115635395 966264796 1229014633 827262231 1276114545 1480412665 2041893083 512565106 1737382276 1045554806 172937528 1746275907 1376570954 1122801782 2013209036 1650561071 1595622894 425898265 770953281 422056706 477352958 1295095933 1783223223 842809023 1939751129 1444043041 1560819338 1810926532 353960897 1128003064 1933682525 1979092040 1987208467 1523445101 174223141 79066913 985640026 798869234 151300097 770795939 1489060367 823126463 1240588773 490645418 832012849 188524191 1034384571 1802169877 150139833 1762370591 1425112310 2121257460 205136626 706737928 265841960 517939268 2070634717 1703052170 1536225470 1511643524 1220003866 714424500 49991283 688093717 1815765740 41049469 529293552 1432086255 1001031015 1792304327 1533146564 399287468 1520421007 153855202 1969342940 742525121 1326187406 1268489176 729430821 1785462100 1180954683 422085275 1578687761 2096405952 1267903266 2105330329 471048135 764314242 459028205 1313062337 1995689086 1786352917 2072560816 282249055 1711434199 1463257872 1497178274 472287065 246628231 1928555152 1908869676 1629894534 885445498 1710706530 1250732374 107768432 524848610 2791827620 1607140095 1820646148 774737399 1808462165 194589252 1051374116 1802033814'.split()))
def xor(vals):
return reduce(xor_, vals)
A = {xor(a)^0xffffffff: a
for a in combinations(nums[:28], 6)}
for b in combinations(nums[28:], 6):
if a := A.get(xor(b)):
break
solution = {*a, *b}
print(f'{solution = }')
print(f'{len(solution) = }')
print(f'{solution.issubset(nums) = }')
print(f'{hex(xor(solution)) = }')

Arrange your numbers into buckets based on the position of the first 1 bit.
To set the first bit to 1, you will have to use an odd number of the items in the corresponding bucket....
As you recurse, try to maintain an invariant that the number of leading 1 bits is increasing and then select the bucket that will change the next 0 to a 1, this will greatly reduce the number of combinations that you have to try.

I have found a possible solution, which could work for my particular task.
As main issue to straitforward approach I see a number of 2E16 combinations.
But, if I want to check if combination of 12 elements equal to 0xFFFFFFFF, I could check if 2 different combinations of 6 elements with opposit values exists.
That will reduce number of combinations to "just" 5E9, which is achievable.
On first attempt I think to store all combinations and then find opposites in the big list. But, in .NET I could not find quick way of storing more then Int32.MaxValue elements.
Taking in account idea with bits from comments and answer, I decided to store at first only xor sums with leftmost bit set to 1, and then by definition I need to check only sums with leftmost bit set to 0 => reducing storage by 2.
In the end it appears that many collisions could appear, so there are many combinations with the same xor sum.
Current version which could find such combinations, need to be compiled in x64 mode and use (any impovements welcomed):
static uint print32(int[] comb, uint[] data)
{
uint p = 0;
for (int i = 0; i < comb.Length; i++)
{
Console.Write("{0} ", comb[i]);
p = p ^ data[comb[i]];
}
Console.WriteLine(" #[{0:X}]", p);
return p;
}
static uint[] data32;
static void Main(string[] args)
{
int n = 128;
int k = 6;
uint p = 0;
uint inv = 0;
long t = 0;
//load n numbers from a file
init(n);
var lookup1x = new Dictionary<uint, List<byte>>();
var lookup0x = new Dictionary<uint, List<byte>>();
Stopwatch watch = new Stopwatch();
watch.Start();
//do not use IEnumerable generator, use function directly to reuse xor value
var hash = new uint[k];
var comb = new int[k];
var stack = new Stack<int>();
stack.Push(0);
while (stack.Count > 0)
{
var index = stack.Count - 1;
var value = stack.Pop();
if (index == 0)
{
p = 0;
Console.WriteLine("Start {0} sequence, combinations found: {1}",value,t);
}
else
{
//restore previous xor value
p = hash[index - 1];
}
while (value < n)
{
//xor and store
p = p ^ data32[value];
hash[index] = p;
//remember current state (combination)
comb[index++] = value++;
if (value < n)
{
stack.Push(value);
}
//combination filled to end
if (index == k)
{
//if xor have MSB set, put it to lookup table 1x
if ((p & 0x8000000) == 0x8000000)
{
lookup1x[p] = comb.Select(i => (byte)i).ToList();
inv = p ^ 0xFFFFFFFF;
if (lookup0x.ContainsKey(inv))
{
var full = lookup0x[inv].Union(lookup1x[p]).OrderBy(x=>x).ToArray();
if (full.Length == 12)
{
print32(full, data32);
}
}
}
else
{
//otherwise put it to lookup table 2, but skip all combinations which are started with 0
if (comb[0] != 0)
{
lookup0x[p] = comb.Select(i => (byte)i).ToList();
inv = p ^ 0xFFFFFFFF;
if (lookup1x.ContainsKey(inv))
{
var full = lookup0x[p].Union(lookup1x[inv]).OrderBy(x=>x).ToArray();
if (full.Length == 12)
{
print32(full, data32);
}
}
}
}
t++;
break;
}
}
}
Console.WriteLine("Check was done in {0} ms ", watch.ElapsedMilliseconds);
//end
}

Scheduler algorithm that fill remaning hours to work

I need to generate all possible values to a scheduler who works like this:
Some hours of the week can be already chosen.
The week of work is defined by the following pattern "???????" question marks can be replaced.
Given a maximum of hours, I need to replace the question marks with digits so that the sum of the scheduled hours match the hours need to work in a week returning a string array with all possible schedules, ordered lexicographically.
Example:
pattern = "08??840",
required_week_hours= 24
In this example, there are only 4 hours left to work.
calling this:
function List<String> GenerateScheduler(int workHours, int dayHours, string pattern){}
public static void Main(){
GenerateScheduler(24, 4, "08??840");
}
This would return the following list of strings:
0804840
0813840
.......
.......
0840840
I'm not very familiar with algorithms, which one I could use to solve this problem?

This sounds like a problem where you have to generate all permutations of a list of a certain amount of numbers that sum up to a certain number. First, you need to sum up the hours you already know. Then you need to count up the number of ? aka the number of shifts/days you do not know about. Using these parameters, this is what the solution will look like,
public List<string> GenerateScheduler(int workHours, int dayHours, string pattern){
int remainingSum = workHours;
int unknownCount = 0;
// first iterate through the pattern to know how many ? characters there are
// as well as the number of hours remaining
for (int i = 0; i < pattern.Length; i++) {
if (pattern[i] == '?') {
unknownCount++;
}
else {
remainingSum -= pattern[i] - '0';
}
}
List<List<int>> permutations = new List<List<int>>();
// get all the lists of work shifts that sum to the remaining number of hours
// the number of work shifts in each list is the number of ? characters in pattern
GeneratePermutations(permutations, workHours, unknownCount);
// after getting all the permutations, we need to iterate through the pattern
// for each permutation to construct a list of schedules to return
List<string> schedules = new List<string>();
foreach (List<int> permutation in permutation) {
StringBuilder newSchedule = new StringBuilder();
int permCount = 0;
for (int i = 0; i < pattern.Length(); i++) {
if (pattern[i] == '?') {
newSchedule.Append(permutation[permCount]);
permCount++;
}
else {
newSchedule.Append(pattern[i]);
}
}
schedules.Add(newSchedule.ToString());
}
return schedules;
}
public void GeneratePermutations(List<List<int>> permutations, int workHours, int unknownCount) {
for (int i = 0; i <= workHours; i++) {
List<int> permutation = new List<int>();
permutation.Add(i);
GeneratePermuationsHelper(permutations, permutation, workHours - i, unknownCount - 1);
}
}
public void GeneratePermutationsHelper(List<List<int>> permutations, List<int> permutation, int remainingHours, int remainingShifts){
if (remainingShifts == 0 && remainingHours == 0) {
permutations.Add(permutation);
return;
}
if (remainingHours <= 0 || remainingShifts <= 0) {
return;
}
for (int i = 0; i <= remainingHours; i++) {
List<int> newPermutation = new List<int>(permutation);
newPermutation.Add(i);
GeneratePermutationsHelper(permutations, newPermutation, remainingHours - i, remainingShifts - 1);
}
}
This can be a lot to digest so I will briefly go over how the permutation recursive helper function works. The parameters go as follows:
a list containing all the permutations
the current permutation being examined
the remaining number of hours needed to reach the total work hour count
the number of remaining shifts (basically number of '?' - permutation.Count)
First, we check to see if the current permutation meets the criteria that the total of its work hours equals the amount of hours remaining needed to complete the pattern and the number of shifts in the permutation equals the number of question marks in the pattern. If it does, then we add this permutation to the list of permutations. If it doesn't, we check to see if the total amount of work hours surpasses the amount of hours remaining or if the number of shifts has reached the number of question marks in the pattern. If so, then the permutation is not added. However, if we can still add more shifts, we will run a loop from i = 0 to remainingHours and make a copy of the permutation while adding i to this copied list in each iteration of the loop. Then, we will adjust the remaining hours and remaining shifts accordingly before calling the helper function recursively with the copied permutation.
Lastly, we can use these permutations to create a list of new schedules, replacing the ? characters in the pattern with the numbers from each permutation.

As per OP, you already know the remaining hours, which I assume is given by the parameter dayHours. So, if you were to break down the problem further, you would need to replace '?' characters with numbers so that, sum of new character(number) is equal to remaining hours(dayHours).
You can do the following.
public IEnumerable<string> GenerateScheduler(int totalHours,int remainingHours,string replacementString)
{
var numberOfPlaces = replacementString.Count(x => x == '?');
var minValue = remainingHours;
var maxValue = remainingHours * Math.Pow(10,numberOfPlaces-1);
var combinations = Enumerable.Range(remainingHours,(int)maxValue)
.Where(x=> SumOfDigit(x) == remainingHours).Select(x=>x.ToString().PadLeft(numberOfPlaces,'0').ToCharArray());
foreach(var item in combinations)
{
var i = 0;
yield return Regex.Replace(replacementString, "[?]", (m) => {return item[i++].ToString(); });
}
}
double SumOfDigit(int value)
{
int sum = 0;
while (value != 0)
{
int remainder;
value = Math.DivRem(value, 10, out remainder);
sum += remainder;
}
return sum;
}

Random Integer, no repeat, within specific range

Detailed scenario:
There are 300 possible numbers that fit into the discription.
That is, the number must between 1123 and 5543 inclusive
The number is unique and not repeated
The number only has the integers 1,2,3,4 and 5.
What I am trying to achieve is a program that can display all those numbers at once in ascending order.
My current code:
var chars = "12345";
var stringChars = new char[4];
var random = new Random();
for (int i = 0; i < stringChars.Length; i++)
{
stringChars[i] = chars[random.Next(chars.Length)];
}
var finalString = new String(stringChars);
Console.WriteLine(finalString)
This code works fine, but I have 2 additional requirements.
Loop 300 times
Display all results in ascending order
EDIT:
Sorry for all confusion. What this code produces is a 4 digit number like I want. But I want to it to do so 300 times. I guess I can use a Loop, but this is my first time using C# (I've only used vb.net in the past). What I mean by random and in ascending order is: Produce a random number, and when all numbers are generated, order them in ascending order before displaying them all.
However, if ordering them will be too complicated, then I'm fine without that.

It's not entirely clear what you're asking, but here is some code that will print the 611 (not 300) numbers matching the description you gave in ascending order.
If I've understood your question correctly, you will want to sample 300 distinct elements from this larger set, sort them, and them print them out. You might look at using a Fisher-Yates shuffle to do this. Shuffle the list, take the first 300, and sort them.
public static void Main (string[] args) {
var digits = new [] { 1, 1, 2, 3 };
for (var num = DigitsToInt(digits); num <= 5543; num = DigitsToInt(digits)) {
Console.WriteLine(num);
for (int i = 3; i >= 0; i--) {
digits[i]++;
if (digits[i] < 6) {
break;
} else {
digits[i] = 1;
}
}
}
}
private static int DigitsToInt(int[] digits) {
return 1000 * digits[0] + 100 * digits[1] + 10 * digits[2] + digits[3];
}

Where is the flaw in my algorithm for consolidating gold mines?

The setup is that, given a list of N objects like
class Mine
{
public int Distance { get; set; } // from river
public int Gold { get; set; } // in tons
}
where the cost of moving the gold from one mine to the other is
// helper function for cost of a move
Func<Tuple<Mine,Mine>, int> MoveCost = (tuple) =>
Math.Abs(tuple.Item1.Distance - tuple.Item2.Distance) * tuple.Item1.Gold;
I want to consolidate the gold into K mines.
I've written an algorithm, thought it over many times, and don't understand why it isn't working. Hopefully my comments help out. Any idea where I'm going wrong?
using System;
using System.Collections.Generic;
using System.IO;
using System.Linq;
class Mine
{
public int Distance { get; set; } // from river
public int Gold { get; set; } // in tons
}
class Solution
{
static void Main(String[] args)
{
// helper function for reading lines
Func<string, int[]> LineToIntArray = (line) => Array.ConvertAll(line.Split(' '), Int32.Parse);
int[] line1 = LineToIntArray(Console.ReadLine());
int N = line1[0], // # of mines
K = line1[1]; // # of pickup locations
// Populate mine info
List<Mine> mines = new List<Mine>();
for(int i = 0; i < N; ++i)
{
int[] line = LineToIntArray(Console.ReadLine());
mines.Add(new Mine() { Distance = line[0], Gold = line[1] });
}
// helper function for cost of a move
Func<Tuple<Mine,Mine>, int> MoveCost = (tuple) =>
Math.Abs(tuple.Item1.Distance - tuple.Item2.Distance) * tuple.Item1.Gold;
// all move combinations
var moves = from m1 in mines
from m2 in mines
where !m1.Equals(m2)
select Tuple.Create(m1,m2);
// moves in ascending order of cost
var ordered = from m in moves
orderby MoveCost(m)
select m;
int sum = 0; // running total of move costs
var spots = Enumerable.Repeat(1, N).ToArray(); // spots[i] = 1 if hasn't been consildated into other mine, 0 otherwise
var iter = ordered.GetEnumerator();
while(iter.MoveNext() && spots.Sum() != K)
{
var move = iter.Current; // move with next smallest cost
int i = mines.IndexOf(move.Item1), // index of source mine in move
j = mines.IndexOf(move.Item2); // index of destination mine in move
if((spots[i] & spots[j]) == 1) // if the source and destination mines are both unconsolidated
{
sum += MoveCost(move); // add this consolidation to the total cost
spots[i] = 0; // "remove" mine i from the list of unconsolidated mines
}
}
Console.WriteLine(sum);
}
}
An example of a test case I'm failing is
3 1
11 3
12 2
13 1
My output is
3
and the correct output is
4

The other answer does point out a flaw in the implementation, but it fails to mention that in your code, you aren't actually changing the Gold values in the remaining Mine objects. So even if you did re-sort the data, it wouldn't help.
Furthermore, at each iteration all you really care about is the minimum value. Sorting the entire list of data is overkill. You can just scan it once to find the minimum-valued item.
You also don't really need the separate array of flags. Just maintain your move objects in a list, and after choosing a move, remove the move objects that include the Mine you would otherwise have flagged as no longer valid.
Here is a version of your algorithm that incorporates the above feedback:
static void Main(String[] args)
{
string input =
#"3 1
11 3
12 2
13 1";
StringReader reader = new StringReader(input);
// helper function for reading lines
Func<string, int[]> LineToIntArray = (line) => Array.ConvertAll(line.Split(' '), Int32.Parse);
int[] line1 = LineToIntArray(reader.ReadLine());
int N = line1[0], // # of mines
K = line1[1]; // # of pickup locations
// Populate mine info
List<Mine> mines = new List<Mine>();
for (int i = 0; i < N; ++i)
{
int[] line = LineToIntArray(reader.ReadLine());
mines.Add(new Mine() { Distance = line[0], Gold = line[1] });
}
// helper function for cost of a move
Func<Tuple<Mine, Mine>, int> MoveCost = (tuple) =>
Math.Abs(tuple.Item1.Distance - tuple.Item2.Distance) * tuple.Item1.Gold;
// all move combinations
var moves = (from m1 in mines
from m2 in mines
where !m1.Equals(m2)
select Tuple.Create(m1, m2)).ToList();
int sum = 0, // running total of move costs
unconsolidatedCount = N;
while (moves.Count > 0 && unconsolidatedCount != K)
{
var move = moves.Aggregate((a, m) => MoveCost(a) < MoveCost(m) ? a : m);
sum += MoveCost(move); // add this consolidation to the total cost
move.Item2.Gold += move.Item1.Gold;
moves.RemoveAll(m => m.Item1 == move.Item1 || m.Item2 == move.Item1);
unconsolidatedCount--;
}
Console.WriteLine("Moves: " + sum);
}
Without more detail in your question, I can't guarantee that this actually meets the specification. But it does produce the value 4 for the sum. :)

When you consolidate mine i into mine j, the amount of gold in the mine j is increased. This makes consolidations from mine j to other mines more expensive potentially making the ordering of the mines by the move cost invalid. To fix this, you could re-sort the list of mines at the beginning of each iteration of your while-loop.

Algorithm uniform distribution of elements in the matrix

I need to generate tasks in the month of the date of the random uniform distribution. For example, for 10 people to 10 tasks so that the dates are spaced not less than two days. Weekends and holidays are not to be used. Essentially uniform random distribution of elements in the multiplicity with the additional condition where the multiplicity - people x dates of month. Suggest, where I can watch the algorithm.

In general decided somehow. However, I must say, a decision I do not like, but nothing better could not think. Parameters are as follows: tasksPerMonth - the number of jobs per month, distanceBetweenTasks - the minimum distance between two adjacent jobs, isTasksInWeekend - take into account whether or not the weekend, minDay - starting from this day (for various reasons, this may not be the first day of the month), listOfDays - initially empty, holidays - a list of days off and holidays, workdaysInMonth - list of working days, random - empty Random (). The rest, I think, in principle, it is clear, the function code is shown below OptimizationTheDistributionOfTasks
public void GenerateRandomTasks(int tasksPerMonth, int distanceBetweenTasks, bool isTasksInWeekend, int minDay, List<int> listOfDays, List<int> holidays, List<int> workdaysInMonth, Random random)
{
if (tasksPerMonth == 0)
tasksPerMonth = 1;
var daysInMonth = workdaysInMonth.Count + holidays.Count;
var tasksDaysInMonth = !isTasksInWeekend ? workdaysInMonth.Count : daysInMonth;
for (int i = 0; i < tasksPerMonth; i++)
{
int maxDayInPeriod;
if (i < tasksPerMonth - 1)
{
maxDayInPeriod = minDay + tasksDaysInMonth / tasksPerMonth;
if (!isTasksInWeekend && holidays.Contains(maxDayInPeriod))
maxDayInPeriod = workdaysInMonth.First(v => v > maxDayInPeriod);
}
else
{
maxDayInPeriod = daysInMonth;
if (!isTasksInWeekend && holidays.Contains(maxDayInPeriod))
maxDayInPeriod = workdaysInMonth.Last();
}
if (minDay > maxDayInPeriod)
minDay = maxDayInPeriod;
var day = random.Next(minDay, maxDayInPeriod);
if ((isTasksInWeekend != true && holidays.Contains(day)))
day = OptimizationTheDistributionOfTasks(minDay, maxDayInPeriod, listOfDays, day, holidays);
if (day > daysInMonth)
day = daysInMonth;
listOfDays.Add(day);
minDay = maxDayInPeriod;
if (minDay <= day + distanceBetweenTasks)
minDay = day + distanceBetweenTasks + 1;
}
}
Method OptimizationTheDistributionOfTasks:
private int OptimizationTheDistributionOfTasks(int minDay, int maxDay, List<int> listDays, int day, List<int> holidays)
{
var listOfDays = new List<DaysForTaskPlan>();
for (int k = minDay; k <= maxDay; k++)
{
var tempCountDays = listDays.Count(d => k == d);
if (!holidays.Contains(k))
listOfDays.Add(new DaysForTaskPlan(k, tempCountDays));
}
if (listOfDays.Any())
{
day = listOfDays.First(p => p.AmountDays == listOfDays.Min(z => z.AmountDays)).CurrDay;
}
listOfDays.Clear();
return day;
}
Good luck to all