Develop Reference

combining two different information in binary code

I have Dictionary<string,T> where string represents the key of record, and I have two other pieces of information about the record that I need to maintain for each record in the dictionary, which are the category of the record and its redundancy (how many times its repeated).
For example: the record XYZ1 is of category 1, and its repeated 1 times. therefore the implementation has to be something like this:
"XYZ1", {1,1}
Now moving on, I may encounter the same record in my dataset, therefore the value of the key has to be updated like:
"XYZ1", {1,2}
"XYZ1", {1,3}
...
Since I am processing big number of records such as 100K, I tried this approach but it seems inefficient because the extra effort of fetching the value from dictionary and then slicing {1,1} and then converting both slices into integer puts lot of overhead on the execution.
I was thinking of using binary digits to represent both category and repatation and maybe bitmask to fetch these pieces.
Edit: I tried to use object with 2 properties, and then Tuple<int,int>. Complexity got worse !
My question: is it possible to do so ?
if not (in terms of complexity) any suggestions?

What is your type T? You could define a custom type which holds the information you need (category and occurences) .
class MyInfo {
public int c { get; set; }
public int o { get; set; }
}
Dictionary<String, MyInfo> data;
Then when traversing your data you can easily check whether some key is already present. If yes, just increment the occurences, else insert a new element.
MyInfo d;
foreach (var e in elements) {
if (!data.TryGet(e.key, out d))
data.Add(e.key, new MyInfo { c = e.cat, o= 1});
else
d.o++;
}
EDIT
You could also combine the category and the number of occurences into one UInt64. For instance take the category in the higher 32 bit (ie you can have 4 billion categories) and the number of occurenes in the lower 32 bit (ie each key can occur 4 billion times)
Dictionary<string, UInt64> data;
UInt64 d;
foreach (var e in elements) {
if (!data.TryGet(e.key, out d))
data[e.key] = (e.cat << 32) + 1;
else
data[e.key] = d + 1;
}
And if you want to get the number of occurrences for one specific key you can just inspect the respective part of the value.
var d = data["somekey"];
var occurrences = d & 0xFFFFFFFF;
var category = d >> 32;

It seems like category never changes. So rather than using a simple string for the key of your dictionary, I would instead do something like:
Dictionary<Tuple<string,int>,int> where the key of the dictionary is a Tuple<string,int> where the string is the record and the int is the category. Then the value in the dictionary is just a count.
A dictionary is probably going to be the fastest data structure for what you're trying to accomplish as it has near constant time O(1) lookup and entry.
You can speed it up a little bit by using the Tuple, as now the category is part of the key and no longer a bit of information you have to access separately.
At the same time you could also keep the string as the key and store a Tuple<int,int> as the value and simply set Item1 as the category and Item2 as the count.
Either way is going to be roughly equivalent in speed. Processing 100k records in such a manner should be pretty fast either way.

Best Way to Check for Used Key with Nhibernate?

on my site I allow people to buy subscriptions to my site in bulk(I call them vouchers). Once they have these vouchers, they give them to whoever and they enter that code into their account to upgrade them.
Right now I am thinking of doing 4 alphanumeric code(upper case, lower case and digits) and will have something like this
var chars = "ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789";
var stringChars = new char[4];
var random = new Random();
for (int i = 0; i < stringChars.Length; i++)
{
stringChars[i] = chars[random.Next(chars.Length)];
}
var finalString = new String(stringChars);
For now I think that will give me more than enough combinations and if I ever do run out I can always up the length of the code. I want to keep it short because I don't want the user to have to type in huge as numbers.
I also don't have the time to make a more elegant solution maybe were they click a link or something in their email and it activates their account and of course this would cut down on someone trying to randomly guess a voucher number.
These are things I would deal with if the site every becomes more popular.
I am wondering though how can I handle the possible duplicate generation of the same voucher. My first thought was to check the database each time a voucher is created and if it exists then make a new one.
However that seems like it could be slow. So I thought also maybe getting all the keys first and store them in memory and they check there but if the list keeps growing I might run into out of memory exceptions and all that great stuff.
So does anyone have any ideas? Or am I stuck doing one of the 2 method I listed above?
I am using nhibernate, asp.net mvc and C#.
Edit
static void Main(string[] args)
{
List<string> hold = new List<string>();
for (int i = 0; i < 10000; i++)
{
HashAlgorithm sha = new SHA1CryptoServiceProvider();
byte[] result = sha.ComputeHash(BitConverter.GetBytes(i));
string hex = null;
foreach (byte x in result)
{
hex += String.Format("{0:x2}", x);
}
hold.Add(hex.Substring(0,3));
Console.WriteLine(hex.Substring(0, 4));
}
Console.WriteLine("Number of Distinct values {0}", hold.Distinct().Count());
}
above is my attempt to try to use hashing. However I think I am missing something as it seems to have quite a bit more duplicates then expected.
Edit 2
I think I added what I was missing but not sure if this is exactly what he meant. I am also not sure what to do in a situation when I moved it as far as I can move it(my has seems to give me a length of 40 places I can move it).
static void Main(string[] args)
{
int subStringLength = 4;
List<string> hold = new List<string>();
for (int i = 0; i < 10000; i++)
{
SHA1CryptoServiceProvider sha = new SHA1CryptoServiceProvider();
byte[] result = sha.ComputeHash(BitConverter.GetBytes(i));
string hex = null;
foreach (byte x in result)
{
hex += String.Format("{0:x2}", x);
}
int startingPositon = 0;
string possibleVoucherCode = hex.Substring(startingPositon,subStringLength);
string voucherCode = Move(subStringLength, hold, hex, startingPositon, possibleVoucherCode);
hold.Add(voucherCode);
}
Console.WriteLine("Number of Distinct values {0}", hold.Distinct().Count());
}
private static string Move(int subStringLength, List<string> hold, string hex, int startingPositon, string possibleVoucherCode)
{
if (hold.Contains(possibleVoucherCode))
{
int newPosition = startingPositon + 1;
if (newPosition <= hex.Length)
{
if ((newPosition + subStringLength) > hex.Length)
{
possibleVoucherCode = hex.Substring(newPosition, subStringLength);
return Move(subStringLength, hold, hex, newPosition, possibleVoucherCode);
}
// return something
return "0";
}
else
{
// return something
return "0";
}
}
else
{
return possibleVoucherCode;
}
}
}

It is going to be slow because you want to generate the vouchers randomly and then check the database for every generated code.
I would create a table vouchers with an id, the code and an is_used column. I would fill that table once with enough random codes. Since this can be done in a separate process, the performance won't be such a big problem. Let it run in the evening and the next day you get a fully filled vouchers-table.
If you want to prevent generating duplicate vouchers, that won't be a problem. You can generate them anyway and put them either in a System.Collections.Generic.HashSet (which prevents adding duplicates without throwing an exception) or call the Linq-method Distinct(), before adding them to that vouchers table.

If you insist on short codes:
Use a GUID as a primary key, generate one random number. How you might want to translate this in to alpha-num is up to you.
Use the last byte or two of the guid and the random number. 1234-684687 This should make it slightly less easy to bruteforce coupons. And handle any (rare) collisions with an exception.
Easy way to shorten an int, change it's base (from 10 to 62). (in VB, and this is old code)
This yields "2lkCB1" when given Int32.MaxValue
''//given intValue as your random integer
Dim result As String = String.Empty
Dim digits as String = "0123456789abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ"
Dim x As Integer
While (intValue > 0)
x = intValue Mod digits.Length
result = digits(x) & result
intValue = intValue - x
intValue = intValue \ digits.Length
End While
Return result
But now we're already answering more than one question.

For a bulk data operation like this, I would recommend not using NHibernate and just doing straight ADO.NET.
Batch Check
Since you anticipate generating big batches of codes at once, you should batch multiple code checks into a single round-trip to the database. If you're using SQL Server 2008 or higher, you could do this using table-valued parameters, checking a whole list of codes at once.
SELECT DISTINCT b.Code
FROM #batch b
WHERE NOT EXISTS (
SELECT v.Code
FROM dbo.Voucher v
WHERE v.Code = b.Code
);
Concurrency
Now, what about concurrency issues? What if two users generate the same code at roughly the same time? Or simply in-between the time when we check the code for uniqueness and when we insert it into the Voucher table?
We can take care of that by modifying the query as follows:
DECLARE #batchid uniqueidentifier;
SET #batchid = NEWID();
INSERT INTO dbo.Voucher (Code, BatchId)
SELECT DISTINCT b.Code, #batchid
FROM #batch b
WHERE NOT EXISTS (
SELECT Code
FROM dbo.Voucher v
WHERE b.Code = v.Code
);
SELECT Code
FROM dbo.Voucher
WHERE BatchId = #batchid;
Executing via .NET
Assuming that you have defined the following table-valued user type...
CREATE TYPE dbo.VoucherCodeList AS TABLE (
Code nvarchar(8) COLLATE SQL_Latin1_General_CP1_CS_AS NOT NULL
/* !!! Remember to specify the collation on your Voucher.Code column too, since you want upper and lower-case codes. */
);
... you could execute this query via .NET code like this:
public ICollection<string> GenerateCodes(int numberOfCodes)
{
var result = new List<string>(numberOfCodes);
while (result.Count < numberOfCodes)
{
var batchSize = Math.Min(_batchSize, numberOfCodes - result.Count);
var batch = Enumerable.Range(0, batchSize)
.Select(x => GenerateRandomCode());
var oldResultCount = result.Count;
result.AddRange(FilterAndSecureBatch(batch));
var filteredBatchSize = result.Count - oldResultCount;
var collisionRatio = ((double)batchSize - filteredBatchSize) / batchSize;
// Automatically increment length of random codes if collisions begin happening too frequently
if (collisionRatio > _collisionThreshold)
CodeLength++;
}
return result;
}
private IEnumerable<string> FilterAndSecureBatch(IEnumerable<string> batch)
{
using (var command = _connection.CreateCommand())
{
command.CommandText = _sqlQuery; // the concurrency-safe query listed above
var metaData = new[] { new SqlMetaData("Code", SqlDbType.NVarChar, 8) };
var param = command.Parameters.Add("#batch", SqlDbType.Structured);
param.TypeName = "dbo.VoucherCodeList";
param.Value = batch.Select(x =>
{
var record = new SqlDataRecord(metaData);
record.SetString(0, x);
return record;
});
using (var reader = command.ExecuteReader())
while (reader.Read())
yield return reader.GetString(0);
}
}
Performance
After implementing all of this (and moving the command and parameter creation out of the loop so it would be re-used between batches), I was able to insert 10,000 codes using a batch size of 500 consistently in approx. 0.5 to 2 seconds, or 5 to 20 codes per millisecond.
Code Density / Collisions / Guessability
The _collisionThreshold field limits the density of your codes. It's a value between 0 and 1. Actually, it must be less than 1 or else you would wind up in an infinite loop when the 4 digit codes were exhausted (probably should add an assertion for this in code). I would recommend never turning it above 0.5 for performance reasons. More than 50% collisions would mean it's spending more time testing already-used codes than actually generating new ones.
Keeping the collision threshold low is how you would control how hard-to-guess your codes are. Setting _collisionThreshold to 0.01 would generate codes such that there's approximately a 1% chance of someone guessing a code.
If collisions occur too frequently, CodeLength (which is used by the GenerateRandomCode() method) will be incremented. This value needs to be persisted somewhere. After executing GenerateCodes(), check CodeLength to see if it has changed and then save the new value.
Source Code
The full code is available here: https://gist.github.com/3217856. I am the author of this code, and am releasing it under the MIT license. I had fun with this little challenge, and also got to learn how to pass a table-valued parameter to an inline parametrized query. I hadn't ever done that before. I've only ever passed them to full-fledged stored procedures.

A possible solution for you is like this:
Find the maximum ID of a voucher (an integer). Then, run any hash function on it, take the first 32 bits and convert to the string you want to show the user (or use a 32bit hash function such as Jenkins hash function). This will probably work, hash collisions are pretty rare. But this solution is very similar to yours, in the point of randomness.
You could run a test which finds the first 10 or 100 collisions (this should be enough for you) and forces the algorithm to "skip" them and use a different starting value. Then, you don't need to check the database at all (well, at least until you reach about 4294967296 vouchers...)

how about utilizing nHibernate's HiLo algorithm?
Here is an example on how you can get the next value (without DB access).

Get closest/next match in .NET Hashtable (or other structure)

I have a scenario at work where we have several different tables of data in a format similar to the following:
Table Name: HingeArms
Hght Part #1 Part #2
33 S-HG-088-00 S-HG-089-00
41 S-HG-084-00 S-HG-085-00
49 S-HG-033-00 S-HG-036-00
57 S-HG-034-00 S-HG-037-00
Where the first column (and possibly more) contains numeric data sorted ascending and represents a range to determine the proper record of data to get (e.g. height <= 33 then Part 1 = S-HG-088-00, height <= 41 then Part 1 = S-HG-084-00, etc.)
I need to lookup and select the nearest match given a specified value. For example, given a height = 34.25, I need to get second record in the set above:
41 S-HG-084-00 S-HG-085-00
These tables are currently stored in a VB.NET Hashtable "cache" of data loaded from a CSV file, where the key for the Hashtable is a composite of the table name and one or more columns from the table that represent the "key" for the record. For example, for the above table, the Hashtable Add for the first record would be:
ht.Add("HingeArms,33","S-HG-088-00,S-HG-089-00")
This seems less than optimal and I have some flexibility to change the structure if necessary (the cache contains data from other tables where direct lookup is possible... these "range" tables just got dumped in because it was "easy"). I was looking for a "Next" method on a Hashtable/Dictionary to give me the closest matching record in the range, but that's obviously not available on the stock classes in VB.NET.
Any ideas on a way to do what I'm looking for with a Hashtable or in a different structure? It needs to be performant as the lookup will get called often in different sections of code. Any thoughts would be greatly appreciated. Thanks.

A hashtable is not a good data structure for this, because items are scattered around the internal array according to their hash code, not their values.
Use a sorted array or List<T> and perform a binary search, e.g.
Setup:
var values = new List<HingeArm>
{
new HingeArm(33, "S-HG-088-00", "S-HG-089-00"),
new HingeArm(41, "S-HG-084-00", "S-HG-085-00"),
new HingeArm(49, "S-HG-033-00", "S-HG-036-00"),
new HingeArm(57, "S-HG-034-00", "S-HG-037-00"),
};
values.Sort((x, y) => x.Height.CompareTo(y.Height));
var keys = values.Select(x => x.Height).ToList();
Lookup:
var index = keys.BinarySearch(34.25);
if (index < 0)
{
index = ~index;
}
var result = values[index];
// result == { Height = 41, Part1 = "S-HG-084-00", Part2 = "S-HG-085-00" }

You can use a sorted .NET array in combination with Array.BinarySearch().
If you get a non negative value this is the index of exact match.
Otherwise, if result is negative use formula
int index = ~Array.BinarySearch(sortedArray, value) - 1
to get index of previous "nearest" match.
The meaning of nearest is defined by a comparer you use. It must be the same you used when sorting the array. See:
http://gmamaladze.wordpress.com/2011/07/22/back-to-the-roots-net-binary-search-and-the-meaning-of-the-negative-number-of-the-array-binarysearch-return-value/

How about LINQ-to-Objects (This is by no means meant to be a performant solution, btw.)
var ht = new Dictionary<string, string>();
ht.Add("HingeArms,33", "S-HG-088-00,S-HG-089-00");
decimal wantedHeight = 34.25m;
var foundIt =
ht.Select(x => new { Height = decimal.Parse(x.Key.Split(',')[1]), x.Key, x.Value }).Where(
x => x.Height < wantedHeight).OrderBy(x => x.Height).SingleOrDefault();
if (foundIt != null)
{
// Do Something with your item in foundIt
}

Weighted Permutations Algorithm

Team:
I am building a business rules engine that is contextually aware -- but it's weighted -- or in other words each business rule has a level of granularity defined by the segments of a key. The segments are not combinatorial in that they cannot be weighted in any order, but rather permutable like a combination lock (interestingly enough improperly named but widely accepted).
However, to reduce the amount of code that's necessary to provide the business rules we are only building exclusion files meaning that each segment could end up with a specific key value or ALL.
So, now that we have an abstract background, let's take a concrete example. The defined segments are as follows:
Line of Business (LOB)
Company
State
Now, let's assume for this example that the LOB is ABC, Company is G and State is WY. If you break that down I should get the following permutations:
ABC_G_WY
ABC_G_ALL
ABC_ALL_WY
ABC_ALL_ALL
ALL_G_WY
ALL_G_ALL
ALL_ALL_WY
ALL_ALL_ALL
However, I need an algorithm to solve that problem. The segments must also be returned in the aforementioned order because you must always find the most finite rule first.
I look forward to your responses and thank you all in advance!

public static void Main(string[] args)
{
List<string> inputValues = new List<string>() { "ABC", "G", "WY" };
List<string> results = new List<string>();
int permutations = (int)Math.Pow(2.0, (double)inputValues.Count);
for (int i = 0; i < permutations; i++)
{
int mask = 1;
Stack<string> lineValues = new Stack<string>();
for (int j = inputValues.Count-1; j >= 0; j--, mask <<= 1)
{
if ((i & mask) == 0)
{
lineValues.Push(inputValues[j]);
}
else
{
lineValues.Push("ALL");
}
}
results.Add(string.Join("_", lineValues.ToArray())); //ToArray can go away in 4.0(?) I've been told. I'm still on 3.5
}
foreach (string s in results)
{
Console.WriteLine(s);
}
Console.WriteLine("Press any key to exit...");
Console.ReadKey(true);
}

If I get the question right, You should:
-Generate all binary strings of length N (there will be 2^N of them)
-sort them by number of bits set
-generate rules. Rule has 'ALL' in position i, if bit number i in the corresponding binary string is set

how to generate a voucher code in c#?

I need to generate a voucher code[ 5 to 10 digit] for one time use only. what is the best way to generate and check if been used?
edited: I would prefer alpha-numeric characters - amazon like gift voucher codes that must be unique.

When generating voucher codes - you should consider whether having a sequence which is predictable is really what you want.
For example, Voucher Codes: ABC101, ABC102, ABC103 etc are fairly predictable. A user could quite easily guess voucher codes.
To protect against this - you need some way of preventing random guesses from working.
Two approaches:
Embed a checksum in your voucher codes.
The last number on a credit card is a checksum (Check digit) - when you add up the other numbers in a certain way, lets you ensure someone has entered a number correctly. See: http://www.beachnet.com/~hstiles/cardtype.html (first link out of google) for how this is done for credit cards.
Have a large key-space, that is only sparsely populated.
For example, if you want to generate 1,000 vouchers - then a key-space of 1,000,000 means you should be able to use random-generation (with duplicate and sequential checking) to ensure it's difficult to guess another voucher code.
Here's a sample app using the large key-space approach:
static Random random = new Random();
static void Main(string[] args)
{
int vouchersToGenerate = 10;
int lengthOfVoucher = 10;
List<string> generatedVouchers = new List<string>();
char[] keys = "ABCDEFGHIJKLMNOPQRSTUVWXYZ01234567890".ToCharArray();
Console.WriteLine("Vouchers: ");
while(generatedVouchers.Count < vouchersToGenerate)
{
var voucher = GenerateVoucher(keys, lengthOfVoucher);
if (!generatedVouchers.Contains(voucher))
{
generatedVouchers.Add(voucher);
Console.WriteLine("\t[#{0}] {1}", generatedVouchers.Count, voucher);
}
}
Console.WriteLine("done");
Console.ReadLine();
}
private static string GenerateVoucher(char[] keys, int lengthOfVoucher)
{
return Enumerable
.Range(1, lengthOfVoucher) // for(i.. )
.Select(k => keys[random.Next(0, keys.Length - 1)]) // generate a new random char
.Aggregate("", (e, c) => e + c); // join into a string
}

Building on the answers from Will Hughes & Shekhar_Pro (and just because I found this question interesting) here's another implementation but I've been a bit liberal with your requirement for the length of the voucher code.
Using a base 32 encoder I found you can use the Tick value to generate alpha-numeric strings. The encoding of a tick count to base 32 produces a 13 character string which can be formatted to make it more readable.
public void GenerateCodes()
{
Random random = new Random();
DateTime timeValue = DateTime.MinValue;
// Create 10 codes just to see the random generation.
for(int i=0; i<10; ++i)
{
int rand = random.Next(3600)+1; // add one to avoid 0 result.
timeValue = timeValue.AddMinutes(rand);
byte[] b = System.BitConverter.GetBytes(timeValue.Ticks);
string voucherCode = Transcoder.Base32Encode(b);
Console.WriteLine(string.Format("{0}-{1}-{2}",
voucherCode.Substring(0,4),
voucherCode.Substring(4,4),
voucherCode.Substring(8,5)));
}
}
Here's the output
AARI-3RCP-AAAAA
ACOM-AAZF-AIAAA
ABIH-LV7W-AIAAA
ADPL-26FL-AMAAA
ABBL-W6LV-AQAAA
ADTP-HFIR-AYAAA
ACDG-JH5K-A4AAA
ADDE-GTST-BEAAA
AAWL-3ZNN-BIAAA
AAGK-4G3Y-BQAAA
If you use a known seed for the Random object and remember how many codes you have already created you can continue to generate codes; e.g. if you need more codes and want to be certain you won't generate duplicates.

Here's one way: Generate a bunch of unique numbers between 10000 and 9999999999 put it in a database. Every time you give one to a user, mark it as used (or delete it if you're trying to save space).
EDIT: Generate the unique alpha-numeric values in the beginning. You'll probably have to keep them around for validation (as others have pointed out).

If your app is limited to using only Numerical digits then i think Timestamps (DateTime.Now.Ticks) can be a good way to get unique code every time. You can use random nums but that will have overhead of checking every number that its been issued already or not. If you can use alphabets also then surely go with GUID.
For checking if its been used or not you need to maintain a database and query it to check for validity.

If you prefer alphanumerical, you could use Guid.NewGuid() method:
Guid g = Guid.NewGuid();
Random rn = new Random();
string gs = g.ToString();
int randomInt = rn.Next(5,10+1);
Console.WriteLine(gs.Substring(gs.Length - randomInt - 1, randomInt));
To check if it was not used store somwhere previously generated codes and compare.

private void AutoPurchaseVouNo1()
{
try
{
int Num = 0;
con.Close();
con.Open();
string incre = "SELECT MAX(VoucherNoint+1) FROM tbl_PurchaseAllCompany";
SqlCommand command = new SqlCommand(incre, con);
if (Convert.IsDBNull(command.ExecuteScalar()))
{
Num = 100;
txtVoucherNoInt1.Text = Convert.ToString(Num);
txtVoucherNo1.Text = Convert.ToString("ABC" + Num);
}
else
{
Num = (int)(command.ExecuteScalar());
txtVoucherNoInt1.Text = Convert.ToString(Num);
txtVoucherNo1.Text = Convert.ToString("ABC" + Num);
}
con.Close();
}
catch (Exception ex)
{
MessageBox.Show("Error: " + ex, "Error !!", MessageBoxButtons.OK, MessageBoxIcon.Error);
}
}
Try this method for creating Voucher Number like ABC100, ABC101, ABC102, etc.

Try this code
var chars = "ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789";
var stringChars = new char[15];
for (int i = 0; i < stringChars.Length; i++)
{
stringChars[i] = chars[random.Next(chars.Length)];
}