I'm using a third party SMS provider and have hit an issue with converting from UCS-2 messages back into readable text.
Their API documentation has this code sample which converts UCS-2 messges into what I'm picking up on the API.
string message = "Это тестовое сообщение юникода";
byte[] ba = Encoding.BigEndianUnicode.GetBytes (message);
var hexString = BitConverter.ToString (ba);
Console.WriteLine ("#U" + hexString.Replace("-",""));
Which converts the message string into
#U042D0442043E00200442043504410442043E0432043E043500200441043E043E043104490435043D043804350020044E043D0438043A043E04340430
This looks like the UCS-2 messages I'm picking up from their API.
Unfortunately they don't give any code samples of how to convert the messages back into a readable form.
I'm sure its not there in the docs because its something simple - but I just seem to figure out how to do it.
To reverse what you have (the string of hex prefixed with #U)
var message = "Это тестовое сообщение юникода";
var ba = Encoding.BigEndianUnicode.GetBytes(message);
var hexString = BitConverter.ToString(ba);
var encoded = "#U" + hexString.Replace("-", "");
Console.WriteLine(encoded);
// reverse
var bytes = Enumerable.Range(2, encoded.Length-2)
.Where(x => x % 2 == 0)
.Select(x => Convert.ToByte(encoded.Substring(x, 2), 16))
.ToArray();
var result = Encoding.BigEndianUnicode.GetString(bytes);
Console.WriteLine(result);
Output
#U042D0442043E00200442043504410442043E0432043E043500200441043E043E043104490435043D043804350020044E043D0438043A043E04340430
Это тестовое сообщение юникода
Demo here
looks like this would be the reverse:
string message = Encoding.BigEndianUnicode.GetString(ba);
The extraction of bytes could be done by such a method:
private IEnumerable<byte> GetTheBytes(string uc2Message)
{
string bytesOnly = uc2Message.Trim('#', 'U');
for (int i = 0; i < bytesOnly.Length-2; i+=2)
{
yield return Convert.ToByte($"{bytesOnly[i]}{bytesOnly[i+1]}", 16);
}
}
Console.WriteLine(Encoding.BigEndianUnicode.GetString(GetTheBytes(uc2Message).ToArray()));
I want to generate so many random unique hexadecimal value by a method.
I want to specify the character length in the parameter of this method.
public string GenerateUniqueHexadecimal(int length)
{
string date = DateTime.Now.ToString("yyMMddHHmmss");
long l = Convert.ToInt64(date);
string hex = l.ToString("X2");
hex = hex.Substring(0, length);
return hex;
}
You could create a randomized hex string like this:
public class StringRandomizer
{
public const string StringChars = "0123456789abcdef";
public string GenerateUniqueHexString(int length)
{
Random rand = new Random();
var charList = StringChars.ToArray();
string hexString = "";
for(int i = 0; i < length; i++)
{
int randIndex = rand.Next(0, charList.Length);
hexString += charList[randIndex];
}
return hexString;
}
}
If you want unique strings then they are not perfectly random -- perfectly random strings may repeat. So you can use a non-random method instead of a random method. One possibility is encryption. Use a block cipher with the right bit size and encrypt the numbers 0, 1, 2, 3, ... for as many unique outputs as you need. Because encryption is one-to-one, you are guaranteed unique outputs as long as the inputs are unique and the key stays the same. The encrypted numbers will appear random, though they are not.
For 64 bits, 16 hex characters, use DES. For 128 bits, 32 hex characters, use AES. For other sizes either use Hasty Pudding cipher, or write a version of the Feistel cipher with the required block size. That last will not be as secure as DES or AES, though that only matters if you require a high level of security.
Well, you have two main possibilities: shuffle when we create all possible Pow(16, length) hexadecimal strings, shuffle and take count of them:
using System.Linq;
...
private static Random s_Generator = new Random();
private static String[] RandomUniqueHexShuffle(int length, int count) {
return Enumerable
.Range(0, 1 << (4 * length))
.OrderBy(i => s_Generator.NextDouble())
.Take(count)
.Select(item => item.ToString($"x{length}"))
.ToArray();
}
And direct generation: where we just create string after string checking them for being unique:
private static String[] RandomUniqueHexDirect(int length, int count) {
HashSet<string> used = new HashSet<string>();
string[] result = new string[count];
for (int i = 0; i < result.Length; ) {
string value = string.Concat(Enumerable
.Range(0, length)
.Select(j => s_Generator.Next(0, 16).ToString("x")));
if (used.Add(value))
result[i++] = value;
}
return result;
}
Which of them should be chosen? For small length and large count use shuffle, otherwise - direct generation
Demo:
// unique random hex strings of length 3 (5 of them)
Console.WriteLine(string.Join(Environment.NewLine, RandomUniqueHexShuffle(3, 5)));
Console.WriteLine();
// unique random hex strings of length 24 (4 of them)
Console.WriteLine(string.Join(Environment.NewLine, RandomUniqueHexShuffle(24, 4)));
Outcome: (may vary since these are random strings)
f6b
106
baf
412
443
b150ab7c723caceccdd6b597
e3c9a9ab066dcea9ae68031c
0b2e1b368e17ed1d88c3600e
33bdf06b109f6b953f5ed2bd
I created a random string. First, I'd like to display the string and after that, I'd like to convert it to a MD5 hash. Can someone help me? I tried a lot but I can't get the right solution.
Here is my Code:
public static string Generate(int lenght)
{
const string chars = "ABCDEFGHIJKLMNOPQRSTUVWXYZ0123456789";
Random random = new Random();
return new string(Enumerable.Repeat(chars, lenght).Select(s => s[random.Next(s.Length)]).ToArray());
}
public static string MD5Hash()
{
StringBuilder hash = new StringBuilder();
MD5CryptoServiceProvider md5provider = new MD5CryptoServiceProvider();
byte[] bytes = md5provider.ComputeHash(new UTF8Encoding().GetBytes(Generate));
for (int i = 0; i < bytes.Length; i++)
{
hash.Append(bytes[i].ToString("x2"));
}
return hash.ToString();
}
Thank you in advance
Best Regards
Either pass a string argument into the MD5Hash method (MD5Hash(string input) for example) OR you can use the Generate method inside the MD5Hash method (string input = Generate(*LengthYouWant*); for example).
Afterwards you can replace the Generate you've put inside of GetBytes currently with the input string.
I'm using this code to generate random strings with given length
public string RandomString(int length)
{
const string valid = "abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ1234567890";
StringBuilder res = new StringBuilder();
Random rnd = new Random();
while (0 < length--)
{
res.Append(valid[rnd.Next(valid.Length)]);
}
return res.ToString();
}
However, I read that RNGCryptoServiceProvideris more secure than Random class. How can I implement RNGCryptoServiceProvider to this function. It should use valid string just like this function.
Since RNGRandomNumberGenerator only returns byte arrays, you have to do it like this:
static string RandomString(int length)
{
const string valid = "abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ1234567890";
StringBuilder res = new StringBuilder();
using (RNGCryptoServiceProvider rng = new RNGCryptoServiceProvider())
{
byte[] uintBuffer = new byte[sizeof(uint)];
while (length-- > 0)
{
rng.GetBytes(uintBuffer);
uint num = BitConverter.ToUInt32(uintBuffer, 0);
res.Append(valid[(int)(num % (uint)valid.Length)]);
}
}
return res.ToString();
}
Note however that this has a flaw, 62 valid characters is equal to 5,9541963103868752088061235991756 bits (log(62) / log(2)), so it won't divide evenly on a 32 bit number (uint).
What consequences does this have?
As a result, the random output won't be uniform. Characters which are lower in value will occur more likely (just by a small fraction, but still it happens).
To be more precise, the first 4 characters of a valid array are 0,00000144354999199840239435286 % more likely to occur.
To avoid this, you should use array lengths that will divide evenly into 64 (Consider using Convert.ToBase64String on the output instead, since you can cleanly match 64 bits to 6 bytes.
You need to generate random bytes using RNGCryptoServiceProvider and append only the valid ones to the returned string:
const string valid = "abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ1234567890";
static string GetRandomString(int length)
{
string s = "";
using (RNGCryptoServiceProvider provider = new RNGCryptoServiceProvider())
{
while (s.Length != length)
{
byte[] oneByte = new byte[1];
provider.GetBytes(oneByte);
char character = (char)oneByte[0];
if (valid.Contains(character))
{
s += character;
}
}
}
return s;
}
You could also use modulo in order to not skip the invalid byte values but that the chances for each character won't be even.
The RNGCryptoServiceProvider returns random numbers in the form of bytes, so you need a way to get a more convenient random number from it:
public static int GetInt(RNGCryptoServiceProvider rnd, int max) {
byte[] r = new byte[4];
int value;
do {
rnd.GetBytes(r);
value = BitConverter.ToInt32(r, 0) & Int32.MaxValue;
} while (value >= max * (Int32.MaxValue / max));
return value % max;
}
Then you can use that in your method:
public static string RandomString(int length) {
const string valid = "abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ1234567890";
StringBuilder res = new StringBuilder();
using (RNGCryptoServiceProvider rnd = new RNGCryptoServiceProvider()) {
while (length-- > 0) {
res.Append(valid[GetInt(rnd, valid.Length)]);
}
}
return res.ToString();
}
(I made the method static, as it doesn't use any instance data.)
Note
I am aware of the deviation with OP's use case, but I think this might help others who does not have the "arbitrary" 62 character limitation and just want to encode bytes using RNGCryptoServiceProvider to generate random string.
TL;DR
just skip to the bottom, the base64 encoded case.
A lot can be said about the reasons to convert a cryptographic byte array into a string, but usually it is for some sort of serialization purposes; and hence, in that case: the selected character set is arbitrary.
So, if it is about serialization, you have tons of options; e.g:
text as HEX representation
text as base64 representation
text as alternative representation
All of these are making use of the same thing: encode numbers in such a way it is suited to be transmitted in a medium that does not support native binary transfer.
I call this "text as ... representation", because in the end, it is text that will be transmitted.
An example in HEX:
//note: using text as HEX makes the result longer
var crypt = new RNGCryptoServiceProvider();
var sb = new StringBuilder();
var buf = new byte[10]; //length: should be larger
crypt.GetBytes(buf);
//gives a "valid" range of: "0123456789ABCDEF"
foreach (byte b in buf)
sb.AppendFormat("{0:x2}", b); //applies "text as hex" encoding
//sb contains a RNGCryptoServiceProvider based "string"
Now you'll say:
but wait: these are only 16 characters where OP's sequence has 62. 62 is more efficient than 16, so, converted to text, your string will be a lot longer.
"Yes", I'll say, "and if that's a problem, why don't you pick a larger number easy-to-read-and-serrializable-characters... 62 ... or 64 perhaps"
The code would be:
//note: added + and / chars. could be any of them
const string valid = "abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ1234567890+/";
var crypt = new RNGCryptoServiceProvider();
var sb = new StringBuilder();
var buf = new byte[10]; //length: should be larger
crypt.GetBytes(buf); //get the bytes
foreach (byte b in buf)
sb.Append(valid[b%64]);
Note: As #Guffa stated; using % is forbidden unless it doesn't alter the distribution. To make this happen, given a evenly distributed set, the subset must fit exactly x times in the original set.
So, expanding your initial valid set with 2 gives a valid result (because: 256 / 64 = 4) --- but, this does not honor OP's 62 character requirements. In fact to get an even distribution you'll need some trickery, addressed in the other answers.
Also note: in all the answers, including this one, the sub-set is smaller than the 256 possibilities of the byte. This means there is less information available in an encoded char than in a byte. This means if you have your string with 4 encoded chars, it's easier to crack the original 4 byte result of the RNGCryptoServiceProvider - So keep in mind, the cryptographic strength is depending on the byte length, not the encoded char length.
Base64
But, now you say:
"Ok, let drop the 62 requirement, and use 64 - why not use 64 base encoding?",
well, if it's suits you, but note trailing =, see Base64 on Wikipedia, it is an additional optional charater which is used.
var crypt = new RNGCryptoServiceProvider();
// = padding characters might be added to make the last encoded block
// contain four Base64 characters.
// which is actually an additional character
var buf = new byte[10];
crypt.GetBytes(buf);
//contains a RNGCryptoServiceProvider random string, which is fairly readable
//and contains max 65 different characters.
//you can limit this to 64, by specifying a different array length.
//because log2(64) = 6, and 24 = 4 x 6 = 3 x 8
//all multiple of 3 bytes are a perfect fit. (e.g.: 3, 6, 15, 30, 60)
string result = Convert.ToBase64String(buf);
My implementation that fixes the issue with 5,9541963103868752088061235991756 bits
public static string RandomString(int length)
{
const string alphabet = "abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ1234567890";
var res = new StringBuilder(length);
using (var rng = new RNGCryptoServiceProvider())
{
int count = (int)Math.Ceiling(Math.Log(alphabet.Length, 2) / 8.0);
Debug.Assert(count <= sizeof(uint));
int offset = BitConverter.IsLittleEndian ? 0 : sizeof(uint) - count;
int max = (int)(Math.Pow(2, count*8) / alphabet.Length) * alphabet.Length;
byte[] uintBuffer = new byte[sizeof(uint)];
while (res.Length < length)
{
rng.GetBytes(uintBuffer, offset, count);
uint num = BitConverter.ToUInt32(uintBuffer, 0);
if (num < max)
{
res.Append(alphabet[(int) (num % alphabet.Length)]);
}
}
}
return res.ToString();
}
see https://bitbucket.org/merarischroeder/number-range-with-no-bias/
I'm sure I have answered this one before with a secure implementation, no bias, and good performance. If so, please comment.
Looking at Tamir's answer, I thought it would be better to use the modulus operation, but trim off the incomplete remainder of byte values. I'm also writing this answer now (possibly again), because I need to reference this solution to a peer.
Approach 1
Support for ranges that are no bigger than 0-255. But it can fall back to approach 2 (which is a little slower)
One byte is always used per value.
Truncate the incomplete remainder if (buffer[i] >= exclusiveLimit)
Modulate the desired range size. After truncation beyond the exclusiveLimit the modulus remains perfectly balanced
(Using a bitmask instead of modulus is a slower approach)
EG. If you want a range 0-16 (that's 17 different values), then 17 can fit into a byte 15 times. There is 1 value that must be discarded [255], otherwise the modulus will be fine.
Code for Approach 1
const string lookupCharacters = "abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ1234567890";
static void TestRandomString()
{
Console.WriteLine("A random string of 100 characters:");
int[] randomCharacterIndexes = new int[100];
SecureRangeOriginal(randomCharacterIndexes, lookupCharacters.Length);
var sb = new StringBuilder();
for (int i = 0; i < randomCharacterIndexes.Length; i++)
{
sb.Append(lookupCharacters[randomCharacterIndexes[i]]);
}
Console.WriteLine(sb.ToString());
Console.WriteLine();
}
static void SecureRangeOriginal(int[] result, int maxInt)
{
if (maxInt > 256)
{
//If you copy this code, you can remove this line and replace it with `throw new Exception("outside supported range");`
SecureRandomIntegerRange(result, 0, result.Length, 0, maxInt); //See git repo for implementation.
return;
}
var maxMultiples = 256 / maxInt; //Finding the byte number boundary above the provided lookup length - the number of bytes
var exclusiveLimit = (maxInt * maxMultiples); //Expressing that boundary (number of bytes) as an integer
var length = result.Length;
var resultIndex = 0;
using (var provider = new RNGCryptoServiceProvider())
{
var buffer = new byte[length];
while (true)
{
var remaining = length - resultIndex;
if (remaining == 0)
break;
provider.GetBytes(buffer, 0, remaining);
for (int i = 0; i < remaining; i++)
{
if (buffer[i] >= exclusiveLimit)
continue;
var index = buffer[i] % maxInt;
result[resultIndex++] = index;
}
}
}
}
Approach 2
Technically ranges from 0 to ulong.Max can be supported
Treat RNGCryptoServiceProvider bytes as a bitstream
Calculate the base2 bit length needed per number
Take the next number from the random bitstream
If that number is still greater than the desired range, discard
Results:
See the repository for the latest results from the test harness
Both approaches appear to have a suitably balanced distribution of numbers
Approach 1 is faster [859ms] but it only works on individual bytes.
Approach 2 is a little slower [3038ms] than Approach 1, but it works across byte boundaries. It discards fewer bits, which can be useful if the random stream input becomes a bottleneck (different algorithm for example).
A hybrid of both approaches gives the best of both worlds: better speed when the byte range is 0-255, support for ranges beyond 255 but a bit slower.
I personally like to use this:
private static string GenerateRandomSecret()
{
var validChars = Enumerable.Range('A', 26)
.Concat(Enumerable.Range('a', 26))
.Concat(Enumerable.Range('0', 10))
.Select(i => (char)i)
.ToArray();
var randomByte = new byte[64 + 1]; // Max Length + Length
using (var rnd = new RNGCryptoServiceProvider())
{
rnd.GetBytes(randomByte);
var secretLength = 32 + (int)(32 * (randomByte[0] / (double)byte.MaxValue));
return new string(
randomByte
.Skip(1)
.Take(secretLength)
.Select(b => (int) ((validChars.Length - 1) * (b / (double) byte.MaxValue)))
.Select(i => validChars[i])
.ToArray()
);
}
}
There shouldn't be any part that needs additional description, but to clarify, this function returns a random string with a random length between 32 and 64 chars and doesn't use % (mod) therefore should keep uniformity a little better.
I use this to create a random salt at program installation and later save it to a file. Therefore security of generated string is not of special concern while the program is running as it is going to get written to an unencrypted file later on anyway.
However, for more serious situations, this shouldn't be used as it is and should be converted to use SecureString class if you are going to keep this value in memory. Read more here:
https://learn.microsoft.com/en-us/dotnet/api/system.security.securestring?redirectedfrom=MSDN&view=netframework-4.7.2
However, even this only applies to NetFramework, for NetCore you need to find another way to secure the value in the memory. Read more here:
https://github.com/dotnet/platform-compat/blob/master/docs/DE0001.md
RNGCryptoServiceProvider is deprecated in .NET 6
Instead, its base class RandomNumberGenerator should be used.
Based on Guffa's answer a solution can be crafted like this:
public static string GetString(int length, string alphabet)
{
var secret = new StringBuilder();
while (length-- > 0)
{
secret.Append(alphabet[RandomNumberGenerator.GetInt32(alphabet.Length)]);
}
return secret.ToString();
}
Whereas alphabet is the desired set of valid characters.
private string sifreuretimi(int sayı) //3
{
Random rastgele = new Random();
StringBuilder sb = new StringBuilder();
char karakter1 = ' ', karakter2 = ' ', karakter3 = ' ';
int ascii1, ascii2, ascii3 = 0;
for (int i = 0; i < sayı/3; i++)
{
ascii1 = rastgele.Next(48,58);
karakter1 = Convert.ToChar(ascii1);
ascii2 = rastgele.Next(65, 91);
karakter2 = Convert.ToChar(ascii2);
ascii3 = rastgele.Next(97, 123);
karakter3 = Convert.ToChar(ascii3);
sb.Append(karakter1);
sb.Append(karakter2);
sb.Append(karakter3);
}
return sb.ToString();
}
This is a nice short solution that will also do the trick, if your goal is just to create a random string, let's say for use as a secure token.
public string RandomTokenString(int tokenLength)
{
var rngCryptoServiceProvider = new RNGCryptoServiceProvider();
var randomBytes = new byte[tokenLength];
rngCryptoServiceProvider.GetBytes(randomBytes);
return BitConverter.ToString(randomBytes);
}
Basically, I'm building a small tracker for experimental purposes. I've gotten quite far, and am now working on the announce part.
What I really can't figure out is how I should decode the info_hash query string provided.
From the specification, it is a urlencoded 20-byte SHA1 hash, which made me write this code,
byte[] foo = Encoding.Default.GetBytes(HttpUtility.UrlDecode(infoHash));
string temp = "";
foreach (byte b in foo)
{
temp += b.ToString("X");
}
Which gives 'temp' the following value,
5D3F3F3F3F5E3F3F3F153FE4033683F55693468
The first and last few characters are correct. This is the raw info_hash,
%5d%96%b6%f6%84%5e%ea%da%c5%15%c4%0e%403h%b9Ui4h
And this is what both uTorrent and my own tracker gives me as info_hash when generating it from the torrent file,
5D96B6F6845EEADAC515C40E403368B955693468
What am I doing wrong?
UrlDecode returns a string, but a SHA1 hash doesn't make sense if interpreted as (ANSI) string.
You need to decode the input string directly to an byte array, without the roundtrip to a string.
var s = "%5d%96%b6%f6%84%5e%ea%da%c5%15%c4%0e%403h%b9Ui4h";
var ms = new MemoryStream();
for (var i = 0; i < s.Length; i++)
{
if (s[i] == '%')
{
ms.WriteByte(
byte.Parse(s.Substring(i + 1, 2), NumberStyles.AllowHexSpecifier));
i += 2;
}
else if (s[i] < 128)
{
ms.WriteByte((byte)s[i]);
}
}
byte[] infoHash = ms.ToArray();
string temp = BitConverter.ToString(infoHash);
// "5D-96-B6-F6-84-5E-EA-DA-C5-15-C4-0E-40-33-68-B9-55-69-34-68"
HttpUtility.UrlDecodeToBytes