I have a question about interesting thing that happened to me when I have tried to convert elements of List<short> to byte[] in C#.
Firstly, I had to read large binary file, which contains 262144 short type signed numbers. I have read the file and build list of numbers with the following code:
byte[] content = null;
content = File.ReadAllBytes(scanName);
List<int> transformed = new List<int>();
for (int n = 0; n < content.Length; n += 2) // 2 bytes
{
short sample = BitConverter.ToInt16(content, n);
transformed.Add(sample);
}
Then I have compressed and decompressed numbers with algorithm and got back same values, which seemed right. The problems occurs when try to convert both lists to byte arrays. This has been done by following method:
private byte[] ToByte(List<short> list){
List<byte> toRet = new List<byte>();
foreach(short s in list)
{
byte[] converted = BitConverter.GetBytes(s);
foreach(byte b in converted)
{
toRet.Add(b);
}
}
return toRet.ToArray();
}
But when I compared both byte arrays with first.SequenceEqual(second), the method returned false. Isn't it strange, because values in both lists are same?
At the end, I have solved the issue. The problem wasn't in converting short to byte, but in the part of code which hasn't been published there. Specifically, I made very beginner mistake, I converted 2D array into 1D array in the wrong way. Now everything works perfectly. Thank you for all your responses and sorry for inconveniences!
So in c#, I have needed a random below given number generator and I found one on StackOverFlow. But near the end, it converts the byte array into a BigInteger. I tried doing the same, though I am using the Deveel-Math lib as it allows me to us BigDeciamals. But I have tried to the array change into a value, and that into a String but I keep getting a "Could not find any recognizable digits." error and as of now I am stumped.
public static BigInteger RandomIntegerBelow1(BigInteger N)
{
byte[] bytes = N.ToByteArray();
BigInteger R;
Random random = new Random();
do
{
random.NextBytes(bytes);
bytes[bytes.Length - 1] &= (byte)0x7F; //force sign bit to positive
R = BigInteger.Parse(BytesToStringConverted(bytes)) ;
//the Param needs a String value, exp: BigInteger.Parse("100")
} while (R >= N);
return R;
}
static string BytesToStringConverted(byte[] bytes)
{
using (var stream = new MemoryStream(bytes))
{
using (var streamReader = new StreamReader(stream))
{
return streamReader.ReadToEnd();
}
}
}
Deveel-Math
Wrong string conversion
You are converting your byte array to a string of characters based on UTF encoding. I'm pretty sure this is not what you want.
If you want to convert a byte array to a string that contains a number expressed in decimal, try this answer using BitConverter.
if (BitConverter.IsLittleEndian)
Array.Reverse(array); //need the bytes in the reverse order
int value = BitConverter.ToInt32(array, 0);
This is way easier
On the other hand, I notice that Deveel-Math's BigInteger has a constructor that takes a byte array as input (see line 226). So you should be able to greatly simplify your code by doing this:
R = new Deveel.Math.BigInteger(1, bytes) ;
However, since Deveel.Math appears to be BigEndian, you may need to reverse the array first:
System.Array.Reverse(bytes);
R = new Deveel.Math.BigInteger(1, bytes);
I have some code that converts a float[] to a Base64 string:
float[] f_elements = <from elsewhere in my code>;
byte[] f_vfeat = f_elements.SelectMany(value => BitConverter.GetBytes(value)).ToArray();
string f_sig = Convert.ToBase64String(f_vfeat);
I also have - basically - the same code that converts an int[] to a Base64 string:
int[] i_elements = <from elsewhere in my code>;
byte[] i_feat = i_elements.SelectMany(value => BitConverter.GetBytes(value)).ToArray();
string i_sig = Convert.ToBase64String(i_feat);
Both of these produce Base64 strings as expected. However, now I need to decode back to an array, and I'm running into trouble.
How can I go from my Base64 string(s), and get the original data array(s). Before I decode the Base64 string, I will know if it is suppose to be an int[] or float[], so I think that will help.
Does anyone know how to do go from Base64 string to float[] or int[]?
You can use BitConverter.ToInt32 or BitConverter.ToSingle to convert part of an array:
byte[] bytes = Convert.FromBase64String();
int[] ints = new int[bytes.Length / 4];
for (int i = 0; i < ints.Length; i++)
{
ints[i] = BitConverter.ToInt32(bytes, i * 4);
}
(And the equivalent for ToSingle, of course.)
In my view, it's a shame that GetBytes doesn't have an overload to write the bytes directly into an existing array, instead of creating a new array on each call...
Is there something wrong with Convert.FromBase64String?
byte[] i_feat = Convert.FromBase64String(i_sig)
Is there some equivalent to the StringBuilder.Insert method for use with byte arrays? I was going to try and use a MemoryStream but an error was thrown telling me that the MemoryStream object is "not expandable". I need to place bytes within specific spots of another (already existing byte array).
Here is the same idea using StringBuilder.
String firstString = "FirstData";
String someString = "string Data";
int Index = 0;
StringBuilder sb = new StringBuilder(firstString);
for(int i = 0; i < someString.Length; i++)
{
sb.Insert(index, someString[i]);
index += 2;
}
Thank you for any help,
Evan
Use a List; which will allow you to insert as needed. If you have an existing array you can call ToList().
List<byte> data1 = new List<byte>() {10, 11, 12};
List<byte> data2 = new List<byte>() {13, 14, 15};
int Index = 0;
for(int i = 0; i < data1.Count; i++)
{
data2.Insert(index, data2[i]);
index += 2;
}
Then to go back to an array simply call ToArray().
You could use a list of bytes as opposed to an array of byte and use the insert method there.
Have you tried using List, it has an Insert method
A "Stream" by definition flows only one way. There is no way to change something once it's flowed out the gate. If you want this ability, you'll have to create a regular in-memory collection of some kind to give yourself a buffer you can modify before attempting to push the data out to a stream.
You can create a byte array, instead of creating some collection of bytes:
System.Text.ASCIIEncoding encoding=new System.Text.ASCIIEncoding();
Byte[] bytes = encoding.GetBytes("yourString");
or:
byte[] utf8 = System.Text.Encoding.UTF8.GetBytes ("yourString");
Use ASCII or UTF8 or any other conversation.
This question already has answers here:
What is the equivalent of memset in C#?
(17 answers)
Closed 8 years ago.
I'm busy rewriting an old project that was done in C++, to C#.
My task is to rewrite the program so that it functions as close to the original as possible.
During a bunch of file-handling the previous developer who wrote this program creates a structure containing a ton of fields that correspond to the set format that a file has to be written in, so all that work is already done for me.
These fields are all byte arrays. What the C++ code then does is use memset to set this entire structure to all spaces characters (0x20). One line of code. Easy.
This is very important as the utility that this file eventually goes to is expecting the file in this format. What I've had to do is change this struct to a class in C#, but I cannot find a way to easily initialize each of these byte arrays to all space characters.
What I've ended up having to do is this in the class constructor:
//Initialize all of the variables to spaces.
int index = 0;
foreach (byte b in UserCode)
{
UserCode[index] = 0x20;
index++;
}
This works fine, but I'm sure there must be a simpler way to do this. When the array is set to UserCode = new byte[6] in the constructor the byte array gets automatically initialized to the default null values. Is there no way that I can make it become all spaces upon declaration, so that when I call my class' constructor that it is initialized straight away like this? Or some memset-like function?
For small arrays use array initialisation syntax:
var sevenItems = new byte[] { 0x20, 0x20, 0x20, 0x20, 0x20, 0x20, 0x20 };
For larger arrays use a standard for loop. This is the most readable and efficient way to do it:
var sevenThousandItems = new byte[7000];
for (int i = 0; i < sevenThousandItems.Length; i++)
{
sevenThousandItems[i] = 0x20;
}
Of course, if you need to do this a lot then you could create a helper method to help keep your code concise:
byte[] sevenItems = CreateSpecialByteArray(7);
byte[] sevenThousandItems = CreateSpecialByteArray(7000);
// ...
public static byte[] CreateSpecialByteArray(int length)
{
var arr = new byte[length];
for (int i = 0; i < arr.Length; i++)
{
arr[i] = 0x20;
}
return arr;
}
Use this to create the array in the first place:
byte[] array = Enumerable.Repeat((byte)0x20, <number of elements>).ToArray();
Replace <number of elements> with the desired array size.
You can use Enumerable.Repeat()
Enumerable.Repeat generates a sequence that contains one repeated value.
Array of 100 items initialized to 0x20:
byte[] arr1 = Enumerable.Repeat((byte)0x20,100).ToArray();
var array = Encoding.ASCII.GetBytes(new string(' ', 100));
If you need to initialise a small array you can use:
byte[] smallArray = new byte[] { 0x20, 0x20, 0x20, 0x20, 0x20, 0x20, 0x20 };
If you have a larger array, then you could use:
byte[] bitBiggerArray Enumerable.Repeat(0x20, 7000).ToArray();
Which is simple, and easy for the next guy/girl to read. And will be fast enough 99.9% of the time.
(Normally will be the BestOption™)
However if you really really need super speed, calling out to the optimized memset method, using P/invoke, is for you:
(Here wrapped up in a nice to use class)
public static class Superfast
{
[DllImport("msvcrt.dll",
EntryPoint = "memset",
CallingConvention = CallingConvention.Cdecl,
SetLastError = false)]
private static extern IntPtr MemSet(IntPtr dest, int c, int count);
//If you need super speed, calling out to M$ memset optimized method using P/invoke
public static byte[] InitByteArray(byte fillWith, int size)
{
byte[] arrayBytes = new byte[size];
GCHandle gch = GCHandle.Alloc(arrayBytes, GCHandleType.Pinned);
MemSet(gch.AddrOfPinnedObject(), fillWith, arrayBytes.Length);
gch.Free();
return arrayBytes;
}
}
Usage:
byte[] oneofManyBigArrays = Superfast.InitByteArray(0x20,700000);
Maybe these could be helpful?
What is the equivalent of memset in C#?
http://techmikael.blogspot.com/2009/12/filling-array-with-default-value.html
Guys before me gave you your answer. I just want to point out your misuse of foreach loop. See, since you have to increment index standard "for loop" would be not only more compact, but also more efficient ("foreach" does many things under the hood):
for (int index = 0; index < UserCode.Length; ++index)
{
UserCode[index] = 0x20;
}
This is a faster version of the code from the post marked as the answer.
All of the benchmarks that I have performed show that a simple for loop that only contains something like an array fill is typically twice as fast if it is decrementing versus if it is incrementing.
Also, the array Length property is already passed as the parameter so it doesn't need to be retrieved from the array properties. It should also be pre-calculated and assigned to a local variable.
Loop bounds calculations that involve a property accessor will re-compute the value of the bounds before each iteration of the loop.
public static byte[] CreateSpecialByteArray(int length)
{
byte[] array = new byte[length];
int len = length - 1;
for (int i = len; i >= 0; i--)
{
array[i] = 0x20;
}
return array;
}
Just to expand on my answer a neater way of doing this multiple times would probably be:
PopulateByteArray(UserCode, 0x20);
which calls:
public static void PopulateByteArray(byte[] byteArray, byte value)
{
for (int i = 0; i < byteArray.Length; i++)
{
byteArray[i] = value;
}
}
This has the advantage of a nice efficient for loop (mention to gwiazdorrr's answer) as well as a nice neat looking call if it is being used a lot. And a lot mroe at a glance readable than the enumeration one I personally think. :)
The fastest way to do this is to use the api:
bR = 0xFF;
RtlFillMemory(pBuffer, nFileLen, bR);
using a pointer to a buffer, the length to write, and the encoded byte. I think the fastest way to do it in managed code (much slower), is to create a small block of initialized bytes, then use Buffer.Blockcopy to write them to the byte array in a loop. I threw this together but haven't tested it, but you get the idea:
long size = GetFileSize(FileName);
// zero byte
const int blocksize = 1024;
// 1's array
byte[] ntemp = new byte[blocksize];
byte[] nbyte = new byte[size];
// init 1's array
for (int i = 0; i < blocksize; i++)
ntemp[i] = 0xff;
// get dimensions
int blocks = (int)(size / blocksize);
int remainder = (int)(size - (blocks * blocksize));
int count = 0;
// copy to the buffer
do
{
Buffer.BlockCopy(ntemp, 0, nbyte, blocksize * count, blocksize);
count++;
} while (count < blocks);
// copy remaining bytes
Buffer.BlockCopy(ntemp, 0, nbyte, blocksize * count, remainder);
This function is way faster than a for loop for filling an array.
The Array.Copy command is a very fast memory copy function. This function takes advantage of that by repeatedly calling the Array.Copy command and doubling the size of what we copy until the array is full.
I discuss this on my blog at https://grax32.com/2013/06/fast-array-fill-function-revisited.html (Link updated 12/16/2019). Also see Nuget package that provides this extension method. http://sites.grax32.com/ArrayExtensions/
Note that this would be easy to make into an extension method by just adding the word "this" to the method declarations i.e. public static void ArrayFill<T>(this T[] arrayToFill ...
public static void ArrayFill<T>(T[] arrayToFill, T fillValue)
{
// if called with a single value, wrap the value in an array and call the main function
ArrayFill(arrayToFill, new T[] { fillValue });
}
public static void ArrayFill<T>(T[] arrayToFill, T[] fillValue)
{
if (fillValue.Length >= arrayToFill.Length)
{
throw new ArgumentException("fillValue array length must be smaller than length of arrayToFill");
}
// set the initial array value
Array.Copy(fillValue, arrayToFill, fillValue.Length);
int arrayToFillHalfLength = arrayToFill.Length / 2;
for (int i = fillValue.Length; i < arrayToFill.Length; i *= 2)
{
int copyLength = i;
if (i > arrayToFillHalfLength)
{
copyLength = arrayToFill.Length - i;
}
Array.Copy(arrayToFill, 0, arrayToFill, i, copyLength);
}
}
You can use a collection initializer:
UserCode = new byte[]{0x20,0x20,0x20,0x20,0x20,0x20};
This will work better than Repeat if the values are not identical.
You could speed up the initialization and simplify the code by using the the Parallel class (.NET 4 and newer):
public static void PopulateByteArray(byte[] byteArray, byte value)
{
Parallel.For(0, byteArray.Length, i => byteArray[i] = value);
}
Of course you can create the array at the same time:
public static byte[] CreateSpecialByteArray(int length, byte value)
{
var byteArray = new byte[length];
Parallel.For(0, length, i => byteArray[i] = value);
return byteArray;
}