Develop Reference

What is the most performant method for setting a subset of sequential items in a large array?

I am working with camera streams. I bring in 1,228,800 bytes per frame, so efficiency is crucial and nanoseconds per byte add up quickly.
I've come up with some example code below to describe the problem as succinctly as possible without seeming too contrived.
There are plenty of inefficiencies in this example code such as defining variables inside of loops, or dividing the brightness value instead of just using a composite value. These aren't my concern, and are just there to make the example simpler.
What I need advice on is the most performant method in C# for setting 3 sequential values at some determined location in a very large array, such as in the case below where I'm setting BGR to 255 while skipping the 4th byte.
Edit: To clarify, the concerning issue is where I'm reindexing Output for each index that is being set. It seems like there should potentially be some method for not traversing the entire array for each value if I already have the location of the previous item.
// Colors are stored as 4 bytes: BGRX where X is always 0
public byte[] Input = new byte[640 * 480 * 4];
public byte[] Output = new byte[640 * 480 * 4];
public int Threshold = 180;
public void ProcessImage() {
for (int i = 0; i < Input.Length; i += 4) {
var brightness = (Input[i] + Input[i + 1] + Input[i + 2]) / 3; // some value under 255
if (brightness > Threshold) {
// What is the most efficient way possible to do this?
Output[i] = 255 - Input[i];
Output[i + 1] = 255 - Input[i + 1];
Output[i + 2] = 255 - Input[i + 2];
}
else {
Output[i] = Input[i];
Output[i + 1] = Input[i + 1];
Output[i + 2] = Input[i + 2];
}
}
}

This (untested, and unsafe) code should be faster, if all you care about is speed:
public void ProcessImage()
{
int ilength = Input.Length;
Debug.Assert(ilength == Output.Length);
Debug.Assert(ilength%4 == 0);
unsafe
{
GCHandle pinned1 = GCHandle.Alloc(Input, GCHandleType.Pinned);
byte* input = (byte*)pinned1.AddrOfPinnedObject();
GCHandle pinned2 = GCHandle.Alloc(Input, GCHandleType.Pinned);
byte* output = (byte*)pinned2.AddrOfPinnedObject();
for (int i = 0; i < ilength; i += 4)
{
var brightness = (*(input) + *(input + 1) + *(input + 2)) / 3;
if (brightness > Threshold)
{
// What is the most efficient way possible to do this?
(*(output)) = (byte)(255 - *(input));
(*(output+1)) = (byte)(255 - *(input+1));
(*(output+2)) = (byte)(255 - *(input+2));
}
else
{
(*(output)) = *(input);
(*(output + 1)) = *(input + 1);
(*(output + 2)) = *(input + 2);
}
input += 4;
output += 4;
}
pinned1.Free();
pinned2.Free();
}
}
Note that I've incorporate the necessary assumptions at the top of the function. I'd suggest you always do this, but whether you prefer Debug.Assert or some other form of validation is up to you.

If you're happy to carry the 4th byte through, it would be quicker to copy Input to Output first with a block copy, then not to perform the else clause of the branch:
Buffer.BlockCopy(Input,0,Output,0,Input.Length);
for (int i = 0; i < Input.Length; i += 4) {
var brightness = (Input[i] + Input[i + 1] + Input[i + 2]) / 3;
if (brightness > Threshold) {
Output[i] = (byte)(255 - Input[i]);
Output[i + 1] = (byte)(255 - Input[i + 1]);
Output[i + 2] = (byte)(255 - Input[i + 2]);
}
}

In terms of the most performant way of setting a single value to multiple array indicies in c#, I think you're looking at it. There's no non-looping way to set the same value to multiple indicies. See How can I assign a value to multiple array indices at once without looping?
If it helps, there's no need for the else statement where you set the 3 indicies to 0. default(byte) is already zero, so every index in the Ouput[] array will initialize to 0.
As a side note, defining variables inside of loops vs outside of loops has no effect on the resulting IL. See Is it better to declare a variable inside or outside a loop?
EDIT: To add on to the comment above, you can use unsafe methods. See https://stackoverflow.com/a/5375552/3290789 and http://www.gutgames.com/post/Using-Unsafe-Code-for-Faster-Image-Manipulation.aspx

Resample loopback capture

I successfully captured sound from Wasapi using the following code:
IWaveIn waveIn = new WasapiLoopbackCapture();
waveIn.DataAvailable += OnDataReceivedFromWaveOut;
What I need to do now, is to resample the in-memory data to pcm with a sample rate of 8000 and 16 bits per sample mono.
I can't use ACMStream to resample the example, because the recorded audio is 32 bits per second.
I have tried this code to convert bytes from 32 to 16 bits, but all I get every time is just blank audio.
byte[] newArray16Bit = new byte[e.BytesRecorded / 2];
short two;
float value;
for (int i = 0, j = 0; i < e.BytesRecorded; i += 4, j += 2)
{
value = (BitConverter.ToSingle(e.Buffer, i));
two = (short)(value * short.MaxValue);
newArray16Bit[j] = (byte)(two & 0xFF);
newArray16Bit[j + 1] = (byte)((two >> 8) & 0xFF);
}
source = newArray16Bit;

I use this routine to resample on the fly from WASAPI IeeeFloat to the format I need in my app, which is 16kHz, 16 bit, 1 channel. My formats are fixed, so I'm hardcoding the conversions I need, but it can be adapted as needed.
private void ResampleWasapi(object sender, WaveInEventArgs e)
{
//the result of downsampling
var resampled = new byte[e.BytesRecorded / 12];
var indexResampled = 0;
//a variable to flag the mod 3-ness of the current sample
var arity = -1;
var runningSamples = new short[3];
for(var offset = 0; offset < e.BytesRecorded; offset += 8)
{
var float1 = BitConverter.ToSingle(e.Buffer, offset);
var float2 = BitConverter.ToSingle(e.Buffer, offset + 4);
//simple average to collapse 2 channels into 1
float mono = (float)((double)float1 + (double)float2) / 2;
//convert (-1, 1) range int to short
short sixteenbit = (short)(mono * 32767);
//the input is 48000Hz and the output is 16000Hz, so we need 1/3rd of the data points
//so save up 3 running samples and then mix and write to the file
arity = (arity + 1) % 3;
//record the value
runningSamples[arity] = sixteenbit;
//if we've hit the third one
if (arity == 2)
{
//simple average of the 3 and put in the 0th position
runningSamples[0] = (short)(((int)runningSamples[0] + (int)runningSamples[1] + (int)runningSamples[2]) / 3);
//copy that short (2 bytes) into the result array at the current location
Buffer.BlockCopy(runningSamples, 0, resampled, indexResampled, 2);
//for next pass
indexResampled += 2;
}
}
//and tell listeners that we've got data
RaiseDataEvent(resampled, resampled.Length);
}

c# screen sharing Efficiency

I'm trying to optimize my screen sharing app. I've already used a few ways to make it faster and stable, such as only sending the deltas between two frames, and using Gzip to compress the data.
This is my client code:
private void Form1_Load(object sender, EventArgs e)
{
Thread th = new Thread(startSend);
th.Start();
}
private void startSend()
{
Bitmap curr;
Bitmap diff;
encoderParams.Param[0] = qualityParam;
Bitmap pre = screenshot();
bmpBytes = imageToByteArray(pre);
SendVarData(handler, bmpBytes);
while (true)
{
curr= screenshot();
diff= Difference(pre, curr);
bmpBytes = imageToByteArray(diff);
SendVarData(handler, bmpBytes);
pre = curr;
}
}
Screenshot:
public Bitmap screenshot()
{
Bitmap screenshot = new Bitmap(SystemInformation.VirtualScreen.Width,
SystemInformation.VirtualScreen.Height,
PixelFormat.Format24bppRgb);
Graphics screenGraph = Graphics.FromImage(screenshot);
screenGraph.CopyFromScreen(0,
0,
0,
0,
SystemInformation.VirtualScreen.Size,
CopyPixelOperation.SourceCopy);
return screenshot;
}
The Difference method:
public Bitmap Difference(Bitmap bmp0, Bitmap bmp1)
{
Bitmap bmp2;
int Bpp = 3;
bmp2 = new Bitmap(bmp0.Width, bmp0.Height, bmp0.PixelFormat);
var bmpData0 = bmp0.LockBits(
new Rectangle(0, 0, bmp0.Width, bmp0.Height),
ImageLockMode.ReadOnly, bmp0.PixelFormat);
var bmpData1 = bmp1.LockBits(
new Rectangle(0, 0, bmp1.Width, bmp1.Height),
ImageLockMode.ReadOnly, bmp1.PixelFormat);
var bmpData2 = bmp2.LockBits(
new Rectangle(0, 0, bmp2.Width, bmp2.Height),
ImageLockMode.ReadWrite, bmp2.PixelFormat);
bmp0.UnlockBits(bmpData0);
bmp1.UnlockBits(bmpData1);
bmp2.UnlockBits(bmpData2);
int len = bmpData0.Height * bmpData0.Stride;
// MessageBox.Show(bmpData0.Stride.ToString());
bool changed=false;
byte[] data0 = new byte[len];
byte[] data1 = new byte[len];
byte[] data2 = new byte[len];
Marshal.Copy(bmpData0.Scan0, data0, 0, len);
Marshal.Copy(bmpData1.Scan0, data1, 0, len);
Marshal.Copy(bmpData2.Scan0, data2, 0, len);
for (int i = 0; i < len; i += Bpp)
{
changed = ((data0[i] != data1[i])
|| (data0[i + 1] != data1[i + 1])
|| (data0[i + 2] != data1[i + 2]));
// this.Invoke(new Action(() => this.Text = changed.ToString()));
data2[i] = changed ? data1[i] : (byte)2; // special markers
data2[i + 1] = changed ? data1[i + 1] : (byte)3; // special markers
data2[i + 2] = changed ? data1[i + 2] : (byte)7; // special markers
if (Bpp == 4) data2[i + 3] =
changed ? (byte)255 : (byte)42; // special markers
}
// this.Invoke(new Action(() => this.Text = changed.ToString()));
Marshal.Copy(data2, 0, bmpData2.Scan0, len);
return bmp2;
}
and the SendVarData function:
int total = 0;
byte[] datasize;
private int SendVarData(Socket s, byte[] data)
{
total = 0;
int size = data.Length;
int dataleft = size;
int sent;
datasize = BitConverter.GetBytes(size);
sent = s.Send(datasize);
sent = s.Send(data, total, dataleft, SocketFlags.None);
total += sent;
dataleft -= sent;
// MessageBox.Show("D");
return total;
}
This is the server - I'm just receiving a full image in the beginning, and then just deltas:
public void startListening()
{
prev = byteArrayToImage(ReceiveVarData(client.Client));
theImage.Image = prev;
while (true)
{
data = ReceiveVarData(client.Client);
curr = byteArrayToImage(data) ;
merge = Merge(prev, curr);
theImage.Image = merge;
count++;
prev = merge;
}
}
public static Bitmap Merge(Bitmap bmp0, Bitmap bmp1)
{
int Bpp = 3;
Bitmap bmp2 = new Bitmap(bmp0.Width, bmp0.Height, bmp0.PixelFormat);
var bmpData0 = bmp0.LockBits(
new System.Drawing.Rectangle(0, 0, bmp0.Width, bmp0.Height),
ImageLockMode.ReadOnly, bmp0.PixelFormat);
var bmpData1 = bmp1.LockBits(
new System.Drawing.Rectangle(0, 0, bmp1.Width, bmp1.Height),
ImageLockMode.ReadOnly, bmp1.PixelFormat);
var bmpData2 = bmp2.LockBits(
new System.Drawing.Rectangle(0, 0, bmp2.Width, bmp2.Height),
ImageLockMode.ReadWrite, bmp2.PixelFormat);
bmp0.UnlockBits(bmpData0);
bmp1.UnlockBits(bmpData1);
bmp2.UnlockBits(bmpData2);
int len = bmpData0.Height * bmpData0.Stride;
byte[] data0 = new byte[len];
byte[] data1 = new byte[len];
byte[] data2 = new byte[len];
Marshal.Copy(bmpData0.Scan0, data0, 0, len);
Marshal.Copy(bmpData1.Scan0, data1, 0, len);
Marshal.Copy(bmpData2.Scan0, data2, 0, len);
for (int i = 0; i < len; i += Bpp)
{
bool toberestored = (data1[i] != 2 && data1[i + 1] != 3 &&
data1[i + 2] != 7 && data1[i + 2] != 42);
if (toberestored)
{
data2[i] = data1[i];
data2[i + 1] = data1[i + 1];
data2[i + 2] = data1[i + 2];
if (Bpp == 4) data2[i + 3] = data1[i + 3];
}
else
{
data2[i] = data0[i];
data2[i + 1] = data0[i + 1];
data2[i + 2] = data0[i + 2];
if (Bpp == 4) data2[i + 3] = data0[i + 3];
}
}
Marshal.Copy(data2, 0, bmpData2.Scan0, len);
return bmp2;
}
I think it's coded fine, but I'm still unable to get more than 6~7fps (of 8kb-100kb) when running on 2 different computers with a fast and stable internet connection, and a maximum of 11fps when running both client and server on the same computer. I think it's because of the complexity of the delta and merging algorithms, but i dont know.
I would very appreciate if anyone could suggest how could optimize it further.

You may organize data2 in a different manner to send less data.
The below "compression" algorithm is very basic but will provide an improved compression compared to your implelmentation.
Search for differences, by identifying start and end of consecutive differences. When you find an interval where all pixels are different, store the length of identical data before that interval using 2 bytes, then store the number of consecutive differences, finally write 3 RGB bytes for each different pixel.
In case of 65535 different pixels, block the max interval length to 65535 and after storing the interval values. The next difference interval starting just after the stored interval, the identical count for next interval will be 0.
In case of 65535 identical pixels, just write $FFFF followed by $0000 indicating an empty sequence of different pixels.
Clarification: What I mean by "identifying start and end of consecutive differences" ?
In the above example, letters identify colors, i.e W(white), P(Pink), O(Orange):
The word "identical" refers to a comparison between data 0 and data1 (not comparing data1[i] with data1[i-1]).
Data0 = WWPWWOWOOOWWOOOPP
Data1 = WWPWPWP OOOWWPP OPP
You have 4 identical pixels (WWPW) followed by an interval (length 3) starting on the 4th character and ending on the 6th where all pixels are different. Then five identical pixels, followed by a new interval with 2 differences. At the end, a few common pixels.
The output for data2 will be (the text in parenthesis are not part of the buffer and explains the previous buffer values :
04 00 (4 identical pixels) 03 00 (3 different pixels coded in next 9 bytes)
Pr Pg Pb (3 bytes RGB code for P) Wr Wg Wb (RGB code for W) Pr Pg Pb (RGB code for P)
05 00 (5 identical pixels) 03 00 (2 different pixels coded in next 6 bytes)
Pr Pg Pb (RGB code for P) ...

The code to write differences would look like to the following.
I let you build the code adressing the opposite action, i.e. reading differences to update the previous image.
Data2Index = 0 ; // next index for additions in data2 ;
int idcount = 0 ;
int diffstart = -1 ;
int diffstart = -1 ;
for (int i = 0; i < len; i += Bpp)
{
changed = ((data0[i] != data1[i])
|| (data0[i + 1] != data1[i + 1])
|| (data0[i + 2] != data1[i + 2]));
if (!changed)
{
if (idcount==ushort.MaxValue)
{ // still identical, but there is a limitation on count
// write to data2 the identical count + differencecount equals to 0
AddIdCountDiffCountAndDifferences(idcount,0,0) ;
idcount = 0 ;
}
if (diffstart>0)
{ // after 0 or more identical values, a change was found
// write to data2 the identical count + difference count + different pixels
AddIdCountDiffCountAndDifferences(idcount,diffcount,diffstart) ;
idcount = 0 ;
diffcount= 0 ;
diffstart=-1 ;
}
else identicalcount++ ; // still identical, continue until difference found
}
else if (diffstart<0)
{ // a difference is found after a sequence of identical pixels, store the index of first difference
diffstart=i ; diffcount=1 ;
}
else
{ // different pixel follows another difference (and limitation not reached)
if (diffcount<ushort.MaxVakue) diffcount++ ;
}
else
{ // limitation reached, i.e. diffcount equals 65535
AddIdCountDiffCountAndDifferences(0,diffcount,diffstart) ;
diffstart+=diffcount ;
diffcount=0 ;
}
The procedure used to fill data2 here:
private int Data2Index = 0 ; // to be reset before
private void AddIdCountDiffCountAndDifferences(int idcount,int diffcount,int diffstart)
{
data2[Data2Index++]=(byte)(idcount && 0xFF) ; // low byte of the int
data2[Data2Index++]=(byte)(idcount >> 8 && 0xFF) ; // second byte of the int
data2[Data2Index++]=(byte)(diffcount && 0xFF) ; // low byte of the int
data2[Data2Index++]=(byte)(diffcount >> 8 && 0xFF) ; // second byte of the int
for (int i=0;i<diffcount;i++)
{
data2[Data2Index++]=data1[diffstart+Bpp*i ] ;
data2[Data2Index++]=data1[diffstart+Bpp*i+1] ;
data2[Data2Index++]=data1[diffstart+Bpp*i+2] ;
}
}

32-bit Grayscale Tiff with floating point pixel values to array using LibTIFF.NET C#

I just started using LibTIFF.NET in my c# application to read Tiff images as heightmaps obtained from ArcGIS servers. All I need is to populate an array with image's pixel values for terrain generation based on smooth gradients. The image is a LZW compressed 32-bit Grayscale Tiff with floating point pixel values representing elevaion in meters.
It's been some days now that I struggle to return right values but all I get is just "0" values assuming it's a total black or white image!
Here's the code so far: (Updated - Read Update 1)
using (Tiff inputImage = Tiff.Open(fileName, "r"))
{
int width = inputImage.GetField(TiffTag.IMAGEWIDTH)[0].ToInt();
int height = inputImage.GetField(TiffTag.IMAGELENGTH)[0].ToInt();
int bytesPerPixel = 4;
int count = (int)inputImage.RawTileSize(0); //Has to be: "width * height * bytesPerPixel" ?
int resolution = (int)Math.Sqrt(count);
byte[] inputImageData = new byte[count]; //Has to be: byte[] inputImageData = new byte[width * height * bytesPerPixel];
int offset = 0;
for (int i = 0; i < inputImage.NumberOfTiles(); i++)
{
offset += inputImage.ReadEncodedTile(i, inputImageData, offset, (int)inputImage.RawTileSize(i));
}
float[,] outputImageData = new float[resolution, resolution]; //Has to be: float[,] outputImageData = new float[width * height];
int length = inputImageData.Length;
Buffer.BlockCopy(inputImageData, 0, outputImageData, 0, length);
using (StreamWriter sr = new StreamWriter(fileName.Replace(".tif", ".txt"))) {
string row = "";
for(int i = 0; i < resolution; i++) { //Change "resolution" to "width" in order to have correct array size
for(int j = 0; j < resolution; j++) { //Change "resolution" to "height" in order to have correct array size
row += outputImageData[i, j] + " ";
}
sr.Write(row.Remove(row.Length - 1) + Environment.NewLine);
row = "";
}
}
}
Sample Files & Results: http://terraunity.com/SampleElevationTiff_Results.zip
Already searched everywhere on internet and couldn't find the solution for this specific issue. So I really appreciate the help which makes it useful for others too.
Update 1:
Changed the code based on Antti Leppänen's answer but got weird results which seems to be a bug or am I missing something? Please see uploaded zip file to see the results with new 32x32 tiff images here:
http://terraunity.com/SampleElevationTiff_Results.zip
Results:
LZW Compressed: RawStripSize = ArraySize = 3081 = 55x55 grid
Unompressed: RawStripSize = ArraySize = 65536 = 256x256 grid
Has to be: RawStripSize = ArraySize = 4096 = 32x32 grid
As you see the results, LibTIFF skips some rows and gives irrelevant orderings and it even gets worse if the image size is not power of 2!

Your example file seems to be tiled tiff and not stripped. Console says:
ElevationMap.tif: Can not read scanlines from a tiled image
I changed your code to read tiles. This way it seems to be reading data.
for (int i = 0; i < inputImage.NumberOfTiles(); i++)
{
offset += inputImage.ReadEncodedTile(i, inputImageData, offset, (int)inputImage.RawTileSize(i));
}

I know it could be late, but I had the same mistake recently and I found the solution, so it could be helpful. The mistake is in the parameter count of the function Tiff.ReadEncodedTile(tile, buffer, offset, count). It must be the decompressed bytes size, not the compressed bytes size. That's the reason why you have not all the information, because you are not saving the whole data in your buffer. See how-to-translate-tiff-readencodedtile-to-elevation-terrain-matrix-from-height.

A fast method to read a floating point tiff.
public static unsafe float[,] ReadTiff(Tiff image)
{
const int pixelStride = 4; // bytes per pixel
int imageWidth = image.GetField(TiffTag.IMAGEWIDTH)[0].ToInt();
int imageHeight = image.GetField(TiffTag.IMAGELENGTH)[0].ToInt();
float[,] result = new float[imageWidth, imageHeight];
int tileCount = image.NumberOfTiles();
int tileWidth = image.GetField(TiffTag.TILEWIDTH)[0].ToInt();
int tileHeight = image.GetField(TiffTag.TILELENGTH)[0].ToInt();
int tileStride = (imageWidth + tileWidth - 1) / tileWidth;
int bufferSize = tileWidth * tileHeight * pixelStride;
byte[] buffer = new byte[bufferSize];
fixed (byte* bufferPtr = buffer)
{
float* array = (float*)bufferPtr;
for (int t = 0; t < tileCount; t++)
{
image.ReadEncodedTile(t, buffer, 0, buffer.Length);
int x = tileWidth * (t % tileStride);
int y = tileHeight * (t / tileStride);
var copyWidth = Math.Min(tileWidth, imageWidth - x);
var copyHeight = Math.Min(tileHeight, imageHeight - y);
for (int j = 0; j < copyHeight; j++)
{
for (int i = 0; i < copyWidth; i++)
{
result[x + i, y + j] = array[j * tileWidth + i];
}
}
}
}
return result;
}

Moving through each pixel for 1bpp Pixel Format

I had a question related to the use of Lockbits method in C#..
I have a 1bpp Image and I'm trying to access all the pixels of the image but some are still left out.
public Bitmap Pixels(Bitmap original)
{
Rectangle rect = new Rectangle(0, 0, original.Width, original.Height);
BitmapData bimgData = original.LockBits(rect, ImageLockMode.ReadWrite, original.PixelFormat);
IntPtr ptr = bimgData.Scan0;
int bytes = bimgData.Stride * bimg.Height;
byte[] Values = new byte[bytes];
Marshal.Copy(ptr, Values, 0, bytes);
int Val;
int stride = bimgData.Stride;
for (int column = 0; column < bimgData.Height; column = column + 1)
for (int row = 0; row < bimgData.Width; row = row +1)
{
c = column;
r = row;
for (int t = 0; t < 8; t++)
{
Val = Values[((c) * stride) + ((r) / 8)] & 2 ^ t;
if (Val == 0)
Values[((c) * stride) + ((r) / 8)] = (byte)(Values[((c) * stride) + ((r) / 8)] + 2 ^ t);
}
}
Marshal.Copy(Values, 0, ptr, bytes);
original.UnlockBits(bimgData);
return original;
}
This code should turn all the pixels white

Val = Values[((c) * stride) + ((r) / 8)] & 2 ^ t;
2 ^ t doesn't do what you hope it does, that's Visual Basic syntax. In the C# language, ^ is the XOR operator. Use the << operator instead and use parentheses to take care of operator precedence. Fix:
Val = Values[((c) * stride) + ((r) / 8)] & (1 << t);
And fix it again when you set the bit.
Do note that turning the entire image to White doesn't require this kind of code at all. Just set all the bytes to 0xff.