I want to add some string in the middle of image metadata block. Under some specific marker. I have to do it on bytes level since .NET has no support for custom metadata fields.
The block is built like 1C 02 XX YY YY ZZ ZZ ZZ ... where XX is the ID of the field I need to append and YY YY is the size of it, ZZ = data.
I imagine it should be more or less possible to read all the image data up to this marker (1C 02 XX) then increase the size bytes (YY YY), add data at the end of ZZ and then add the rest of the original file? Is this correct?
How should I go on with it? It needs to work as fast as possible with 4-5 MB JPEG files.
In general there is no way to speed up this operation. You have to read at least portion that needs to be moved and write it again in updated file. Creating new file and copying content to it may be faster if you can parallelize read and write operations.
Note: In you particular case it may not be possible to just insert content in the middle of the file as most of file formats are not designed with such modifcations in mind. Often there are offsets to portions of the file that will be invalid when you shift part of the file. Specifying what file format you trying to work with may help other people to provide better approaches.
Solved the problem with this code:
List<byte> dataNew = new List<byte>();
byte[] data = File.ReadAllBytes(jpegFilePath);
int j = 0;
for (int i = 1; i < data.Length; i++)
{
if (data[i - 1] == (byte)0x1C) // 1C IPTC
{
if (data[i] == (byte)0x02) // 02 IPTC
{
if (data[i + 1] == (byte)fileByte) // IPTC field_number, i.e. 0x78 = IPTC_120
{
j = i;
break;
}
}
}
}
for (int i = 0; i < j + 2; i++) // add data from file before this field
dataNew.Add(data[i]);
int countOld = (data[j + 2] & 255) << 8 | (data[j + 3] & 255); // curr field length
int countNew = valueToAdd.Length; // new string length
int newfullSize = countOld + countNew; // sum
byte[] newSize = BitConverter.GetBytes((Int16)newfullSize); // Int16 on 2 bytes (to use 2 bytes as size)
Array.Reverse(newSize); // changes order 10 00 to 00 10
for (int i = 0; i < newSize.Length; i++) // add changed size
dataNew.Add(newSize[i]);
for (int i = j + 4; i < j + 4 + countOld; i++) // add old field value
dataNew.Add(data[i]);
byte[] newString = ASCIIEncoding.ASCII.GetBytes(valueToAdd);
for (int i = 0; i < newString.Length; i++) // append with new field value
dataNew.Add(newString[i]);
for (int i = j + 4 + newfullSize; i < data.Length; i++) // add rest of the file
dataNew.Add(data[i]);
byte[] finalArray = dataNew.ToArray();
File.WriteAllBytes(Path.Combine(Path.GetDirectoryName(jpegFilePath), "newfile.jpg"), finalArray);
Here is an easy and quite fast solution. It moves all bytes after given offset to their new position according to given extraBytes, so you can insert your data.
public void ExpandFile(FileStream stream, long offset, int extraBytes)
{
// http://stackoverflow.com/questions/3033771/file-io-with-streams-best-memory-buffer-size
const int SIZE = 4096;
var buffer = new byte[SIZE];
var length = stream.Length;
// Expand file
stream.SetLength(length + extraBytes);
var pos = length;
int to_read;
while (pos > offset)
{
to_read = pos - SIZE >= offset ? SIZE : (int)(pos - offset);
pos -= to_read;
stream.Position = pos;
stream.Read(buffer, 0, to_read);
stream.Position = pos + extraBytes;
stream.Write(buffer, 0, to_read);
}
Need to be checked, though...
Related
I successfully captured sound from Wasapi using the following code:
IWaveIn waveIn = new WasapiLoopbackCapture();
waveIn.DataAvailable += OnDataReceivedFromWaveOut;
What I need to do now, is to resample the in-memory data to pcm with a sample rate of 8000 and 16 bits per sample mono.
I can't use ACMStream to resample the example, because the recorded audio is 32 bits per second.
I have tried this code to convert bytes from 32 to 16 bits, but all I get every time is just blank audio.
byte[] newArray16Bit = new byte[e.BytesRecorded / 2];
short two;
float value;
for (int i = 0, j = 0; i < e.BytesRecorded; i += 4, j += 2)
{
value = (BitConverter.ToSingle(e.Buffer, i));
two = (short)(value * short.MaxValue);
newArray16Bit[j] = (byte)(two & 0xFF);
newArray16Bit[j + 1] = (byte)((two >> 8) & 0xFF);
}
source = newArray16Bit;
I use this routine to resample on the fly from WASAPI IeeeFloat to the format I need in my app, which is 16kHz, 16 bit, 1 channel. My formats are fixed, so I'm hardcoding the conversions I need, but it can be adapted as needed.
private void ResampleWasapi(object sender, WaveInEventArgs e)
{
//the result of downsampling
var resampled = new byte[e.BytesRecorded / 12];
var indexResampled = 0;
//a variable to flag the mod 3-ness of the current sample
var arity = -1;
var runningSamples = new short[3];
for(var offset = 0; offset < e.BytesRecorded; offset += 8)
{
var float1 = BitConverter.ToSingle(e.Buffer, offset);
var float2 = BitConverter.ToSingle(e.Buffer, offset + 4);
//simple average to collapse 2 channels into 1
float mono = (float)((double)float1 + (double)float2) / 2;
//convert (-1, 1) range int to short
short sixteenbit = (short)(mono * 32767);
//the input is 48000Hz and the output is 16000Hz, so we need 1/3rd of the data points
//so save up 3 running samples and then mix and write to the file
arity = (arity + 1) % 3;
//record the value
runningSamples[arity] = sixteenbit;
//if we've hit the third one
if (arity == 2)
{
//simple average of the 3 and put in the 0th position
runningSamples[0] = (short)(((int)runningSamples[0] + (int)runningSamples[1] + (int)runningSamples[2]) / 3);
//copy that short (2 bytes) into the result array at the current location
Buffer.BlockCopy(runningSamples, 0, resampled, indexResampled, 2);
//for next pass
indexResampled += 2;
}
}
//and tell listeners that we've got data
RaiseDataEvent(resampled, resampled.Length);
}
I just started using LibTIFF.NET in my c# application to read Tiff images as heightmaps obtained from ArcGIS servers. All I need is to populate an array with image's pixel values for terrain generation based on smooth gradients. The image is a LZW compressed 32-bit Grayscale Tiff with floating point pixel values representing elevaion in meters.
It's been some days now that I struggle to return right values but all I get is just "0" values assuming it's a total black or white image!
Here's the code so far: (Updated - Read Update 1)
using (Tiff inputImage = Tiff.Open(fileName, "r"))
{
int width = inputImage.GetField(TiffTag.IMAGEWIDTH)[0].ToInt();
int height = inputImage.GetField(TiffTag.IMAGELENGTH)[0].ToInt();
int bytesPerPixel = 4;
int count = (int)inputImage.RawTileSize(0); //Has to be: "width * height * bytesPerPixel" ?
int resolution = (int)Math.Sqrt(count);
byte[] inputImageData = new byte[count]; //Has to be: byte[] inputImageData = new byte[width * height * bytesPerPixel];
int offset = 0;
for (int i = 0; i < inputImage.NumberOfTiles(); i++)
{
offset += inputImage.ReadEncodedTile(i, inputImageData, offset, (int)inputImage.RawTileSize(i));
}
float[,] outputImageData = new float[resolution, resolution]; //Has to be: float[,] outputImageData = new float[width * height];
int length = inputImageData.Length;
Buffer.BlockCopy(inputImageData, 0, outputImageData, 0, length);
using (StreamWriter sr = new StreamWriter(fileName.Replace(".tif", ".txt"))) {
string row = "";
for(int i = 0; i < resolution; i++) { //Change "resolution" to "width" in order to have correct array size
for(int j = 0; j < resolution; j++) { //Change "resolution" to "height" in order to have correct array size
row += outputImageData[i, j] + " ";
}
sr.Write(row.Remove(row.Length - 1) + Environment.NewLine);
row = "";
}
}
}
Sample Files & Results: http://terraunity.com/SampleElevationTiff_Results.zip
Already searched everywhere on internet and couldn't find the solution for this specific issue. So I really appreciate the help which makes it useful for others too.
Update 1:
Changed the code based on Antti Leppänen's answer but got weird results which seems to be a bug or am I missing something? Please see uploaded zip file to see the results with new 32x32 tiff images here:
http://terraunity.com/SampleElevationTiff_Results.zip
Results:
LZW Compressed: RawStripSize = ArraySize = 3081 = 55x55 grid
Unompressed: RawStripSize = ArraySize = 65536 = 256x256 grid
Has to be: RawStripSize = ArraySize = 4096 = 32x32 grid
As you see the results, LibTIFF skips some rows and gives irrelevant orderings and it even gets worse if the image size is not power of 2!
Your example file seems to be tiled tiff and not stripped. Console says:
ElevationMap.tif: Can not read scanlines from a tiled image
I changed your code to read tiles. This way it seems to be reading data.
for (int i = 0; i < inputImage.NumberOfTiles(); i++)
{
offset += inputImage.ReadEncodedTile(i, inputImageData, offset, (int)inputImage.RawTileSize(i));
}
I know it could be late, but I had the same mistake recently and I found the solution, so it could be helpful. The mistake is in the parameter count of the function Tiff.ReadEncodedTile(tile, buffer, offset, count). It must be the decompressed bytes size, not the compressed bytes size. That's the reason why you have not all the information, because you are not saving the whole data in your buffer. See how-to-translate-tiff-readencodedtile-to-elevation-terrain-matrix-from-height.
A fast method to read a floating point tiff.
public static unsafe float[,] ReadTiff(Tiff image)
{
const int pixelStride = 4; // bytes per pixel
int imageWidth = image.GetField(TiffTag.IMAGEWIDTH)[0].ToInt();
int imageHeight = image.GetField(TiffTag.IMAGELENGTH)[0].ToInt();
float[,] result = new float[imageWidth, imageHeight];
int tileCount = image.NumberOfTiles();
int tileWidth = image.GetField(TiffTag.TILEWIDTH)[0].ToInt();
int tileHeight = image.GetField(TiffTag.TILELENGTH)[0].ToInt();
int tileStride = (imageWidth + tileWidth - 1) / tileWidth;
int bufferSize = tileWidth * tileHeight * pixelStride;
byte[] buffer = new byte[bufferSize];
fixed (byte* bufferPtr = buffer)
{
float* array = (float*)bufferPtr;
for (int t = 0; t < tileCount; t++)
{
image.ReadEncodedTile(t, buffer, 0, buffer.Length);
int x = tileWidth * (t % tileStride);
int y = tileHeight * (t / tileStride);
var copyWidth = Math.Min(tileWidth, imageWidth - x);
var copyHeight = Math.Min(tileHeight, imageHeight - y);
for (int j = 0; j < copyHeight; j++)
{
for (int i = 0; i < copyWidth; i++)
{
result[x + i, y + j] = array[j * tileWidth + i];
}
}
}
}
return result;
}
I have a circuit that sends me two different data from sensors. Data is coming as packets. First data is '$' to separate one packet to another. After '$' it sends 16 bytes microphone data and 1 byte pulse sensor data. I have an array to store incoming data and after plotting the data in each 20 ms, i start to write new bytes from zero index of array. I need to plot these data to different graphs using ZedGraph. However i could not separate those data correctly. Sometimes one or more data of audio are shown in other graph. Here is my code:
for (int i = 0; i < 4; i++)
{
if (data[i * 18] == Convert.ToByte('$'))
{
for (int x = ((i * 18) + 1); x < ((i * 18) + 17); x++)
{
listAuido.Add(time, data[x]);
}
for (int a = ((i * 18) + 17); a < ((i * 18) + 18); a++)
{
listPulse.Add(time, data[a]);
}
}
}
How can i solve this issue?
Circuit settings: BaudRate: 38400, Frequency: 200hz, CommunicationType: RS232.
Port Settings:ReadTimeOut=5 WrtieTimeOut=5;
While reading data i am using codes below. Read_Data1 refers data[] the code above. I have a counter and after plotting the data its value equals zero and i prevent my buffer index out of range exception
byte[] Read_Data1 = new byte[1000];
private void myPort_DataReceived(object sender, SerialDataReceivedEventArgs e)
{
if (!myPort.IsOpen)
return;
if (myPort.BytesToRead > 0)
{
byte[] buffer = new byte[myPort.BytesToRead];
myPort.Read(buffer, 0, buffer.Length);
DataConversion.bytetobyte(Read_Data1, buffer, buffer.Length, count);
count += buffer.Length;
DataRecord.SaveBytesToFile(buffer, save.FileName);
}
}
public static void bytetobyte(byte[] Storage, byte[] databyte, int datacount, int count)
{
int abc;
for (abc = 0; abc < datacount; abc++)
{
Storage[abc + count] = databyte[abc];
}
}
Without seeing the data stream its hard to understand exactly what's going on but when you multiply by 18 the offset will be different as i increases. in order for that to work you need to be sure that exactly 4 packets are actually in the buffer at this time or things might get weird. Having worked a far bit with rs232 connected measurement hardware I often find that their software is not all that consistent, so I'd be careful about assuming the data is there :)
Look into how and when you're reading data into the data buffer, are you sure it contains fresh data everytime you call your code?
The loop looks correct but its a little difficult to read, I'd rewrite it as
for (int i = 0; i < 4; i++)
{
if (data[i * 18] == Convert.ToByte('$'))
{
for (int x = 0; x < 16; x++)
{
listAuido.Add(time, data[(i * 18) +1 + x]);
}
for (int a = 0; a < 1; a++) // this doesn't really need to be a loop
{
listPulse.Add(time, data[ ((i * 18) + 17)+ a]);
}
}
I looked at the code you added but its not immediately clear how the first bit of code is called by the second. I still think that there is something with the buffer handling that is causing your issue, but you could possibly eliminate that but using the buffer built into the serial port and just one byte at the time:
while(true){ //you could do this on a separate thread or on a timer
if(port.ReadByte() == Convert.ToByte('$')){
for (int x = 0; x < 16; x++)
listAuido.Add(time, port.ReadByte());
listPulse.Add(time, port.ReadByte());
}
}
I'm kindda new to c# and i was looking for some ideas on 2 thing. I have looked far and wide for answers but haven't found an answer to my exact problem.
I have a byte array (called BA) within a for loop which keeps over-writting itself and there is no way for my to be able print it as a whole array. Is there a way to export it outside the for loop (maybe with a different name) so i can use it later on in the program? i just want something like this:
byte[] BA2 = {3 187,3 203,111 32, ...etc}; //(groups of 2bytes separated by commas).
From the original
string hexValues = "03 BB,03 CB,6F 20,57 6F,72 6C,64 21";
(and also to represent this information in bits (boolean) so {00000011 10111011,00000011 11001011, ...etc})
The second thing i must do is to shift these two bytes by 4 and apply and AND gate with FFF0 (which is the same as multiplying the first byte * 1, and the second by 0xF0). Then put this in a ushort[ ] (unsigned short array) which holds the transformed bytes in binary format and then from there convert it back to HEX.
I understand that this might be unclear (my code is kind of messy), and pretty complex. but i was hoping some of you c# guru's could lend me hand.
Here's my code so far, i have put in comments the bits that don't work so the code runs. but i desperatly need to fix them.
class Program
{
static void Main(string[] args)
{
string hexValues = "03 BB,03 CB,6F 20,57 6F,72 6C,64 21";
string[] hex2byte = hexValues.Split(',');
for (int j = 0; j < 6; j++)
{
Console.WriteLine("\n2 byte String is: "+ hex2byte[j]);
string[] hex1byte = hex2byte[j].Split(' ');
for (int k = 0; k < 2; k++)
{
Console.WriteLine("byte " + hex1byte[k]);
byte[] BA = StringToByteArray((hex1byte[k]));
//bool[] AA = BitConverter.ToBoolean(BA); // I'm essentially stuck here. I need somehting which actually works.
//for (int i2 = 0; i2 < 2; i2++); // This is my attemp to perform de shift and AND.
//{
// ushort[] X = new ushort[1];
// X[0] = (ushort)((ushort)(BA[0] << 4) + (ushort)((BA[1] & 0xF0) >> 4)); // They have to be in this order: ((1stByte & 0xFF) << 4) + ((2byte & 0xF0) >> 4); first to the left then the right.
//}
Console.WriteLine("Converted " + BA[0]);
}
}
//Console.WriteLine(BA[4]); // it says: Does not exist in current context. Can it only b accesed in the for loop?
Console.ReadKey();
} // Main method finishes.
// Define StringToByteArray method.
public static byte[] StringToByteArray(String hex)
{
int NumberChars = hex.Length;
byte[] bytes = new byte[NumberChars / 2];
for (int i = 0; i < NumberChars; i += 2)
{
bytes[i / 2] = Convert.ToByte(hex.Substring(i, 2), 16);
}
return bytes;
}
}
Is this what you are looking for?
string[] hexValues = new string[] { "03BB", "03CB", "6F20", "576F", "726C", "6421" };
ushort result = 0;
for (int i = 0; i < hexValues.Length; i++)
{
Console.WriteLine("2 byte String is {0}", hexValues[i]);
result = ushort.Parse(hexValues[i], NumberStyles.AllowHexSpecifier);
Console.WriteLine("converted: {0}", result.ToString());
Console.WriteLine("converted: {0}", result.ToString("x")); // "x" format in ToString -> very useful for creating hex strings.
}
For your shifting you can use the << and >> operators, and | and & for bitwise operations.
The Microsoft website has the code snippet:
using (FileStream fsSource = new FileStream(pathSource,
FileMode.Open, FileAccess.Read))
{
// Read the source file into a byte array.
byte[] bytes = new byte[fsSource.Length];
int numBytesToRead = (int)fsSource.Length;
int numBytesRead = 0;
while (numBytesToRead > 0)
{
// Read may return anything from 0 to numBytesToRead.
int n = fsSource.Read(bytes, numBytesRead, numBytesToRead);
// Break when the end of the file is reached.
if (n == 0)
break;
numBytesRead += n;
numBytesToRead -= n;
}
}
What concerns me is that fsSource.Length is a long, whereas numBytesRead is an int so at most only 2 * int.MaxValue can be read into bytes (the head and the tail of the stream). So my questions are:
Is there some reason that this is OK?
If not, how should you read a FileStream into a byte[].
In this situation I wouldn't even bother processing the FileStream manually; use File.ReadAllBytes instead:
byte[] bytes = File.ReadAllBytes(pathSource);
To answer your question:
The sample code is good for most of applications where we are not reaching extremes.
If you have really long stream like say a video, use BufferedStream. Sample code is available at MSDN site
Example using ReadAllBytes:
private byte[] m_cfgBuffer;
m_cfgBuffer = File.ReadAllBytes(m_FileName);
StringBuilder PartNbr = new StringBuilder();
StringBuilder Version = new StringBuilder();
int i, j;
byte b;
i = 356; // We know that the cfg file header ends at position 356 (1st hex(80))
b = m_cfgBuffer[i];
while (b != 0x80) // Scan for 2nd hex(80)
{
i++;
b = m_cfgBuffer[i];
}
// Now extract the part number - 6 bytes after hex(80)
m_PartNbrPos = i + 5;
for (j = m_PartNbrPos; j < m_PartNbrPos + 6; j++)
{
char cP = (char)m_cfgBuffer[j];
PartNbr.Append(cP);
}
m_PartNbr = PartNbr.ToString();
// Now, extract version number - 6 bytes after part number
m_VersionPos = (m_PartNbrPos + 6) + 6;
for (j = m_VersionPos; j < m_VersionPos + 2; j++)
{
char cP = (char)m_cfgBuffer[j];
Version.Append(cP);
}
m_Version = Version.ToString();