Develop Reference

JFIF and EXIF data structures

I'm creating a program which reads and prints out metadata from an image, but I'm struggling to get my head around the JFIF and EXIF marker structures.
According to the wikipedia page for JFIF, the JFIF marker should be as follows:
FF E0 s1 s2 4A 46 49 46 00 ii ii jj XX XX YY YY xx yy
Where:
FF E0 is the start of the JFIF marker
s1 and s2 combined give the size of the segment (excluding the APP0 marker)
4A 46 49 46 00 is the identifier (literally JFIF in ascii)
i is the version of JFIF (2 bytes)
j is the density uni for DPI measurement
X is the Horizontal DPI
Y is the Vertical DPI
x is the Horizontal THUMBNAIL DPI
y is the Vertical THUMBNAIL DPI
However, when running an image through my program, I get this:
ff e0 20 10 4a 46 49 46 20 1 1 20 20 48 20 48 20 20 ff e1
The marker start is there, but the segment size looks well off (0x2010??) seeing as the next Marker for EXIF data starts just 15 bytes later! (FF E1)
I think that hex values of 0x00 aren't being printed (hence why my image prints the JFIF identifier without the zeros) which may be adding to the confusion, but even then, how is the JFIF version 20 20?
If anyone on here has any experience looking at image metadata, I'd really appreciate your help! There's not a lot of resources that I can find that break down JFIF/EXIF data very clearly.
If you need me to post any code in here then I can, though apart from not printing 0x00 values to the console, it seems to being working as expected, so I think my main issue is actually understanding the meta data
Here is the code for taking the byte stream and then converting it into hex:
fileLocation = Console.ReadLine();
var fileDataAsBytes = File.ReadAllBytes(fileLocation);
var headers = fileDataAsBytes
.Select((b, i) => (b, i))
.Where(tuple => tuple.b == 0xFF
&& fileDataAsBytes[tuple.i + 1] == 0xE1)
.Select(tuple => $"{tuple.i}: {tuple.b:x}
{fileDataAsBytes[tuple.i + 1]:x}");
Console.WriteLine(String.Join(",", headers));
DealWithMarkers(fileDataAsBytes);
DisplayAllConversions(fileDataAsBytes);
public static void DisplayAllConversions(byte[] fileDataAsBytes)
{
DisplayBytes(fileDataAsBytes);
DisplayHex(fileDataAsBytes);
DisplayString(fileDataAsBytes);
}
public static void DisplayHex(byte[] fileDataAsBytes)
{
Console.WriteLine($"\n\n\n\t*\t*\t*\t(As Hex)\t*\t*\t*\n");
for (int i = 0; (i < 1000) && (i < fileDataAsBytes.Length); i++)
{
Console.Write($"{fileDataAsBytes[i]:x} ");
}
}
I tried this with a different image and it actually prints out the 0 value bytes correctly, so there must be something weird with the image file I was analyzing!
Image file with "incorrect" 0 bytes
Image file with "correct" 0 bytes

FlatBuffers: Encoding in C++ versus C#, decoding in C# end-to-end example

Imagine a schema:
namespace MyEvents;
table EventAddress
{
id:uint;
timestamp:ulong;
adress:string;
}
table EventSignalStrength
{
id:uint;
timestamp:ulong;
strength:float;
}
table EventStatus
{
status:string;
}
union Events {EventAddress, EventSignalStrength, EventStatus}
table EventHolder
{
theEvent:Events;
}
root_type EventHolder;
For status message "EXIT", in C++ I encode and send over the wire like:
std::string message("EXIT");
flatbuffers::FlatBufferBuilder builder;
auto messageString= builder.CreateString(message); // Message to send.
auto statusEvent= MyEvents::CreateEventStatus(builder, messageString);
auto eventHolder= MyEvents::CreateEventHolder(builder, MyEvents::Events_EventStatus, statusEvent.Union());
builder.Finish(eventHolder);
// Code to decode to check my work omitted, but the data decode properly in my real-world application.
ret= sendto(m_udpSocket, reinterpret_cast<const char*>(builder.GetBufferPointer()), static_cast<int>(builder.GetSize()), 0, reinterpret_cast<SOCKADDR *>(&m_destination), sizeof(m_destination));
For the same message, "EXIT", in C# I encode and send over the wire like:
string message= "EXIT";
FlatBufferBuilder builder = new FlatBufferBuilder(1);
StringOffset messageOffset = builder.CreateString(message);
EventStatus.StartEventStatus(builder);
EventStatus.AddStatus(builder, messageOffset);
Offset<EventStatus> eventStatusOffset = EventStatus.EndEventStatus(builder);
EventHolder.StartEventHolder(builder);
EventHolder.AddTheEventType(builder, Events.EventStatus);
EventHolder.AddTheEvent(builder, eventStatusOffset.Value);
Offset<EventHolder> eventHolderOffset = EventHolder.EndEventHolder(builder);
EventHolder.FinishEventHolderBuffer(builder, eventHolderOffset);
// Test the encoding by decoding:
EventHolder flatBuffer = EventHolder.GetRootAsEventHolder(builder.DataBuffer);
Events flatBufferType = flatBuffer.TheEventType; // Type looks good.
EventStatus decodedEvent= new EventStatus();
flatBuffer.GetDataObject<EventStatus>(decodedEvent); // decodedEvent.Status looks good.
// This code seems to send the correct data:
Byte[] sendSized = builder.SizedByteArray();
udpClient.Send(sendSized, sendSized.Length);
// This code does not seem to send the correct data:
//ByteBuffer sendByteBuffer = builder.DataBuffer;
//udpClient.Send(sendByteBuffer.Data, sendByteBuffer.Data.Length);
In my client application, written in C#, I decode as:
Byte[] receiveBytes = udpClient.Receive(ref m_remoteEndpoint);
ByteBuffer flatBufferBytes= new ByteBuffer(receiveBytes);
EventHolder flatBuffer = EventHolder.GetRootAsEventHolder(flatBufferBytes);
Events flatBufferType= flatBuffer.DataObjectType;
EventAddress eventAddress = null;
EventSignalStrength eventSignalStrength = null;
EventStatus eventStatus = null;
switch (flatBufferType)
{
case Events.EventAddress:
{
eventAddress = new EventAddress();
flatBuffer.GetDataObject<EventAddress>(eventAddress);
ProcessEventAddress(eventAddress);
break;
}
case Events.EventSignalStrength:
{
eventSignalStrength = new EventSignalStrength();
flatBuffer.GetDataObject<EventSignalStrength>(eventSignalStrength);
ProcessEventSignalStrength(eventSignalStrength);
break;
}
case Events.EventStatus:
{
eventStatus= new EventStatus();
flatBuffer.GetDataObject<EventStatus>(eventStatus);
Console.WriteLine("\nStatus Message: {0}", eventStatus.status);
break;
}
}
When I receive EventStatus messages from the C++ application, they decode properly.
When I receive EventStatus messages from the C# sending application, they decode properly.
When I dump the buffers sent from the applications, they are (in decimal):
C++ - 12 0 0 0 8 0 14 0 7 0 8 0 8 0 0 0 0 0 0 4 12 0 0 0 0 0 6 0 8 0 4 0 6 0 0 0 4 0 0 0 4 0 0 0 69 88 73 84 0 0 0 0
C# - 12 0 0 0 8 0 10 0 9 0 4 0 8 0 0 0 12 0 0 0 0 4 6 0 8 0 4 0 6 0 0 0 4 0 0 0 4 0 0 0 69 88 73 84 0 0 0 0
Originally, the messages from the C# sender were not decoding properly - now they are. I had made a change to the sender, so maybe had not rebuilt.
I am a little mystified that the received C++ buffer and the C# buffer are different, yet they decode properly to the same result.
My real-world schema is much more complex - am I following the proper procedure for decoding on the C# side?
Am I following the correct procedure for reducing the flatbuffer to Byte[] for sending over the wire in C#? It looks like I am, but it did not seem to work for awhile....
Any input appreciated.

The ByteBuffer contains the buffer, but not necessarily at offset 0, so yes, turning it into a byte array (or sending the bytebuffer contents from its starting offset) are the only correct ways of sending it.
The encoding may differ between languages, as implementations may serialize things in different orders. Here, the C++ implementation decides to write the union type field before the offset, which happens to be inefficient for alignment, so it is a bit bigger. C# does the opposite.

Matlab fwrite in C#

I have a piece of code that i am trying to implement in C#. The code writes a file using the frwite command using Matlab. I have tried looking at the documentation and doing some examples to understand how does frwite works.
I tried the following but no success.
Here is the code:
line_vectors = [5;10;15;20;25]
sampPeriod=100000;
[filename,permission,machineformat] = fopen(outputfile);
fwrite(outputfile,sampPeriod,'int32');
fwrite(outputfile,line_vectors(:),'float32');
Output using fread():
160
134
1
0
0
0
160
64
0
0
32
65
0
0
112
65
0
0
160
65
0
0
200
65
I tried to implement a similar code in C#:
using (BinaryWriter writer = new BinaryWriter(file))
{
writer.Write(100000);
writer.Write(5);
writer.Write(10);
writer.Write(15);
writer.Write(20);
}
Output using fread() in Matlab:
160
134
1
0
5
0
0
0
10
0
0
0
15
0
0
0
20
0
0
0
If anybody could help me in mapping the fwrite functionality in C#.

If you want all numbers after the 1st to be written as float32, you can indicate the value type for the other values, like this:
using (BinaryWriter writer = new BinaryWriter(file))
{
writer.Write(100000);
writer.Write(5.0f);
writer.Write(10.0f);
writer.Write(15.0f);
writer.Write(20.0f);
}
BinaryWriter.Write is an overloaded function, with many possible input types. Depending on the type passed as the input variable, the specific version of the function writes the bytes that represent the value as that type. Since your initial code provided no other information, the default is to assume int, and a 4 byte integer representation was used.

C# Translating a few smaller arrays to one big

Title may not explain fully what I want to do, so I made an image.
You can see there are 4 1D arrays(red numbers, black colored numbers are indexes), each of this array goes from 0 to 63. I want to somehow translate them, that for example, index 16 will point to first index of second array.
What I was thinking of, is a function where I give List of arrays and index that I want to get as input, and its returns me the index of array and exact index in this array as output.
I would like to have some hints or suggestions on how to proceed here to achieve this functionality.

Ok, your image shows an interleaved data of 16 elements, so you want to have (showing an example of only two matrices because of space :D)
Global index
0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32
-----------------------------------------------------------------------------------------------------
Array0 indexes - Array1 indexes
0 1 2 3 4 5 6 7 8 9 A B C D E F - 0 1 2 3 4 5 6 7 8 9 A B C D E F
-------------------------------------------------------------------------------------------------
Global index
33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65
-----------------------------------------------------------------------------------------------------
Array0 indexes - Array1 indexes
10 11 12 13 14 15 16 17 18 19 1A 1B 1C 1D 1E 1F - 10 11 12 13 14 15 16 17 18 19 1A 1B 1C 1D 1E 1F
To get it you can do something like this:
public class CompositeIndex
{
public int ArrayNumber { get; set; }
public int ElementNumber { get; set; }
}
public static CompositeIndex GetIndex(int GlobalIndex, int ArrayCount, int ElementsToInterleave)
{
CompositeIndex index = new CompositeIndex();
int fullArrays = GlobalIndex / ElementsToInterleave; //In your example: 16 / 16 = 1;
index.ArrayNumber = fullArrays % ArrayCount; //In your example: 1 mod 4 = 1;
index.ElementNumber = GlobalIndex - (fullArrays * ElementsToInterleave); //In your example: 16 - (1 * 16) = 0;
return index;
}
Then, if you have 4 matrices and want to get the "global" index 16 you do:
var index = GetIndex(16, 4, 16);
This function allows you to use an indeterminated number of arrays and interleaved elements.
BTW, another time ask better your question, a lot more people will help you if they don't have to solve a puzzles to understand what you want...

Spliting into lines then into variables

i have this text file below:
001 Bulbasaur 45 49 49 65 65 45 Grass Poison
002 Ivysaur 60 62 63 80 80 60 Grass Poison
003 Venusaur 80 82 83 100 100 80 Grass Poison
004 Charmander 39 52 43 60 50 65 Fire
005 Charmeleon 58 64 58 80 65 80 Fire
I have written this piece of code to split it into lines then into variables but it refuses to work, my apology's if i am asking this in the wrong place. (unity C# question).
var lines = textFile.text.Split("\n"[0]);
allMonsters = new Monsters[lines.Length];
List<string> lineSplit = new List<string>();
for (int i = 0; i < lines.Length; i++) {
Debug.Log(lines[i]);
lineSplit.Clear();
lineSplit = lines[i].Split(' ').ToList ();
int ID = int.Parse(lineSplit[0]);
string Name = lineSplit[1].ToString();
float HP = float.Parse(lineSplit[2]);
float ATK = float.Parse(lineSplit[3]);
float DEF = float.Parse(lineSplit[4]);
float SPATK = float.Parse(lineSplit[5]);
float SpDEF = float.Parse(lineSplit[6]);
float speed = float.Parse(lineSplit[7]);
FirstType Ft = (FirstType)System.Enum.Parse(typeof(FirstType),lineSplit[8]);
SecondType ST = (SecondType)System.Enum.Parse(typeof(SecondType),lineSplit[9]); }
The code works for the first line but then on the second run of this code i get null reference to an object error, please help me.
Note, variables are assigned to so they aren't overwritten after code.
EDIT: LineSplit variable is over 1200 elements long, so i do not think unity is clearing the array properly could this be the issue?

I'm not sure about the second iteration, but on the 4th iteration line 004 Charmander 39 52 43 60 50 65 Fire have only 9 parameters (by split of space) and you using lineSplit[9], so there you will get NullPointerException.

When working with text files, linefeed often comes along with carriage return.
Try
var lines = textFile.text.Split(new string[]{"\r\n"}, System.StringSplitOptions.None);
in the first line.