Develop Reference

How Can I Parse a Pcapng File in C#?

I'm new to Pcapng files. I've read the 40+ page whitepaper and I'm still scratching my head and sweating. I understand that the Pcapng file is:
Made up of a Section Header Block - This is the start of every Pcapng file.
Question 1: How large is this?
It appears that it's BlockType (4 Bytes) + BlockTotalLength (4 bytes) + Byte Order Magic (4 Bytes) + Mahor and Minor Version (4 bytes total, 2 bytes each) + Section Length (4 bytes) + Options (Variable) + Block Total length (again, 4 bytes).
If I'm building a parser, how would I know how many bytes I need to skip to arrive at my first data frame block?
Question 2: Where is the data stored? By data I mean the entire frame that contains Ethernet, IP, and TCP Data, as shown in the picture below (Figure 1).
The documentation states that:
A section includes data delimited by two section header blocks.
When doing a manual inspection (yes, I went byte by byte over a file to see how many bytes lie in between two frames :'( ), I noticed there were 35 bytes in between each message (each message shown on wireshark had 35 bytes in between). Are these bytes related to a pcapng block?
Once I understand how to get to the first tcp frame, and how many bytes I need to skip to get to the next, I can build my parser.
I'm willing to send Bitcoin/Monero to anyone who can help me understand how I can best parse these pcapng messages. Thanks!

If I'm building a parser, how would I know how many bytes I need to skip to arrive at my first data frame block?
That's not how you do it.
If you're building a parser, note that a parser must look at more than just the first data frame block.
First of all, it must look at the Section Header Block (SHB), to determine the byte order of the data in all the subsequent blocks by looking at the Byte-Order Magic field.
After that, you need to look at all subsequent blocks, looking for Interface Description Blocks and Enhanced Packet Blocks (EPBs), Simple Packet Blocks (SPBs), and possibly Packet Blocks (PBs) (those are obsolete, so no program should write them, but programs should be prepared to read them). Each EPB or PB has an interface ID that refers to an IDB, which must have appeared before the EPB or PB in question; an SPB implicitly refers to the first IDB, which, again, must have appeared before the SPB in question.
The format of the packet data in an EPB, SPB, or PB depends on the link-layer type specified by the IDB to which it refers, so you need to have read the IDB in question.
And, as the above indicates, there is no fixed number of bytes between the SHB and the first EPB, SPB, or PB, so there is no simple fixed number of bytes to skip to get to the first data frame block. For one thing, there's a variable number of bytes, which you can only determine by reading all the blocks before the first EPB, SPB, or PB. For another thing, you can't skip them, you have to read them to get enough information to interpret the packet data in them.
Where is the data stored? By data I mean the entire frame that contains Ethernet, IP, and TCP Data, as shown in the picture below (Figure 1).
It's stored in EPBs, SPBs, or PBs. See the descriptions of those three block types; frames are in the "Packet Data" fields of those blocks.
So I'm at my Interface Description Block and the 64 bit number that contains both a Timestamp Resolution of 9 (10^-9, Nanoseconds?) and 6 (10^-6, Microseconds).
As Christopher Maynard indicated, the 9 isn't a timestamp resolution, it's an option type. Pcapng blocks have both fixed information at the beginning and options; an option begins with an option type and option value length, followed by the option data. An IDB if_tsresol option has
2 bytes of option type, with the value 9;
2 bytes of option value length, with the value 1;
1 byte of option value, with the value as specified in the description of that option.
A value of 6 means the time stamp resolution is 1/10^6 of a second, which means 1 microsecond.

I think #tee-zad-awk found an answer that helped over at https://ask.wireshark.org/question/15159/how-can-i-display-as-much-pcapng-information-as-possible/, but for the benefit of anyone else looking for an answer to this question, I've linked it here and have provided my answer below, just in case the link is ever broken someday.
It seems that, after reading the 40 page whitepaper on Pcapng ...
The current PCAP Next Generation (pcapng) Capture File Format draft document is 52 pages, so perhaps you're not looking at the most recent version? Other versions do exist, such as those at https://datatracker.ietf.org/doc/html/draft-tuexen-opswg-pcapng-00, https://pcapng.github.io/pcapng/ or https://www.tcpdump.org/pcap/pcap.html and probably others, but they're all obsolete.
If you're looking for a pcapng parser to help you decipher the file, then look no further than Wireshark itself. If you've loaded a pcapng file into Wireshark, you can use "View -> Reload as File Format/Capture" (Ctrl+Shift+F) to cause Wireshark to load and display the raw file contents itself rather than to load and display the packets from the file. This should cause you to be able to see the various pcapng blocks and be able to drill down into them. For example:
Frame 1: 184 bytes on wire (1472 bits), 184 bytes captured (1472 bits)
MIME file
PCAPNG File Format
Block: Section Header Block 1
Block: Interface Description Block 0
Block: Enhanced Packet Block 1
You can also have a look at the Wireshark source code, such as the epan/dissectors/file-pcapng.c and wiretap/pcapng.c files.
By the way, if you're looking to support all extensions, the Wireshark [PcapNg wiki page] (https://wiki.wireshark.org/Development/PcapNg) has a link to Augmented PCAP Next Generation Dump File Format page that you might also want to take a look at. I don't know how many other extensions may have been implemented but not included in the main pcapng file format specification, but hopefully not many, as this could quickly become problematic with different projects possibly using the same block type for different blocks. That practice should be highly discouraged.

In order to find it out, it is helpful to read the specifications of the protocol of the network device and the package that has been sent. For example, we need to know the frame description of an Ethernet device and the package description of a TCP/IP package in order to understand the raw data. Having studied this, we record some traffic in Wireshark and select a block in the upper window of Wireshark. The middle window will tell you in clear text what Wireshark has received. On clicking on any of the lines in the middle window, Wireshark will mark the bytes of the raw data in the lower window that bear the information of the clicked line. Also, you can click on the raw data and then the clear text is marked. Moreover, the status line also informs you about it. This is very helpful for understanding the data.
I needed to read the TCP / IPv4 packages of Ethernet traffic. The block starts with the identification block type = 0x00000006 and the length of the block. The device was Ethernet so that I had the link type LINKTYPE_ETHERNET. The section length can be taken from byte 16-23. The other entries of the block header can be taken from here.
After the block header or after 28 bytes , the Ethernet frame came with the following entries (see here for a description):
mac address destination, 6 bytes
mac address source, 6 bytes
type: 0x0800 for IPv4, 0x0806 for ARP, 0x86DD for IPv6, 0x8100 for the presence of an IEEE 802.1Q tag.
For an IPv4 package or type = 0x0800, the following bytes are the IPv4 header (see here for a description):
IP version and header length, 1 byte
differentiated services field, 1 byte
total length, 2 bytes
identification, 2 bytes
flags, 1 byte
fragment offset, 1 byte
time to live, 1 byte
protocol with 0x06 for TCP, 1 byte
header checksum, 2 bytes
source IP address
destination IP address
options
The total length is very important: the byte that follows the last byte of the IPv4 + TCP package is located at total length bytes after the entry IP version and header length. However, this entry can be tricky. I head an entry with length 0 though the IP header length already had 20 bytes. In this case, Wireshark was helpful. It reported
[Total Length: 1547 bytes (reported as 0, presumed to be because of "TCP segmentation offload" (TSO))]
A detailed description of this phenomenon can be taken from here. In this case I could compute the payload length by the section length from above minus the length of the Ethernet frame (14 bytes) minus the length of the IP Header minus the length of the TCP header. However, padding problems could arise though I did not have these problems. A padding problem occurs when the package length is extended to a multiple of 4 bytes or something.
If the protocol of the IPv4 header is 0x06, the TCP package follows. The details of an TCP package can be taken from here. Of course, Wireshark also helps you interpreting the TCP package: just click on the lines in the middle window that belong to the TCP package or click on the raw data.
As outlined here, the interpretation of a pcapng file has many ifs and whens.

Anybody have experience reading ISOBUS (ISO 11783-10) binary timelog files?

I am trying to open and read a bunch of geo-referenced timelog files that are in binary format. They supposedly follow the ISO-11783 (ISOBUS) standard for agricultural machinery, but after reading 100s of pages of the standard I cannot figure out how to read the files either with a hex editor or programmatically with .NET c#. I know the timelog comes in file-pairs: an xml file and a binary file. The binary file, for example, is named TLG00004.bin and in notepad it looks like this (partial):
and when I open that file in Visual Studio 2015 (Community) as a binary file the hex looks like this:
which does not help me. I don't even know how to begin reading this as a byte stream in code (or anything else for that matter).
I know the file is supposed to look like this in human readable form:
(TimeStart, PositionNorth, PositionEast, PositionStatus, # DLV, DLV 0, PDV 0, DLV 1, PDV 1, DLV 2, PDV 2,...) it can have up to 255 DLV-PDV pairs which I believe are 32-bit integers. An example was shown as: (2005-05-02T16:32:00,51.00678,6.03489,1,2,0,10,1,15)
Little hints I have seen in the documentation indicate to me this must be utf-8 and perhaps base64 encoding with little endian and no Byte Order Mark. But I tried opening this in the free version of Hexinator and can't (human) read it using any of the dozens of encodings in that app, including utf-8, 16, 32...
I know this is not normal programming stuff but am throwing it out there to see if I'm lucky enough that someone has done this before and sees this. Any hints or resource-pointing would find me grateful, and I would be very thankful if someone can share any code that reads this kind of file.

Your data seems to follow the ISO 11783-10 standard for "Log data binary file structure" data exchange.
You will need to unpack your binary data into data types according to the specification. For example, the first 32 bits of the data are the milliseconds since midnight stored as a 32 bit unsigned integer. The next 16 bits are the days since 1980-01-01 stored as a 16 bit unsigned integer.
Unpacking binary data is programming language specific and some programming languages have useful libraries to assist in shifting through binary data.
As your question is about the general parsing of ISOBUS and I'm not proficient in your given language (C#), I can only give you an initial pointer.
BinaryReader looks to be the ideal way of unpacking a binary file by reading a number of bits from a stream and advancing the pointer through it:
using (BinaryReader reader = new BinaryReader(File.Open(fileName, FileMode.Open)))
{
milliSecondsSinceMidnight = reader.ReadUInt32();
daysSince1980 = reader.ReadUInt16();
}
If you need further help, you can now ask a specific question about byte parsing in C#.

how to encode data for iso 8583 to transfer socket c#

I don't understand exactly how to send data over c# socket.send( byte[]),
I mean they say I need to send 0800 (Network Management Request) for an echo test, how to convert.
Please I've been programming for a while but I don't understand the instructions.
Thanks

First of all you need to have an understanding of the ISO8583 message format. For echo test messages, in the 87 revision, your MTID should be 0800 and field 70, the Network Management Information code, should be set to 301, indicating echo test.
Building an ISO message is quite tricky. (shameless plug coming up) I have released OpenIso8583.Net, a message parser/builder for .Net which can be extended into the particular flavor of ISO you are using

You should first understand the spec that you're working to; I expect you have something more specific than the bare ISO8583 message spec, something that is specific about the fields required and the content. The important thing is the way you build and deblock the ISO8583 fields from to and from the message based on the bitmap that specifies which fields are present.
When I've built ISO8583 test clients in C# in the past I first put together a set of classes that could build and deblock a message bitmap. Once you have that you need some code to build and deblock your messages. These will set (or test) bits in the bitmap and then extract or insert the expected fields into a byte buffer.
Once you have this working the actual sending and receiving of the byte buffer messages is trivial.

Looking at the spec, it would be impossible to provide a full answer here - but to get you started, you basically need to create the various messages and send them down the pipe. As the Socket class requires an array of bytes rather than a string, you can use one of the Encoding classes to get at the raw bytes of a string. If I am reading the info correctly from Wikipedia:
byte[] echo = System.Text.Encoding.ASCII.GetBytes("0100");
socket.Send(echo);
Disclaimer: I have never had to implement ISO 8583 and the reference I looked at wasn't clear if the codes were in fact simple ASCII characters (though I am betting they are). Hopefully someone more familiar with the standard will clarify or confirm that assumption.

//***************additional encoders***************
UnicodeEncoding encoderUnicode = new UnicodeEncoding();
UTF32Encoding encoder32 = new UTF32Encoding();
UTF7Encoding encoder7 = new UTF7Encoding();
UTF8Encoding encoder8 = new UTF8Encoding();
//*************end of additionals*************
ASCIIEncoding encoder = new ASCIIEncoding();
about the parser, do some google on iso 8583, preferably of 2003

Differences in length in TagLib# (C#) and TagLib (C++)

I am currently in the process of moving my C# application over to Qt / C++. I'm running into problems with lengths from TagLib. I find it odd that TagLib# returns audio durations in milliseconds, while TagLib returns its (incorrect) durations in seconds. TagLib just returns zero for the length values, while TagLib# remains correct.
Here is my source in C# / TagLib#...
TagLib.File tagfile = TagLib.File.Create(path);
uint milliseconds = (uint)tagfile.Properties.Duration.TotalMilliseconds;
And here is what should be nearly equivalent in C++ / TagLib. I've even forced it to read accurately. No success.
TagLib::FileName fn(path);
TagLib::FileRef fr(fn, true, TagLib::AudioProperties::Accurate);
uint length = fr.audioProperties()->length();
It works as expected for a good majority of my media files. However, a select few audio files fail to return any audio properties (the rest of the tag information reads fine!). The exact same audio properties are returned with no issues on TagLib#.
Any ideas are appreciated. Thanks.
Does anyone have any more ideas before the bounty ends?

Hi there is a patch to taglib that calculate the length in milliseconds, this guy added a method (lengthMilliseconds()) that return the length in milliseconds, maybe that could be useful for you:
http://web.archiveorange.com/archive/v/sF3Pjr01lSQjsqjrAC7L

A lot has changed in TagLib#'s parsing of audio files since it was originally ported, so its hard to say where exactly the difference would occur. You may check your C++ program for debug messages.
My guess is that the difference is in how the two libraries react to invalid headers. It appears that if the first frame header it finds is invalid, TagLib won't calculate any audio property values. TagLib#, on the other hand, looks for the first valid header in the first 16KiB of the audio part of the file. If the first header it encounters is corrupt, it will scan for the next one. If I remember correctly, an incorrectly saved ID3v2 tag could result in 0xFF FF FF FF appearing in the beginning of the audio section of the file. This would trigger the type of failure described above.
The problem is at line 166 of taglib/mpeg/mpegproperties.cpp. This could be solved using the same approach as lines 171 to 191, but you would want to update the code to give up after a point in case it really isn't an MP3 file.

As of this writing, TagLib 1.11 BETA 2 natively supports getting the length of audio in milliseconds. You can do so with the following code:
TagLib::FileRef f(path);
int lengthInMillis = f.audioProperties()->lengthInMilliseconds();

read/write SLE4442 memory card with WinSCard API in c#

A bit of background information:
Inorder to read/write to SLE4442 memory cards, my app is currently using an Omnikey Cardman 3021 USB card reader, a Sumbsembly Smartcard API (external dll) which is capable of wrapping CT-API calls (directed to omnikey's dll) so that I can read/write the memory card in my c# app.
The only problem here is that Omnikey only provides a 32-bit dll of their CT-API. I asked if they are going to produce a 64-bit version, but they couldn't be bothered.
Current situation:
Inorder to make my application 64-bit capable, I must rewrite it using Windows WinSCard API. The problem here is that there are no specific examples on the web how to do it. Also getting hold of working APDU commands is nearly impossible, but I've managed to aquire two slightly different versions that sort of work.
I have googled this a hundred times over many months and with what I have managed to gobble together I can finally read the SLE4442 memory card. But for the life of me I can't get writing to work.
The code:
I'm not going to post the entire code into this first post (if need be I can do it later or provide a link to the source code).
But I'll outline the basic steps.
1) SCardEstablishContext
2) Get the reader name via SCardListReaders
3) SCardConnect
4) Read entire memory with SCardTransmit and APDU new byte[] { 0xFF, 0xB0, 0, 0, 0 };
5) Verify pin with SCardTransmit and APDU new byte[] { 0xFF, 0x20, 0, 0, 3, 0xFF, 0xFF, 0xFF }; (Note that this does return 0x90;0x00 as a response, which means the verification should have been succesful)
6) Try to write with ScardTransmit and APDU new byte[] { 0xFF, 0xD6, 0, 0, 50, 1 }; (try to write value 1 at memory position 50) - I have also tried using an APDU with the first parameter being 0x00 and/or the second byte being 0xD0. The response has never been 0x90;0x00 so I assume there is an error during writing, but I haven't been able to find any meaning to the error codes returned.
Possible causes:
Because I can read a memory card with the WinSCard API then it must be possible to also write to one (side note - the memory card(s) that I try to write to are in in working condition, I haven't locked them down by failing to verify the PIN 3 times).
1) Maybe the write APDU command is wrong. Could be that the instruction byte (second byte) is incorrect, or the memory location uses some sort of an extended coding scheme.
2) Maybe the verify command didn't actually verify. As in the command itself is fine, which is why 0x90 was returned, but I must call or setup something first.
3) Just a hunch, but I think that this is the real culprit. While googling I did find some vague references to having to call the SCardControl method with parameter IOCTL_SMARTCARD_SET_CARD_TYPE and setting the card type to SLE4442. But again no working examples anywhere and my trial-and-error testing resulted in failures. I got "One or more of the supplied parameters could not be properly interpreted." and some other error messages as well, can't remember what they all were. Assuming the code I copy-pasted from google code has the right descriptions for the error codes.
What I need:
What I need is someone to post or direct me to a site that has full+working code in c# for read/write SLE4442 using WinSCard API and it must work in both 32-bit and 64-bit enviroments.
The code doesn't have to be foolproof - eg. handling every possible error situation nicely. I should be able to do that myself. But if it is (including the APDU command result descriptions - eg. 0x90;0x00 is success, but 0x6B;0x4D is... etc...) then all the better.

APDU for writing to card, in your example, should be:
FF D6 00 50 01 01

Our Omnikey (3121) terminal writes data on SLE4442/SLE4442 cards perfectly, try this APDU:
FF D6 00 04 10 00 01 02 03 04 05 06 07 08 09 0A 0B 0C 0D 0E AA
see also:
http://acs.com.hk/drivers/eng/API_ACR122U.pdf, chapter 5.4

FF,D2,00,01,03,01,02,03 (new pin = 1,2,3) does not work (also after verifying existing PIN:
6D,00,FF,20,00,00 03,FF,FF,FF) - It returns error code 6D00.
For us (yet) not a huge problem, as we code our cards now with the Xiring XiMax terminal (programmable terminal with it's own SDK, the terminal can store data in flash memory).
However, we need to find a solution soon. I am interested in your (future) findings.
Do you have the Omnikey SDK? We can send you working examples in C++ that can change PSC (pin) and data on SLE4442/5542 cards.
This week (so not tomorrow, as previously stated, we have many things going on and are constantly ) we will test the (so far) working code with the Omnikey 1021). Hope to help you out.
We always have 50+ Omnikey 3121 readers on stock and we can offer you these (1+) for a far batter price. Let me know if you are interested.
I am also interested what where you are located and what applications you are developing.