ASAsense Binary Table (ABT) format

The ASAsense Binary Table (ABT) format is an alternative to the CSV format, but for binary data with fixed row lengths. The structure is as follows:

name	length (bytes)	type	allowed values	explanation
file_type	1	u8	1	The internal number of the file type and version
n_cols	4	u32	1-u32::max	The number of columns
col_widths	4 × n_cols	u32[n_cols]	1-u32::max per column	n_cols u32 integers describing the byte width of the columns
metadata_length	4	u32	0-u32::max	The length of the metadata field (can be 0 if it is to be skipped)
metadata	metadata_length	char[]	string of fixed length (so no '\0' character marks the end of the string)	JSON-encoded metadata
data	variable, until EOF	byte[]	any	The actual data

The JSON metadata should be an object with the following structure:

field	required	type	explanation
n_rows	false	number	The number of rows in the file, if known upfront (can be omitted for streaming data)
comment	false	string	A free string
columns	true	a list of column definition objects	A list of objects describing the columns in more detail (should have n_cols entries)
extra	false	JSON object	A JSON object that can be specified freely

A "column definition object" should have the following format:

field	required	type	explanation
name	false	string	An optional name of the column
comment	false	string	A free string
datatype	true	string	An identifier for the datatype of the column

The datatype identifiers can be anything, but the reader should support at least the following:

identifier	description
utf8	Interpret the data as a UTF-8 string (with fixed length; it does not need to be terminated with '\0')
int	Interpret the data as a signed integer
uint	Interpret the data as an unsigned integer
float	Interpret the data as floating-point data (only possible for 4- or 8-byte fields)
bool	Boolean value (0 = false, anything else = true)
numtype/div	"div" should be replaced by any floating-point number. The data is first interpreted as "numtype" (int, uint or float) and then divided by "div". This is mainly useful for storing decimal numbers in integer fields.
unix_timestamp(numtype/div)	Interprets the data as specified within the brackets (so any numtype, with optional divider), but takes into account that it is a Unix timestamp (seconds since the Unix epoch, 1 January 1970 at UTC)

Other, unknown datatype identifiers should be treated by the reader as opaque binary data (bytes).

Common extra metadata fields

In many ABT files used by ASAsense, the following extra metadata fields will be found:

Field name	Field type	Default value	Description
sorted_by_timestamp	bool	true	Indicates whether the data in the ABT file is sorted by timestamp (column 1)
frame_width_s	float	NaN	Indicates the frame width of one sample (for example the sampling period or spectrogram frame width)

CLI tool for ABT handling

To make the ABT format easier to use, a few precompiled binaries are available:

• Linux
  • Ubuntu 22.04 x86_64
  • Ubuntu 20.04 x86_64
• Windows 11 x64

The MD5 sums of the ZIP files are as follows:

adcd6a227861bb6b188ef8c5748faae8  abt-cli-ubuntu-20.04-x86_64.zip
6f4c63cbd7416ead026f170c97b3ab89  abt-cli-ubuntu-22.04-x86_64.zip
39db54827d2634262d9d666b4262b212  abt-cli-win11-x64.zip