snapraid/snapraid_log.txt

                              ===================
                              SnapRAID Log Format
                              ===================


1 DESCRIPTION
=============

This document describes the structured log tags output by SnapRAID,
which are designed to be used by tools monitoring the SnapRAID
processing state. The tags provide information in an easy-to-parse
format, ideal for automation and integration.

To capture these structured log tags, you must use the `--log` option
with SnapRAID.

You can direct the output to a specific file, or force the output to
the standard output or standard error using the special arguments
`>&1` or `>&2`. Note that these arguments often need to be quoted on
the command line to prevent the shell from interpreting them.

Example:

    snapraid status --log ">&2"

The output tags strictly adhere to the following format:

NAME:VALUE:VALUE:VALUE...<EOL>

    NAME
        This is the primary tag identifier, always a lowercase string
        without spaces (e.g., `summary`, `block`, `info_time`).
    VALUE
        Following the `NAME`, these can be additional sub-identifiers,
        descriptive names (like a disk name), or numeric/string data
        values. They are separated by a colon `:`.
    <EOL>
        Each tag is terminated by an end-of-line character.

This document provides a summary of the tags generated by SnapRAID.
For the most detailed and up-to-date information regarding the meaning
and context of the values, it is recommended to check the source
code's `log_tag()` statements.


2 STATUS
========

This section describe the tags output with the `status` command.

All tags start with a prefix indicating the context, typically `summary:`
for overall statistics or a disk/block context.

---- 2.0.1 General Summary Tags ----
These tags provide general information about the array configuration and overall state.

    summary:block_size:<uint>
        The size of a block in bytes.
    summary:parity_block_count:<uint>
        The total number of blocks allocated for all parity levels.
    summary:parity_block_total:<name>:<uint>
        Total blocks available for a specific parity level.
    summary:parity_block_free:<name>:<uint>
        Free blocks available for a specific parity level.
    summary:parity_block_free_min:<uint>
        The minimum number of free blocks across all parity levels.
    summary:file_count:<uint>
        Total number of files in the array.
    summary:file_block_count:<uint64>
        Total number of data blocks used by files.
    summary:fragmented_file_count:<uint>
        Total number of fragmented files.
    summary:excess_fragment_count:<uint>
        Total number of excess fragments (one less than the total fragments).
    summary:zerosubsecond_file_count:<uint>
        Total number of files with a zero sub-second timestamp.
    summary:file_size:<uint64>
        Total size of all files in bytes.
    summary:parity_size:<uint64>
        Total size of allocated parity blocks in bytes.
    summary:parity_size_max:<uint64>
        Maximum possible parity size (allocated + min free) in bytes.
    summary:hash:<name>
        The current hash algorithm in use.
    summary:prev_hash:<name>
        The previous hash algorithm used.
    summary:best_hash:<name>
        The best hash algorithm for optimal performance.
    summary:total_wasted:<uint64>
        Total wasted space (space on data disks exceeding parity capacity) in bytes.
    summary:total_used:<uint64>
        Total used space by data files in bytes.
    summary:total_free:<uint64>
        Total free space (usable by data) in bytes.
    summary:total_use_percent:<uint>
        Overall array use percentage.
    summary:has_unsynced:<uint>
        Count of blocks that are unsynced (need a sync).
    summary:has_unscrubbed:<uint>
        Count of blocks that are unscrubbed (just synced).
    summary:has_rehash:<uint>
        Count of blocks needing a rehash.
    summary:has_bad:<count>:<first>:<last>
        Count of bad blocks, along with the first and last bad block index.
    summary:scrub_oldest_days:<uint>
        Days ago the oldest block was last scrubbed.
    summary:scrub_median_days:<uint>
        Days ago the median block was last scrubbed.
    summary:scrub_newest_days:<uint>
        Days ago the newest block was last scrubbed.

---- 2.0.2 Per-Disk Tags ----
These tags provide statistics for individual data disks. `<name>`
is the disk name.

    summary:disk_file_count:<name>:<uint>
        Number of files on the disk.
    summary:disk_block_count:<name>:<uint>
        Number of blocks used by files on the disk.
    summary:disk_fragmented_file_count:<name>:<uint>
        Number of fragmented files on the disk.
    summary:disk_excess_fragment_count:<name>:<uint>
        Number of excess fragments on the disk.
    summary:disk_zerosubsecond_file_count:<name>:<uint>
        Number of files with zero sub-second timestamp on the disk.
    summary:disk_file_size:<name>:<uint64>
        Total size of files on the disk in bytes.
    summary:disk_block_allocated:<name>:<uint>
        Highest block index used on the disk + 1.
    summary:disk_block_total:<name>:<uint>
        Total blocks on the disk (physical size).
    summary:disk_block_free:<name>:<uint>
        Free blocks on the disk.
    summary:disk_block_max_by_space:<name>:<uint>
        Total blocks by disk space (used + free blocks).
    summary:disk_block_max_by_parity:<name>:<uint>
        Total blocks limited by parity space (parity size + min parity free blocks).
    summary:disk_block_max:<name>:<uint>
        Maximum usable blocks (minimum of `..._by_space` and `..._by_parity`).
    summary:disk_space_wasted:<name>:<int64>
        Wasted space on the disk in bytes (positive if disk is larger than usable parity space).
    summary:disk_used:<name>:<uint64>
        Used space on the disk in bytes.
    summary:disk_free:<name>:<uint64>
        Free usable space on the disk in bytes.
    summary:disk_use_percent:<name>:<uint>
        Disk use percentage relative to `disk_block_max`.

---- 2.0.3 File Timestamp Tags ----
These tags list files with zero sub-second timestamps.

    zerosubsecond:<disk_name>:<file_path>
        Lists files that have a zero sub-second timestamp.
        Up to 50 are logged per disk.

---- 2.0.4 Block Information Tags ----
These tags are generated when the GUI option is enabled (`state->opt.gui`).

    block_count:<uint>
        The total number of blocks in the array.
    block:<index>:<time>:<used>:<unsynced>:<bad>:<rehash>
        Status of a specific block.

    <index> - Block index.
    <time> - Last scrub time (Unix timestamp).
    <used> - "used" if the block contains data.
    <unsynced> - "unsynced" if the block needs a sync.
    <bad> - "bad" if the block has errors.
    <rehash> - "rehash" if the block needs a rehash.

    block_noinfo:<index>:<used>:<unsynced>
        Status of an unused block (no info recorded).

---- 2.0.5 Scrub History Tags ----
These tags provide details on the scrub/sync time distribution.

    info_count:<uint>
        Total number of info recorded about past scrub and sync
        operations.

    info_time:<time>:<count>:<status>
        All the info records.

    <time> - Unix timestamp of the scrub/sync.
    <count> - Number of blocks processed at this timestamp.
    <status> - "scrubbed" or "new" (for synced blocks).

    scrub_graph_range:<max_columns>:<max_height>
        Dimensions of the scrub history graph obtained from the
        `info_time` records. This provides the boundary for the
        precomputed graph data available in the `scrub_graph_bar` tags.

    <max_columns> - The number of columns in the graph (constant
        GRAPH_COLUMN).
    <max_height> - The maximum height (peak count) of blocks found
        in any column (the highest sum of scrubbed and new blocks).

    scrub_graph_bar:<index>:<days_ago>:<scrubbed>:<new>
        Data for a column of the scrub history graph.

    <index> - Column index. From 0 to max_columns - 1.
    <days_ago> - Days ago for this column's time range.
    <scrubbed> - Count of scrubbed blocks in that day.
    <new> - Count of newly synced (unscrubbed) blocks in that day.