mirror of
https://github.com/trapexit/mergerfs.git
synced 2026-11-19 22:46:34 -05:00
docs/performance: explain mergerfs IO performance vs RAID
Describe why mergerfs does not aggregate branch performance like RAID/ZFS (whose aggregate throughput is also capped by shared interconnects) and instead provides independent per-branch performance: best case A+B+C across concurrent streams, worst case single-stream limited to one branch's speed.
This commit is contained in:
parent
8ea47df13e
commit
515b0549ef
1 changed files with 59 additions and 0 deletions
|
|
@ -48,6 +48,65 @@ before changing them to understand how functionality will change.
|
||||||
* use LVM and LVM cache to place a SSD in front of your HDDs
|
* use LVM and LVM cache to place a SSD in front of your HDDs
|
||||||
|
|
||||||
|
|
||||||
|
## Understanding mergerfs IO Performance
|
||||||
|
|
||||||
|
A common source of confusion when evaluating **mergerfs**' performance
|
||||||
|
is comparing it to traditional RAID or filesystem level pooling
|
||||||
|
technologies such as mdadm, ZFS, btrfs, or striped LVM. Those
|
||||||
|
solutions aggregate devices into a single virtual device and spread IO
|
||||||
|
across all members. Under favorable conditions that means the
|
||||||
|
performance of the pool is the sum of the performance of its members:
|
||||||
|
striped reads and writes hit multiple drives at once and the aggregate
|
||||||
|
throughput scales with the number of devices, up to the limits of the
|
||||||
|
interconnects (PCIe, SATA, network, etc.) which every member must
|
||||||
|
share.
|
||||||
|
|
||||||
|
**mergerfs** does not aggregate or stripe data. Every file lives
|
||||||
|
wholly on one branch and **mergerfs** simply routes access to it. As a
|
||||||
|
result **mergerfs** does not aggregate the performance of its branches
|
||||||
|
in the way RAID does. What it provides instead is **independent
|
||||||
|
performance across branches**.
|
||||||
|
|
||||||
|
* **Best case:** workloads which access different branches
|
||||||
|
concurrently perform independently of one another. If you have 3
|
||||||
|
filesystems on 3 drives and each is being actively used, the total
|
||||||
|
throughput can be A + B + C: the sum of each branch performing at
|
||||||
|
full speed. No single drive is a bottleneck for the others.
|
||||||
|
* **Worst case:** a single stream of IO - one process reading or
|
||||||
|
writing one file - performs at the speed of the single branch the
|
||||||
|
file is on, no differently than a lone filesystem. Additional
|
||||||
|
branches do not make a single file faster.
|
||||||
|
|
||||||
|
This means that when reasoning about expected performance the key
|
||||||
|
question is whether your workload is **single stream** or **multi
|
||||||
|
stream**:
|
||||||
|
|
||||||
|
* Single large file transfers, video playback, or one process doing
|
||||||
|
sequential IO will never exceed the speed of one branch regardless
|
||||||
|
of how many are mounted. If a single stream must be faster, put a
|
||||||
|
faster device (tiered cache, SSD, LVM cache) in front of the
|
||||||
|
branches or use an actual striped RAID layer beneath **mergerfs**'
|
||||||
|
branches.
|
||||||
|
* Concurrency across branches is where **mergerfs** shines. Many
|
||||||
|
parallel downloads, seeding torrents, multiple rsync streams,
|
||||||
|
multiple applications reading different files: these naturally
|
||||||
|
spread across branches and their throughput adds up.
|
||||||
|
|
||||||
|
There are practical upsides to this non-aggregated design as well:
|
||||||
|
|
||||||
|
* Drives of different sizes and speeds don't reduce the whole pool to
|
||||||
|
the slowest member. Each branch performs at its own speed.
|
||||||
|
* The failure of one drive only affects the files on that branch
|
||||||
|
rather than the integrity of the entire pool.
|
||||||
|
|
||||||
|
Finally, keep in mind that **mergerfs** is a proxy above the branches,
|
||||||
|
so measured throughput is the lesser of the branch's native performance
|
||||||
|
and any overhead introduced by **mergerfs** and FUSE. Use
|
||||||
|
[nullrw](config/options.md) or a ram disk as a branch to measure that
|
||||||
|
overhead separately, and see the [benchmarking
|
||||||
|
section](benchmarking.md) for how to benchmark meaningfully.
|
||||||
|
|
||||||
|
|
||||||
## Additional Reading
|
## Additional Reading
|
||||||
|
|
||||||
* [Benchmarking](benchmarking.md)
|
* [Benchmarking](benchmarking.md)
|
||||||
|
|
|
||||||
Loading…
Add table
Add a link
Reference in a new issue