Rework config, centralize fuse config (#1547)
Some checks are pending
CodeQL / Analyze (push) Waiting to run
mkdocs / deploy (push) Waiting to run

This commit is contained in:
trapexit 2025-10-24 08:20:29 -05:00 • committed by GitHub
parent 00e5467e99
commit 727c522fb8
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
46 changed files with 1294 additions and 1316 deletions

View file

@ -41,10 +41,10 @@ transparently enable page caching when mmap is requested. This means
it should be safe to set `cache.files=off`. However, on Linux v6.5 and
below you will need to configure `cache.files` as you need.
If [passthrough](passthrough.md) is enabled so must be page
caching. mergerfs will set `cache.files=auto-full` if `passthrough` is
enabled. And when using `passthrough` the there is no double page
caching since it is in fact passing through the IO.
If [passthrough.io](passthrough.md) is enabled so must page
caching. mergerfs will set `cache.files=auto-full` if `passthrough.io`
is enabled. And when using `passthrough.io` the there is no double
page caching since it is in fact passing through the IO.
[^1]: This is not unique to mergerfs and affects all FUSE

View file

@ -3,19 +3,21 @@
These are old, deprecated options which may no longer have any
function or have been replaced. **They should not be used.**
* **direct_io**: Bypass page cache. Use `cache.files=off`
instead.
* **kernel_cache**: Do not invalidate data cache on file open. Use
`cache.files=full` instead.
* **auto_cache**: Invalidate data cache if file mtime or
size change. Use `cache.files=auto-full` instead. (default: false)
* **async_read**: Perform reads asynchronously. Use
`async_read=true` instead.
* **sync_read**: Perform reads synchronously. Use
`async_read=false` instead.
* **splice_read**: Does nothing.
* **splice_write**: Does nothing.
* **splice_move**: Does nothing.
* **allow_other**: mergerfs v2.35.0 and above sets this FUSE option
automatically if running as root.
* **use_ino**: Effectively replaced with `inodecalc`.
* **async_read**: Use `async_read=true`.
* **atomic_o_trunc**: Does nothing.
* **big_writes**: Does nothing.
* **attr_timeout**: Use `cache.attr`.
* **auto_cache**: Use `cache.files=auto-full`.
* **direct_io**: Use `cache.files=off`.
* **entry_timeout**: Use `cache.entry`.
* **hard_remove**: Does nothing.
* **kernel_cache**: Use `cache.files=full`.
* **negative_entry**: Use `cache.negative_entry`.
* **nonempty**: Does nothing.
* **splice_move**: Does nothing.
* **splice_read**: Does nothing.
* **splice_write**: Does nothing.
* **sync_read**: Use `async_read=false`.
* **use_ino**: Use `inodecalc`.

View file

@ -30,9 +30,11 @@ config file.
## mount options
* **config**: Path to a config file. Same arguments as below in
key=val / ini style format.
`key` or `key=val` style format. `#` used for comments.
* **[branches](branches.md)**: Colon delimited list of branches. Used
primarily in config file.
primarily in a config file.
* **mountpoint=DIRPATH**: The path to the mountpoint. Used primarily
in a config file.
* **[branches-mount-timeout](branches-mount-timeout.md)=UINT**: Number
of seconds to wait at startup for branches to be a mount other than
the mountpoint's filesystem. (default: 0)
@ -95,7 +97,7 @@ config file.
xattrs. Default is to passthrough xattr requests. 'noattr' will
short circuit as if nothing exists. 'nosys' will respond with ENOSYS
as if xattrs are not supported or disabled. (default: passthrough)
* **[link_cow](link_cow.md)=BOOL**: When enabled if a regular file is
* **[link-cow](link_cow.md)=BOOL**: When enabled if a regular file is
opened which has a link count > 1 it will copy the file to a
temporary file and rename over the original. Breaking the link and
providing a basic copy-on-write function similar to
@ -130,11 +132,11 @@ config file.
* **[fuse_msg_size](fuse_msg_size.md)=PAGESIZE**: Set the max number of
pages per FUSE message. Only available on Linux >= 4.20 and ignored
otherwise. (min: 1; max: 65535; default: "1M")
* **[threads](threads.md)=INT**: Number of threads to use. When used
alone (`process-thread-count=-1`) it sets the number of threads
reading and processing FUSE messages. When used together it sets the
number of threads reading from FUSE. When set to zero it will
attempt to discover and use the number of logical cores. If the
* **[read-thread-count](threads.md)=INT**: Number of threads to
use. When used alone (`process-thread-count=-1`) it sets the number
of threads reading and processing FUSE messages. When used together
it sets the number of threads reading from FUSE. When set to zero it
will attempt to discover and use the number of logical cores. If the
thread count is set negative it will look up the number of cores
then divide by the absolute value. ie. threads=-2 on an 8 core
machine will result in 8 / 2 = 4 threads. There will always be at
@ -142,7 +144,6 @@ config file.
`process-thread-count` then it will try to pick reasonable values
based on CPU thread count. NOTE: higher number of threads increases
parallelism but usually decreases throughput. (default: 0)
* **[read-thread-count](threads.md)=INT**: Alias for `threads`.
* **[process-thread-count](threads.md)=INT**: Enables separate thread
pool to asynchronously process FUSE requests. In this mode
`read-thread-count` refers to the number of threads reading FUSE
@ -153,7 +154,7 @@ config file.
time. Meaning the total memory usage of the queues is queue depth
multiplied by the number of process threads plus read thread
count. 0 sets the depth to the same as the process thread
count. (default: 0)
count. (default: 2)
* **[pin-threads](pin-threads.md)=STR**: Selects a strategy to pin
threads to CPUs (default: unset)
* **[flush-on-close](flush-on-close.md)=never|always|opened-for-write**:
@ -170,9 +171,9 @@ config file.
* **[func.FUNC](functions_categories_policies.md)=POLICY**: Sets the
specific FUSE function's policy. See below for the list of value
types. Example: **func.getattr=newest**
* **[func.readdir](func_readdir.md)=seq|cosr|cor|cosr:INT|cor:INT**: Sets `readdir`
policy. INT value sets the number of threads to use for
concurrency. (default: seq)
* **[func.readdir](func_readdir.md)=seq|cosr|cor|cosr:INT|cor:INT**:
Sets `readdir` policy. INT value sets the number of threads to use
for concurrency. (default: seq)
* **[category.action](functions_categories_policies.md)=POLICY**: Sets
policy of all FUSE functions in the action category. (default:
epall)
@ -206,15 +207,23 @@ config file.
with `cache.files=per-process` (if the process is not in
`process-names`) or `cache.files=off`. (Is a kernel feature added in
v6.2) (default: true)
* **[passthrough](passthrough.md)**: Enable [FUSE IO
* **[passthrough.io](passthrough.md)=ENUM**: Enable [FUSE IO
passthrough](https://kernelnewbies.org/Linux_6.9#Faster_FUSE_I.2FO)
if available. (default: off)
* **gid-cache-expire-timeout**: Number of seconds till supplemental
group data is refreshed in the [GID
* **passthrough.max-stack-depth=INT**: Set to `1` another filesystem
can be stacked on mergerfs. Set to `2` to have mergerfs stacked over
another filesystem. (default: 1)
* **gid-cache.expire-timeout=INT**: Number of seconds till
supplemental group data is refreshed in the [GID
cache](../known_issues_bugs.md#supplemental-user-groups). (default:
3600)
* **gid-cache-remove-timeout**: Number of seconds to wait till cached
data is removed due to lack of usage. (default: 43200)
* **gid-cache.remove-timeout=INT**: Number of seconds to wait till
cached data is removed due to lack of usage. (default: 43200)
* **remember-nodes=INT**: The number of seconds to keep the internal
representation of a file once the OS tells mergerfs it is no longer
needed. Really only needed for [exporting mergerfs via
NFS](../remote_filesystems.md) (default: 0)
* **noforget**: Effectively sets `remember-nodes` to infinity.
**NOTE:** Options are evaluated in the order listed so if the options
are **func.rmdir=rand,category.action=ff** the **action** category

View file

@ -1,4 +1,4 @@
# passthrough
# passthrough.io
* default: `off`
* arguments:
@ -14,7 +14,7 @@ to act as an active proxy for all read and write requests. This
results in, at times, significant overhead compared to direct
interaction with the underlying filesystems. Not because `mergerfs` is
doing anything particularly slow but because the additional
communication and data transfers required. With the `passthrough`
communication and data transfers required. With the `passthrough.io`
feature `mergerfs` is able to instruct the kernel to perform reads and
writes directly on the underlying file. Bypassing `mergerfs` entirely
(specifically and only for reads and writes) and thereby providing
@ -37,48 +37,47 @@ features related to read/write are affected.
requests being sent to `mergerfs` there is little reason to have
larger message sizes since the only other message larger than 1
page, directory reading, currently has a small fixed buffer size.
* parallel-direct-writes: Not only is `direct-io` and `passthrough`
* parallel-direct-writes: Not only is `direct-io` and `passthrough.io`
mutually exclusive but since `mergerfs` no longer receives write
requests in passthrough mode then there is no parallel writes
possible.
* [cache.writeback](cache.md): FUSE's writeback caching is
incompatible with `passthrough`. If `cache.writeback=true` when
enabling `passthrough` it will be reset to `false`.
* [cache.files](cache.md): Must be enabled for `passthrough` to
incompatible with `passthrough.io`. If `cache.writeback=true` when
enabling `passthrough.io` it will be reset to `false`.
* [cache.files](cache.md): Must be enabled for `passthrough.io` to
work. When `cache.files=off` FUSE's `direct-io` mode is enabled
which overrides `passthrough.` Meaning `cache.files` should be set
to `partial`, `full` or `auto-full` to use `passthrough`. If
`cache.files` is set to `off` when using `passthrough` it will
which overrides `passthrough.io`. Meaning `cache.files` should be
set to `partial`, `full` or `auto-full` to use `passthrough.io`. If
`cache.files` is set to `off` when using `passthrough.io` it will
reset it to `auto-full`.
Technically `mergerfs` has the ability to choose options like
`passthrough`, `direct-io`, and page caching independently for every
file opened. However, at the moment there is no use case for picking
and choosing which to enable outside `cache.files=per-process` (which
is largely unnecessary on Linux v6.6 and above. See
`passthrough.io`, `direct-io`, and page caching independently for
every file opened. However, at the moment there is no use case for
picking and choosing which to enable outside `cache.files=per-process`
(which is largely unnecessary on Linux v6.6 and above. See
[direct-io-allow-mmap](options.md)) If such a use case arises please
[reach out to the author](../support.md) to discuss.
Unlike [preload.so](../tooling.md#preloadso), `passthrough` will work for
any software interacting with `mergerfs`. However, `passthrough`
requires Linux v6.9 or above to work.
Unlike [preload.so](../tooling.md#preloadso), `passthrough.io` will
work for any software interacting with `mergerfs`. However,
`passthrough.io` requires Linux v6.9 or above to work.
**NOTE:** This feature will **ONLY** work if `mergerfs` is running as
`root` as currently only `root` is allowed to leverage the kernel
feature.
**NOTE:** If a file has been opened and `passthrough` enabled, while
that file is open, if another open request is made `mergerfs` must
also enable `passthrough` for the second open request. This is a
limitation of how the passthrough feature works. Though there is no
known usecase where this is useful.
**NOTE:** If a file has been opened and `passthrough.io` enabled,
while that file is open, if another open request is made `mergerfs`
must also enable `passthrough.io` for the second open request. This is
a limitation of how the passthrough feature works.
**NOTE:** In order to add `passthrough` feature to `mergerfs` it was
necessary to remove the "feature" where mergerfs could open the same
file on different branches. Such as using `func.open=rand` and having
multiple files at the same relative path across different
**NOTE:** In order to add `passthrough.io` feature to `mergerfs` it
was necessary to remove the "feature" where mergerfs could open the
same file on different branches. Such as using `func.open=rand` and
having multiple files at the same relative path across different
branches. This "feature" was very very rarely used and it was
impossible to support `passthrough` without changing the behavior.
impossible to support `passthrough.io` without changing that behavior.
## Alternatives

View file

@ -33,11 +33,11 @@ proxying may be added in a future release.
## Conflicts With Other Options
If using [IO passthrough](passthrough.md) there is no reason to set
this value to `true`. However, since `passthrough` leads to mergerfs
IO calls being bypassed entirely will have no impact on performance or
behavior regardless.
this value to `true`. However, since `passthrough.io` leads to
mergerfs IO calls being bypassed entirely will have no impact on
performance or behavior regardless.
When using `passthrough` the IO priority of the client app would be
When using `passthrough.io` the IO priority of the client app would be
used directly given the IO is passed through.

View file

@ -18,37 +18,71 @@ mergerfs [does not impact](usage_and_functionality.md) the underlying
filesystems and can be added or removed without any impact it is
extremely easy to test and experiment with different settings.
Additional reading:
* [Config/Options](../config/options.md)
## What create policy should I use?
If it isn't clear to you then you should use `pfrd` as described in
the [quickstart guide](../quickstart.md).
## Why is pfrd the default create policy?
Originally the default was `epmfs` however it was found to cause
significant confusion among new users who expected mergerfs to always
choose a branch with available space. For mergerfs v2.41.0 it was
decided to change the default to the less restrictive policy `pfrd`.
significant confusion among new users who assumed mergerfs to behave
like other software which always choose a branch with most available
space without any filtering or restrictions. The reason for using
`epmfs` was to allow people to use mergerfs without impacting the
layout of their data across the branches. Keeping the total entropy of
the system in check. This, however, was not understood by users who
did not read the documentation and caused significant support burdens.
For mergerfs v2.41.0 it was decided to change the default to `pfrd` to
minimize the support burden.
`pfrd` was chosen because it prioritizes placement to branches based
on free space (percentage wise) without overloading a specific branch
as `mfs`, `lus`, or other policies could when a singular branch has
significantly more free space. Since it is impossible to know the
usage and access patterns of the filesystem spreading out files across
the branches also spreads the load and therefore good for general use
cases making it a good default.
Additional reading:
* [functions, categories and policies](../config/functions_categories_policies.md)
## How can I ensure files are collocated on the same branch?
Many people like the idea of ensuring related files, such as all the
files to a TV show season or songs in an album, are stored on the same
storage device. However, most people have no actual need for this
storage device. However, most people have no technical need for this
behavior.
1. If you backup your data it is extremely likely your backup solution
can restore only those files you are missing.
2. Software such as **Sonarr** can manage the downloading and post
processing of bespoke episodes which may be missing in a
season. Either by downloading the episode individually if available
or by downloading a full season.
can restore only those files you are missing. There is no need to
store data in related units to ensure restoring data is easy.
2. Software such as [Sonarr](https://sonarr.tv) can manage the
downloading and post processing of bespoke files / episodes which
may be missing. Either by downloading the episode individually if
available or by downloading a full season.
3. There is no benefit to keeping files collocated with regard to
drive spinup, caching, or other secondary concern.
drive spinup, caching, or other secondary concern. In fact
depending on usage patterns placing the data on the same underlying
device could harm performance.
The main use case for wanting collocation is where the branch is going
to be removed from the pool and you wish to have all data from some
logical set on that device. Such as you intend to take a drive out of
the pool to take on a trip and want a whole show on the
drive. However, even in these situations you typically end up needing
to curate the files anyway because it has show A but not show B.
to be removed from the pool and used in isolation temporarily and you
wish to have all data from some logical set on that device. Such as
you intend to take a drive out of the pool to take on a trip and want
a whole show or artist's works on the drive. However, even in these
situations you typically end up needing to curate the files anyway
because it has show A but not show B. In such cases it is more
convient to use a separate devices and sync what you wish to it.
All that said you can accomplish collocation to varying degrees using
the following methods:
@ -56,33 +90,37 @@ the following methods:
1. Use
[mergerfs.consolidate](https://github.com/trapexit/mergerfs-tools/blob/master/src/mergerfs.consolidate)
when consolidation is needed.
2. Use a `msp` create policy.
2. Use a `msp` create policy to do a best effort collocation without
needing explicit directory curation.
3. Use `epmfs` or other `ep` create policy and manually create paths
on the branches directly.
4. Use a `ep` `create` policy and `rand` for `mkdir`.
on the branches directly as you need/desire.
4. Use a `ep` `create` policy and `pfrd` or `rand` for `mkdir`.
## How can I balance files across the pool?
Similar to collocation there is generally little reason to balance
files.
files across the pool branches.
1. Since prediction of a filesystem's death or loss of data is near
impossible there is little reason to balance in hopes of limiting
data loss.
1. Since prediction of a device's/filesystem's death or loss of data
is near impossible there is little reason to balance in hopes of
limiting data loss.
2. While performance could be impacted by having too much reading or
writing happen to singular underlying filesystems balancing won't
help unless you have the ability to manage the access patterns to
the pool.
3. Over time most configurations will lead to a random distribution of
files across the branches which is effectively "balancing."
3. Over time the churn of adding and removing files will lead to a
random distribution of files across the branches which is
effectively "balancing" if using create policies which do not favor
any particular branch.
If you wish to move files around or balance the pool you can:
All that said if you wish to move files around or balance the pool you
can:
1. Use `rand` or `pfrd` create policies and just use your system as
normal.
2. Write simple scripts using rsync or similar to move files around as
you wish.
2. Write scripts using rsync or similar to move files around as you
wish.
3. Use
[mergerfs.balance](https://github.com/trapexit/mergerfs-tools/blob/master/src/mergerfs.balance). Keep
in mind that this tool is really just an example of how to

View file

@ -2,6 +2,9 @@
## What should mergerfs NOT be used for?
* Situations where you need large amounts of contiguous space beyond
that available on any singular device. Such as putting 10GiB file on
2 6GiB filesystems.
* databases: Even if the database stored data in separate files
(mergerfs wouldn't offer much otherwise) the higher latency of the
indirection will really harm performance. If it is a lightly used
@ -13,9 +16,9 @@
should stick with RAID. However, it is fine to put a filesystem
which is on a RAID setup in mergerfs.
**However, if using [passthrough](../config/passthrough.md) the above
situations are less likely to be a concern. Best to do testing for
your specific use case.**
**However, if using [passthrough](../config/passthrough.md) the
performance related issues above are less likely to be a concern. Best
to do testing for your specific use case.**
## It's mentioned that there are some security issues with mhddfs. What are they? How does mergerfs address them?

View file

@ -8,7 +8,7 @@
block to change credentials as required by numerous filesystem
functions. This impacts performance.
* FreeBSD's FUSE implementation is lacking many features of Linux.
* passthrough
* IO passthrough
* statx
* lazy umount
* oom_score_adj

View file

@ -64,6 +64,15 @@ branches](config/branches.md#branch-setup).
### Command Line
```
# Two positional options: `:` colon separated branches and mountpoint
mergerfs -o opt[,opt...] /branch0:/branch1 /mountpoint
# or list of positional options where the last is the mountpoint
mergerfs -o opt[,opt...] /branch0 /branch1 /mountpoint
# or set mountpoint via -o mountpoint
mergerfs -o mountpoint=/mountpoint /branch0 /branch1
```
```
mergerfs -o cache.files=off,category.create=pfrd,func.getattr=newest,dropcacheonclose=false /mnt/hdd0:/mnt/hdd1 /media
```
@ -129,7 +138,8 @@ ExecStart=/usr/bin/mergerfs \
-o category.create=pfrd \
-o func.getattr=newest \
-o dropcacheonclose=false \
/mnt/hdd0:/mnt/hdd1 \
/mnt/hdd0 \
/mnt/hdd1 \
/media
ExecStop=/usr/bin/umount /media
# Or if you need fusermount
@ -195,7 +205,8 @@ ExecStart=/usr/bin/mergerfs \
-o category.create=pfrd \
-o func.getattr=newest \
-o dropcacheonclose=false \
/mnt/hdd0:/mnt/hdd1 \
/mnt/hdd0 \
/mnt/hdd1 \
/media
ExecStop=/usr/bin/umount /media
# Or if you need fusermount

View file

@ -76,19 +76,21 @@ but if you run into problems it may be worth trying Samba/SMB.
**mergerfs settings:**
* `noforget`
* `noforget` or `remember-nodes=SECONDS`
* `inodecalc=path-hash`
* `lazy-umount-mountpoint=false`
`noforget` is needed because NFS uses the `name_to_handle_at` and
`noforget` (or `remember-nodes` with a large value such as `86400` for
a full day) is needed because NFS uses the `name_to_handle_at` and
`open_by_handle_at` functions which allow a program to keep a
reference to a file without technically having it open in the typical
sense. The problem is that FUSE has no way to know that NFS has a
handle that it will later use to open the file again. As a result, it
is possible for the kernel to tell mergerfs to forget about the file
node and should NFS ever ask for that node's details in the future it
would have nothing to respond with. Keeping nodes around forever is
not ideal but at the moment the only way to manage the situation.
would have nothing to respond with. Keeping nodes around forever (or
however long is set with `remember-nodes`) is not ideal but at the
moment the only way to manage the situation.
`inodecalc=path-hash` is needed because NFS is sensitive to
out-of-band changes. FUSE doesn't care if a file's inode value changes

View file

@ -50,8 +50,8 @@ OPTIONS:
EXPERIMENTAL
For some time there has been work to enable
[passthrough](config/passthrough.md) IO in FUSE. Passthrough IO would
For some time there has been work to enable IO
[passthrough](config/passthrough.md) in FUSE. IO Passthrough would
allow for near native performance with regards to reads and writes (at
the expense of certain mergerfs features.) In Linux v6.9 that feature
made its way into the kernel however in a somewhat limited form which