mergerfs/mkdocs/docs/faq/technical_behavior_and_limitations.md
Antonio S. Musumeci 8eed256747 Restore splice zero-copy I/O as runtime options
Reinstate the splice read/write/move paths removed in 84592a9f, now
gated behind runtime options with negotiated pipe sizing and fallback
logging.

- Transport: msgbuf-piped splice receive, fd-referenced splice reply
  (vmsplice + splice), splice-move pipe-bounce, and restored
  fuse_buf_copy/fuse_buf_flags plumbing for copy fallbacks
- Negotiation: new pipe-max-size option (auto-raised at INIT when
  possible; the sysctl read/write is factored into src/pipe_max_size.cpp
  so fuse_init.cpp and option_parser.cpp share one implementation);
  fuse-msg-size clamped to what the pipe can hold; splice is disabled
  at INIT with a warning if the sysctl can't fit even the minimum,
  retracting the FUSE_CAP_SPLICE_* bits already advertised to the
  kernel in that case
- Read replies and splice-move are gated at 128 KiB (below that the
  syscalls cost more than the copy they avoid); every fallback path
  degrades to the copy path and logs once per process via SysLog
- msgbuf pipe cache is process-global and mutex-guarded (mutex_lockguard)
  so pipes can move between read and worker threads under
  process-thread mode
- API simplified after review: collapsed the three per-axis options
  (splice-read/write/move) into a single splice boolean (default on);
  no-splice-read/write/move and the bare splice-read/write/move
  spellings stay registered as accepted no-ops so existing mount
  options don't fail the mount, but they do NOT alter the splice
  knob - upgraders with no-splice-* in their config get splice
  enabled and must set splice=false explicitly (see
  deprecated_options.md)
- Fixed several correctness issues found in parallel review: a
  dropped write_buf registration that silently corrupted writes, a
  dirty-pipe reuse bug in msgbuf_free, an iov double-send in
  fuse_reply_data's splice-move fallback, and pipe-capacity math that
  assumed a 4096-byte page size
- Modernized the transport/negotiation code to this codebase's C++
  style: RAII (unique_ptr/DEFER) instead of malloc/free and manual fd
  close(), std::from_chars/to_chars instead of stoul/snprintf,
  _ReplyPipe's pipe ops as methods instead of free functions closing
  over a thread_local global, and vendored/libfuse/lib/buffer.cpp (a
  near-verbatim port of upstream's buffer.c) brought in line with the
  rest of the fork
- Documented splice and pipe-max-size options (mkdocs, FAQ,
  deprecated-options list); added tools/bench-splice.sh
- Benchmarked on tmpfs: seq_write +21%, seq_read +52%, small_write
  +5%, small_read +12%; validated with a 1h soak (~271 GiB
  hash-verified I/O, zero integrity failures), a live-mount round-trip
  of I/O above the splice threshold, and the existing unit/smoke
  suites at their pre-existing baseline

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-09-07 11:24:16 -04:00

13 KiB

Technical Behavior and Limitations

Yes. links (otherwise known as hard links) are fundamentally supported by mergerfs. There is no option to enable or disable links. All comments elsewhere claiming they do not work are related to the setup of the user's system.

See the Intro to Filesystems page for more details on what a link is.

A limitation of a link is that it must reference a file/inode within the same filesystem or device. When a request is made to the operating system to link (or rename) a file to a target location the OS will first check if the source and target are the same filesystem / device. If not it will return the error EXDEV meaning that the request is not able to be made due to the result being "cross device".

What catches many users off guard is that this "cross device" check is not very deep or involved. Even the same filesystem bind mounted to another location will typically be considered a different device. So when mergerfs is creating a union of other filesystems there is no way to link or rename from one of the underlying branches into mergerfs. They are different filesystems even if logically they seem related or the same. And similarly it is impossible for mergerfs to truly make a link between two branches which are not in fact the same mounted filesystem.

This is relevant to mergerfs users because bind mounts are very common when using container runtimes such as Docker, Podman, or Kubernetes. When you add a volume or mount to your container that is in fact a bind mount of that path. Meaning if you were to add /mnt/mergerfs/downloads and /mnt/mergerfs/movies as separate mounts into your container any request to link between the two locations within the container will fail with the EXDEV error even though it will work perfectly fine on the host. If you wish for link and rename to work you MUST mount into the container a point in the directory structure above what you wish to use. In this case /mnt/mergerfs.

Another situation where people will hit unexpected EXDEV errors is when they use a policy which makes file creation decisions based on existing paths. If the source of a link or rename is on branch A but the target directory is on branch B, given the whole point of the policy is to strictly enforce behavior based on what branches already exist, it must return EXDEV. Doing otherwise would fundamentally break the purpose of the policy. Read rename & link for more details on how mergerfs handles those two functions.

When in doubt it is trivial to test link:

$ cd /mnt/mergerfs/
$ touch foo
$ ln -v foo bar
'bar' => 'foo'
$ stat foo bar | grep Inode
Device: 0,173   Inode: 123456  Links: 2
Device: 0,173   Inode: 123456  Links: 2

For the more technical: within mergerfs the inode of a file is not necessarily indicative of two file names linking to the same underlying data. For more details as to why that is see the docs on inodecalc.

Are there any size limitations to files, directories, etc.?

Whatever the limits are of the underlying filesystems.

Unfortunately not. FUSE, the technology mergerfs is based on, does not support the clone_file_range feature needed for it to work. mergerfs won't even know such a request is made. The kernel will simply return an error back to the application making the request.

Should FUSE gain the ability mergerfs will be updated to support it.

That said: mergerfs itself will attempt to use FICLONE when copying data between files in the limited cases where it copies/moves files.

How does mergerfs handle moving and copying of files?

This is a very common mistaken assumption regarding how filesystems work. There is no such thing as "move" or "copy." These concepts are high level behaviors made up of numerous independent steps and not individual filesystem functions.

A "move" can include a "copy" so lets describe copy first.

When an application copies a file from "source" to "destination" it can do so in a number of ways but the basics are the following.

  1. open the source file.
  2. create the destination file.
  3. read a chunk of data from source and write to destination. Continue till it runs out of data to copy.
  4. Copy file metadata (stat) such as ownership (chown), permissions (chmod), timestamps (utimes), extended attributes (getxattr, setxattr), etc.
  5. close source and destination files.

"move" is typically a rename(src,dst) and if that errors with EXDEV (meaning the source and destination are on different filesystems) the application will "copy" the file as described above and then it removes (unlink) the source.

The rename(src,dst), open(src), create(dst), data copying, metadata copying, unlink(src), etc. are entirely distinct and separate events. There is really no practical way to know that what is ultimately occurring is the "copying" of a file or what the source file would be. Since the source is not known there is no way to know how large a created file is destined to become. This is why it is impossible for mergerfs to choose the branch for a create based on file size. The only context provided when a file is created, besides the name, is the permissions, if it is to be read and/or written, and some low level settings for the operating system.

All of this means that mergerfs can not make decisions when a file is created based on file size or the source of the data. That information is simply not available. At best mergerfs could respond to files reaching a certain size when writing data, a file is closed, or renamed.

Related: if a user wished to have mergerfs perform certain activities based on the name of a file it is common and even best practice for a program to write to a temporary file first and then rename to its final destination. That temporary file name will typically be random and have no indication of the type of file being written. At best something could be done on rename.

Additional reading: Intro to Filesystems: Workflows

Why does the total available space in mergerfs not equal outside?

Are you using ext2/3/4? With reserve for root? mergerfs uses available space for statfs calculations. If you've reserved space for root then it won't show up.

You can remove the reserve by running: tune2fs -m 0 <device>

I notice massive slowdowns of writes when enabling cache.files.

When file caching is enabled in any form (cache.files!=off) it may issue getxattr requests for security.capability prior to every single write. This will usually result in performance degradation, especially when using a network filesystem (such as NFS or SMB.) Unfortunately at this moment, the kernel is not caching the response.

To work around this situation mergerfs offers a few solutions.

  1. Set security_capability=false. It will short circuit any call and return ENOATTR. This still means though that mergerfs will receive the request before every write but at least it doesn't get passed through to the underlying filesystem.
  2. Set xattr=noattr. Same as above but applies to all calls to getxattr. Not just security.capability. This will not be cached by the kernel either but mergerfs' runtime config system will still function.
  3. Set xattr=nosys. Results in mergerfs returning ENOSYS which will be cached by the kernel. No future xattr calls will be forwarded to mergerfs. The downside is that also means the xattr based config and query functionality won't work either.
  4. Disable file caching. If you aren't using applications which use mmap it's probably simpler to just disable it altogether. The kernel won't send the requests when caching is disabled.

Why use FUSE? Why not a kernel based solution?

As with any solution to a problem, there are advantages and disadvantages to each one.

A FUSE based solution has all the downsides of FUSE:

  • Higher IO latency due to the trips in and out of kernel space (though now minimized with passthrough IO)
  • Higher general overhead due to trips in and out of kernel space
  • Double caching when using page caching
  • Misc limitations due to FUSE's design

But FUSE also has a lot of upsides:

  • Easier to offer a cross platform solution
  • Easier forward and backward compatibility
  • Easier updates for users
  • Easier and faster release cadence
  • Allows more flexibility in design and features
  • Overall easier to write, secure, and maintain
  • Much lower barrier to entry (getting code into the kernel takes a lot of time and effort initially)

Is my OS's libfuse needed for mergerfs to work?

No. Normally mount.fuse is needed to get mergerfs (or any FUSE filesystem to mount using the mount command but in vendoring the libfuse library the mount.fuse app has been renamed to mount.mergerfs meaning the filesystem type in fstab can simply be mergerfs. That said there should be no harm in having it installed and continuing to using fuse.mergerfs as the type in /etc/fstab.

If mergerfs doesn't work as a type it could be due to how the mount.mergerfs tool was installed. Must be in /sbin/ with proper permissions.

Why was splice support removed and then restored?

Splice support was originally removed after testing showed it, at best, provided equivalent performance and in some cases worse. It was restored in 2026 after a rewrite for the current transport layer: request receive staging, fd-referenced read replies, and a 128 KiB threshold below which read replies stay on the copy path (write requests stage their payload through a pipe at any size). On tmpfs branches the restored paths benchmark at seq_read +52%, seq_write +21%. See splice and pipe-max-size in the options documentation. Splice remains unavailable on platforms that lack it; the copy path is the fallback everywhere.

How does mergerfs handle credentials?

mergerfs is a multithreaded application in order to handle requests from the kernel concurrently. Each FUSE message has a header with certain details about the request including the process ID (pid) of the requesting application, the process' effective user id (uid), and group id (gid).

FUSE and the kernel have two ways of managing permissions. A kernel side default_permissions option and leaving it to the FUSE server. When default permissions is enabled the kernel will do the entitlement checks and only allow requests through which should be allowed according to normal POSIX permissions. When not enabled it is the responsibility of the FUSE server, in this case mergerfs, to do whatever is necessary to manage entitlements.

Prior to mergerfs v2.42.0 it would enable default_permissions but also leveraged the uid and gid available in each FUSE request. The thread actioning the request would change its credentials to match those of the requesting application to ensure permissions were properly handled as well as dealing with some quirks of non-POSIX compatible filesystems. However, this strategy had two major issues. First, it was not compatible with the new allow-idmap feature of FUSE which allows a filesystem to advertise it can be used with id mapping which is often used with containers. Secondly, it caused permission issues when the kernel would say something was allowed but due to the way mergerfs was changing creds would fail. This mostly came in the form of chrooted setups like used in containers. Neither of these tended to impact casual users but did break some niche or power user use cases. Since the casual user won't notice the subtle changes and it would enable new use cases it was decided that the credential handling would change.

As of v2.42.0 mergerfs now runs as root more generally. Either changing credentials when creating files (Linux) or chown'ing them after creation (FreeBSD).

As a result of this change it is now necessary for the FUSE default_permissions feature be used for proper entitlements management. mergerfs does allow it to be disabled but it should only be done so for debugging purposes.

Does mergerfs support idmap?

Yes, by setting allow-idmap=true (which is the default.)

Requires that kernel-permissions-check be enabled (the default.)

What happens if a branch filesystem blocks?

POSIX filesystem calls unfortunately are entirely synchronous. If the underlying filesystem mergerfs is interacting with freezes up or blocks then the thread issuing the request will block same as any other piece of software. If enough threads block then mergerfs will block. There are no timeouts or ways to truly work around this situation. If mergerfs spawned new threads it is very likely they too would end up blocked.

What information does mergerfs have when making decisions?

Every incoming request contains:

  • The tid or pid of the calling thread/process
  • The uid and gid of the calling thread/process
  • The umask of the calling thread/process
  • The arguments of the filesystem call in question

Naturally the mergerfs config as well as anything queriable from the operating system or filesystems are also available.

Why does running mount -a result in new instances of mergerfs?

See lazy-umount-mountpoint