Compare commits

..

No commits in common. "master" and "1.2.2" have entirely different histories.

1142 changed files with 6844 additions and 154081 deletions

2
.github/FUNDING.yml vendored
View file

@ -1,2 +0,0 @@
github: [trapexit]
custom: ['https://github.com/trapexit/support']

View file

@ -1,77 +0,0 @@
---
name: Bug report
about: For the reporting of behavior not as described
title: ''
labels: bug, investigating
assignees: ''
---
See also https://trapexit.github.io/mergerfs/latest/support
**Describe the bug**
A clear and concise description of the unexpected behavior.
**Please be sure to use latest release of mergerfs to ensure the issue
still exists. Not your distro's latest but the latest official
release. Support will NOT be provided for old versions.**
The `master` branch is **not** to be considered production ready. Feel
free to file bug reports but do so indicating clearly that you are
testing unreleased code.
**To Reproduce**
Provide steps to reproduce the behavior. List **all** steps to
reproduce. **All** settings. The version of all software involved.
Please simplify the reproduction as much as possible.
* Unless it is dependenat on multiple branches, use a single branch
* Reproduce with standard tooling if possible
(touch,truncate,rm,rmdir,ln,etc.) Having to install 3rd party
software will make debugging more difficult.
**Expected behavior**
A clear and concise description of the expected behavior.
**Workarounds already tested**
Please list all workarounds attempted and why they were ruled out as
options.
**System information:**
* Attach output from
[mergerfs.collect-info](https://trapexit.github.io/mergerfs/latest/tooling/#mergerfscollect-info)
tool.
* Alternatively:
* OS: `lsb_release -a`
* kernel version: `uname -a`
* mergerfs version: `mergerfs --version`
* mergerfs settings / arguments: from fstab, systemd unit, command
line, OMV plugin, etc.
* List of drives, filesystems, & sizes:
* `df -h`
* `lsblk -o NAME,FSTYPE,FSSIZE,SIZE,MOUNTPOINTS,RM,RO,ROTA`
* All information about the relevant paths and files: permissions,
ownership, etc.
* All information about the application making the requests:
version, uid/gid
* Is mergerfs running within a container or used with container
technologies?
* Are the client apps using mergerfs running in a container?
* A strace of the application having a problem:
* `strace -fvTtt -s 256 -o /tmp/app.strace.txt <cmd>`
* `strace -fvTtt -s 256 -o /tmp/app.strace.txt -p <appPID>`
* strace of mergerfs while app tried to do it's thing:
* `strace -fvTtt -s 256 -p <mergerfsPID> -o /tmp/mergerfs.strace.txt`
**Additional context**
Add any other context about the problem here.

View file

@ -1,20 +0,0 @@
---
name: Feature request
about: Suggest an idea for this project
title: ''
labels: feature, investigating
assignees: ''
---
**Is your feature request related to a problem? Please describe.**
A clear and concise description of what the problem is.
**Describe the solution you'd like**
A clear and concise description of what you want to happen.
**Describe alternatives you've considered**
A clear and concise description of any alternative solutions or features you've considered.
**Additional context**
Add any other context or screenshots about the feature request here.

View file

@ -1,10 +0,0 @@
---
name: Question
about: When you're looking for information or general support
title: ''
labels: investigating, question
assignees: ''
---
For general questions and support please use [GitHub Discussions](https://github.com/trapexit/mergerfs/discussions) or [Discord](https://discord.gg/MpAr69V).

20
.gitignore vendored
View file

@ -11,23 +11,3 @@
*.lai
*.la
*.a
# build artifacts
mergerfs
build/
src/version.hpp
VERSION
# Debian files
debian/files
debian/changelog
debian/debhelper-build-stamp
debian/mergerfs.debhelper.log
debian/mergerfs.substvars
# RPM files
rpmbuild/
# mkdocs files
mkdocs/venv
__pycache__/

View file

@ -1,17 +0,0 @@
## Libraries
### included in repo
* libfuse: https://github.com/libfuse/libfuse (heavily modified fork of v2.x)
* rapidhash: https://github.com/Nicoshev/rapidhash
* fmt: https://github.com/fmtlib/fmt
* concurrentqueue: https://github.com/cameron314/concurrentqueue
* scope_guard: https://github.com/Neargye/scope_guard
* subprocess: https://github.com/arun11299/cpp-subprocess
* CLI11: https://github.com/CLIUtils/CLI11
* string-lite: https://github.com/martinmoene/string-lite
* boost
* download boost
* ./bootstrap.sh
* ./b2 tools/bcp
* ./dist/bin/bcp unordered/concurrent_flat_map.hpp <target_dir>

30
LICENSE
View file

@ -1,17 +1,23 @@
/*
ISC License
The MIT License (MIT)
Copyright (c) 2026, Antonio SJ Musumeci <trapexit@spawn.link>
Copyright (c) 2014 Antonio SJ Musumeci <trapexit@spawn.link>
Permission to use, copy, modify, and/or distribute this software for any
purpose with or without fee is hereby granted, provided that the above
copyright notice and this permission notice appear in all copies.
Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:
THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES
WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF
MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR
ANY SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES
WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN
ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF
OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE.
The above copyright notice and this permission notice shall be included in
all copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
THE SOFTWARE.
*/

470
Makefile
View file

@ -1,409 +1,145 @@
# Copyright (c) 2025, Antonio SJ Musumeci <trapexit@spawn.link>
# The MIT License (MIT)
#
# Permission to use, copy, modify, and/or distribute this software for any
# purpose with or without fee is hereby granted, provided that the above
# copyright notice and this permission notice appear in all copies.
# Copyright (c) 2014 Antonio SJ Musumeci <trapexit@spawn.link>
#
# THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES
# WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF
# MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR
# ANY SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES
# WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN
# ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF
# OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE.
# Permission is hereby granted, free of charge, to any person obtaining a copy
# of this software and associated documentation files (the "Software"), to deal
# in the Software without restriction, including without limitation the rights
# to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
# copies of the Software, and to permit persons to whom the Software is
# furnished to do so, subject to the following conditions:
#
# The above copyright notice and this permission notice shall be included in
# all copies or substantial portions of the Software.
#
# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
# AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
# THE SOFTWARE.
$(info mergerfs MAKEFLAGS: $(MAKEFLAGS))
PKGCONFIG = $(shell which pkg-config)
GIT = $(shell which git)
TAR = $(shell which tar)
MKDIR = $(shell which mkdir)
TOUCH = $(shell which touch)
CP = $(shell which cp)
RM = $(shell which rm)
FIND = $(shell which find)
INSTALL = $(shell which install)
MKTEMP = $(shell which mktemp)
STRIP = $(shell which strip)
PANDOC = $(shell which pandoc)
ifeq ($(shell id -u),0)
FAKEROOT ?=
ifeq ($(PKGCONFIG),"")
$(error "pkg-config not installed"
endif
CP ?= cp
FAKEROOT ?= fakeroot
FIND ?= find
GIT ?= git
GIT2DEBCL ?= buildtools/git2debcl
INSTALL ?= install
LN ?= ln
MKDIR ?= mkdir
MKTEMP ?= mktemp
MV ?= mv
RM ?= rm
RPMBUILD ?= rpmbuild
RPMBUILD_FLAGS ?=
SED ?= sed
STRIP ?= strip
TAR ?= tar
TOUCH ?= touch
BUILDDIR := build
GITHUB_REPO ?= "https://github.com/trapexit/mergerfs"
DEFAULT_TARGET := debian:13.amd64
ifndef GIT_REPO
ifneq ($(shell $(GIT) --version 2> /dev/null),)
ifeq ($(shell test -e .git; echo $$?),0)
GIT_REPO = 1
endif
endif
ifeq ($(PANDOC),"")
$(warning "pandoc does not appear available: manpage won't be buildable")
endif
USE_XATTR ?= 1
XATTR_AVAILABLE = $(shell test ! -e /usr/include/attr/xattr.h; echo $$?)
ifdef SANITIZE
ifeq ($(SANITIZE),1)
override SANITIZE := -fsanitize=address,undefined,leak
else ifeq ($(filter -fsanitize=%,$(SANITIZE)),)
override SANITIZE := -fsanitize=$(SANITIZE)
endif
FUSE_AVAILABLE = $(shell ! pkg-config --exists fuse; echo $$?)
ifeq ($(FUSE_AVAILABLE),0)
FUSE_AVAILABLE = $(shell test ! -e /usr/include/fuse.h; echo $$?)
endif
COMMON_OPT_FLAGS += \
-fstack-protector-strong \
-Wextra \
-Wno-unused-parameter
DEBUG_OPT_FLAGS += \
-O0 \
-g3 \
-fno-omit-frame-pointer \
$(SANITIZE)
ifdef RELEASE
OPT_FLAGS := -O3 $(COMMON_OPT_FLAGS)
else
OPT_FLAGS ?= $(DEBUG_OPT_FLAGS) $(COMMON_OPT_FLAGS)
ifeq ($(FUSE_AVAILABLE),0)
$(error "FUSE development package doesn't appear available")
endif
ifdef STATIC
STATIC_FLAGS := -static
else
STATIC_FLAGS :=
OPTS = -O2
SRC = $(wildcard src/*.cpp)
OBJ = $(SRC:src/%.cpp=obj/%.o)
DEPS = $(OBJ:obj/%.o=obj/%.d)
TARGET = mergerfs
MANPAGE = $(TARGET).1
CFLAGS = -g -Wall \
$(OPTS) \
$(shell $(PKGCONFIG) fuse --cflags) \
-DFUSE_USE_VERSION=26 \
-MMD
LDFLAGS = $(shell $(PKGCONFIG) fuse --libs)
BINDIR = $(PREFIX)/usr/bin
MANDIR = $(PREFIX)/usr/share/man/man1
INSTALLTARGET = $(DESTDIR)/$(BINDIR)/$(TARGET)
MANTARGET = $(DESTDIR)/$(MANDIR)/$(MANPAGE)
ifeq ($(XATTR_AVAILABLE),0)
$(warning "xattr not available: disabling")
CFLAGS += -DWITHOUT_XATTR
endif
ifdef LTO
LTO_FLAGS := -flto
else
LTO_FLAGS :=
KERNEL = $(shell uname -s)
ifeq ($(KERNEL),Linux)
CFLAGS += -DLINUX
endif
ifeq ($(KERNEL),Darwin)
CFLAGS += -DOSX
endif
SRC := $(wildcard src/*.cpp)
OBJS := $(SRC:src/%.cpp=build/.objs/%.cpp.o)
DEPS := $(SRC:src/%.cpp=build/.objs/%.cpp.d)
all: $(TARGET)
TESTS := $(wildcard tests/*.cpp)
TESTS_OBJS := $(filter-out build/.objs/mergerfs.cpp.o,$(OBJS))
TESTS_OBJS += $(TESTS:tests/%.cpp=build/.test_objs/%.cpp.o)
TESTS_DEPS := $(TESTS:tests/%.cpp=build/.test_objs/%.cpp.d)
TESTS_DEPS += $(DEPS)
MANPAGE := mergerfs.1
CPPFLAGS ?=
override CPPFLAGS += \
-D_FILE_OFFSET_BITS=64 \
-DBOOST_UNORDERED_DISABLE_PARALLEL_ALGORITHMS
CFLAGS ?= \
$(OPT_FLAGS) \
-Wall \
-Wno-unused-result \
-pipe
override CFLAGS += \
-MMD \
-MP
CXXFLAGS ?= \
$(OPT_FLAGS) \
$(STATIC_FLAGS) \
$(LTO_FLAGS) \
-Wall \
-Wpessimizing-move \
-Werror=pessimizing-move \
-Wredundant-move \
-Werror=redundant-move \
-Wno-unused-result \
-pipe
override CXXFLAGS += \
-std=c++20 \
-MMD \
-MP
override INC_FLAGS := \
-Isrc \
-Ivendored \
-Ivendored/libfuse/include
override MFS_FLAGS := \
-DUSE_XATTR=$(USE_XATTR)
override TESTS_FLAGS := \
-Isrc \
-Ivendored \
-Ivendored/acutest \
-DTESTS
LIBFUSE := vendored/libfuse/$(BUILDDIR)/libfuse.a
LDFLAGS ?=
LDLIBS := \
-lrt \
-pthread
override LDLIBS += \
$(LIBFUSE)
ifeq ($(CXX),g++)
GCC_VERSION := $(shell $(CXX) -dumpversion | cut -f1 -d.)
ifeq ($(shell test $(GCC_VERSION) -lt 9; echo $$?),0)
override LDLIBS += -lstdc++fs
endif
endif
# https://www.gnu.org/prep/standards/html_node/Directory-Variables.html
DESTDIR ?=
PREFIX ?= /usr/local
EXEC_PREFIX ?= $(PREFIX)
DATAROOTDIR ?= $(PREFIX)/share
DATADIR ?= $(DATAROOTDIR)
BINDIR ?= $(EXEC_PREFIX)/bin
SBINDIR ?= $(EXEC_PREFIX)/sbin
LIBDIR ?= $(EXEC_PREFIX)/lib
MANDIR ?= $(DATAROOTDIR)/man
MAN1DIR ?= $(MANDIR)/man1
INSTALLBINDIR ?= $(DESTDIR)$(BINDIR)
INSTALLSBINDIR ?= $(DESTDIR)$(SBINDIR)
INSTALLLIBDIR ?= $(DESTDIR)$(LIBDIR)/mergerfs
INSTALLMAN1DIR ?= $(DESTDIR)$(MAN1DIR)
.PHONY: all
all: libfuse $(BUILDDIR)/mergerfs $(BUILDDIR)/fsck.mergerfs $(BUILDDIR)/mergerfs.collect-info
.PHONY: help
help:
@echo "usage: make ARG\n"
@echo "USE_XATTR=0 - build program without xattrs functionality"
@echo "STATIC=1 - build static binary"
@echo "LTO=1 - build with link time optimization"
@echo "SANITIZE=1 - build with sanitizers (address,undefined,leak)"
@echo "SANITIZE=list - build with custom sanitizers (comma separated)"
@echo "CLEANUP=1 - cleanup images between release build process"
@echo "PKGDIR=/dir/ - location for release build pkgs"
@echo "GITREF=gitref - gitref to use for release builds"
@echo "usage: make"
@echo "make XATTR_AVAILABLE=0 - to build program without xattrs functionality (auto discovered otherwise)"
$(TARGET): obj/obj-stamp $(OBJ)
$(CXX) $(CFLAGS) $(OBJ) -o $@ $(LDFLAGS)
$(BUILDDIR)/mergerfs: $(LIBFUSE) src/version.hpp $(OBJS)
$(CXX) $(CXXFLAGS) $(INC_FLAGS) $(MFS_FLAGS) $(CPPFLAGS) $(OBJS) -o $@ $(LDFLAGS) $(LDLIBS)
$(BUILDDIR)/fsck.mergerfs: $(BUILDDIR)/mergerfs
$(LN) -sf "mergerfs" $@
$(BUILDDIR)/mergerfs.collect-info: $(BUILDDIR)/mergerfs
$(LN) -sf "mergerfs" $@
$(BUILDDIR)/tests: $(BUILDDIR)/mergerfs $(TESTS_OBJS)
$(CXX) $(CXXFLAGS) $(TESTS_FLAGS) $(INC_FLAGS) $(MFS_FLAGS) $(CPPFLAGS) $(TESTS_OBJS) -o $@ $(LDFLAGS) $(LDLIBS)
.PHONY: libfuse
libfuse: $(LIBFUSE)
$(MAKE) -C vendored/libfuse
$(LIBFUSE):
$(MAKE) -C vendored/libfuse $(BUILDDIR)/libfuse.a RELEASE=$(RELEASE)
tests: $(BUILDDIR)/tests
.PHONY:
changelog:
ifdef GIT_REPO
$(GIT2DEBCL) --name mergerfs > ChangeLog
else
@echo "WARNING: need git repo to generate ChangeLog"
endif
$(GIT) log --pretty --numstat --summary | git2cl > ChangeLog
.PHONY: version
version: src/version.hpp
src/version.hpp:
./buildtools/update-version
$(BUILDDIR)/stamp:
$(MKDIR) -p $(BUILDDIR)/.objs
$(MKDIR) -p $(BUILDDIR)/.test_objs
obj/obj-stamp:
$(MKDIR) -p obj
$(TOUCH) $@
$(BUILDDIR)/.objs/%.cpp.o: src/%.cpp $(BUILDDIR)/stamp
$(CXX) $(CXXFLAGS) $(INC_FLAGS) $(MFS_FLAGS) $(CPPFLAGS) -c $< -o $@
obj/%.o: src/%.cpp
$(CXX) $(CFLAGS) -c $< -o $@
$(BUILDDIR)/.test_objs/%.cpp.o: tests/%.cpp | $(BUILDDIR)/stamp
$(CXX) $(CXXFLAGS) $(TESTS_FLAGS) $(INC_FLAGS) $(MFS_FLAGS) $(CPPFLAGS) -c $< -o $@
clean:
$(RM) -rf obj "$(TARGET)" "$(MANPAGE)"
$(FIND) -name "*~" -delete
$(BUILDDIR)/preload.so: $(BUILDDIR)/stamp tools/preload.c
$(CC) -shared -fPIC $(CFLAGS) -o $@ tools/preload.c
preload: $(BUILDDIR)/preload.so
.PHONY: clean
clean: rpm-clean
$(RM) -rf $(BUILDDIR)
$(FIND) . -name "*~" -delete
$(MAKE) -C vendored/libfuse clean
.PHONY: distclean
distclean: clean
ifdef GIT_REPO
$(GIT) clean -xfd
endif
$(GIT) clean -ifd
.PHONY: install
install: install-base install-mount-tools install-preload install-man
install: $(TARGET) $(MANPAGE)
$(INSTALL) -m 0755 -D "$(TARGET)" "$(INSTALLTARGET)"
$(INSTALL) -m 0644 -D "$(MANPAGE)" "$(MANTARGET)"
.PHONY: install-base
install-base: all
$(MKDIR) -p "$(INSTALLBINDIR)"
$(INSTALL) -v -m 0755 "$(BUILDDIR)/mergerfs" "$(INSTALLBINDIR)/mergerfs"
$(LN) -fs "mergerfs" "${INSTALLBINDIR}/fsck.mergerfs"
$(LN) -fs "mergerfs" "${INSTALLBINDIR}/mergerfs.collect-info"
install-strip: install
$(STRIP) "$(INSTALLTARGET)"
.PHONY: install-mount-tools
install-mount-tools: install-base
$(MKDIR) -p "$(INSTALLBINDIR)"
$(MAKE) -C vendored/libfuse install
uninstall:
$(RM) "$(INSTALLTARGET)"
$(RM) "$(MANTARGET)"
.PHONY: install-man
install-man: man/$(MANPAGE)
$(MKDIR) -p "$(INSTALLMAN1DIR)"
$(INSTALL) -v -m 0644 "man/$(MANPAGE)" "$(INSTALLMAN1DIR)/$(MANPAGE)"
$(MANPAGE): README.md
$(PANDOC) -s -t man -o $(MANPAGE) README.md
.PHONY: install-preload
install-preload: preload
$(MKDIR) -p "$(INSTALLLIBDIR)"
$(INSTALL) -v -m 444 "$(BUILDDIR)/preload.so" "$(INSTALLLIBDIR)/preload.so"
man: $(MANPAGE)
.PHONY: install-strip
install-strip: install-base
$(STRIP) "$(INSTALLBINDIR)/mergerfs"
.PHONY: uninstall
uninstall: uninstall-base uninstall-mount.mergerfs uninstall-preload uninstall-man
uninstall-base:
$(RM) -f "$(INSTALLBINDIR)/mergerfs"
$(RM) -f "$(INSTALLBINDIR)/fsck.mergerfs"
$(RM) -f "$(INSTALLBINDIR)/mergerfs.collect-info"
uninstall-mount.mergerfs:
$(RM) -f "$(INSTALLBINDIR)/mount.mergerfs"
uninstall-man:
$(RM) -f "$(INSTALLMAN1DIR)/$(MANPAGE)"
uninstall-preload:
$(RM) -f "$(INSTALLLIBDIR)/preload.so"
.PHONY: tarball
tarball: changelog version
$(eval VERSION := $(shell cat VERSION))
$(eval VERSION := $(subst -,_,$(VERSION)))
$(eval FILENAME := mergerfs-$(VERSION))
tarball: clean changelog man
$(eval VERSION := $(shell $(GIT) describe --always --tags --dirty))
$(eval FILENAME := $(TARGET)-$(VERSION))
$(eval TMPDIR := $(shell $(MKTEMP) --tmpdir -d .$(FILENAME).XXXXXXXX))
$(MKDIR) $(TMPDIR)/$(FILENAME)
$(CP) -ar . $(TMPDIR)/$(FILENAME)
$(TAR) --exclude=.git -cz -C $(TMPDIR) -f $(FILENAME).tar.gz $(FILENAME)
$(TAR) --exclude=.git -cz -C $(TMPDIR) -f ../$(FILENAME).tar.gz $(FILENAME)
$(RM) -rf $(TMPDIR)
.PHONY: debian-changelog
debian-changelog:
ifdef GIT_REPO
$(GIT2DEBCL) --name mergerfs > debian/changelog
else
$(CP) -v ChangeLog debian/changelog
$(eval DISTRO := $(shell lsb_release -i -s | tr '[:upper:]' '[:lower:]'))
$(eval CODENAME := $(shell lsb_release -c -s | tr '[:upper:]' '[:lower:]'))
sed -i 's/~unknown-unknown/~$(DISTRO)-$(CODENAME)/g' debian/changelog
sed -i 's/ unknown; / $(CODENAME); /g' debian/changelog
endif
.PHONY: signed-deb
signed-deb:
$(MAKE) distclean
$(MAKE) debian-changelog
$(FAKEROOT) dpkg-buildpackage -nc
.PHONY: deb
deb:
$(MAKE) distclean
$(MAKE) debian-changelog
$(MAKE) version
$(FAKEROOT) dpkg-buildpackage -nc -uc -us
$(MKDIR) -p ./$(BUILDDIR)/pkgs/
$(MV) -v ../mergerfs*deb ./$(BUILDDIR)/pkgs/
$(eval VERSION := $(shell $(GIT) describe --always --tags --dirty))
$(GIT) dch --auto --release --new-version="$(VERSION)"
$(GIT) buildpackage --git-ignore-new
.PHONY: rpm-clean
rpm-clean:
$(RM) -rf rpmbuild
.PHONY: all clean install help
.PHONY: rpm
rpm: tarball
$(eval VERSION := $(shell cat VERSION))
$(eval VERSION := $(subst -,_,$(VERSION)))
$(MKDIR) -p rpmbuild/BUILD rpmbuild/RPMS rpmbuild/SOURCES
$(SED) 's/__VERSION__/$(VERSION)/g' mergerfs.spec > \
rpmbuild/SOURCES/mergerfs.spec
cp -ar mergerfs-$(VERSION).tar.gz rpmbuild/SOURCES
$(RPMBUILD) $(RPMBUILD_FLAGS) -ba rpmbuild/SOURCES/mergerfs.spec \
--define "_topdir $(CURDIR)/rpmbuild"
.PHONY: install-build-pkgs
install-build-pkgs:
./buildtools/install-build-pkgs
.PHONY: install-build-tools
install-build-tools:
./buildtools/install-build-tools
define build_release
$(eval GITREF ?= $(shell git describe --exact-match --tags HEAD 2>/dev/null || git symbolic-ref --short HEAD 2>/dev/null || git rev-parse --short HEAD))
@echo "GITREF=$(GITREF)"
./buildtools/build-release \
--target=$(1) \
$(if $(CLEANUP),--cleanup) \
$(if $(PKGDIR),--pkgdirpath=$(PKGDIR)) \
--git-repo=$(if $(REMOTE_REPO),$(GITHUB_REPO),".") \
--branch=$(GITREF)
endef
.PHONY: release release-all
.PHONY: release-amd64 release-arm64 release-armhf release-riscv64
.PHONY: release-sample release-static release-tarball release-installer
release:
$(MAKE) RELEASE=1 CPPFLAGS="$(CPPFLAGS) -DNDEBUG" all
release-all:
$(call build_release,"all")
release-sample:
$(if $(TARGET),,$(eval TARGET := $(shell ./buildtools/detect-local-target 2>/dev/null)))
$(call build_release,$(if $(TARGET),$(TARGET),$(DEFAULT_TARGET)))
release-amd64:
$(call build_release,"amd64")
release-arm64:
$(call build_release,"arm64")
release-armhf:
$(call build_release,"armhf")
release-riscv64:
$(call build_release,"riscv64")
release-static:
$(call build_release,"static")
release-tarball:
$(call build_release,"tarball")
release-installer:
$(call build_release,"installer")
container:
$(eval GITREF ?= $(shell git describe --exact-match --tags HEAD 2>/dev/null || git symbolic-ref --short HEAD 2>/dev/null || git rev-parse --short HEAD))
@echo "GITREF=$(GITREF)"
./buildtools/build-containerimage "$(GITREF)"
.PHONY: tags
tags:
rm -fv TAGS
find . -name "*.c" -print | etags --append -
find . -name "*.h" -print | etags --append -
find . -name "*.cpp" -print | etags --append -
find . -name "*.hpp" -print | etags --append -
-include $(DEPS)
include $(wildcard obj/*.d)

201
README.md
View file

@ -1,125 +1,136 @@
# mergerfs - a featureful union filesystem
% mergerfs(1) mergerfs user manual
% Antonio SJ Musumeci <trapexit@spawn.link>
% June 9, 2014
**mergerfs** is a
[FUSE](https://en.wikipedia.org/wiki/Filesystem_in_Userspace) based
[union filesystem](https://en.wikipedia.org/wiki/Union_mount) geared
towards simplifying storage and management of files across numerous
commodity storage devices. It is similar to [**mhddfs**, **unionfs**,
**aufs**, **DrivePool**,
etc.](https://trapexit.github.io/mergerfs/latest/project_comparisons/)
# NAME
mergerfs - another FUSE union filesystem
## Features
# SYNOPSIS
* Logically combine numerous filesystems/paths into a single
mount point (JBOFS: Just a Bunch of FileSystems)
* Combine paths of the same or different filesystems
* Ability to add or remove filesystems/paths without impacting the
rest of the data
* Unaffected by individual filesystem failure
* Configurable file selection and creation placement
* File IO
[passthrough](https://trapexit.github.io/mergerfs/latest/config/passthrough/)
for near native IO performance (where supported)
* Works with filesystems of any size
* Works with filesystems of [almost any
type](https://trapexit.github.io/mergerfs/latest/faq/compatibility_and_integration/#what-filesystems-can-be-used-as-branches)
* Ignore read-only filesystems when creating files
* [Hard links](https://trapexit.github.io/mergerfs/latest/faq/technical_behavior_and_limitations/#do-hard-links-work)
* Hard link [copy-on-write /
CoW](https://trapexit.github.io/mergerfs/latest/config/link_cow/)
* [Runtime configurable](https://trapexit.github.io/mergerfs/latest/runtime_interface/)
* Support for extended attributes (xattrs)
* Support for file attributes (chattr)
* Support for POSIX ACLs
mergerfs -ocreate=epmfs,search=ff,action=ff &lt;srcpoints&gt; &lt;mountpoint&gt;
# DESCRIPTION
## Non-features
mergerfs is similar to mhddfs, unionfs, and aufs. Like mhddfs in that it too uses FUSE. Like aufs in that it provides multiple policies for how to handle behavior.
* Read/write overlay on top of read-only filesystem like OverlayFS
* File whiteout
* RAID like parity calculation (see [SnapRAID](https://www.snapraid.it))
* Redundancy
* Splitting of files across branches
Why create mergerfs when those exist? mhddfs isn't really maintained or flexible. There are also issues with running as root. aufs is more flexible but contains some hard to debug inconsistencies in behavior. Neither support file attributes ([chattr](http://linux.die.net/man/1/chattr)).
# OPTIONS
## How it works
###options###
mergerfs logically merges multiple filesystem paths together. Not
block devices, not filesystem mounts, just paths. It acts as a proxy
to the underlying filesystem paths. Combining the behaviors of some
functions and being a selector for others.
| Option | Default |
|--------|--------|
| search | ff |
| action | ff |
| create | epmfs |
When the contents of a directory are requested mergerfs combines the
list of files from each directory, deduplicating entries, and returns
that list.
###srcpoints###
When a file or directory is created a policy is first run to determine
which branch will be selected for the creation.
For functions which change attributes or remove the file the behavior
may be applied to all instances found.
The way in which mergerfs behaves is controlled by the
[config/options/settings](https://trapexit.github.io/mergerfs/latest/config/options). More
specifically by
[policies](https://trapexit.github.io/mergerfs/latest/config/functions_categories_policies/).
### Visualization
The source points argument is a colon (':') delimited list of paths. To make it simplier to include multiple source points without having to modify your [fstab](http://linux.die.net/man/5/fstab) we also support [globbing](http://linux.die.net/man/7/glob).
```
A + B = C
/disk1 /disk2 /merged
| | |
+-- /dir1 +-- /dir1 +-- /dir1
| | | | | |
| +-- file1 | | | +-- file1
| | +-- file2 | +-- file2
| | +-- file3 | +-- file3
| | |
+-- /dir2 | +-- /dir2
| | | | |
| +-- file4 | | +-- file4
| | |
| +-- /dir3 +-- /dir3
| | | | |
| | +-- file5 | +-- file5
| | |
+-- file6 | +-- file6
+-- file7 +-- file7 +-- file7
$ mergerfs /mnt/disk*:/mnt/cdrom /media/drives
```
The above line will use all points in /mnt prefixed with *disk* and the directory *cdrom*.
## QuickStart
In /etc/fstab it'd look like the following:
https://trapexit.github.io/mergerfs/latest/quickstart/
```
# <file system> <mount point> <type> <options> <dump> <pass>
/mnt/disk*:/mnt/cdrom /media/drives fuse.mergerfs allow_other 0 0
```
**NOTE:** the globbing is done at mount time. If a new directory is added matching the glob after the fact it will not be included.
# POLICIES
Filesystem calls are broken up into 4 functional categories: search, action, create, and none. These categories can be assigned a policy which dictates how [mergerfs](http://github.com/trapexit/mergerfs) behaves while when action on the filesystem. Any policy can be assigned to a category though some aren't terribly practical. For instance: rand (Random) may be useful for **create** but could lead to very odd behavior if used for **search** or **action**. Since the input for any policy is the source mounts and fusepath and the output a vector of targets the choice was made to simplify the implementation and allow a policies usage in any category. **NOTE:** In any policy which can return more than one location (currently only **all**) the first value will be used in **search** and **create** policies since they can only ever act on 1 filepath.
#### Functional classifications ####
| Class | FUSE calls |
|-------|------------|
| search | access, getattr, getxattr, listxattr, open, readlink |
| action | chmod, link, removexattr, rmdir, setxattr, truncate, unlink, utimens |
| create | create, mkdir, mknod |
| none | fallocate, fgetattr, fsync, ftruncate, ioctl, read, readdir, rename, statfs, symlink, write, release |
#### Policy descriptions ####
| Policy | Description |
|--------------|-------------|
| ff (first found) | Given the order the paths were provided at mount time act on the first one found (regardless if stat would return EACCES). |
| ffwp (first found w/ permissions) | Given the order the paths were provided at mount time act on the first one found which you have access (stat does not error with EACCES). |
| newest (newest file) | If multiple files exist return the one with the most recent mtime. |
| all (all files found) | Attempt to apply the call to each file found. If any sub call succeeds the entire operation succeeds and other errors ignored. If all fail then the last error is reported. |
| mfs (most free space) | Assuming the path is found to exist (ENOENT would not be returned) use the drive with the most free space available. |
| epmfs (existing path, most free space) | If the path exists in multiple locations use the one with the most free space. Otherwise fall back to mfs. |
| rand (random) | Pick a destination at random. Again the dirname of the full path must exist somewhere. |
#### statvfs ####
It normalizes the source drives based on the fragment size and sums the number of adjusted blocks and inodes. This means you will see the combined space of all sources. Total, used, and free. The sources however are dedupped based on the drive so multiple points on the same drive will not result in double counting it's space.
**NOTE:** create is really a search for existence and then create. The 'search' policy applies to the first part. If the [dirname](http://linux.die.net/man/3/dirname) of the full path is not found to exist [ENOENT](http://linux.die.net/man/3/errno) is returned.
# BUILDING
* Need to install FUSE development libraries (libfuse-dev).
* Optionally need libattr1 (libattr1-dev).
## Documentation
```
[trapexit:~/dev/mergerfs] $ make help
usage: make
make XATTR_AVAILABLE=0 - to build program without xattrs functionality (auto discovered otherwise)
```
https://trapexit.github.io/mergerfs
# Runtime Settings
#### /.mergerfs pseudo file ####
```
<mountpoint>/.mergerfs
```
## Support
There is a pseudo file available at the mountpoint which allows for the runtime modification of policies. The file will not show up in readdirs but can be stat'ed, read, and writen. Most other calls will fail with EPERM, EINVAL, or whatever may be appropriate for that call. Anything not understood while writing will result in EINVAL otherwise the number of bytes written will be returned.
https://trapexit.github.io/mergerfs/latest/support/
Reading the file will result in a newline delimited list of current settings as followed:
```
[trapexit:/tmp/mount] $ cat .mergerfs
action=ff
create=epmfs
search=ff
```
## Sponsorship and Donations
Writing to the file is buffered and waits till a newline to process. Meaning echo works well.
[https://github.com/trapexit/support](https://github.com/trapexit/support)
```
[trapexit:/tmp/mount] $ echo "search=newest" >> .mergerfs
[trapexit:/tmp/mount] $ cat .mergerfs
action=ff
create=epmfs
search=newest
```
Development and support of a project like mergerfs requires a
significant amount of time and effort. The software is released under
the very liberal [ISC](https://opensource.org/license/isc-license-txt)
license and is therefore free to use for personal or commercial uses.
*NOTE:* offset is not supported and ignored in both read and write. There is also a safety check which limits buffered + incoming length to a max of 1024 bytes.
If you are a non-commercial user and find mergerfs and its support valuable
and would like to support the project financially it would be very
much appreciated.
#### xattrs ####
If you are using mergerfs commercially please consider sponsoring the
project to ensure it continues to be maintained and receive
updates. If custom features are needed feel free to [contact me
directly](mailto:support@spawn.link).
If xattrs has been enabled you can also use [{list,get,set}xattrs](http://linux.die.net/man/2/listxattr) on the pseudo file **.mergerfs** to modify the policies. The keys are **mergerfs.action**, **mergerfs.create**, and **mergerfs.search**.
```
[trapexit:/tmp/mount] $ attr -l .mergerfs
Attribute "mergerfs.action" has a 2 byte value for .mergerfs
Attribute "mergerfs.create" has a 5 byte value for .mergerfs
Attribute "mergerfs.search" has a 2 byte value for .mergerfs
[trapexit:/tmp/mount] $ attr -g mergerfs.action .mergerfs
Attribute "mergerfs.action" had a 2 byte value for .mergerfs:
ff
[trapexit:/tmp/mount] 1 $ attr -s mergerfs.action -V ffwp .mergerfs
Attribute "mergerfs.action" set to a 4 byte value for .mergerfs:
ffwp
```

View file

@ -1,16 +0,0 @@
#!/bin/sh
GIT_BRANCH="${1:-master}"
set -x
podman \
build \
--squash \
--no-cache \
--force-rm \
--build-arg="GIT_REPO=file:///mnt" \
--build-arg="BRANCH=${GIT_BRANCH}" \
-v "${PWD}":/mnt:ro \
-f "buildtools/containerimage/Containerfile" \
--tag "ghcr.io/trapexit/mergerfs:${GIT_BRANCH}" \
buildtools/

View file

@ -1,76 +0,0 @@
#!/bin/sh
set -e
if [ -e /tmp/build-env ]; then
. /tmp/build-env
fi
rpmbuild_flags()
{
case "$(command -v "${CXX:-g++}")" in
/opt/rh/*)
printf '%s' '--undefine _annotated_build'
;;
esac
}
if [ $# -ne 2 ]; then
echo "usage: $0 <branch> <git_repo>"
echo "argv: $@"
echo "argc: $#"
exit 1
fi
GIT_REPO="${1}"
BRANCH="${2}"
BUILDDIR="/tmp/build"
echo "Clone source from ${GIT_REPO} to ${BUILDDIR}"
git \
clone \
--single-branch \
--branch="${BRANCH}" \
"${GIT_REPO}" \
"${BUILDDIR}"
cd "${BUILDDIR}"
git log HEAD^1..
mkdir "/build"
if [ -e /usr/bin/apt-get ]; then
make deb
elif [ -e /usr/bin/dnf ]; then
make RPMBUILD_FLAGS="$(rpmbuild_flags)" rpm
elif [ -e /usr/bin/yum ]; then
make RPMBUILD_FLAGS="$(rpmbuild_flags)" rpm
elif [ -e /sbin/apk ]; then
echo "NOT YET SUPPORTED"
exit 1
elif [ -e /usr/sbin/pkg ]; then
echo "NOT YET SUPPORTED"
exit 1
elif [ -e /usr/bin/pacman ]; then
make tarball
else
echo "NOT YET SUPPORTED"
exit 1
fi
echo "Copy packages to host..."
if [ -e /usr/bin/pacman ]; then
find "${BUILDDIR}" \
-type f \
-name "*.tar.gz" \
-exec cp -v {} /build/ \;
else
find "${BUILDDIR}" \
-type f \
\( -name "*.deb" -or -name "*.rpm" \) \
-not -name "*dbgsym*" \
-not -name "*sym*" \
-not -name "*src.rpm" \
-exec cp -v {} /build/ \;
fi
exit 0

View file

@ -1,178 +0,0 @@
#!/usr/bin/env python3
import subprocess
import sys
import os
import argparse
import time
def build(git_repo,
branch,
containerfile,
pkgdirpath):
timestamp = time.time_ns()
args = ['podman',
'build',
'--pull=always',
'--force-rm']
if os.path.exists(git_repo):
git_repo = os.path.realpath(git_repo)
args += ['-v',f'{git_repo}:/tmp/mergerfs.git:ro:z']
args += ['--build-arg=GIT_REPO=file:///tmp/mergerfs.git']
else:
args += [f'--build-arg=GIT_REPO={git_repo}']
os.makedirs(pkgdirpath,exist_ok=True)
args += ['-v',f'{os.path.realpath(pkgdirpath)}:/pkgs:ro']
args += ['-o',pkgdirpath,
'-f',containerfile,
f'--build-arg=BRANCH={branch}',
f'--build-arg=BUILD_TIMESTAMP={timestamp}',
'buildtools/']
# TODO: Capture output and write to log
print(args)
rv = subprocess.run(args)
report_filepath = os.path.join(pkgdirpath,"build-report.txt")
with open(report_filepath,"a+") as f:
build = os.path.basename(containerfile)
f.write(build + ": ")
f.write(f"branch={branch}; ")
f.write(f"timestamp={timestamp}; ")
f.write("rv=")
if rv.returncode == 0:
f.write("success;")
else:
f.write("fail;")
f.write("\n")
def setup():
# Detect distribution
if os.path.exists('/usr/bin/apt-get'):
args = ['sudo',
'apt-get',
'install',
'-fy',
'qemu-user-static',
'qemu-user-binfmt']
elif os.path.exists('/usr/bin/pacman'):
args = ['sudo',
'pacman',
'-Sy',
'--noconfirm',
'qemu-user-static']
else:
print("Unsupported distribution for setup")
sys.exit(1)
print(args)
subprocess.run(args)
setup_binfmt()
def setup_binfmt():
"""Register qemu-user-static binfmt handlers (needed on Arch, etc.)."""
binfmt_conf = """\
:qemu-aarch64:M::\\x7fELF\\x02\\x01\\x01\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x02\\x00\\xb7\\x00:\\xff\\xff\\xff\\xff\\xff\\xff\\xff\\x00\\xff\\xff\\xff\\xff\\xff\\xff\\xff\\xff\\xfe\\xff\\xff\\xff:/usr/bin/qemu-aarch64-static:F
:qemu-arm:M::\\x7fELF\\x01\\x01\\x01\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x02\\x00\\x28\\x00:\\xff\\xff\\xff\\xff\\xff\\xff\\xff\\x00\\xff\\xff\\xff\\xff\\xff\\xff\\xff\\xff\\xfe\\xff\\xff\\xff:/usr/bin/qemu-arm-static:F
:qemu-armeb:M::\\x7fELF\\x01\\x02\\x01\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x02\\x00\\x28:\\xff\\xff\\xff\\xff\\xff\\xff\\xff\\x00\\xff\\xff\\xff\\xff\\xff\\xff\\xff\\xff\\xff\\xfe\\xff\\xff:/usr/bin/qemu-armeb-static:F
:qemu-riscv64:M::\\x7fELF\\x02\\x01\\x01\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x02\\x00\\xf3\\x00:\\xff\\xff\\xff\\xff\\xff\\xff\\xff\\x00\\xff\\xff\\xff\\xff\\xff\\xff\\xff\\xff\\xfe\\xff\\xff\\xff:/usr/bin/qemu-riscv64-static:F
:qemu-i386:M::\\x7fELF\\x01\\x01\\x01\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x02\\x00\\x03\\x00:\\xff\\xff\\xff\\xff\\xff\\xff\\xff\\x00\\xff\\xff\\xff\\xff\\xff\\xff\\xff\\xff\\xfe\\xff\\xff\\xff:/usr/bin/qemu-i386-static:F
"""
conf_path = '/etc/binfmt.d/qemu-static.conf'
print(f"Writing {conf_path}")
try:
with open(conf_path, 'w') as f:
f.write(binfmt_conf)
except PermissionError:
print(f"Permission denied writing {conf_path}; trying with sudo...")
subprocess.run(['sudo', 'tee', conf_path],
input=binfmt_conf.encode(), check=True)
print("Restarting systemd-binfmt...")
subprocess.run(['sudo', 'systemctl', 'restart', 'systemd-binfmt'])
def podman_cleanup():
args = ['podman',
'system',
'prune',
'-af']
print(args)
subprocess.run(args)
def parse_args():
p = argparse.ArgumentParser()
p.add_argument('--target',
default='debian:13.amd64')
p.add_argument('--pkgdirpath',
default='build/pkgs/')
p.add_argument('--branch',
default='master')
p.add_argument('--setup',
required=False,
action='store_true')
p.add_argument('--cleanup',
required=False,
action='store_true')
p.add_argument('--git-repo',
required=False,
type=str)
return p.parse_args()
def should_skip(filename):
if filename.endswith('~'):
return True
return False
def main():
args = parse_args()
print(args)
if args.setup:
setup()
sys.exit(0)
if not args.git_repo:
print("error: the following arguments are required: --git-repo")
sys.exit(1)
containerfiles = []
basepath = 'buildtools/containerfiles'
if os.path.exists(f'{basepath}/{args.target}'):
containerfiles.append(f'{basepath}/{args.target}')
elif args.target == 'all':
for root,dirnames,filenames in os.walk(basepath):
filenames.sort()
for filename in filenames:
if should_skip(filename):
continue
containerfile = f'buildtools/containerfiles/{filename}'
containerfiles.append(containerfile)
else:
for root,dirnames,filenames in os.walk(basepath):
filenames.sort()
for filename in filenames:
if args.target not in filename:
continue
if should_skip(filename):
continue
containerfile = f'buildtools/containerfiles/{filename}'
containerfiles.append(containerfile)
for containerfile in containerfiles:
if args.cleanup:
podman_cleanup()
build(git_repo=args.git_repo,
branch=args.branch,
containerfile=containerfile,
pkgdirpath=args.pkgdirpath)
sys.exit(0)
if __name__ == '__main__':
main()

View file

@ -1,10 +0,0 @@
FROM --platform=amd64 debian:11 as build
COPY install-build-pkgs build-mergerfs /tmp/
RUN /tmp/install-build-pkgs
ARG BUILD_TIMESTAMP=0
ARG GIT_REPO
ARG BRANCH
RUN /tmp/build-mergerfs $GIT_REPO $BRANCH
FROM scratch
COPY --from=build /build/ /

View file

@ -1,10 +0,0 @@
FROM --platform=arm64 debian:11 as build
COPY install-build-pkgs build-mergerfs /tmp/
RUN /tmp/install-build-pkgs
ARG BUILD_TIMESTAMP=0
ARG GIT_REPO
ARG BRANCH
RUN /tmp/build-mergerfs $GIT_REPO $BRANCH
FROM scratch
COPY --from=build /build/ /

View file

@ -1,10 +0,0 @@
FROM --platform=armhf debian:11 as build
COPY install-build-pkgs build-mergerfs /tmp/
RUN /tmp/install-build-pkgs
ARG BUILD_TIMESTAMP=0
ARG GIT_REPO
ARG BRANCH
RUN /tmp/build-mergerfs $GIT_REPO $BRANCH
FROM scratch
COPY --from=build /build/ /

View file

@ -1,10 +0,0 @@
FROM --platform=i386 debian:11 as build
COPY install-build-pkgs build-mergerfs /tmp/
RUN /tmp/install-build-pkgs
ARG BUILD_TIMESTAMP=0
ARG GIT_REPO
ARG BRANCH
RUN /tmp/build-mergerfs $GIT_REPO $BRANCH
FROM scratch
COPY --from=build /build/ /

View file

@ -1,10 +0,0 @@
FROM --platform=amd64 debian:12 as build
COPY install-build-pkgs build-mergerfs /tmp/
RUN /tmp/install-build-pkgs
ARG BUILD_TIMESTAMP=0
ARG GIT_REPO
ARG BRANCH
RUN /tmp/build-mergerfs $GIT_REPO $BRANCH
FROM scratch
COPY --from=build /build/ /

View file

@ -1,10 +0,0 @@
FROM --platform=arm64 debian:12 as build
COPY install-build-pkgs build-mergerfs /tmp/
RUN /tmp/install-build-pkgs
ARG BUILD_TIMESTAMP=0
ARG GIT_REPO
ARG BRANCH
RUN /tmp/build-mergerfs $GIT_REPO $BRANCH
FROM scratch
COPY --from=build /build/ /

View file

@ -1,10 +0,0 @@
FROM --platform=armhf debian:12 as build
COPY install-build-pkgs build-mergerfs /tmp/
RUN /tmp/install-build-pkgs
ARG BUILD_TIMESTAMP=0
ARG GIT_REPO
ARG BRANCH
RUN /tmp/build-mergerfs $GIT_REPO $BRANCH
FROM scratch
COPY --from=build /build/ /

View file

@ -1,10 +0,0 @@
FROM --platform=i386 debian:12 as build
COPY install-build-pkgs build-mergerfs /tmp/
RUN /tmp/install-build-pkgs
ARG BUILD_TIMESTAMP=0
ARG GIT_REPO
ARG BRANCH
RUN /tmp/build-mergerfs $GIT_REPO $BRANCH
FROM scratch
COPY --from=build /build/ /

View file

@ -1,10 +0,0 @@
FROM --platform=amd64 debian:13 as build
COPY install-build-pkgs build-mergerfs /tmp/
RUN /tmp/install-build-pkgs
ARG BUILD_TIMESTAMP=0
ARG GIT_REPO
ARG BRANCH
RUN /tmp/build-mergerfs $GIT_REPO $BRANCH
FROM scratch
COPY --from=build /build/ /

View file

@ -1,10 +0,0 @@
FROM --platform=arm64 debian:13 as build
COPY install-build-pkgs build-mergerfs /tmp/
RUN /tmp/install-build-pkgs
ARG BUILD_TIMESTAMP=0
ARG GIT_REPO
ARG BRANCH
RUN /tmp/build-mergerfs $GIT_REPO $BRANCH
FROM scratch
COPY --from=build /build/ /

View file

@ -1,10 +0,0 @@
FROM --platform=armhf debian:13 as build
COPY install-build-pkgs build-mergerfs /tmp/
RUN /tmp/install-build-pkgs
ARG BUILD_TIMESTAMP=0
ARG GIT_REPO
ARG BRANCH
RUN /tmp/build-mergerfs $GIT_REPO $BRANCH
FROM scratch
COPY --from=build /build/ /

View file

@ -1,10 +0,0 @@
FROM --platform=i386 debian:13 as build
COPY install-build-pkgs build-mergerfs /tmp/
RUN /tmp/install-build-pkgs
ARG BUILD_TIMESTAMP=0
ARG GIT_REPO
ARG BRANCH
RUN /tmp/build-mergerfs $GIT_REPO $BRANCH
FROM scratch
COPY --from=build /build/ /

View file

@ -1,10 +0,0 @@
FROM --platform=riscv64 debian:13 as build
COPY install-build-pkgs build-mergerfs /tmp/
RUN /tmp/install-build-pkgs
ARG BUILD_TIMESTAMP=0
ARG GIT_REPO
ARG BRANCH
RUN /tmp/build-mergerfs $GIT_REPO $BRANCH
FROM scratch
COPY --from=build /build/ /

View file

@ -1,10 +0,0 @@
FROM --platform=amd64 fedora:40 as build
COPY install-build-pkgs build-mergerfs /tmp/
RUN /tmp/install-build-pkgs
ARG BUILD_TIMESTAMP=0
ARG GIT_REPO
ARG BRANCH
RUN /tmp/build-mergerfs $GIT_REPO $BRANCH
FROM scratch
COPY --from=build /build/ /

View file

@ -1,10 +0,0 @@
FROM --platform=arm64 fedora:40 as build
COPY install-build-pkgs build-mergerfs /tmp/
RUN /tmp/install-build-pkgs
ARG BUILD_TIMESTAMP=0
ARG GIT_REPO
ARG BRANCH
RUN /tmp/build-mergerfs $GIT_REPO $BRANCH
FROM scratch
COPY --from=build /build/ /

View file

@ -1,10 +0,0 @@
FROM --platform=amd64 fedora:41 as build
COPY install-build-pkgs build-mergerfs /tmp/
RUN /tmp/install-build-pkgs
ARG BUILD_TIMESTAMP=0
ARG GIT_REPO
ARG BRANCH
RUN /tmp/build-mergerfs $GIT_REPO $BRANCH
FROM scratch
COPY --from=build /build/ /

View file

@ -1,10 +0,0 @@
FROM --platform=arm64 fedora:41 as build
COPY install-build-pkgs build-mergerfs /tmp/
RUN /tmp/install-build-pkgs
ARG BUILD_TIMESTAMP=0
ARG GIT_REPO
ARG BRANCH
RUN /tmp/build-mergerfs $GIT_REPO $BRANCH
FROM scratch
COPY --from=build /build/ /

View file

@ -1,10 +0,0 @@
FROM --platform=amd64 fedora:42 as build
COPY install-build-pkgs build-mergerfs /tmp/
RUN /tmp/install-build-pkgs
ARG BUILD_TIMESTAMP=0
ARG GIT_REPO
ARG BRANCH
RUN /tmp/build-mergerfs $GIT_REPO $BRANCH
FROM scratch
COPY --from=build /build/ /

View file

@ -1,10 +0,0 @@
FROM --platform=arm64 fedora:42 as build
COPY install-build-pkgs build-mergerfs /tmp/
RUN /tmp/install-build-pkgs
ARG BUILD_TIMESTAMP=0
ARG GIT_REPO
ARG BRANCH
RUN /tmp/build-mergerfs $GIT_REPO $BRANCH
FROM scratch
COPY --from=build /build/ /

View file

@ -1,10 +0,0 @@
FROM --platform=amd64 fedora:43 as build
COPY install-build-pkgs build-mergerfs /tmp/
RUN /tmp/install-build-pkgs
ARG BUILD_TIMESTAMP=0
ARG GIT_REPO
ARG BRANCH
RUN /tmp/build-mergerfs $GIT_REPO $BRANCH
FROM scratch
COPY --from=build /build/ /

View file

@ -1,10 +0,0 @@
FROM --platform=arm64 fedora:43 as build
COPY install-build-pkgs build-mergerfs /tmp/
RUN /tmp/install-build-pkgs
ARG BUILD_TIMESTAMP=0
ARG GIT_REPO
ARG BRANCH
RUN /tmp/build-mergerfs $GIT_REPO $BRANCH
FROM scratch
COPY --from=build /build/ /

View file

@ -1,10 +0,0 @@
FROM --platform=amd64 fedora:44 as build
COPY install-build-pkgs build-mergerfs /tmp/
RUN /tmp/install-build-pkgs
ARG BUILD_TIMESTAMP=0
ARG GIT_REPO
ARG BRANCH
RUN /tmp/build-mergerfs $GIT_REPO $BRANCH
FROM scratch
COPY --from=build /build/ /

View file

@ -1,10 +0,0 @@
FROM --platform=arm64 fedora:44 as build
COPY install-build-pkgs build-mergerfs /tmp/
RUN /tmp/install-build-pkgs
ARG BUILD_TIMESTAMP=0
ARG GIT_REPO
ARG BRANCH
RUN /tmp/build-mergerfs $GIT_REPO $BRANCH
FROM scratch
COPY --from=build /build/ /

View file

@ -1,10 +0,0 @@
FROM --platform=amd64 quay.io/rockylinux/rockylinux:10 as build
COPY install-build-pkgs build-mergerfs /tmp/
RUN /tmp/install-build-pkgs
ARG BUILD_TIMESTAMP=0
ARG GIT_REPO
ARG BRANCH
RUN /tmp/build-mergerfs $GIT_REPO $BRANCH
FROM scratch
COPY --from=build /build/ /

View file

@ -1,10 +0,0 @@
FROM --platform=arm64 quay.io/rockylinux/rockylinux:10 as build
COPY install-build-pkgs build-mergerfs /tmp/
RUN /tmp/install-build-pkgs
ARG BUILD_TIMESTAMP=0
ARG GIT_REPO
ARG BRANCH
RUN /tmp/build-mergerfs $GIT_REPO $BRANCH
FROM scratch
COPY --from=build /build/ /

View file

@ -1,10 +0,0 @@
FROM --platform=amd64 quay.io/rockylinux/rockylinux:8 as build
COPY install-build-pkgs build-mergerfs /tmp/
RUN /tmp/install-build-pkgs
ARG BUILD_TIMESTAMP=0
ARG GIT_REPO
ARG BRANCH
RUN /tmp/build-mergerfs $GIT_REPO $BRANCH
FROM scratch
COPY --from=build /build/ /

View file

@ -1,10 +0,0 @@
FROM --platform=arm64 quay.io/rockylinux/rockylinux:8 as build
COPY install-build-pkgs build-mergerfs /tmp/
RUN /tmp/install-build-pkgs
ARG BUILD_TIMESTAMP=0
ARG GIT_REPO
ARG BRANCH
RUN /tmp/build-mergerfs $GIT_REPO $BRANCH
FROM scratch
COPY --from=build /build/ /

View file

@ -1,10 +0,0 @@
FROM --platform=amd64 quay.io/rockylinux/rockylinux:9 as build
COPY install-build-pkgs build-mergerfs /tmp/
RUN /tmp/install-build-pkgs
ARG BUILD_TIMESTAMP=0
ARG GIT_REPO
ARG BRANCH
RUN /tmp/build-mergerfs $GIT_REPO $BRANCH
FROM scratch
COPY --from=build /build/ /

View file

@ -1,10 +0,0 @@
FROM --platform=arm64 quay.io/rockylinux/rockylinux:9 as build
COPY install-build-pkgs build-mergerfs /tmp/
RUN /tmp/install-build-pkgs
ARG BUILD_TIMESTAMP=0
ARG GIT_REPO
ARG BRANCH
RUN /tmp/build-mergerfs $GIT_REPO $BRANCH
FROM scratch
COPY --from=build /build/ /

View file

@ -1,15 +0,0 @@
FROM --platform=linux/amd64 alpine:latest as build
COPY install-build-pkgs /tmp/
RUN /tmp/install-build-pkgs
ARG BUILD_TIMESTAMP=0
ARG GIT_REPO
ARG BRANCH
RUN git clone --single-branch --branch="${BRANCH}" ${GIT_REPO} /tmp/mergerfs
WORKDIR /tmp/mergerfs
RUN make NDEBUG=1 LTO=1 STATIC=1 DESTDIR=/tmp -j$(nproc) install
RUN mkdir /build
RUN /tmp/usr/local/bin/mergerfs --version | head -n1 | cut -c 11- | tee /tmp/version
RUN tar cvfz /build/mergerfs-$(cat /tmp/version)-static-linux_amd64.tar.gz --directory=/tmp usr sbin
FROM scratch
COPY --from=build /build/ /

View file

@ -1,15 +0,0 @@
FROM --platform=linux/arm64 alpine:latest as build
COPY install-build-pkgs /tmp/
RUN /tmp/install-build-pkgs
ARG BUILD_TIMESTAMP=0
ARG GIT_REPO
ARG BRANCH
RUN git clone --single-branch --branch="${BRANCH}" ${GIT_REPO} /tmp/mergerfs
WORKDIR /tmp/mergerfs
RUN make NDEBUG=1 LTO=1 STATIC=1 DESTDIR=/tmp -j$(nproc) install
RUN mkdir /build
RUN /tmp/usr/local/bin/mergerfs --version | head -n1 | cut -c 11- | tee /tmp/version
RUN tar cvfz /build/mergerfs-$(cat /tmp/version)-static-linux_arm64.tar.gz --directory=/tmp usr sbin
FROM scratch
COPY --from=build /build/ /

View file

@ -1,15 +0,0 @@
FROM --platform=linux/armhf alpine:latest as build
COPY install-build-pkgs /tmp/
RUN /tmp/install-build-pkgs
ARG BUILD_TIMESTAMP=0
ARG GIT_REPO
ARG BRANCH
RUN git clone --single-branch --branch="${BRANCH}" ${GIT_REPO} /tmp/mergerfs
WORKDIR /tmp/mergerfs
RUN make NDEBUG=1 LTO=1 STATIC=1 DESTDIR=/tmp -j$(nproc) install
RUN mkdir /build
RUN /tmp/usr/local/bin/mergerfs --version | head -n1 | cut -c 11- | tee /tmp/version
RUN tar cvfz /build/mergerfs-$(cat /tmp/version)-static-linux_armhf.tar.gz --directory=/tmp usr sbin
FROM scratch
COPY --from=build /build/ /

View file

@ -1,15 +0,0 @@
FROM --platform=linux/i386 alpine:latest as build
COPY install-build-pkgs /tmp/
RUN /tmp/install-build-pkgs
ARG BUILD_TIMESTAMP=0
ARG GIT_REPO
ARG BRANCH
RUN git clone --single-branch --branch="${BRANCH}" ${GIT_REPO} /tmp/mergerfs
WORKDIR /tmp/mergerfs
RUN make NDEBUG=1 LTO=1 STATIC=1 DESTDIR=/tmp -j$(nproc) install
RUN mkdir /build
RUN /tmp/usr/local/bin/mergerfs --version | head -n1 | cut -c 11- | tee /tmp/version
RUN tar cvfz /build/mergerfs-$(cat /tmp/version)-static-linux_i386.tar.gz --directory=/tmp usr sbin
FROM scratch
COPY --from=build /build/ /

View file

@ -1,14 +0,0 @@
ARG BUILD_TIMESTAMP=0
FROM --platform=linux/amd64 ubuntu:latest as build
COPY install-build-pkgs /tmp/
RUN /tmp/install-build-pkgs && \
DEBIAN_FRONTEND=noninteractive apt-get -qy --no-install-suggests --no-install-recommends install \
makeself
RUN mkdir -p /tmp/mergerfs/
RUN ls -lhd /pkgs
RUN cp -v /pkgs/*static*.gz /tmp/mergerfs/
RUN mkdir -p /build
RUN makeself /tmp/mergerfs/ /build/mergerfs-installer.run "mergerfs" echo "foo"
FROM scratch
COPY --from=build /build/ /

View file

@ -1,15 +0,0 @@
FROM --platform=linux/riscv64 alpine:latest as build
COPY install-build-pkgs /tmp/
RUN /tmp/install-build-pkgs
ARG BUILD_TIMESTAMP=0
ARG GIT_REPO
ARG BRANCH
RUN git clone --single-branch --branch="${BRANCH}" ${GIT_REPO} /tmp/mergerfs
WORKDIR /tmp/mergerfs
RUN make NDEBUG=1 LTO=1 STATIC=1 DESTDIR=/tmp -j$(nproc) install
RUN mkdir /build
RUN /tmp/usr/local/bin/mergerfs --version | head -n1 | cut -c 11- | tee /tmp/version
RUN tar cvfz /build/mergerfs-$(cat /tmp/version)-static-linux_riscv64.tar.gz --directory=/tmp usr sbin
FROM scratch
COPY --from=build /build/ /

View file

@ -1,17 +0,0 @@
FROM alpine:latest as tarball
COPY install-build-pkgs /tmp/
RUN /tmp/install-build-pkgs
ARG BUILD_TIMESTAMP=0
ARG BRANCH=master
ARG GIT_REPO
RUN git clone --single-branch "${GIT_REPO}" /tmp/mergerfs --branch="${BRANCH}"
RUN cd /tmp/mergerfs && make version tarball
RUN mkdir /build
RUN cp -v /tmp/mergerfs/*.tar.gz /build/
FROM scratch
COPY --from=tarball /build/ /

View file

@ -1,10 +0,0 @@
FROM --platform=amd64 ubuntu:22.04 as build
COPY install-build-pkgs build-mergerfs /tmp/
RUN /tmp/install-build-pkgs
ARG BUILD_TIMESTAMP=0
ARG GIT_REPO
ARG BRANCH
RUN /tmp/build-mergerfs $GIT_REPO $BRANCH
FROM scratch
COPY --from=build /build/ /

View file

@ -1,10 +0,0 @@
FROM --platform=arm64 ubuntu:22.04 as build
COPY install-build-pkgs build-mergerfs /tmp/
RUN /tmp/install-build-pkgs
ARG BUILD_TIMESTAMP=0
ARG GIT_REPO
ARG BRANCH
RUN /tmp/build-mergerfs $GIT_REPO $BRANCH
FROM scratch
COPY --from=build /build/ /

View file

@ -1,10 +0,0 @@
FROM --platform=armhf ubuntu:22.04 as build
COPY install-build-pkgs build-mergerfs /tmp/
RUN /tmp/install-build-pkgs
ARG BUILD_TIMESTAMP=0
ARG GIT_REPO
ARG BRANCH
RUN /tmp/build-mergerfs $GIT_REPO $BRANCH
FROM scratch
COPY --from=build /build/ /

View file

@ -1,10 +0,0 @@
FROM --platform=riscv64 ubuntu:22.04 as build
COPY install-build-pkgs build-mergerfs /tmp/
RUN /tmp/install-build-pkgs
ARG BUILD_TIMESTAMP=0
ARG GIT_REPO
ARG BRANCH
RUN /tmp/build-mergerfs $GIT_REPO $BRANCH
FROM scratch
COPY --from=build /build/ /

View file

@ -1,10 +0,0 @@
FROM --platform=amd64 ubuntu:24.04 as build
COPY install-build-pkgs build-mergerfs /tmp/
RUN /tmp/install-build-pkgs
ARG BUILD_TIMESTAMP=0
ARG GIT_REPO
ARG BRANCH
RUN /tmp/build-mergerfs $GIT_REPO $BRANCH
FROM scratch
COPY --from=build /build/ /

View file

@ -1,10 +0,0 @@
FROM --platform=arm64 ubuntu:24.04 as build
COPY install-build-pkgs build-mergerfs /tmp/
RUN /tmp/install-build-pkgs
ARG BUILD_TIMESTAMP=0
ARG GIT_REPO
ARG BRANCH
RUN /tmp/build-mergerfs $GIT_REPO $BRANCH
FROM scratch
COPY --from=build /build/ /

View file

@ -1,10 +0,0 @@
FROM --platform=armhf ubuntu:24.04 as build
COPY install-build-pkgs build-mergerfs /tmp/
RUN /tmp/install-build-pkgs
ARG BUILD_TIMESTAMP=0
ARG GIT_REPO
ARG BRANCH
RUN /tmp/build-mergerfs $GIT_REPO $BRANCH
FROM scratch
COPY --from=build /build/ /

View file

@ -1,10 +0,0 @@
FROM --platform=riscv64 ubuntu:24.04 as build
COPY install-build-pkgs build-mergerfs /tmp/
RUN /tmp/install-build-pkgs
ARG BUILD_TIMESTAMP=0
ARG GIT_REPO
ARG BRANCH
RUN /tmp/build-mergerfs $GIT_REPO $BRANCH
FROM scratch
COPY --from=build /build/ /

View file

@ -1,10 +0,0 @@
FROM --platform=amd64 ubuntu:25.04 as build
COPY install-build-pkgs build-mergerfs /tmp/
RUN /tmp/install-build-pkgs
ARG BUILD_TIMESTAMP=0
ARG GIT_REPO
ARG BRANCH
RUN /tmp/build-mergerfs $GIT_REPO $BRANCH
FROM scratch
COPY --from=build /build/ /

View file

@ -1,10 +0,0 @@
FROM --platform=arm64 ubuntu:25.04 as build
COPY install-build-pkgs build-mergerfs /tmp/
RUN /tmp/install-build-pkgs
ARG BUILD_TIMESTAMP=0
ARG GIT_REPO
ARG BRANCH
RUN /tmp/build-mergerfs $GIT_REPO $BRANCH
FROM scratch
COPY --from=build /build/ /

View file

@ -1,10 +0,0 @@
FROM --platform=armhf ubuntu:25.04 as build
COPY install-build-pkgs build-mergerfs /tmp/
RUN /tmp/install-build-pkgs
ARG BUILD_TIMESTAMP=0
ARG GIT_REPO
ARG BRANCH
RUN /tmp/build-mergerfs $GIT_REPO $BRANCH
FROM scratch
COPY --from=build /build/ /

View file

@ -1,10 +0,0 @@
FROM --platform=riscv64 ubuntu:25.04 as build
COPY install-build-pkgs build-mergerfs /tmp/
RUN /tmp/install-build-pkgs
ARG BUILD_TIMESTAMP=0
ARG GIT_REPO
ARG BRANCH
RUN /tmp/build-mergerfs $GIT_REPO $BRANCH
FROM scratch
COPY --from=build /build/ /

View file

@ -1,10 +0,0 @@
FROM --platform=amd64 ubuntu:25.10 as build
COPY install-build-pkgs build-mergerfs /tmp/
RUN /tmp/install-build-pkgs
ARG BUILD_TIMESTAMP=0
ARG GIT_REPO
ARG BRANCH
RUN /tmp/build-mergerfs $GIT_REPO $BRANCH
FROM scratch
COPY --from=build /build/ /

View file

@ -1,10 +0,0 @@
FROM --platform=arm64 ubuntu:25.10 as build
COPY install-build-pkgs build-mergerfs /tmp/
RUN /tmp/install-build-pkgs
ARG BUILD_TIMESTAMP=0
ARG GIT_REPO
ARG BRANCH
RUN /tmp/build-mergerfs $GIT_REPO $BRANCH
FROM scratch
COPY --from=build /build/ /

View file

@ -1,10 +0,0 @@
FROM --platform=armhf ubuntu:25.10 as build
COPY install-build-pkgs build-mergerfs /tmp/
RUN /tmp/install-build-pkgs
ARG BUILD_TIMESTAMP=0
ARG GIT_REPO
ARG BRANCH
RUN /tmp/build-mergerfs $GIT_REPO $BRANCH
FROM scratch
COPY --from=build /build/ /

View file

@ -1,10 +0,0 @@
FROM --platform=riscv64 ubuntu:25.10 as build
COPY install-build-pkgs build-mergerfs /tmp/
RUN /tmp/install-build-pkgs
ARG BUILD_TIMESTAMP=0
ARG GIT_REPO
ARG BRANCH
RUN /tmp/build-mergerfs $GIT_REPO $BRANCH
FROM scratch
COPY --from=build /build/ /

View file

@ -1,10 +0,0 @@
FROM --platform=amd64 ubuntu:26.04 as build
COPY install-build-pkgs build-mergerfs /tmp/
RUN /tmp/install-build-pkgs
ARG BUILD_TIMESTAMP=0
ARG GIT_REPO
ARG BRANCH
RUN /tmp/build-mergerfs $GIT_REPO $BRANCH
FROM scratch
COPY --from=build /build/ /

View file

@ -1,10 +0,0 @@
FROM --platform=arm64 ubuntu:26.04 as build
COPY install-build-pkgs build-mergerfs /tmp/
RUN /tmp/install-build-pkgs
ARG BUILD_TIMESTAMP=0
ARG GIT_REPO
ARG BRANCH
RUN /tmp/build-mergerfs $GIT_REPO $BRANCH
FROM scratch
COPY --from=build /build/ /

View file

@ -1,10 +0,0 @@
FROM --platform=armhf ubuntu:26.04 as build
COPY install-build-pkgs build-mergerfs /tmp/
RUN /tmp/install-build-pkgs
ARG BUILD_TIMESTAMP=0
ARG GIT_REPO
ARG BRANCH
RUN /tmp/build-mergerfs $GIT_REPO $BRANCH
FROM scratch
COPY --from=build /build/ /

View file

@ -1,10 +0,0 @@
FROM --platform=riscv64 ubuntu:26.04 as build
COPY install-build-pkgs build-mergerfs /tmp/
RUN /tmp/install-build-pkgs
ARG BUILD_TIMESTAMP=0
ARG GIT_REPO
ARG BRANCH
RUN /tmp/build-mergerfs $GIT_REPO $BRANCH
FROM scratch
COPY --from=build /build/ /

View file

@ -1,19 +0,0 @@
FROM --platform=linux/amd64 alpine:latest as build
ARG GIT_REPO
ARG BRANCH
RUN <<EOF
apk add git gcc g++ make linux-headers
git clone --single-branch --branch="${BRANCH}" "${GIT_REPO}" /tmp/mergerfs
cd /tmp/mergerfs
make NDEBUG=1 LTO=1 STATIC=1 DESTDIR=/tmp/install -j$(nproc) install
tar cvfz /tmp/mergerfs.tgz --directory=/tmp/install usr sbin
EOF
FROM --platform=linux/amd64 alpine:latest
RUN --mount=from=build,source=/tmp/mergerfs.tgz,target=/tmp/mergerfs.tgz <<EOF
tar xvf /tmp/mergerfs.tgz -C /
EOF
ENTRYPOINT ["/usr/local/bin/mergerfs"]
CMD ["--help"]

View file

@ -1,14 +0,0 @@
#!/bin/sh
set -x
BASEPATH="/tmp"
for x in $(seq -w 2)
do
FILEPATH="${BASEPATH}/mergerfs-${x}.img"
MOUNTPOINT="${BASEPATH}/mergerfs-${x}"
truncate -s 1G "${FILEPATH}"
mkdir -p "${MOUNTPOINT}"
mkfs.ext4 -m0 -L "mergerfs${x}" "${FILEPATH}"
sudo mount "${FILEPATH}" "${MOUNTPOINT}"
done

View file

@ -1,119 +0,0 @@
#!/bin/bash
# Detects local OS and maps to containerfiles target
# Returns target if found in buildtools/containerfiles/, empty string otherwise
set -euo pipefail
CONTAINERFILES_DIR="buildtools/containerfiles"
# Detect system architecture and map to containerfiles format
detect_arch() {
local arch
arch=$(uname -m)
case "$arch" in
x86_64) echo "amd64" ;;
aarch64) echo "arm64" ;;
armv7l) echo "armhf" ;;
i686|i386) echo "i386" ;;
riscv64) echo "riscv64" ;;
*) echo "" ;;
esac
}
# Detect distro and version from os-release
detect_distro() {
local os_release="${1:-}"
if [[ ! -r "$os_release" ]]; then
return 1
fi
# Source the file safely - ID and VERSION_ID are environment variables
. "$os_release"
local distro="$ID"
local version="$VERSION_ID"
# Map distro ID to containerfiles naming convention
case "$distro" in
ubuntu|debian|fedora) ;;
rocky|rhel|centos|almalinux) distro="rockylinux" ;;
*)
# Fallback to ID_LIKE for derivatives
if [[ -n "${ID_LIKE:-}" ]]; then
case "$ID_LIKE" in
*ubuntu*) distro="ubuntu" ;;
*debian*) distro="debian" ;;
*fedora*) distro="fedora" ;;
esac
fi
;;
esac
# Only return if we have valid distro and version
if [[ -n "$distro" && -n "$version" ]]; then
echo "$distro:$version"
else
return 1
fi
}
# Try to find a matching containerfile target
find_containerfile() {
local target="$1"
local containerfile="${CONTAINERFILES_DIR}/${target}"
# Direct match
if [[ -d "$containerfile" || -f "$containerfile" ]]; then
echo "$target"
return 0
fi
return 1
}
# Main detection logic
main() {
local arch
# Detect architecture
arch=$(detect_arch)
if [[ -z "$arch" ]]; then
echo "WARNING: Could not detect system architecture" >&2
exit 1
fi
# Try to detect distro from multiple locations
local distro_ver
distro_ver=$(detect_distro "/etc/os-release") || \
distro_ver=$(detect_distro "/usr/lib/os-release") || {
echo "WARNING: Could not detect distro from os-release" >&2
exit 1
}
# Build target string
local target="${distro_ver}.${arch}"
# Try to find matching containerfile
if find_containerfile "$target"; then
exit 0
fi
# Try with partial version matching (e.g., 25.10 → 25)
local distro="${distro_ver%%:*}"
local version="${distro_ver##*:}"
# Strip last minor version number and try again
local short_version="${version%.*}"
if [[ -n "$short_version" ]]; then
target="${distro}:${short_version}.${arch}"
if find_containerfile "$target"; then
exit 0
fi
fi
# No match found
echo "WARNING: No containerfile found for target: ${distro_ver}.${arch}" >&2
exit 1
}
main "$@"

View file

@ -1,27 +0,0 @@
#!/bin/sh
input_file="${1}"
while IFS= read -r line || [ -n "$line" ]; do
# Skip empty lines
if [ -z "$line" ]; then
continue
fi
image=$(echo "$line" | cut -d' ' -f1)
platform=$(echo "$line" | cut -d' ' -f2-)
echo "${image} ${platform}"
cat <<EOF > "containerfiles/${image##*/}.${platform}"
FROM --platform=${platform} ${image} as build
COPY install-build-pkgs build-mergerfs /tmp/
RUN /tmp/install-build-pkgs
ARG BUILD_TIMESTAMP=0
ARG GIT_REPO
ARG BRANCH
RUN /tmp/build-mergerfs \$GIT_REPO \$BRANCH
FROM scratch
COPY --from=build /build/ /
EOF
done < "$input_file"

View file

@ -1,202 +0,0 @@
#!/bin/sh
# Copyright (c) 2016, Antonio SJ Musumeci <trapexit@spawn.link>
# Permission to use, copy, modify, and/or distribute this software for any
# purpose with or without fee is hereby granted, provided that the above
# copyright notice and this permission notice appear in all copies.
# THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES
# WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF
# MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR
# ANY SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES
# WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN
# ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF
# OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE.
# Default values
NAME=""
VERSION="::guess::"
DISTRO="::guess::"
CODENAME="::guess::"
URGENCY="medium"
# Function to display usage
usage() {
cat << EOF
Usage: $0 --name NAME [OPTIONS]
Generated debian/changelog from git log
Options:
--name NAME Name of package (required)
--version VERSION Place in git history to include upto (default: ::guess::)
--distro DISTRO Distribution name (default: ::guess::)
--codename NAME Distribution codename (default: ::guess::)
--urgency URGENCY Urgency (default: medium)
-h, --help Show this help message
EOF
exit 1
}
# Parse command line arguments
while [ $# -gt 0 ]; do
case "$1" in
--name)
NAME="$2"
shift 2
;;
--version)
VERSION="$2"
shift 2
;;
--distro)
DISTRO="$2"
shift 2
;;
--codename)
CODENAME="$2"
shift 2
;;
--urgency)
URGENCY="$2"
shift 2
;;
-h|--help)
usage
;;
*)
echo "Unknown option: $1" >&2
usage
;;
esac
done
# Check required arguments
if [ -z "$NAME" ]; then
echo "Error: --name is required" >&2
usage
fi
# Function to get git tags sorted in reverse order
git_tags() {
git tag -l | sort -rV
}
# Function to get git log between two tags
git_log() {
local fromtag="$1"
local totag="$2"
git log --no-merges --oneline "${fromtag}...${totag}" 2>/dev/null
}
# Function to get author and time for a tag
git_author_and_time() {
local tag="$1"
git log -1 --format="-- %an <%ae> %cD" "$tag"
}
# Function to get git version
git_version() {
git describe --always --tags --dirty 2>/dev/null || echo "unknown"
}
# Function to guess distribution
guess_distro() {
if command -v lsb_release >/dev/null 2>&1; then
lsb_release -i -s 2>/dev/null | tr '[:upper:]' '[:lower:]'
else
echo "unknown"
fi
}
# Function to guess codename
guess_codename() {
if command -v lsb_release >/dev/null 2>&1; then
lsb_release -c -s 2>/dev/null | tr '[:upper:]' '[:lower:]'
else
echo "unknown"
fi
}
# Handle guessing of values
if [ "$DISTRO" = "::guess::" ]; then
DISTRO=$(guess_distro)
fi
if [ "$CODENAME" = "::guess::" ]; then
CODENAME=$(guess_codename)
fi
if [ "$VERSION" = "::guess::" ]; then
VERSION=$(git_version)
fi
# Version suffix
VERSUFFIX="~${DISTRO}-${CODENAME}"
# Get all tags
TAGS=$(git_tags)
# Create temporary file for tag pairs
TMPFILE=$(mktemp)
trap 'rm -f "$TMPFILE"' EXIT
# Check if version is in tags
if echo "$TAGS" | grep -q "^${VERSION}$"; then
# If version is a tag, start from that tag
echo "$TAGS" | awk -v ver="$VERSION" 'BEGIN{found=0} {if($0==ver)found=1; if(found)print}' > "$TMPFILE"
else
# If version is not a tag, use HEAD as starting point
echo "$VERSION HEAD" > "$TMPFILE"
echo "$TAGS" >> "$TMPFILE"
fi
# Process tags and generate changelog
PREV_TAG=""
PREV_REF=""
FIRST=1
while IFS= read -r line; do
if [ -z "$line" ]; then
continue
fi
# Handle the special case of VERSION HEAD
if [ "$line" = "$VERSION HEAD" ]; then
TAG="$VERSION"
REF="HEAD"
else
TAG="$line"
REF="$line"
fi
if [ -n "$PREV_TAG" ]; then
# Get log entries between current and previous tag
LOG_ENTRIES=$(git_log "$REF" "$PREV_REF")
# Skip if no changes
if [ -z "$LOG_ENTRIES" ] || [ "$LOG_ENTRIES" = "" ]; then
PREV_TAG="$TAG"
PREV_REF="$REF"
continue
fi
# Print package header
printf "%s (%s%s) %s; urgency=%s\n\n" "$NAME" "$PREV_TAG" "$VERSUFFIX" "$CODENAME" "$URGENCY"
# Print each log entry
echo "$LOG_ENTRIES" | while IFS= read -r entry; do
if [ -n "$entry" ]; then
printf " * %s\n" "$entry"
fi
done
# Print author and time
AUTHOR_TIME=$(git_author_and_time "$PREV_REF")
printf "\n %s\n\n" "$AUTHOR_TIME"
fi
PREV_TAG="$TAG"
PREV_REF="$REF"
done < "$TMPFILE"

View file

@ -1,162 +0,0 @@
#!/bin/sh
set -e
BUILD_ENV=/tmp/build-env
write_gcc_env()
{
{
printf 'export CC=%s\n' "$1"
printf 'export CXX=%s\n' "$2"
} > "$BUILD_ENV"
}
write_toolset_env()
{
{
printf '. %s\n' "$1"
printf 'export CC=gcc\n'
printf 'export CXX=g++\n'
} > "$BUILD_ENV"
}
latest_apt_gxx()
{
apt-cache --names-only search '^g[+][+]-[0-9]' \
| sed -n 's/^\(g++-[0-9][0-9]*\) .*/\1/p' \
| sort -t- -k2,2V \
| tail -n 1
}
latest_dnf_toolset()
{
dnf -q list --showduplicates 'gcc-toolset-*-gcc-c++' 2>/dev/null \
| sed -n 's/^gcc-toolset-\([0-9][0-9]*\)-gcc-c++\..*/\1/p' \
| sort -V \
| tail -n 1
}
latest_yum_toolset()
{
yum -q list available 'devtoolset-*-gcc-c++' 2>/dev/null \
| sed -n 's/^devtoolset-\([0-9][0-9]*\)-gcc-c++\..*/\1/p' \
| sort -V \
| tail -n 1
}
version_gt()
{
highest="$(printf '%s\n%s\n' "$1" "$2" | sort -V | tail -n 1)"
[ "$highest" = "$1" ] && [ "$1" != "$2" ]
}
check_cxx20()
{
cxx="${CXX:-g++}"
test_src=/tmp/mergerfs-cxx20-check.cpp
test_bin=/tmp/mergerfs-cxx20-check
cat > "$test_src" <<'EOF'
template <typename T>
concept Incrementable = requires(T value) { value + 1; };
static_assert(Incrementable<int>);
int main()
{
return 0;
}
EOF
if ! "$cxx" -std=c++20 "$test_src" -o "$test_bin" >/dev/null 2>&1; then
echo "error: $cxx does not support C++20" >&2
exit 1
fi
rm -f "$test_src" "$test_bin"
}
: > "$BUILD_ENV"
if [ -e /usr/bin/apt-get ]; then
export DEBIAN_FRONTEND=noninteractive
apt-get -qy update
cxx_pkg="$(latest_apt_gxx)"
if [ -n "$cxx_pkg" ]; then
gcc_pkg="gcc-${cxx_pkg#g++-}"
compiler_pkgs="$gcc_pkg $cxx_pkg"
write_gcc_env "$gcc_pkg" "$cxx_pkg"
else
compiler_pkgs="g++"
write_gcc_env gcc g++
fi
apt-get -qy --force-yes --no-install-suggests --no-install-recommends \
install \
ca-certificates \
build-essential \
dpkg-dev \
git \
$compiler_pkgs \
debhelper \
lsb-release \
fakeroot
. "$BUILD_ENV"
check_cxx20
elif [ -e /usr/bin/dnf ]; then
dnf -y update
toolset="$(latest_dnf_toolset)"
dnf -y install \
git rpm-build gcc-c++ make which libatomic
default_gxx_major="$(g++ -dumpversion | cut -f1 -d.)"
if [ -n "$toolset" ] && version_gt "$toolset" "$default_gxx_major"; then
dnf -y install \
"gcc-toolset-${toolset}-gcc" \
"gcc-toolset-${toolset}-gcc-c++"
write_toolset_env "/opt/rh/gcc-toolset-${toolset}/enable"
else
write_gcc_env gcc g++
fi
. "$BUILD_ENV"
check_cxx20
elif [ -e /usr/bin/yum ]; then
yum -y update
yum -y install \
git rpm-build gcc-c++ make which
yum -y install centos-release-scl || true
yum -y update
toolset="$(latest_yum_toolset)"
default_gxx_major="$(g++ -dumpversion | cut -f1 -d.)"
if [ -n "$toolset" ] && version_gt "$toolset" "$default_gxx_major"; then
yum -y install \
"devtoolset-${toolset}-gcc" \
"devtoolset-${toolset}-gcc-c++"
write_toolset_env "/opt/rh/devtoolset-${toolset}/enable"
else
write_gcc_env gcc g++
fi
. "$BUILD_ENV"
check_cxx20
elif [ -e /usr/bin/zypper ]; then
zypper update -y
zypper install -y \
git rpm-build gcc-c++ make which
write_gcc_env gcc g++
. "$BUILD_ENV"
check_cxx20
elif [ -e /sbin/apk ]; then
apk add \
abuild git gcc g++ make \
linux-headers
write_gcc_env gcc g++
. "$BUILD_ENV"
check_cxx20
elif [ -e /usr/sbin/pkg ]; then
pkg install \
git gmake gcc
elif [ -e /usr/bin/pacman ]; then
pacman -Sy --noconfirm base-devel git fakeroot
write_gcc_env gcc g++
. "$BUILD_ENV"
check_cxx20
fi

View file

@ -1,32 +0,0 @@
#!/bin/sh
if [ "$(id -u)" -eq 0 ]; then
SUDO=
elif command -v sudo >/dev/null 2>&1; then
SUDO=sudo
elif command -v doas >/dev/null 2>&1; then
SUDO=doas
else
printf '%s\n' "error: need root privileges; run as root or install sudo/doas" >&2
exit 1
fi
if [ -e /usr/bin/apt-get ]; then
export DEBIAN_FRONTEND=noninteractive
${SUDO} apt-get -qy update
${SUDO} apt-get -qy --no-install-suggests --no-install-recommends install \
qemu-user-static \
podman
elif [ -e /usr/bin/dnf ]; then
${SUDO} dnf -y update
${SUDO} dnf -y install \
qemu-user-static \
podman
elif [ -e /sbin/apk ]; then
${SUDO} apk add \
python3 \
qemu-user-static \
podman
elif [ -e /usr/bin/pacman ]; then
${SUDO} pacman -Sy --noconfirm qemu-user-static podman
fi

View file

@ -1,39 +0,0 @@
debian:11 amd64
debian:11 arm64
debian:11 armhf
debian:11 i386
debian:12 amd64
debian:12 arm64
debian:12 armhf
debian:12 i386
debian:13 amd64
debian:13 arm64
debian:13 armhf
debian:13 i386
debian:13 riscv64
fedora:40 amd64
fedora:40 arm64
fedora:41 amd64
fedora:41 arm64
fedora:42 amd64
fedora:42 arm64
fedora:43 amd64
fedora:43 arm64
quay.io/rockylinux/rockylinux:10 amd64
quay.io/rockylinux/rockylinux:10 arm64
quay.io/rockylinux/rockylinux:8 amd64
quay.io/rockylinux/rockylinux:8 arm64
quay.io/rockylinux/rockylinux:9 amd64
quay.io/rockylinux/rockylinux:9 arm64
ubuntu:22.04 amd64
ubuntu:22.04 arm64
ubuntu:22.04 armhf
ubuntu:22.04 riscv64
ubuntu:24.04 amd64
ubuntu:24.04 arm64
ubuntu:24.04 armhf
ubuntu:24.04 riscv64
ubuntu:25.04 amd64
ubuntu:25.04 arm64
ubuntu:25.04 armhf
ubuntu:25.04 riscv64

View file

@ -1,20 +0,0 @@
#!/bin/sh
VERSION=$(git describe --always --tags --dirty)
if [ $? -ne 0 ]; then
VERSION=$(cat VERSION)
fi
if [ "${VERSION}" = "" ]; then
VERSION="unknown"
fi
echo "version = ${VERSION}"
echo -n "${VERSION}" > VERSION
grep -sq \"${VERSION}\" src/version.hpp
RV=$?
if [ $RV -ne 0 ]; then
echo "Writing 'src/version.hpp'"
echo "#pragma once" > src/version.hpp
echo "static const char MERGERFS_VERSION[] = \"${VERSION}\";" >> src/version.hpp
fi

27
debian/changelog vendored Normal file
View file

@ -0,0 +1,27 @@
mergerfs (1.1.0) trusty; urgency=medium
* initial code commit
* fix building without libattr
* update build instructions in readme
* use {get,list,set}xattr to modify runtime
* use long instead of int to limit possibility of overflow in switch, closes #1
* add FS_IOC_{GET}VERSION to ioctl
* remove statfs policy
* fs::make_path should check for forward slashes, add if missing
* remove stat'ing of files in readdir. closes #3
* use eaccess to determine permissions for ffwp. closes #2
* rework policy code
* add fgetattr. closes #5
* when readdir's filler returns non-zero return ENOMEM. closes #7
* enhance Makefile
* further Makefile enhancements
* fix free space calculations. closes #8
* add debian package building
-- Antonio SJ Musumeci <trapexit@spawn.link> Fri, 30 May 2014 21:32:19 -0400
mergerfs (1.0.0) trusty; urgency=medium
* Initial commit
-- Antonio SJ Musumeci <trapexit@spawn.link> Fri, 30 May 2014 21:28:09 -0400

1
debian/compat vendored Normal file
View file

@ -0,0 +1 @@
9

13
debian/control vendored
View file

@ -2,7 +2,10 @@ Source: mergerfs
Section: utils
Priority: optional
Maintainer: Antonio SJ Musumeci <trapexit@spawn.link>
Build-Depends: debhelper-compat (= 10)
Build-Depends: debhelper (>= 8.0.0),
libfuse-dev,
libattr1-dev,
pkg-config
Standards-Version: 3.9.4
Homepage: http://github.com/trapexit/mergerfs
@ -10,8 +13,8 @@ Package: mergerfs
Architecture: any
Depends: ${shlibs:Depends},
${misc:Depends},
fuse [linux-any] | fuse4bsd [kfreebsd-any]
libfuse2
Description: another FUSE union filesystem
mergerfs is a union filesystem geared towards simplifying storage and
management of files across numerous commodity storage devices. It is
similar to mhddfs, unionfs, and aufs.
mergerfs is similar to mhddfs, unionfs, and aufs. Like mhddfs in that it too
uses FUSE. Like aufs in that it provides multiple policies for how to handle
behavior.

36
debian/copyright vendored
View file

@ -4,22 +4,28 @@ Source: http://github.com/trapexit/mergerfs
Files: *
Copyright: 2014 Antonio SJ Musumeci <trapexit@spawn.link>
License: ISC
License: MIT
Files: debian/*
Copyright: 2014 Antonio SJ Musumeci <trapexit@spawn.link>
License: ISC
License: MIT
License: ISC
Permission to use, copy, modify, and/or distribute this software for any
purpose with or without fee is hereby granted, provided that the above
copyright notice and this permission notice appear in all copies.
.
THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES
WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF
MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR
ANY SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES
WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN
ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF
OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE.
n
License: MIT
Permission is hereby granted, free of charge, to any person obtaining a
copy of this software and associated documentation files (the "Software"),
to deal in the Software without restriction, including without limitation
the rights to use, copy, modify, merge, publish, distribute, sublicense,
and/or sell copies of the Software, and to permit persons to whom the
Software is furnished to do so, subject to the following conditions:
.
The above copyright notice and this permission notice shall be included
in all copies or substantial portions of the Software.
.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS
OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT.
IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY
CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT,
TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE
SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
n

1
debian/files vendored Normal file
View file

@ -0,0 +1 @@
mergerfs_1.1.0_amd64.deb utils optional

View file

@ -1,9 +0,0 @@
#!/bin/sh
set -e
case "${1}" in
configure)
chmod 4755 /usr/bin/mergerfs-fusermount
;;
esac

12
debian/rules vendored
View file

@ -1,16 +1,8 @@
#!/usr/bin/make -f
# -*- makefile -*-
export DH_VERBOSE=1
# Uncomment this to turn on verbose mode.
#export DH_VERBOSE=1
%:
dh $@
override_dh_auto_build:
$(MAKE) release
override_dh_auto_install:
$(MAKE) \
DESTDIR=$(CURDIR)/debian/mergerfs \
PREFIX=/usr \
install

View file

@ -1,120 +0,0 @@
.TH MERGERFS 1 "May 2025" "mergerfs" "User Commands"
.SH NAME
mergerfs \- a featureful union filesystem
.SH SYNOPSIS
.B mergerfs
[options] <branches> <mountpoint>
.SH DESCRIPTION
.B mergerfs
is a FUSE-based union filesystem geared towards simplifying storage and management of files across numerous commodity storage devices. It is similar to
.BR mhddfs ,
.BR unionfs ,
and
.BR aufs .
.SH QUICKSTART
.UR https://trapexit.github.io/mergerfs/latest/quickstart
.LI https://trapexit.github.io/mergerfs/latest/quickstart
.UE
.SH DOCUMENTATION
.UR https://trapexit.github.io/mergerfs
.LI https://trapexit.github.io/mergerfs
.UE
.SH SUPPORT
.UR https://trapexit.github.io/mergerfs/latest/support
.LI https://trapexit.github.io/mergerfs/latest/support
.UE
.SH FEATURES
.IP \[bu] 2
Logically combine numerous filesystems/paths into a single mount point
.IP \[bu]
Combine paths of the same or different filesystems
.IP \[bu]
Ability to add or remove filesystems/paths without impacting the rest of the data
.IP \[bu]
Unaffected by individual filesystem failure
.IP \[bu]
Configurable file selection and creation placement
.IP \[bu]
Works with filesystems of any size
.IP \[bu]
Works with filesystems of almost any type
.IP \[bu]
Ignore read-only filesystems when creating files
.IP \[bu]
Hard link copy-on-write / CoW
.IP \[bu]
Runtime configurable
.IP \[bu]
Support for extended attributes (xattrs)
.IP \[bu]
Support for file attributes (chattr)
.IP \[bu]
Support for POSIX ACLs
.SH NON-FEATURES
.IP \[bu] 2
Read/write overlay on top of read-only filesystem like OverlayFS
.IP \[bu]
File whiteout
.IP \[bu]
RAID-like parity calculation
.IP \[bu]
Redundancy
.IP \[bu]
Splitting of files across branches
.SH HOW IT WORKS
.B mergerfs
logically merges multiple filesystem paths together. It acts as a proxy to the underlying filesystem paths, combining the behaviors of some functions and being a selector for others.
When the contents of a directory are requested,
.B mergerfs
combines the list of files from each directory, deduplicating entries, and returns that list.
When a file or directory is created, a policy is first run to determine which branch will be selected for the creation.
For functions which change attributes or remove the file, the behavior may be applied to all instances found.
Read more about policies here:
.UR https://trapexit.github.io/mergerfs/latest/config/functions_categories_policies
.LI https://trapexit.github.io/mergerfs/latest/config/functions_categories_policies
.UE
.SH VISUALIZATION
.nf
A + B = C
/disk1 /disk2 /merged
| | |
+-- /dir1 +-- /dir1 +-- /dir1
| | | | | |
| +-- file1 | +-- file2 | +-- file1
| | +-- file3 | +-- file2
+-- /dir2 | | +-- file3
| | +-- /dir3 |
| +-- file4 | +-- /dir2
| +-- file5 | |
+-- file6 | +-- file4
|
+-- /dir3
| |
| +-- file5
|
+-- file6
.fi
.SH SPONSORSHIP AND DONATIONS
Development and support of a project like
.B mergerfs
require a significant amount of time and effort. The software is released under the very liberal ISC license and is therefore free to use for personal or commercial uses.
If you are a non-commercial user and find
.B mergerfs
and its support valuable and would like to support the project financially, it would be very much appreciated.
If you are using
.B mergerfs
commercially, please consider sponsoring the project to ensure it continues to be maintained and receive updates. If custom features are needed, feel free to contact the maintainer directly.
.UR https://github.com/trapexit/support
.LI https://github.com/trapexit/support
.UE
.SH LICENSE
ISC License
.SH AUTHOR
Antonio SJ Musumeci (aka trapexit)

View file

@ -1,59 +0,0 @@
Name: mergerfs
Version: __VERSION__
Release: 1%{?dist}
Summary: A featureful FUSE based union filesystem
Group: Applications/System
License: ISC
URL: https://github.com/trapexit/mergerfs
Source: mergerfs-%{version}.tar.gz
BuildRequires: gcc-c++
# rpmbuild driven by the Makefile uses git to generate a version number
BuildRequires: git
%global debug_package %{nil}
%undefine _debuginfo_subpackages
%global _enable_debug_packages 0
%prep
%setup -q
%description
mergerfs is a union filesystem geared towards simplifying storage and
management of files across numerous commodity storage devices. It is
similar to mhddfs, unionfs, and aufs.
%build
make %{?_smp_mflags} CFLAGS="%{optflags}" CXXFLAGS="%{optflags}" LDFLAGS="%{__global_ldflags}"
%install
make install PREFIX=%{_prefix} DESTDIR=%{buildroot}
%files
/usr/bin/mergerfs
/usr/bin/mergerfs-fusermount
/usr/bin/fsck.mergerfs
/usr/bin/mergerfs.collect-info
/sbin/mount.mergerfs
/usr/lib/mergerfs/preload.so
%doc %{_mandir}/*
%changelog
* Fri Apr 26 2019 Antonio SJ Musumeci <trapexit@spawn.link>
- Update description
* Mon Jan 25 2016 Antonio SJ Musumeci <trapexit@spawn.link>
- Remove sbin files
* Sat Sep 05 2015 Antonio SJ Musumeci <trapexit@spawn.link>
- Include PREFIX to install
* Mon Dec 29 2014 Joe Lawrence <joe.lawrence@stratus.com>
- Tweak rpmbuild to archive current git HEAD into a tarball, then (re)build in
the rpmbuild directory -- more complicated but seemingly better suited to
generate source and debug rpms.
* Fri Jun 20 2014 Joe Lawrence <joe.lawrence@stratus.com>
- Initial rpm spec file.

View file

@ -1,23 +0,0 @@
# mkdocs
## Getting started
```bash
python3 -m venv venv
source venv/bin/activate
pip3 install --no-cache-dir mkdocs mkdocs-material pymdown-extensions mike
mike serve
```
## Deploy
```bash
mike deploy --push <version_name> --allow-empty
```
## References
* https://www.mkdocs.org/
* https://squidfunk.github.io/mkdocs-material/

View file

@ -1,7 +0,0 @@
#!/bin/sh
set -x
set -e
python3 -m venv venv
. ./venv/bin/activate
pip3 install --no-cache-dir 'mkdocs<2.0' mkdocs-material pymdown-extensions mike

View file

@ -1,111 +0,0 @@
# Benchmarking
Filesystems are complicated. They do many things and many of those are
interconnected. Additionally, the OS, drivers, hardware, etc. can all
impact performance. Therefore, when benchmarking, it is **necessary**
that the test focuses as narrowly as possible.
For most throughput is the key benchmark. To test throughput `dd` is
useful but **must** be used with the correct settings in order to
ensure the filesystem or device is actually being tested. The OS can
and will cache data. Without forcing synchronous reads and writes
and/or disabling caching the values returned will not be
representative of the device's true performance.
When benchmarking through mergerfs ensure you only use 1 branch to
remove any possibility of the policies or differences in underlying
filesystems complicating the situation. Benchmark the underlying
filesystem first and then mount mergerfs with that same filesystem and
that. Throughput **will be** lower but if you are experiencing speeds
below your expectation you will need to narrow down precisely which
component is leading to the slowdown. Preferably test the following in
the order listed (but not combined).
1. Enable `nullrw` mode with `nullrw=true`. This will effectively make
reads and writes no-ops. Removing the underlying device /
filesystem from the equation. This will give us the top theoretical
speeds.
2. Configure mergerfs to use a `tmpfs` branch. `tmpfs` is a RAM
disk. Extremely high speed and very low latency. This is a more
realistic best case scenario. Example: `mount -t tmpfs -o size=2G
tmpfs /tmp/tmpfs`
3. Configure mergerfs to use a local device filesystem branch. NVMe,
SSD, HDD, etc. Test them individually. If you have different
interconnects / controllers use the same storage device when
testing to ensure consistency.
4. Configure mergerfs to use any network filesystems you plan to use
one at a time. It may also be worth trying a different network
filesystem. `NFS` vs `CIFS/SMB/Samba` vs `sshfs`, etc.
Once you find the component which has the performance issue you can do
further testing with different options to see if they impact
performance. If you find a specific system, device, filesystem,
controller, etc. that performs poorly contact the author so it can be
investigated further.
Sometimes the problem is really the application accessing or writing
data through mergerfs. Some software use small buffer sizes which can
lead to more requests and therefore greater overhead. You can test
this out yourself by replacing `bs=1M` in the examples below with
`ibs` or `obs` and using a size of `512` instead of `1M`. In one
example test using `nullrw` the write speed dropped from 4.9GB/s to
69.7MB/s when moving from `1M` to `512`. Similar results were had when
testing reads. Small writes overhead may be improved by leveraging a
write cache but in casual tests little gain was found. If you have an
app that appears slow with mergerfs it could be due to this. `strace`
can be used with the app in question or mergerfs to see the size of
read/writes. Contact the software author or worse case the mergerfs
author so it may be investigated further.
### Write Benchmark
```
$ dd if=/dev/zero of=/mnt/mergerfs/16GB.file bs=1M count=16384 oflag=nocache conv=fdatasync status=progress
```
### Read Benchmark
```
$ dd if=/mnt/mergerfs/16GB.file of=/dev/null bs=1M iflag=nocache conv=fdatasync status=progress
```
### fio
`fio` is a popular and flexible benchmarking tool.
Below are some basic tests but also try setting `--direct=0`, changing
the `ioengine`, or playing with other options.
#### Write
```
$ fio --name=writetest --filename=/mnt/mergerfs/16GB.file --size=16Gb --rw=write --bs=1M --direct=1 --numjobs=1 --iodepth=8 --group_reporting --runtime=60 --startdelay=0 --ioengine=psync
```
#### Read
```
$ fio --name=readtest --filename=/mnt/mergerfs/16GB.file --size=16Gb --rw=read --bs=1M --direct=1 --numjobs=1 --iodepth=8 --group_reporting --runtime=60 --startdelay=0 --ioengine=psync
```
### Other Benchmarks
If you are attempting to benchmark other behaviors you must ensure you
clear kernel caches before runs. In fact it would be a good deal to
run before the read and write benchmarks as well just in case.
```
sync
echo 3 | sudo tee /proc/sys/vm/drop_caches
```
## Additional Reading
* [Tweaking Performance](performance.md)
* [Options](config/options.md)
* [Tips and Notes](tips_notes.md)

View file

@ -1,89 +0,0 @@
# branches-mount-timeout
* type: `UINT` (seconds)
* default: `0`
mergerfs is compatible with any path, whether it be a mount point or a
directory on your [root
filesystem.](https://en.wikipedia.org/wiki/Root_directory) It doesn’t
require branch paths to be mounted or to match a specific filesystem
at startup, and it operates effectively without needing details about
the intended filesystem. This flexibility eliminates the need to
manage mount ordering, which is particularly advantageous on modern
systems where filesystems are mounted asynchronously, resulting in
unpredictable mount sequences. While
[systemd](https://www.freedesktop.org/software/systemd/man/latest/systemd.mount.html)
offers a way to enforce mount dependencies using the
[x-systemd.requires=PATH](https://www.freedesktop.org/software/systemd/man/latest/systemd.mount.html#x-systemd.requires=)
option in /etc/fstab, applying this to every branch path is both
cumbersome and susceptible to errors.
`branches-mount-timeout` will cause mergerfs to first attempt to mount
each branch if the branch appears to be on the same filesystem as the
`mergerfs` mount point and then wait for all branches to become
"mounted." The logic to determine "mounted" is as follows.
1. It is mounted if the branch directory has the extended attribute
`user.mergerfs.branch`
2. It is mounted if the branch directory contains a file named
`.mergerfs.branch`
3. It is **not** mounted if the branch directory has the extended
attribute `user.mergerfs.branch_mounts_here`
4. It is **not** mounted if the branch directory contains
a file named `.mergerfs.branch_mounts_here`
5. It is mounted if the `st_dev` value of the mergerfs mount point is
different from the branch path.
These files and xattrs are used to help `mergerfs` know whether or not
a directory happens to be a branch or is the underlying directory
mount point.
## Paths which appear after startup
The [`branches` resolution process](branches.md#resolution) runs before
this option starts waiting. A missing literal path is retained as a
branch, so a configuration such as `/mnt/hdd/disk0/foobar` can work
when `/mnt/hdd/disk0` is mounted asynchronously and `foobar` only
becomes visible afterward, provided the path then meets one of the
"mounted" conditions above.
For each branch which is not ready, mergerfs first runs `mount` once
with the full branch path as the target and then polls that same path
until the timeout expires. It does not create the path, try to mount
its parent, or rerun `mount`. In the example above, the initial
`mount /mnt/hdd/disk0/foobar` may fail while the path is absent. If
another service mounts `/mnt/hdd/disk0` and
`/mnt/hdd/disk0/foobar` appears and meets one of the "mounted"
conditions before the deadline, mergerfs detects it during polling
and continues.
Globs are different. If `/mnt/hdd/disk0/*` has no matches during
branch resolution, the literal path `/mnt/hdd/disk0/*` is retained.
The mount wait does not expand it again after `/mnt/hdd/disk0`
becomes available. Ensure the parent filesystem is mounted before
mergerfs, configure a literal branch path, or update `branches`
through the runtime API once the paths exist.
**NOTE:** If on a `systemd` based system and using `fstab` it is a
good idea to set the mount option
[x-systemd.mount-timeout](https://www.freedesktop.org/software/systemd/man/latest/systemd.mount.html#x-systemd.mount-timeout=)
to some value longer than `branches-mount-timeout` otherwise `systemd`
might kill `mergerfs` before it finishes waiting.
**NOTE:** The way `mergerfs` attempts to mount the branch is by
calling out to the system command `mount` with the branch as the
target. If that path is configured in `/etc/fstab` as a mount point
then `mount` should attempt to mount it with the settings defined in
`fstab`.
## branches-mount-timeout-fail
Default: `false`
When set to `true` mergerfs will fail entirely after
`branches-mount-timeout` expires without all branches being
mounted. If set to `false` it will simply ignore the mount status of
the branches and continue on. The details will be
[logged](../error_handling_and_logging.md).

View file

@ -1,237 +0,0 @@
# branches
The `branches` argument is a colon (':') delimited list of paths to be
pooled together. It does not matter if the paths are on the same or
different filesystems nor does it matter [the filesystem
type](../faq/compatibility_and_integration.md/#what-filesystems-can-be-used-as-branches)
(within reason). Used and available space metrics will not be
duplicated for paths on the same filesystem and any features which
aren't supported by the underlying filesystem (such as file attributes
or extended attributes) will return the appropriate errors.
Branches currently have two options which can be set. A
[mode](#branch-mode) which impacts whether or not the branch is
included in a policy calculation and a individual
[minfreespace](#minfreespace) value. The values are set by
prepending an `=` at the end of a branch designation and using commas
as delimiters. Example: `/mnt/hdd/disk0=RW,1234`
### branch mode
- RW: (read/write) - Default behavior. Will be eligible in all policy
categories.
- RO: (read-only) - Will be excluded from `create` and `action`
policies. Same as a read-only mounted filesystem would be (though
faster to process).
- NC: (no-create) - Will be excluded from `create` policies. You can't
create on that branch but you can change or delete.
### minfreespace
Same purpose and syntax as the [global option](minfreespace.md) but
specific to the branch. Defaults to the global value.
### resolution
Each colon-delimited branch designation is resolved independently when
the `branches` option is parsed, both at startup and when it is updated
through the runtime API:
1. The path portion is expanded as a glob.
2. If the glob matches one or more directories, each match becomes a
branch.
3. If the glob has no matches, mergerfs logs a notice and keeps the
configured path unchanged as a single, literal branch.
4. mergerfs resolves each resulting path with `realpath`. If that
succeeds, the canonical path is stored. If it fails, typically
because the path does not exist yet, the unchanged path is stored.
5. A path which exists but is not a directory is logged and skipped. A
path which does not exist is logged but retained.
Consequently, a literal branch such as `/mnt/hdd/disk0/foobar` may be
configured before it exists and can become usable after
`/mnt/hdd/disk0` is mounted. Branch resolution is not repeated in the
background, however. If `/mnt/hdd/disk0/*` has no matches when it is
resolved, mergerfs stores the literal path `/mnt/hdd/disk0/*`;
directories which match it later are not discovered automatically.
Update `branches` through the runtime API or restart mergerfs to
resolve the glob again.
### globbing
To make it easier to include multiple branches mergerfs supports
[globbing](http://linux.die.net/man/7/glob). **The globbing tokens
MUST be escaped when using via the shell else the shell itself will
apply the glob itself.**
```
# mergerfs /mnt/hdd/\*:/mnt/ssd/\* /media
```
The above line will use all directories within `/mnt/hdd` and
`/mnt/ssd`.
To have the pool mounted at boot or otherwise accessible from related
tools use `/etc/fstab`.
```
# <file system> <mount point> <type> <options> <dump> <pass>
/mnt/hdd/*:/mnt/ssd/* /media mergerfs minfreespace=16G 0 0
```
**NOTE:** The globbing is done at mount or when updated using the
runtime API. If a new directory is added matching the glob after the
fact it will not be automatically included.
A convenient way to configure branches is to use
[symlinks](../quickstart.md/#etcfstab-w-config-file).
## branch setup
`mergerfs` does **not** require any special setup of branch paths in
order to be used however here are some suggestions.
### layout
When you have a large collection of storage devices, types of storage,
and types of interconnects it can be useful to have a verbose naming
convention. My preference is to create a directory within `/mnt/` for
each major type of storage device and/or interconnect. `hdd` for
traditional hard drives; `ssd` for slower SATA based SSDs; `nvme` for
M.2 or U.2 NVME SSDs; `remote` for NFS, SMB, sshfs, etc.
The mount points within those directories are named after the size of
the storage and its serial number in the case of physical storage and
the hostname and remote filesystem for remote.
```
$ ls -lh /mnt/
total 16K
drwxr-xr-x 8 root root 4.0K Aug 18 2024 hdd
drwxr-xr-x 6 root root 4.0K Oct 8 2024 nvme
drwxr-xr-x 3 root root 4.0K Aug 24 2024 remote
drwxr-xr-x 3 root root 4.0K Jul 14 2024 ssd
$ ls -lh /mnt/hdd/
total 8K
d--------- 2 root root 4.0K Apr 14 15:58 10T-01234567
d--------- 2 root root 4.0K Apr 12 20:51 20T-12345678
$ ls -lh /mnt/nvme/
total 8K
d--------- 2 root root 4.0K Apr 14 16:00 1T-ABCDEFGH
d--------- 2 root root 4.0K Apr 14 23:24 1T-BCDEFGHI
$ ls -lh /mnt/remote/
total 8K
d--------- 2 root root 4.0K Apr 12 20:23 foo-sshfs
d--------- 2 root root 4.0K Apr 12 20:24 bar-nfs
# You can find the serial number of a drive using lsblk
$ lsblk -d -o NAME,PATH,SIZE,SERIAL
NAME PATH SIZE SERIAL
sda /dev/sda 9.1T 01234567
sdb /dev/sdb 18.2T 12345678
nvme0n1 /dev/nvme0n1 953.9G ABCDEFGH
nvme1n1 /dev/nvme1n1 953.9G BCDEFGHI
```
### permissions and mode
#### mount points
To ensure the directory is **only** used as a point to mount another
filesystem it is good to lock it down as much as possible. Be sure to
do this **before** mounting a filesystem to it.
```
$ sudo chown root:root /mnt/hdd/10T-XYZ
$ sudo chmod 0000 /mnt/hdd/10T-XYZ
$ sudo setfattr -n user.mergerfs.branch_mounts_here
$ sudo chattr +i /mnt/hdd/10T-XYZ
```
The extended attribute `user.mergerfs.branch_mounts_here` is used by
the [branches-mount-timeout](branches-mount-timeout.md) option to
recognize whether or not a mergerfs branch path points to the intended
filesystem.
The `chattr` is likely to only work on EXT{2,3,4} filesystems but will
restrict even `root` from modifying the directory or its content but
is still able to be a mount point.
#### mounted filesystems
For those new to Linux, intending to be the primary individual logged
into the system, or simply want to simplify permissions it is
recommended to set the root of mounted filesystems like `/tmp/` is set
to. Owned by `root`, `ugo+rwx` and [sticky
bit](https://en.wikipedia.org/wiki/Sticky_bit) set.
This must be done **after** mounting the filesystem to the target
mount point.
```
$ sudo chown root:root /mnt/hdd/10T-SERIALNUM
$ sudo chmod 1777 /mnt/hdd/10T-SERIALNUM
$ sudo setfattr -n user.mergerfs.branch /mnt/hdd/10T-SERIALNUM
```
### formatting
While even less relevant to mergerfs than the details above the topic
does com up regularly with mergerfs users. When it comes to
partitioning and formatting it is suggested to keep things
simple. Most users of mergerfs will be using the whole drive capacity
and as such have a singular partition and filesystem on that
partition. Something many people don't realize is that the partition
is not necessary and actually can become problematic.
Rather than creating paritions just format the block device. Let's use
`/dev/sda` as an example.
```
$ lsblk -d -o NAME,PATH,SERIAL /dev/sda
NAME PATH SERIAL
sda /dev/sda 01234567
$ sudo mkfs.ext4 -m0 -L 01234567 /dev/sda
mke2fs 1.47.0 (5-Feb-2023)
Discarding device blocks: done
Creating filesystem with 262144 4k blocks and 65536 inodes
Filesystem UUID: ede8bf96-9aff-464e-a4fb-a4ab8a197cd7
Superblock backups stored on blocks:
32768, 98304, 163840, 229376
Allocating group tables: done
Writing inode tables: done
Creating journal (8192 blocks): done
Writing superblocks and filesystem accounting information: done
```
You can then use the serial number as the identifier in `fstab`.
```
# <file system> <mount point> <type> <options> <dump> <pass>
LABEL=01234567 /mnt/hdd/10TB-01234567 auto nofail 0 2
```
Benefits of doing it this way?
* One less thing (partitions) to configure and worry about.
* Guarantees alignment of blocks
* Makes it easier to use the drive with different enclosures. Some
SATA to USB adapters use 4K blocks while the drive itself is using
512b. If you create partitions with one block size and move the
drive to a controller that uses the other the offset on the device
where the filesystem starts will be misinterpreted. It is possible
to manually fix this but it isn't well documented and avoidable.

View file

@ -1,193 +0,0 @@
# caching
## cache.files
Controls how [page caching](https://en.wikipedia.org/wiki/Page_cache)
works for mergerfs itself. Not the underlying filesystems.
* `cache.files=off`: Disables page caching for mergerfs.
* `cache.files=partial`: Enables page caching. Files are cached
while open.
* `cache.files=full`: Enables page caching. Files are cached across
opens.
* `cache.files=auto-full`: Enables page caching. Files are cached
across opens if mtime and size are unchanged since previous
open. Cache is dropped if mtime or size change on open.
* `cache.files=per-process`: Enable page caching (equivalent to
`cache.files=partial`) only for processes whose 'comm' name matches
one of the values defined in cache.files.process-names. If the name
does not match the file open is equivalent to `cache.files=off`.
Generally, enabling the page cache actually *harms*
performance[^1]. In part because it can lead to buffer bloat due to
the kernel caching both the underlying filesystem's file content as
well as the file through mergerfs. However, if you want to confirm
performance differences it is recommended that you perform some
benchmark to confirm which option works best for your setup.
Why then would you want to enable page caching if it consumes ~2x the
RAM as normal and is on average slower? Because it is the only way to
support
[mmap](https://man7.org/linux/man-pages/man2/mmap.2.html). `mmap` is a
way for programs to treat a file as if it is a contiguous RAM buffer
which is regularly used by a number of programs such as those that
leverage **sqlite3**. Despite `mmap` not being supported by all
filesystems it is unfortunately common for software to not have an
option to use regular file IO instead of `mmap`.
The good thing is that in Linux v6.6[^2] and above FUSE can now
transparently enable page caching when mmap is requested. This means
it should be safe to set `cache.files=off`. However, on Linux v6.5 and
below you will need to configure `cache.files` as you need.
If [passthrough.io](passthrough.md) is enabled so must page
caching. mergerfs will set `cache.files=auto-full` if `passthrough.io`
is enabled. And when using `passthrough.io` the there is no double
page caching since it is in fact passing through the IO.
[^1]: This is not unique to mergerfs and affects all FUSE
filesystems. It is something that the FUSE community hopes to
investigate at some point but as of early 2025 there are a number
of major reworking going on with FUSE which needs to be finished
first.
[^2]: [https://kernelnewbies.org/Linux_6.6#FUSE](https://kernelnewbies.org/Linux_6.6#FUSE)
## cache.entry
* `cache.entry=UINT`: Sets the number of seconds to cache
entry queries. Defaults to `1`.
The kernel must ask mergerfs about the existence of files. The entry
cache caches that those details which limits the number of requests
sent to mergerfs.
The risk of setting this value, as with most any cache, is related to
[out-of-band](https://en.wikipedia.org/wiki/Out-of-band) changes. If
the filesystems are changed outside mergerfs there is a risk of files
which have been removed continuing to show as available. It will fail
gracefully if a phantom file is actioned on in some way so there is
little risk in setting the value much higher. Especially if there are
no out-of-band changes.
## cache.negative-entry
* `cache.negative-entry=UINT`: Sets the number of seconds to cache
negative entry queries. Defaults to `1`.
This is a cache for negative entry query responses. Such as when a
file which does not exist is referenced.
The risk of setting this value, as with most any cache, is related to
[out-of-band](https://en.wikipedia.org/wiki/Out-of-band) changes. If
the filesystems are changed outside mergerfs there is a risk of files
which have been added outside mergerfs not appearing correctly till
the cache entry times out if there had been a request for the same
name within mergerfs which didn't exist. This is mostly an
inconvenience.
## cache.attr
* `cache.attr=UINT`: Sets the number of seconds to cache file
attributes. Defaults to `1`.
This is a cache for file attributes and metadata such as that which is
collected by the
[stat](https://man7.org/linux/man-pages/man2/stat.2.html) system call
which is used when you run commands such as `find` or `ls -lh`.
As with other caches the risk of enabling the attribute cache is if
changes are made to the file out-of-band there could be
inconsistencies between the actual file and the cached details which
could result in different issues depending on how the data is used. If
the simultaneous writing of a file from inside and outside is unlikely
then you should be safe. That said any simultaneous, uncoordinated
manipulation of a file can lead to unexpected results.
## cache.statfs
* `cache.statfs=UINT`: Sets the number of seconds to cache `statfs`
calls used by policies. Defaults to `0`.
A number of policies require looking up the available space of the
branches being considered. This is accomplished by calling
[statfs](https://man7.org/linux/man-pages/man2/statfs.2.html). This
call however is a bit expensive so this cache reduces the overhead by
limiting how often the calls are actually made.
This will mean that if the available space of branches changed
somewhat rapidly there is a risk of `create` or `mkdir` calls made
within the timeout period ending up on the same branch. This however
should even itself out over time.
## cache.symlinks
* `cache.symlinks=BOOL`: Enable kernel caching of symlink
values. Defaults to `false`.
As of Linux v4.20 there is an ability to cache the value of symlinks
so that the kernel does not need to make a request to mergerfs every
single time a
[readlink](https://man7.org/linux/man-pages/man2/readlink.2.html)
request is made. While not a common usage pattern, if software very
regularly queries symlink values, the use of this cache could
significantly improve performance.
mergerfs will not error if the kernel used does not support symlink
caching.
As with other caches the main risk in enabling it is if you are
manipulating symlinks from both within and without the mergerfs
mount. Should the value be changed outside of mergerfs then it will
not be reflected in the mergerfs mount till the cached value is
invalidated.
## cache.readdir
* `cache.readdir=BOOL`: Enable kernel caching of readdir
results. Defaults to `false`.
As of Linux v4.20 it supports readdir caching. This can have a
significant impact on directory traversal. Especially when combined
with entry (`cache.entry`) and attribute (`cache.attr`) caching. If
the kernel doesn't support readdir caching setting the option to true
has no effect. This option is configurable at runtime via xattr
user.mergerfs.cache.readdir.
## cache.writeback
* `cache.writeback=BOOL`: Enable writeback cache. Defaults to `false`.
When `cache.files` is enabled the default is for it to perform
writethrough caching. This behavior won't help improve performance as
each write still goes one for one through the filesystem. By enabling
the FUSE writeback cache small writes *may* be aggregated by the
kernel and then sent to mergerfs as one larger request. This can
greatly improve the throughput for apps which write to files
inefficiently. The amount the kernel can aggregate is limited by the
size of a FUSE message. Read the [fuse-msg-size](fuse-msg-size.md)
section for more details.
There is a side effect as a result of enabling writeback
caching. Underlying files won't ever be opened with O_APPEND or
O_WRONLY. The former because the kernel then manages append mode and
the latter because the kernel may request file data from mergerfs to
populate the write cache. The O_APPEND change means that if a file is
changed outside of mergerfs it could lead to corruption as the kernel
won't know the end of the file has changed. That said any time you use
caching you should keep from writing to the same file outside of
mergerfs at the same time.
Note that if an application is properly sizing writes then writeback
caching will have little or no effect. It will only help with writes
of sizes below the FUSE message size (128K on older kernels, 1M on
newer). Even then its effectiveness might not be great. Given the side
effects of enabling this feature it is recommended that its benefits
be proved out with benchmarks.

View file

@ -1,26 +0,0 @@
# Deprecated Options
These are old, deprecated options which may no longer have any
function or have been replaced. **They should not be used.**
* **allow_other**: mergerfs v2.35.0 and above sets this FUSE option
automatically if running as root.
* **async_read**: Use `async-read=true`.
* **atomic_o_trunc**: Does nothing.
* **attr_timeout**: Use [cache.attr](cache.md).
* **auto_cache**: Use [cache.files=auto-full](cache.md).
* **big_writes**: Does nothing.
* **direct_io**: Use [cache.files=off](cache.md).
* **entry_timeout**: Use [cache.entry](cache.md).
* **hard_remove**: Does nothing.
* **kernel_cache**: Use [cache.files=full](cache.md).
* **negative_entry**: Use [cache.negative-entry](cache.md).
* **remember**: Use `never-forget-nodes`.
* **remember-nodes**: Use `never-forget-nodes`.
* **noforget**: Use `never-forget-nodes`.
* **nonempty**: Does nothing.
* **splice_move**: Does nothing.
* **splice_read**: Does nothing.
* **splice_write**: Does nothing.
* **sync_read**: Use `async-read=false`.
* **use_ino**: Use [inodecalc](inodecalc.md).

View file

@ -1,15 +0,0 @@
# export-support
* `export-support=true|false`
* Default: `true`
In theory, this flag should not be exposed to the end user. It is a
low-level FUSE flag which indicates whether or not the kernel can send
certain kinds of messages to it for the purposes of using it with
NFS. mergerfs does support these messages but due to bugs and quirks
found in the kernel and mergerfs this option is provided just in case
it is needed for debugging.
Given that this flag is set when the FUSE connection is first
initiated it is not possible to change during run time.

View file

@ -1,21 +0,0 @@
# flush-on-close
By default, FUSE would issue a flush before the release of a file
descriptor. This was considered a bit aggressive and a feature added
to give the FUSE server the ability to choose when that happens.
* `flush-on-close=always`
* `flush-on-close=never`
* `flush-on-close=opened-for-write`
* Defaults to `opened-for-write`.
For now it defaults to `opened-for-write` which is less aggressive
than the behavior before this feature was added. It should not be a
problem because the flush is really only relevant when a file is
written to. Given flush is irrelevant for many filesystems in the
future a branch specific flag may be added so only files opened on a
specific branch would be flushed on close.
## References
* [https://lkml.kernel.org/linux-fsdevel/20211024132607.1636952-1-amir73il@gmail.com/T/](https://lkml.kernel.org/linux-fsdevel/20211024132607.1636952-1-amir73il@gmail.com/T/)

View file

@ -1,23 +0,0 @@
# follow-symlinks
This feature, when enabled, will cause symlinks to be interpreted by
mergerfs as their target.
When there is a getattr/stat request for a file mergerfs will check if
the file is a symlink and depending on the `follow-symlinks` setting
will replace the information about the symlink with that of that which
it points to.
When unlink'ing or rmdir'ing the followed symlink it will remove the
symlink itself and not that which it points to.
* `follow-symlinks=never`: Behave as normal. Symlinks are treated as such.
* `follow-symlinks=directory`: Resolve symlinks only which point to directories.
* `follow-symlinks=regular`: Resolve symlinks only which point to regular files.
* `follow-symlinks=all`: Resolve all symlinks to that which they point
to. Symlinks which do not point to anything are left as is.
* Defaults to `never`.
**WARNING:** This feature should be considered experimental. There
might be edge cases yet found. If you find any odd behaviors please
file a ticket on [github](https://github.com/trapexit/mergerfs/issues/new?assignees=&labels=bug%2C+investigating&projects=&template=bug_report.md&title=).

View file

@ -1,43 +0,0 @@
# func.readdir
* type: `STR`
* default: `seq`
* examples: `func.readdir=seq`, `func.readdir=cor:4`
`readdir` has policies to control how to read directory content.
| Policy | Description |
| ------ | ----------- |
| seq | "sequential" : Iterate sequentially over branches in the order defined in `branches`. This is the default and traditional behavior found prior to the readdir policy introduction. This will be increasingly slower as more branches are added to the pool. Especially if needing to wait for drives to spin up or network filesystems to respond. |
| cosr:N:M | "concurrent open, sequential read" : Concurrently open branch directories using a thread pool and process them in the order defined in `branches`. This keeps memory and CPU usage low while also reducing the time spent waiting on branches to respond. `N` is the number of threads. If negative it will be the core count divided by `abs(N)`. `M` is the queue depth. If either value is `0` it will be decided based on system configuration. |
| cosr:N | cosr:N:M with M = 0 |
| cosr | cosr:N:M with N = 0 and M = 0 |
| cor:N:M | "concurrent open and read" : Concurrently open branch directories and immediately start reading their contents using a thread pool. This will result in slightly higher memory and CPU usage but reduced latency. Particularly when using higher latency / slower speed network filesystem branches. Unlike `seq` and `cosr` the order of files could change due the async nature of the thread pool. This should not be a problem since the order of files returned in not guaranteed. `N` is the number of threads. If negative it will be the core count divided by `abs(N)`. `M` is the queue depth. If either value is `0` it will be decided based on system configuration. |
| cor:N | cosr:N:M with M = 0 |
| cor | cosr:N:M with N = 0 and M = 0 |
Keep in mind that `readdir` mostly just provides a list of file names
in a directory and possibly some basic metadata about said files. To
know details about the files, as one would see from commands like
`find` or `ls`, it is required to call `stat` on the file which is
controlled by `fuse.getattr`.
Until Linux v6.16, despite being able to change the size of FUSE
messages, `readdir` requests were limited to 1 page (4KiB) in size per
message. This limitation limited throughput and therefore performance
of the `readdir` call. In v6.16 that was changed to be able to grow to
the [fuse-msg-size](fuse-msg-size.md) which allows a lot more data to
be sent per request and therefore much better performance in certain
situations.
## Technical Details
* On Linux
[getdents](https://man7.org/linux/man-pages/man2/getdents.2.html) is
used instead of
[readdir](https://man7.org/linux/man-pages/man2/readdir.2.html) in
order to leverage larger buffers for better performance on larger
directory reads as well as have more control in concurrent policies.
* [readdir](https://man7.org/linux/man-pages/man2/readdir.2.html) is
used on FreeBSD.

View file

@ -1,150 +0,0 @@
# functions, categories and policies
The POSIX filesystem API is made up of a number of functions. `creat`,
`stat`, `chown`, etc. For ease of configuration in mergerfs, most of
the core functions are grouped into 3 categories: `action`, `create`,
and `search`. These functions and categories can be assigned a policy
which dictates which branch is chosen when performing that
function. IE. When an application requests that a file or directory is
created how should mergerfs decide which branch to choose to attempt
the create? That is the policy.
Some functions, listed in the category `N/A` below, can not be
assigned the normal policies because they are directly related to a
file which has already been opened.
## Functions and their Category classifications
| Category | Functions |
| -------- | --------- |
| action | chmod, chown, link, removexattr, rename, rmdir, setxattr, truncate, unlink, utimens |
| create | create, mkdir, mknod, symlink |
| search | access, getattr, getxattr, ioctl (directories), listxattr, open, readlink |
| N/A | fchmod, fchown, futimens, ftruncate, fallocate, fgetattr, fsync, ioctl (files), read, readdir, release, statfs, write, copy_file_range |
In cases where something may be searched for (such as a path to clone)
`getattr` will usually be used.
## Policies
See below for [available policies and their descriptions](#policy-descriptions).
A policy is an algorithm designed to select one or more branches for a
function to operate on.
Policies do not actually manage the filesystem or layout. They are
strictly responsible for deciding which files or branches will be
worked on in relation to the function being performed. Once the branch
is chosen other parts of the system does what is necessary to
accomplish the function. Such as the cloning of directories between
branches.
When using policies which are based on a branch's available space the
branch base path provided is used. Not the full path to the file or
directory in question. Meaning that mounts within the branch will not
be considered in the space calculations.
**NOTE:** While any policy can be assigned to a function or category,
some may not be very useful in practice. For instance: `rand` (random)
may be useful for file creation but could lead to very odd behavior if
used for `chmod` if there were more than one copy of the file. Unless
users find this flexibility useful it will likely be removed in the
future.
## Filtering
Most policies search branches and create a list of files / paths for
functions to work on. The policy is responsible for filtering and
sorting the branches. Filters include [minfreespace](minfreespace.md),
whether or not a branch is mounted read-only, and the branch mode
(RO,NC,RW). These filters are applied across most policies.
- No `search` function policies filter.
- All `action` function policies filter out branches which are
mounted **read-only** or mode is **RO (read-only)**.
- All `create` function policies filter out branches which are
mounted **read-only**, mode **RO (read-only)** or **NC (no
create)**, or has available space less than
[minfreespace](minfreespace.md).
Policies may have their own additional filtering such as those that
require relative paths to be present.
If all branches are filtered an error will be returned. Typically
`EROFS` (read-only filesystem) or `ENOSPC` (no space left on
device) depending on the most recent reason for filtering a
branch.
If `create`, `mkdir`, `mknod`, or `symlink` fail with `EROFS`
or other fundamental errors then mergerfs will mark any branch found
to be read-only as such (IE will set the mode `RO`) and will rerun the
policy and try again. This is mostly for `ext4` filesystems that can
suddenly become read-only when it encounters an error.
## Path Preservation
Policies, as described below, are of two basic classifications. `path
preserving` and `non-path preserving`.
All policies which start with `ep` (`epff`, `eplfs`, `eplus`, `epmfs`,
`eprand`) are `path preserving`. `ep` stands for `existing path`.
A path preserving policy will only consider branches where the relative
path being accessed already exists.
With the `msp` or `most shared path` policies they are defined as
`path preserving` for the purpose of controlling `link` and `rename`'s
behaviors since `ignorepponrename` is available to disable that
behavior.
## Policy descriptions
A policy's behavior differs, as mentioned above, based on the function
it is used with. Sometimes it really might not make sense to even
offer certain policies because they are literally the same as others
but it makes things a bit more uniform.
| Policy | Description |
| --------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| pfrd (percentage free random distribution) | Selects a branch randomly, with the probability of selection proportional to its available space relative to the total available space across all branches. For instance, if Branch A has 100 GB free and Branch B has 50 GB free, Branch A is twice as likely to be chosen. |
| rand (random) | Calls **all** and then randomizes. Returns 1 branch. |
| mfs (most free space) | Pick the branch with the most available free space. |
| ff (first found) | Given the order of the branches, as defined at mount time or configured at runtime, act on the first one found. |
| lfs (least free space) | Pick the branch with the least available free space. |
| lup (least used percent) | Pick the branch with the least used percentage of space |
| lus (least used space) | Pick the branch with the least used space. |
| all | Search: For **mkdir**, **mknod**, and **symlink** it will apply to all branches. **create** works like **ff**. |
| msppfrd (most shared path, percentage free random distribution) | Like **eppfrd** but if it fails to find a branch it will try again with the parent directory. Continues this pattern till finding one. |
| mspmfs (most shared path, most free space) | Like **epmfs** but if it fails to find a branch it will try again with the parent directory. Continues this pattern till finding one. |
| msplfs (most shared path, least free space) | Like **eplfs** but if it fails to find a branch it will try again with the parent directory. Continues this pattern till finding one. |
| msplus (most shared path, least used space) | Like **eplus** but if it fails to find a branch it will try again with the parent directory. Continues this pattern till finding one. |
| eppfrd (existing path, percentage free random distribution) | Like **pfrd** but limited to existing paths. |
| epmfs (existing path, most free space) | Of all the branches on which the relative path exists choose the branch with the most free space. |
| eprand (existing path, random) | Calls **epall** and then randomizes. Returns 1. |
| epff (existing path, first found) | Given the order of the branches, as defined at mount time or configured at runtime, act on the first one found where the relative path exists. |
| eplfs (existing path, least free space) | Of all the branches on which the relative path exists choose the branch with the least free space. |
| eplus (existing path, least used space) | Of all the branches on which the relative path exists choose the branch with the least used space. |
| epall (existing path, all) | For **mkdir**, **mknod**, and **symlink** it will apply to all found. **create** works like **epff** (but more expensive because it doesn't stop after finding a valid branch). |
| newest | Pick the file / directory with the largest mtime. |
**NOTE:** If you are using an underlying filesystem that reserves
blocks such as ext2, ext3, or ext4 be aware that mergerfs respects the
reservation by using `f_bavail` (number of free blocks for
unprivileged users) rather than `f_bfree` (number of free blocks) in
policy calculations. **df** does NOT use `f_bavail`, it uses
`f_bfree`, so direct comparisons between **df** output and mergerfs'
policies is not appropriate.
## Defaults
| Category | Policy |
| -------- | ------ |
| action | epall |
| create | pfrd |
| search | ff |

View file

@ -1,50 +0,0 @@
# fuse-msg-size
* type: `UINT|SIZE`
* default: `1M`
* example: `fuse-msg-size=1M`
* If the value is a `SIZE` in the form of `xB`, `xK`, `xM`, `xG`, or
`xT` then it will choose the closest multiple of the system page
size. If the value is just an integer without a size multipler suffix
then it is treated as a page count
FUSE applications communicate with the kernel over a special character
device: `/dev/fuse`. A large portion of the overhead associated with
FUSE is the cost of going back and forth between user space and kernel
space over that device. Generally speaking, the fewer trips needed the
better the performance will be. Reducing the number of trips can be
done a number of ways. Kernel level caching and increasing message
sizes being two significant ones. When it comes to reads and writes if
the message size is doubled the number of trips are approximately
halved.
In Linux v4.20 a new feature was added allowing the negotiation of the
max message size. Since the size is in multiples of
[pages](https://en.wikipedia.org/wiki/Page_(computer_memory)) the
feature is called `max_pages`. In versions of Linux prior to v6.13
there is a maximum `max_pages` value of 256 (1MiB) and minimum of 1
(4KiB). In [Linux
v6.13](https://web.git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=2b3933b1e0a0a4b758fbc164bb31db0c113a7e2c)
and above the max value supported by the kernel can range from 1
(4KiB) to 65535 (~256MiB) (assuming a page size of 4KiB.) The default
used by Linux >= 4.20, and hard coded value used before 4.20, is 32
(128KiB). In mergerfs it is referred to as `fuse-msg-size` to make it
clear what it impacts and provide some abstraction.
If the `fuse-msg-size` value provided is more than the system wide
maximum mergerfs will attempt to increase the system wide value to keep
the user from needing to set the value using `sysctl`,
`/etc/sysctl.conf`, or `/proc/sys/fs/fuse/max_pages_limit`.
The main downside to increasing the value is that memory usage will
increase approximately relative to the number of [processing
threads](threads.md) configured. Keep this in mind for systems with
lower amounts of memory like most SBCs. Performance improvements seem
to peak around 4MiB.
On kernels before v4.20 the option has no effect. On kernels between
v4.20 and v6.13 the max value is 256. On kernels >= v6.13 the maximum
value is 65535.
NOTE: If you intend to enable `cache.files` you should also set
[readahead](readahead.md) to match `fuse-msg-size`.

View file

@ -1,80 +0,0 @@
# inodecalc
[Inodes](https://en.wikipedia.org/wiki/Inode) (`st_ino`) are unique
identifiers for files within a filesystem. Each mounted filesystem
also has a (typically) unique device ID (`st_dev`) as well and
together they can uniquely identify a file on the whole of the
system. Entries on the same device with the same inode are (generally)
in fact references to the same underlying file. It is a many to one
relationship between names and an inode. Directories, however, do not
have multiple links on most systems due to the complexity that would
add.
FUSE allows the server (in this case `mergerfs`) to set inode values
but not device IDs. Creating an inode value is somewhat complex in
mergerfs' case as files aren't really in its control. If a policy
changes what directory or file is to be selected or changes are made
to branches out of band it becomes unclear what value should be
used. Most software does not to care what the values are but those
that do often break if a value changes unexpectedly. For instance: the
tool `find` will abort a directory walk if it sees a directory inode
change. NFS can return stale handle errors if the inode changes
out-of-band. File dedup tools will usually leverage device ids and
inodes as a shortcut in searching for duplicate files and would resort
to full file comparisons should it find different inode values.
mergerfs offers multiple ways to calculate the inode in hopes of
covering different usecases.
* `passthrough`: Passes through the underlying inode value. Mostly
intended for testing as using this does not address any of the
problems mentioned above and could confuse file deduplication
software as inodes from different filesystems can be the same.
* `path-hash`: Hashes the relative path of the entry in question. The
underlying file's values are completely ignored. This means the
inode value will always be the same for that file path. This is
useful when using NFS and you make changes out of band such as copy
data between branches. This also means that entries that do point to
the same file will not be recognizable via inodes. That does not
mean hard links don't work. They will.
* `path-hash32`: 32bit version of path-hash.
* `devino-hash`: Hashes the branch path and inode of the underlying
entry. This won't prevent issues with NFS should the policy pick a
different file or files move out of band but will present the same
inode for underlying files that share the same inode. Previous
releases of mergerfs used the device ID but that changes on
reboot. The downside to using the branch path is inodes for files on
the same underlying filesystem will be different.
* `devino-hash32`: 32bit version of devino-hash.
* `hybrid-hash`: Performs path-hash on directories and devino-hash on
other file types. Since directories can't have hard links the static
value won't make a difference and the files will get values useful
for finding duplicates. Probably the best to use if not using
NFS. As such it is the default.
* `hybrid-hash32`: 32bit version of hybrid-hash.
32bit versions are provided as there is some software which does not
handle 64bit inodes well.
While there is a risk of hash collision in tests of a couple of
million entries there were zero collisions. Unlike a typical
filesystem FUSE filesystems can reuse inodes and not refer to the same
entry. The internal identifier used to reference a file in FUSE is
different from the inode value presented. The former is the nodeid and
is actually a tuple of 2 64bit values: nodeid and generation. This
tuple is not client facing. The inode that is presented to the client
is passed through the kernel uninterpreted.
From FUSE docs for `use_ino`:
> Honor the st_ino field in the functions getattr() and
> fill_dir(). This value is used to fill in the st_ino field
> in the stat(2), lstat(2), fstat(2) functions and the d_ino
> field in the readdir(2) function. The filesystem does not
> have to guarantee uniqueness, however some applications
> rely on this value being unique for the whole filesystem.
> Note that this does *not* affect the inode that libfuse
> and the kernel use internally (also called the "nodeid").
**NOTE:** As of version 2.35.0 the use_ino option has been
removed. mergerfs should always be managing inode values.

Some files were not shown because too many files have changed in this diff Show more