mirror of
https://github.com/trapexit/mergerfs.git
synced 2026-10-26 19:01:56 -04:00
Compare commits
No commits in common. "master" and "1.2.2" have entirely different histories.
1142 changed files with 6844 additions and 154081 deletions
2
.github/FUNDING.yml
vendored
2
.github/FUNDING.yml
vendored
|
|
@ -1,2 +0,0 @@
|
|||
github: [trapexit]
|
||||
custom: ['https://github.com/trapexit/support']
|
||||
77
.github/ISSUE_TEMPLATE/bug_report.md
vendored
77
.github/ISSUE_TEMPLATE/bug_report.md
vendored
|
|
@ -1,77 +0,0 @@
|
|||
---
|
||||
name: Bug report
|
||||
about: For the reporting of behavior not as described
|
||||
title: ''
|
||||
labels: bug, investigating
|
||||
assignees: ''
|
||||
|
||||
---
|
||||
|
||||
See also https://trapexit.github.io/mergerfs/latest/support
|
||||
|
||||
**Describe the bug**
|
||||
|
||||
A clear and concise description of the unexpected behavior.
|
||||
|
||||
**Please be sure to use latest release of mergerfs to ensure the issue
|
||||
still exists. Not your distro's latest but the latest official
|
||||
release. Support will NOT be provided for old versions.**
|
||||
|
||||
The `master` branch is **not** to be considered production ready. Feel
|
||||
free to file bug reports but do so indicating clearly that you are
|
||||
testing unreleased code.
|
||||
|
||||
|
||||
**To Reproduce**
|
||||
|
||||
Provide steps to reproduce the behavior. List **all** steps to
|
||||
reproduce. **All** settings. The version of all software involved.
|
||||
|
||||
Please simplify the reproduction as much as possible.
|
||||
* Unless it is dependenat on multiple branches, use a single branch
|
||||
* Reproduce with standard tooling if possible
|
||||
(touch,truncate,rm,rmdir,ln,etc.) Having to install 3rd party
|
||||
software will make debugging more difficult.
|
||||
|
||||
|
||||
**Expected behavior**
|
||||
|
||||
A clear and concise description of the expected behavior.
|
||||
|
||||
|
||||
**Workarounds already tested**
|
||||
|
||||
Please list all workarounds attempted and why they were ruled out as
|
||||
options.
|
||||
|
||||
|
||||
**System information:**
|
||||
|
||||
* Attach output from
|
||||
[mergerfs.collect-info](https://trapexit.github.io/mergerfs/latest/tooling/#mergerfscollect-info)
|
||||
tool.
|
||||
* Alternatively:
|
||||
* OS: `lsb_release -a`
|
||||
* kernel version: `uname -a`
|
||||
* mergerfs version: `mergerfs --version`
|
||||
* mergerfs settings / arguments: from fstab, systemd unit, command
|
||||
line, OMV plugin, etc.
|
||||
* List of drives, filesystems, & sizes:
|
||||
* `df -h`
|
||||
* `lsblk -o NAME,FSTYPE,FSSIZE,SIZE,MOUNTPOINTS,RM,RO,ROTA`
|
||||
* All information about the relevant paths and files: permissions,
|
||||
ownership, etc.
|
||||
* All information about the application making the requests:
|
||||
version, uid/gid
|
||||
* Is mergerfs running within a container or used with container
|
||||
technologies?
|
||||
* Are the client apps using mergerfs running in a container?
|
||||
* A strace of the application having a problem:
|
||||
* `strace -fvTtt -s 256 -o /tmp/app.strace.txt <cmd>`
|
||||
* `strace -fvTtt -s 256 -o /tmp/app.strace.txt -p <appPID>`
|
||||
* strace of mergerfs while app tried to do it's thing:
|
||||
* `strace -fvTtt -s 256 -p <mergerfsPID> -o /tmp/mergerfs.strace.txt`
|
||||
|
||||
**Additional context**
|
||||
|
||||
Add any other context about the problem here.
|
||||
20
.github/ISSUE_TEMPLATE/feature_request.md
vendored
20
.github/ISSUE_TEMPLATE/feature_request.md
vendored
|
|
@ -1,20 +0,0 @@
|
|||
---
|
||||
name: Feature request
|
||||
about: Suggest an idea for this project
|
||||
title: ''
|
||||
labels: feature, investigating
|
||||
assignees: ''
|
||||
|
||||
---
|
||||
|
||||
**Is your feature request related to a problem? Please describe.**
|
||||
A clear and concise description of what the problem is.
|
||||
|
||||
**Describe the solution you'd like**
|
||||
A clear and concise description of what you want to happen.
|
||||
|
||||
**Describe alternatives you've considered**
|
||||
A clear and concise description of any alternative solutions or features you've considered.
|
||||
|
||||
**Additional context**
|
||||
Add any other context or screenshots about the feature request here.
|
||||
10
.github/ISSUE_TEMPLATE/question.md
vendored
10
.github/ISSUE_TEMPLATE/question.md
vendored
|
|
@ -1,10 +0,0 @@
|
|||
---
|
||||
name: Question
|
||||
about: When you're looking for information or general support
|
||||
title: ''
|
||||
labels: investigating, question
|
||||
assignees: ''
|
||||
|
||||
---
|
||||
|
||||
For general questions and support please use [GitHub Discussions](https://github.com/trapexit/mergerfs/discussions) or [Discord](https://discord.gg/MpAr69V).
|
||||
20
.gitignore
vendored
20
.gitignore
vendored
|
|
@ -11,23 +11,3 @@
|
|||
*.lai
|
||||
*.la
|
||||
*.a
|
||||
|
||||
# build artifacts
|
||||
mergerfs
|
||||
build/
|
||||
src/version.hpp
|
||||
VERSION
|
||||
|
||||
# Debian files
|
||||
debian/files
|
||||
debian/changelog
|
||||
debian/debhelper-build-stamp
|
||||
debian/mergerfs.debhelper.log
|
||||
debian/mergerfs.substvars
|
||||
|
||||
# RPM files
|
||||
rpmbuild/
|
||||
|
||||
# mkdocs files
|
||||
mkdocs/venv
|
||||
__pycache__/
|
||||
|
|
|
|||
17
DEPENDENCIES
17
DEPENDENCIES
|
|
@ -1,17 +0,0 @@
|
|||
## Libraries
|
||||
|
||||
### included in repo
|
||||
|
||||
* libfuse: https://github.com/libfuse/libfuse (heavily modified fork of v2.x)
|
||||
* rapidhash: https://github.com/Nicoshev/rapidhash
|
||||
* fmt: https://github.com/fmtlib/fmt
|
||||
* concurrentqueue: https://github.com/cameron314/concurrentqueue
|
||||
* scope_guard: https://github.com/Neargye/scope_guard
|
||||
* subprocess: https://github.com/arun11299/cpp-subprocess
|
||||
* CLI11: https://github.com/CLIUtils/CLI11
|
||||
* string-lite: https://github.com/martinmoene/string-lite
|
||||
* boost
|
||||
* download boost
|
||||
* ./bootstrap.sh
|
||||
* ./b2 tools/bcp
|
||||
* ./dist/bin/bcp unordered/concurrent_flat_map.hpp <target_dir>
|
||||
30
LICENSE
30
LICENSE
|
|
@ -1,17 +1,23 @@
|
|||
/*
|
||||
ISC License
|
||||
The MIT License (MIT)
|
||||
|
||||
Copyright (c) 2026, Antonio SJ Musumeci <trapexit@spawn.link>
|
||||
Copyright (c) 2014 Antonio SJ Musumeci <trapexit@spawn.link>
|
||||
|
||||
Permission to use, copy, modify, and/or distribute this software for any
|
||||
purpose with or without fee is hereby granted, provided that the above
|
||||
copyright notice and this permission notice appear in all copies.
|
||||
Permission is hereby granted, free of charge, to any person obtaining a copy
|
||||
of this software and associated documentation files (the "Software"), to deal
|
||||
in the Software without restriction, including without limitation the rights
|
||||
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
|
||||
copies of the Software, and to permit persons to whom the Software is
|
||||
furnished to do so, subject to the following conditions:
|
||||
|
||||
THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES
|
||||
WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF
|
||||
MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR
|
||||
ANY SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES
|
||||
WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN
|
||||
ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF
|
||||
OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE.
|
||||
The above copyright notice and this permission notice shall be included in
|
||||
all copies or substantial portions of the Software.
|
||||
|
||||
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
|
||||
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
|
||||
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
|
||||
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
|
||||
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
|
||||
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
|
||||
THE SOFTWARE.
|
||||
*/
|
||||
|
|
|
|||
470
Makefile
470
Makefile
|
|
@ -1,409 +1,145 @@
|
|||
# Copyright (c) 2025, Antonio SJ Musumeci <trapexit@spawn.link>
|
||||
# The MIT License (MIT)
|
||||
#
|
||||
# Permission to use, copy, modify, and/or distribute this software for any
|
||||
# purpose with or without fee is hereby granted, provided that the above
|
||||
# copyright notice and this permission notice appear in all copies.
|
||||
# Copyright (c) 2014 Antonio SJ Musumeci <trapexit@spawn.link>
|
||||
#
|
||||
# THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES
|
||||
# WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF
|
||||
# MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR
|
||||
# ANY SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES
|
||||
# WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN
|
||||
# ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF
|
||||
# OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE.
|
||||
# Permission is hereby granted, free of charge, to any person obtaining a copy
|
||||
# of this software and associated documentation files (the "Software"), to deal
|
||||
# in the Software without restriction, including without limitation the rights
|
||||
# to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
|
||||
# copies of the Software, and to permit persons to whom the Software is
|
||||
# furnished to do so, subject to the following conditions:
|
||||
#
|
||||
# The above copyright notice and this permission notice shall be included in
|
||||
# all copies or substantial portions of the Software.
|
||||
#
|
||||
# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
|
||||
# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
|
||||
# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
|
||||
# AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
|
||||
# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
|
||||
# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
|
||||
# THE SOFTWARE.
|
||||
|
||||
$(info mergerfs MAKEFLAGS: $(MAKEFLAGS))
|
||||
PKGCONFIG = $(shell which pkg-config)
|
||||
GIT = $(shell which git)
|
||||
TAR = $(shell which tar)
|
||||
MKDIR = $(shell which mkdir)
|
||||
TOUCH = $(shell which touch)
|
||||
CP = $(shell which cp)
|
||||
RM = $(shell which rm)
|
||||
FIND = $(shell which find)
|
||||
INSTALL = $(shell which install)
|
||||
MKTEMP = $(shell which mktemp)
|
||||
STRIP = $(shell which strip)
|
||||
PANDOC = $(shell which pandoc)
|
||||
|
||||
ifeq ($(shell id -u),0)
|
||||
FAKEROOT ?=
|
||||
ifeq ($(PKGCONFIG),"")
|
||||
$(error "pkg-config not installed"
|
||||
endif
|
||||
|
||||
CP ?= cp
|
||||
FAKEROOT ?= fakeroot
|
||||
FIND ?= find
|
||||
GIT ?= git
|
||||
GIT2DEBCL ?= buildtools/git2debcl
|
||||
INSTALL ?= install
|
||||
LN ?= ln
|
||||
MKDIR ?= mkdir
|
||||
MKTEMP ?= mktemp
|
||||
MV ?= mv
|
||||
RM ?= rm
|
||||
RPMBUILD ?= rpmbuild
|
||||
RPMBUILD_FLAGS ?=
|
||||
SED ?= sed
|
||||
STRIP ?= strip
|
||||
TAR ?= tar
|
||||
TOUCH ?= touch
|
||||
|
||||
BUILDDIR := build
|
||||
GITHUB_REPO ?= "https://github.com/trapexit/mergerfs"
|
||||
|
||||
DEFAULT_TARGET := debian:13.amd64
|
||||
|
||||
ifndef GIT_REPO
|
||||
ifneq ($(shell $(GIT) --version 2> /dev/null),)
|
||||
ifeq ($(shell test -e .git; echo $$?),0)
|
||||
GIT_REPO = 1
|
||||
endif
|
||||
endif
|
||||
ifeq ($(PANDOC),"")
|
||||
$(warning "pandoc does not appear available: manpage won't be buildable")
|
||||
endif
|
||||
|
||||
USE_XATTR ?= 1
|
||||
XATTR_AVAILABLE = $(shell test ! -e /usr/include/attr/xattr.h; echo $$?)
|
||||
|
||||
ifdef SANITIZE
|
||||
ifeq ($(SANITIZE),1)
|
||||
override SANITIZE := -fsanitize=address,undefined,leak
|
||||
else ifeq ($(filter -fsanitize=%,$(SANITIZE)),)
|
||||
override SANITIZE := -fsanitize=$(SANITIZE)
|
||||
endif
|
||||
FUSE_AVAILABLE = $(shell ! pkg-config --exists fuse; echo $$?)
|
||||
|
||||
ifeq ($(FUSE_AVAILABLE),0)
|
||||
FUSE_AVAILABLE = $(shell test ! -e /usr/include/fuse.h; echo $$?)
|
||||
endif
|
||||
|
||||
COMMON_OPT_FLAGS += \
|
||||
-fstack-protector-strong \
|
||||
-Wextra \
|
||||
-Wno-unused-parameter
|
||||
|
||||
DEBUG_OPT_FLAGS += \
|
||||
-O0 \
|
||||
-g3 \
|
||||
-fno-omit-frame-pointer \
|
||||
$(SANITIZE)
|
||||
|
||||
ifdef RELEASE
|
||||
OPT_FLAGS := -O3 $(COMMON_OPT_FLAGS)
|
||||
else
|
||||
OPT_FLAGS ?= $(DEBUG_OPT_FLAGS) $(COMMON_OPT_FLAGS)
|
||||
ifeq ($(FUSE_AVAILABLE),0)
|
||||
$(error "FUSE development package doesn't appear available")
|
||||
endif
|
||||
|
||||
ifdef STATIC
|
||||
STATIC_FLAGS := -static
|
||||
else
|
||||
STATIC_FLAGS :=
|
||||
OPTS = -O2
|
||||
SRC = $(wildcard src/*.cpp)
|
||||
OBJ = $(SRC:src/%.cpp=obj/%.o)
|
||||
DEPS = $(OBJ:obj/%.o=obj/%.d)
|
||||
TARGET = mergerfs
|
||||
MANPAGE = $(TARGET).1
|
||||
CFLAGS = -g -Wall \
|
||||
$(OPTS) \
|
||||
$(shell $(PKGCONFIG) fuse --cflags) \
|
||||
-DFUSE_USE_VERSION=26 \
|
||||
-MMD
|
||||
LDFLAGS = $(shell $(PKGCONFIG) fuse --libs)
|
||||
|
||||
BINDIR = $(PREFIX)/usr/bin
|
||||
MANDIR = $(PREFIX)/usr/share/man/man1
|
||||
INSTALLTARGET = $(DESTDIR)/$(BINDIR)/$(TARGET)
|
||||
MANTARGET = $(DESTDIR)/$(MANDIR)/$(MANPAGE)
|
||||
|
||||
ifeq ($(XATTR_AVAILABLE),0)
|
||||
$(warning "xattr not available: disabling")
|
||||
CFLAGS += -DWITHOUT_XATTR
|
||||
endif
|
||||
|
||||
ifdef LTO
|
||||
LTO_FLAGS := -flto
|
||||
else
|
||||
LTO_FLAGS :=
|
||||
KERNEL = $(shell uname -s)
|
||||
ifeq ($(KERNEL),Linux)
|
||||
CFLAGS += -DLINUX
|
||||
endif
|
||||
ifeq ($(KERNEL),Darwin)
|
||||
CFLAGS += -DOSX
|
||||
endif
|
||||
|
||||
SRC := $(wildcard src/*.cpp)
|
||||
OBJS := $(SRC:src/%.cpp=build/.objs/%.cpp.o)
|
||||
DEPS := $(SRC:src/%.cpp=build/.objs/%.cpp.d)
|
||||
all: $(TARGET)
|
||||
|
||||
TESTS := $(wildcard tests/*.cpp)
|
||||
TESTS_OBJS := $(filter-out build/.objs/mergerfs.cpp.o,$(OBJS))
|
||||
TESTS_OBJS += $(TESTS:tests/%.cpp=build/.test_objs/%.cpp.o)
|
||||
TESTS_DEPS := $(TESTS:tests/%.cpp=build/.test_objs/%.cpp.d)
|
||||
TESTS_DEPS += $(DEPS)
|
||||
|
||||
MANPAGE := mergerfs.1
|
||||
CPPFLAGS ?=
|
||||
override CPPFLAGS += \
|
||||
-D_FILE_OFFSET_BITS=64 \
|
||||
-DBOOST_UNORDERED_DISABLE_PARALLEL_ALGORITHMS
|
||||
CFLAGS ?= \
|
||||
$(OPT_FLAGS) \
|
||||
-Wall \
|
||||
-Wno-unused-result \
|
||||
-pipe
|
||||
override CFLAGS += \
|
||||
-MMD \
|
||||
-MP
|
||||
CXXFLAGS ?= \
|
||||
$(OPT_FLAGS) \
|
||||
$(STATIC_FLAGS) \
|
||||
$(LTO_FLAGS) \
|
||||
-Wall \
|
||||
-Wpessimizing-move \
|
||||
-Werror=pessimizing-move \
|
||||
-Wredundant-move \
|
||||
-Werror=redundant-move \
|
||||
-Wno-unused-result \
|
||||
-pipe
|
||||
override CXXFLAGS += \
|
||||
-std=c++20 \
|
||||
-MMD \
|
||||
-MP
|
||||
override INC_FLAGS := \
|
||||
-Isrc \
|
||||
-Ivendored \
|
||||
-Ivendored/libfuse/include
|
||||
override MFS_FLAGS := \
|
||||
-DUSE_XATTR=$(USE_XATTR)
|
||||
override TESTS_FLAGS := \
|
||||
-Isrc \
|
||||
-Ivendored \
|
||||
-Ivendored/acutest \
|
||||
-DTESTS
|
||||
|
||||
LIBFUSE := vendored/libfuse/$(BUILDDIR)/libfuse.a
|
||||
LDFLAGS ?=
|
||||
LDLIBS := \
|
||||
-lrt \
|
||||
-pthread
|
||||
override LDLIBS += \
|
||||
$(LIBFUSE)
|
||||
ifeq ($(CXX),g++)
|
||||
GCC_VERSION := $(shell $(CXX) -dumpversion | cut -f1 -d.)
|
||||
ifeq ($(shell test $(GCC_VERSION) -lt 9; echo $$?),0)
|
||||
override LDLIBS += -lstdc++fs
|
||||
endif
|
||||
endif
|
||||
|
||||
# https://www.gnu.org/prep/standards/html_node/Directory-Variables.html
|
||||
DESTDIR ?=
|
||||
PREFIX ?= /usr/local
|
||||
EXEC_PREFIX ?= $(PREFIX)
|
||||
DATAROOTDIR ?= $(PREFIX)/share
|
||||
DATADIR ?= $(DATAROOTDIR)
|
||||
BINDIR ?= $(EXEC_PREFIX)/bin
|
||||
SBINDIR ?= $(EXEC_PREFIX)/sbin
|
||||
LIBDIR ?= $(EXEC_PREFIX)/lib
|
||||
MANDIR ?= $(DATAROOTDIR)/man
|
||||
MAN1DIR ?= $(MANDIR)/man1
|
||||
|
||||
INSTALLBINDIR ?= $(DESTDIR)$(BINDIR)
|
||||
INSTALLSBINDIR ?= $(DESTDIR)$(SBINDIR)
|
||||
INSTALLLIBDIR ?= $(DESTDIR)$(LIBDIR)/mergerfs
|
||||
INSTALLMAN1DIR ?= $(DESTDIR)$(MAN1DIR)
|
||||
|
||||
|
||||
.PHONY: all
|
||||
all: libfuse $(BUILDDIR)/mergerfs $(BUILDDIR)/fsck.mergerfs $(BUILDDIR)/mergerfs.collect-info
|
||||
|
||||
.PHONY: help
|
||||
help:
|
||||
@echo "usage: make ARG\n"
|
||||
@echo "USE_XATTR=0 - build program without xattrs functionality"
|
||||
@echo "STATIC=1 - build static binary"
|
||||
@echo "LTO=1 - build with link time optimization"
|
||||
@echo "SANITIZE=1 - build with sanitizers (address,undefined,leak)"
|
||||
@echo "SANITIZE=list - build with custom sanitizers (comma separated)"
|
||||
@echo "CLEANUP=1 - cleanup images between release build process"
|
||||
@echo "PKGDIR=/dir/ - location for release build pkgs"
|
||||
@echo "GITREF=gitref - gitref to use for release builds"
|
||||
@echo "usage: make"
|
||||
@echo "make XATTR_AVAILABLE=0 - to build program without xattrs functionality (auto discovered otherwise)"
|
||||
|
||||
$(TARGET): obj/obj-stamp $(OBJ)
|
||||
$(CXX) $(CFLAGS) $(OBJ) -o $@ $(LDFLAGS)
|
||||
|
||||
$(BUILDDIR)/mergerfs: $(LIBFUSE) src/version.hpp $(OBJS)
|
||||
$(CXX) $(CXXFLAGS) $(INC_FLAGS) $(MFS_FLAGS) $(CPPFLAGS) $(OBJS) -o $@ $(LDFLAGS) $(LDLIBS)
|
||||
|
||||
$(BUILDDIR)/fsck.mergerfs: $(BUILDDIR)/mergerfs
|
||||
$(LN) -sf "mergerfs" $@
|
||||
|
||||
$(BUILDDIR)/mergerfs.collect-info: $(BUILDDIR)/mergerfs
|
||||
$(LN) -sf "mergerfs" $@
|
||||
|
||||
$(BUILDDIR)/tests: $(BUILDDIR)/mergerfs $(TESTS_OBJS)
|
||||
$(CXX) $(CXXFLAGS) $(TESTS_FLAGS) $(INC_FLAGS) $(MFS_FLAGS) $(CPPFLAGS) $(TESTS_OBJS) -o $@ $(LDFLAGS) $(LDLIBS)
|
||||
|
||||
.PHONY: libfuse
|
||||
libfuse: $(LIBFUSE)
|
||||
$(MAKE) -C vendored/libfuse
|
||||
|
||||
$(LIBFUSE):
|
||||
$(MAKE) -C vendored/libfuse $(BUILDDIR)/libfuse.a RELEASE=$(RELEASE)
|
||||
|
||||
tests: $(BUILDDIR)/tests
|
||||
|
||||
.PHONY:
|
||||
changelog:
|
||||
ifdef GIT_REPO
|
||||
$(GIT2DEBCL) --name mergerfs > ChangeLog
|
||||
else
|
||||
@echo "WARNING: need git repo to generate ChangeLog"
|
||||
endif
|
||||
$(GIT) log --pretty --numstat --summary | git2cl > ChangeLog
|
||||
|
||||
.PHONY: version
|
||||
version: src/version.hpp
|
||||
|
||||
src/version.hpp:
|
||||
./buildtools/update-version
|
||||
|
||||
$(BUILDDIR)/stamp:
|
||||
$(MKDIR) -p $(BUILDDIR)/.objs
|
||||
$(MKDIR) -p $(BUILDDIR)/.test_objs
|
||||
obj/obj-stamp:
|
||||
$(MKDIR) -p obj
|
||||
$(TOUCH) $@
|
||||
|
||||
$(BUILDDIR)/.objs/%.cpp.o: src/%.cpp $(BUILDDIR)/stamp
|
||||
$(CXX) $(CXXFLAGS) $(INC_FLAGS) $(MFS_FLAGS) $(CPPFLAGS) -c $< -o $@
|
||||
obj/%.o: src/%.cpp
|
||||
$(CXX) $(CFLAGS) -c $< -o $@
|
||||
|
||||
$(BUILDDIR)/.test_objs/%.cpp.o: tests/%.cpp | $(BUILDDIR)/stamp
|
||||
$(CXX) $(CXXFLAGS) $(TESTS_FLAGS) $(INC_FLAGS) $(MFS_FLAGS) $(CPPFLAGS) -c $< -o $@
|
||||
clean:
|
||||
$(RM) -rf obj "$(TARGET)" "$(MANPAGE)"
|
||||
$(FIND) -name "*~" -delete
|
||||
|
||||
$(BUILDDIR)/preload.so: $(BUILDDIR)/stamp tools/preload.c
|
||||
$(CC) -shared -fPIC $(CFLAGS) -o $@ tools/preload.c
|
||||
|
||||
preload: $(BUILDDIR)/preload.so
|
||||
|
||||
.PHONY: clean
|
||||
clean: rpm-clean
|
||||
$(RM) -rf $(BUILDDIR)
|
||||
$(FIND) . -name "*~" -delete
|
||||
$(MAKE) -C vendored/libfuse clean
|
||||
|
||||
.PHONY: distclean
|
||||
distclean: clean
|
||||
ifdef GIT_REPO
|
||||
$(GIT) clean -xfd
|
||||
endif
|
||||
$(GIT) clean -ifd
|
||||
|
||||
.PHONY: install
|
||||
install: install-base install-mount-tools install-preload install-man
|
||||
install: $(TARGET) $(MANPAGE)
|
||||
$(INSTALL) -m 0755 -D "$(TARGET)" "$(INSTALLTARGET)"
|
||||
$(INSTALL) -m 0644 -D "$(MANPAGE)" "$(MANTARGET)"
|
||||
|
||||
.PHONY: install-base
|
||||
install-base: all
|
||||
$(MKDIR) -p "$(INSTALLBINDIR)"
|
||||
$(INSTALL) -v -m 0755 "$(BUILDDIR)/mergerfs" "$(INSTALLBINDIR)/mergerfs"
|
||||
$(LN) -fs "mergerfs" "${INSTALLBINDIR}/fsck.mergerfs"
|
||||
$(LN) -fs "mergerfs" "${INSTALLBINDIR}/mergerfs.collect-info"
|
||||
install-strip: install
|
||||
$(STRIP) "$(INSTALLTARGET)"
|
||||
|
||||
.PHONY: install-mount-tools
|
||||
install-mount-tools: install-base
|
||||
$(MKDIR) -p "$(INSTALLBINDIR)"
|
||||
$(MAKE) -C vendored/libfuse install
|
||||
uninstall:
|
||||
$(RM) "$(INSTALLTARGET)"
|
||||
$(RM) "$(MANTARGET)"
|
||||
|
||||
.PHONY: install-man
|
||||
install-man: man/$(MANPAGE)
|
||||
$(MKDIR) -p "$(INSTALLMAN1DIR)"
|
||||
$(INSTALL) -v -m 0644 "man/$(MANPAGE)" "$(INSTALLMAN1DIR)/$(MANPAGE)"
|
||||
$(MANPAGE): README.md
|
||||
$(PANDOC) -s -t man -o $(MANPAGE) README.md
|
||||
|
||||
.PHONY: install-preload
|
||||
install-preload: preload
|
||||
$(MKDIR) -p "$(INSTALLLIBDIR)"
|
||||
$(INSTALL) -v -m 444 "$(BUILDDIR)/preload.so" "$(INSTALLLIBDIR)/preload.so"
|
||||
man: $(MANPAGE)
|
||||
|
||||
.PHONY: install-strip
|
||||
install-strip: install-base
|
||||
$(STRIP) "$(INSTALLBINDIR)/mergerfs"
|
||||
|
||||
.PHONY: uninstall
|
||||
uninstall: uninstall-base uninstall-mount.mergerfs uninstall-preload uninstall-man
|
||||
|
||||
uninstall-base:
|
||||
$(RM) -f "$(INSTALLBINDIR)/mergerfs"
|
||||
$(RM) -f "$(INSTALLBINDIR)/fsck.mergerfs"
|
||||
$(RM) -f "$(INSTALLBINDIR)/mergerfs.collect-info"
|
||||
|
||||
uninstall-mount.mergerfs:
|
||||
$(RM) -f "$(INSTALLBINDIR)/mount.mergerfs"
|
||||
|
||||
uninstall-man:
|
||||
$(RM) -f "$(INSTALLMAN1DIR)/$(MANPAGE)"
|
||||
|
||||
uninstall-preload:
|
||||
$(RM) -f "$(INSTALLLIBDIR)/preload.so"
|
||||
|
||||
.PHONY: tarball
|
||||
tarball: changelog version
|
||||
$(eval VERSION := $(shell cat VERSION))
|
||||
$(eval VERSION := $(subst -,_,$(VERSION)))
|
||||
$(eval FILENAME := mergerfs-$(VERSION))
|
||||
tarball: clean changelog man
|
||||
$(eval VERSION := $(shell $(GIT) describe --always --tags --dirty))
|
||||
$(eval FILENAME := $(TARGET)-$(VERSION))
|
||||
$(eval TMPDIR := $(shell $(MKTEMP) --tmpdir -d .$(FILENAME).XXXXXXXX))
|
||||
$(MKDIR) $(TMPDIR)/$(FILENAME)
|
||||
$(CP) -ar . $(TMPDIR)/$(FILENAME)
|
||||
$(TAR) --exclude=.git -cz -C $(TMPDIR) -f $(FILENAME).tar.gz $(FILENAME)
|
||||
$(TAR) --exclude=.git -cz -C $(TMPDIR) -f ../$(FILENAME).tar.gz $(FILENAME)
|
||||
$(RM) -rf $(TMPDIR)
|
||||
|
||||
.PHONY: debian-changelog
|
||||
debian-changelog:
|
||||
ifdef GIT_REPO
|
||||
$(GIT2DEBCL) --name mergerfs > debian/changelog
|
||||
else
|
||||
$(CP) -v ChangeLog debian/changelog
|
||||
$(eval DISTRO := $(shell lsb_release -i -s | tr '[:upper:]' '[:lower:]'))
|
||||
$(eval CODENAME := $(shell lsb_release -c -s | tr '[:upper:]' '[:lower:]'))
|
||||
sed -i 's/~unknown-unknown/~$(DISTRO)-$(CODENAME)/g' debian/changelog
|
||||
sed -i 's/ unknown; / $(CODENAME); /g' debian/changelog
|
||||
endif
|
||||
|
||||
.PHONY: signed-deb
|
||||
signed-deb:
|
||||
$(MAKE) distclean
|
||||
$(MAKE) debian-changelog
|
||||
$(FAKEROOT) dpkg-buildpackage -nc
|
||||
|
||||
.PHONY: deb
|
||||
deb:
|
||||
$(MAKE) distclean
|
||||
$(MAKE) debian-changelog
|
||||
$(MAKE) version
|
||||
$(FAKEROOT) dpkg-buildpackage -nc -uc -us
|
||||
$(MKDIR) -p ./$(BUILDDIR)/pkgs/
|
||||
$(MV) -v ../mergerfs*deb ./$(BUILDDIR)/pkgs/
|
||||
$(eval VERSION := $(shell $(GIT) describe --always --tags --dirty))
|
||||
$(GIT) dch --auto --release --new-version="$(VERSION)"
|
||||
$(GIT) buildpackage --git-ignore-new
|
||||
|
||||
.PHONY: rpm-clean
|
||||
rpm-clean:
|
||||
$(RM) -rf rpmbuild
|
||||
.PHONY: all clean install help
|
||||
|
||||
.PHONY: rpm
|
||||
rpm: tarball
|
||||
$(eval VERSION := $(shell cat VERSION))
|
||||
$(eval VERSION := $(subst -,_,$(VERSION)))
|
||||
$(MKDIR) -p rpmbuild/BUILD rpmbuild/RPMS rpmbuild/SOURCES
|
||||
$(SED) 's/__VERSION__/$(VERSION)/g' mergerfs.spec > \
|
||||
rpmbuild/SOURCES/mergerfs.spec
|
||||
cp -ar mergerfs-$(VERSION).tar.gz rpmbuild/SOURCES
|
||||
$(RPMBUILD) $(RPMBUILD_FLAGS) -ba rpmbuild/SOURCES/mergerfs.spec \
|
||||
--define "_topdir $(CURDIR)/rpmbuild"
|
||||
|
||||
.PHONY: install-build-pkgs
|
||||
install-build-pkgs:
|
||||
./buildtools/install-build-pkgs
|
||||
|
||||
.PHONY: install-build-tools
|
||||
install-build-tools:
|
||||
./buildtools/install-build-tools
|
||||
|
||||
define build_release
|
||||
$(eval GITREF ?= $(shell git describe --exact-match --tags HEAD 2>/dev/null || git symbolic-ref --short HEAD 2>/dev/null || git rev-parse --short HEAD))
|
||||
@echo "GITREF=$(GITREF)"
|
||||
./buildtools/build-release \
|
||||
--target=$(1) \
|
||||
$(if $(CLEANUP),--cleanup) \
|
||||
$(if $(PKGDIR),--pkgdirpath=$(PKGDIR)) \
|
||||
--git-repo=$(if $(REMOTE_REPO),$(GITHUB_REPO),".") \
|
||||
--branch=$(GITREF)
|
||||
endef
|
||||
|
||||
.PHONY: release release-all
|
||||
.PHONY: release-amd64 release-arm64 release-armhf release-riscv64
|
||||
.PHONY: release-sample release-static release-tarball release-installer
|
||||
release:
|
||||
$(MAKE) RELEASE=1 CPPFLAGS="$(CPPFLAGS) -DNDEBUG" all
|
||||
release-all:
|
||||
$(call build_release,"all")
|
||||
release-sample:
|
||||
$(if $(TARGET),,$(eval TARGET := $(shell ./buildtools/detect-local-target 2>/dev/null)))
|
||||
$(call build_release,$(if $(TARGET),$(TARGET),$(DEFAULT_TARGET)))
|
||||
release-amd64:
|
||||
$(call build_release,"amd64")
|
||||
release-arm64:
|
||||
$(call build_release,"arm64")
|
||||
release-armhf:
|
||||
$(call build_release,"armhf")
|
||||
release-riscv64:
|
||||
$(call build_release,"riscv64")
|
||||
release-static:
|
||||
$(call build_release,"static")
|
||||
release-tarball:
|
||||
$(call build_release,"tarball")
|
||||
release-installer:
|
||||
$(call build_release,"installer")
|
||||
|
||||
container:
|
||||
$(eval GITREF ?= $(shell git describe --exact-match --tags HEAD 2>/dev/null || git symbolic-ref --short HEAD 2>/dev/null || git rev-parse --short HEAD))
|
||||
@echo "GITREF=$(GITREF)"
|
||||
./buildtools/build-containerimage "$(GITREF)"
|
||||
|
||||
.PHONY: tags
|
||||
tags:
|
||||
rm -fv TAGS
|
||||
find . -name "*.c" -print | etags --append -
|
||||
find . -name "*.h" -print | etags --append -
|
||||
find . -name "*.cpp" -print | etags --append -
|
||||
find . -name "*.hpp" -print | etags --append -
|
||||
|
||||
|
||||
-include $(DEPS)
|
||||
include $(wildcard obj/*.d)
|
||||
|
|
|
|||
201
README.md
201
README.md
|
|
@ -1,125 +1,136 @@
|
|||
# mergerfs - a featureful union filesystem
|
||||
% mergerfs(1) mergerfs user manual
|
||||
% Antonio SJ Musumeci <trapexit@spawn.link>
|
||||
% June 9, 2014
|
||||
|
||||
**mergerfs** is a
|
||||
[FUSE](https://en.wikipedia.org/wiki/Filesystem_in_Userspace) based
|
||||
[union filesystem](https://en.wikipedia.org/wiki/Union_mount) geared
|
||||
towards simplifying storage and management of files across numerous
|
||||
commodity storage devices. It is similar to [**mhddfs**, **unionfs**,
|
||||
**aufs**, **DrivePool**,
|
||||
etc.](https://trapexit.github.io/mergerfs/latest/project_comparisons/)
|
||||
# NAME
|
||||
|
||||
mergerfs - another FUSE union filesystem
|
||||
|
||||
## Features
|
||||
# SYNOPSIS
|
||||
|
||||
* Logically combine numerous filesystems/paths into a single
|
||||
mount point (JBOFS: Just a Bunch of FileSystems)
|
||||
* Combine paths of the same or different filesystems
|
||||
* Ability to add or remove filesystems/paths without impacting the
|
||||
rest of the data
|
||||
* Unaffected by individual filesystem failure
|
||||
* Configurable file selection and creation placement
|
||||
* File IO
|
||||
[passthrough](https://trapexit.github.io/mergerfs/latest/config/passthrough/)
|
||||
for near native IO performance (where supported)
|
||||
* Works with filesystems of any size
|
||||
* Works with filesystems of [almost any
|
||||
type](https://trapexit.github.io/mergerfs/latest/faq/compatibility_and_integration/#what-filesystems-can-be-used-as-branches)
|
||||
* Ignore read-only filesystems when creating files
|
||||
* [Hard links](https://trapexit.github.io/mergerfs/latest/faq/technical_behavior_and_limitations/#do-hard-links-work)
|
||||
* Hard link [copy-on-write /
|
||||
CoW](https://trapexit.github.io/mergerfs/latest/config/link_cow/)
|
||||
* [Runtime configurable](https://trapexit.github.io/mergerfs/latest/runtime_interface/)
|
||||
* Support for extended attributes (xattrs)
|
||||
* Support for file attributes (chattr)
|
||||
* Support for POSIX ACLs
|
||||
mergerfs -ocreate=epmfs,search=ff,action=ff <srcpoints> <mountpoint>
|
||||
|
||||
# DESCRIPTION
|
||||
|
||||
## Non-features
|
||||
mergerfs is similar to mhddfs, unionfs, and aufs. Like mhddfs in that it too uses FUSE. Like aufs in that it provides multiple policies for how to handle behavior.
|
||||
|
||||
* Read/write overlay on top of read-only filesystem like OverlayFS
|
||||
* File whiteout
|
||||
* RAID like parity calculation (see [SnapRAID](https://www.snapraid.it))
|
||||
* Redundancy
|
||||
* Splitting of files across branches
|
||||
Why create mergerfs when those exist? mhddfs isn't really maintained or flexible. There are also issues with running as root. aufs is more flexible but contains some hard to debug inconsistencies in behavior. Neither support file attributes ([chattr](http://linux.die.net/man/1/chattr)).
|
||||
|
||||
# OPTIONS
|
||||
|
||||
## How it works
|
||||
###options###
|
||||
|
||||
mergerfs logically merges multiple filesystem paths together. Not
|
||||
block devices, not filesystem mounts, just paths. It acts as a proxy
|
||||
to the underlying filesystem paths. Combining the behaviors of some
|
||||
functions and being a selector for others.
|
||||
| Option | Default |
|
||||
|--------|--------|
|
||||
| search | ff |
|
||||
| action | ff |
|
||||
| create | epmfs |
|
||||
|
||||
When the contents of a directory are requested mergerfs combines the
|
||||
list of files from each directory, deduplicating entries, and returns
|
||||
that list.
|
||||
###srcpoints###
|
||||
|
||||
When a file or directory is created a policy is first run to determine
|
||||
which branch will be selected for the creation.
|
||||
|
||||
For functions which change attributes or remove the file the behavior
|
||||
may be applied to all instances found.
|
||||
|
||||
The way in which mergerfs behaves is controlled by the
|
||||
[config/options/settings](https://trapexit.github.io/mergerfs/latest/config/options). More
|
||||
specifically by
|
||||
[policies](https://trapexit.github.io/mergerfs/latest/config/functions_categories_policies/).
|
||||
|
||||
|
||||
### Visualization
|
||||
The source points argument is a colon (':') delimited list of paths. To make it simplier to include multiple source points without having to modify your [fstab](http://linux.die.net/man/5/fstab) we also support [globbing](http://linux.die.net/man/7/glob).
|
||||
|
||||
```
|
||||
A + B = C
|
||||
/disk1 /disk2 /merged
|
||||
| | |
|
||||
+-- /dir1 +-- /dir1 +-- /dir1
|
||||
| | | | | |
|
||||
| +-- file1 | | | +-- file1
|
||||
| | +-- file2 | +-- file2
|
||||
| | +-- file3 | +-- file3
|
||||
| | |
|
||||
+-- /dir2 | +-- /dir2
|
||||
| | | | |
|
||||
| +-- file4 | | +-- file4
|
||||
| | |
|
||||
| +-- /dir3 +-- /dir3
|
||||
| | | | |
|
||||
| | +-- file5 | +-- file5
|
||||
| | |
|
||||
+-- file6 | +-- file6
|
||||
+-- file7 +-- file7 +-- file7
|
||||
$ mergerfs /mnt/disk*:/mnt/cdrom /media/drives
|
||||
```
|
||||
|
||||
The above line will use all points in /mnt prefixed with *disk* and the directory *cdrom*.
|
||||
|
||||
## QuickStart
|
||||
In /etc/fstab it'd look like the following:
|
||||
|
||||
https://trapexit.github.io/mergerfs/latest/quickstart/
|
||||
```
|
||||
# <file system> <mount point> <type> <options> <dump> <pass>
|
||||
/mnt/disk*:/mnt/cdrom /media/drives fuse.mergerfs allow_other 0 0
|
||||
```
|
||||
|
||||
**NOTE:** the globbing is done at mount time. If a new directory is added matching the glob after the fact it will not be included.
|
||||
|
||||
# POLICIES
|
||||
|
||||
Filesystem calls are broken up into 4 functional categories: search, action, create, and none. These categories can be assigned a policy which dictates how [mergerfs](http://github.com/trapexit/mergerfs) behaves while when action on the filesystem. Any policy can be assigned to a category though some aren't terribly practical. For instance: rand (Random) may be useful for **create** but could lead to very odd behavior if used for **search** or **action**. Since the input for any policy is the source mounts and fusepath and the output a vector of targets the choice was made to simplify the implementation and allow a policies usage in any category. **NOTE:** In any policy which can return more than one location (currently only **all**) the first value will be used in **search** and **create** policies since they can only ever act on 1 filepath.
|
||||
|
||||
#### Functional classifications ####
|
||||
| Class | FUSE calls |
|
||||
|-------|------------|
|
||||
| search | access, getattr, getxattr, listxattr, open, readlink |
|
||||
| action | chmod, link, removexattr, rmdir, setxattr, truncate, unlink, utimens |
|
||||
| create | create, mkdir, mknod |
|
||||
| none | fallocate, fgetattr, fsync, ftruncate, ioctl, read, readdir, rename, statfs, symlink, write, release |
|
||||
|
||||
#### Policy descriptions ####
|
||||
| Policy | Description |
|
||||
|--------------|-------------|
|
||||
| ff (first found) | Given the order the paths were provided at mount time act on the first one found (regardless if stat would return EACCES). |
|
||||
| ffwp (first found w/ permissions) | Given the order the paths were provided at mount time act on the first one found which you have access (stat does not error with EACCES). |
|
||||
| newest (newest file) | If multiple files exist return the one with the most recent mtime. |
|
||||
| all (all files found) | Attempt to apply the call to each file found. If any sub call succeeds the entire operation succeeds and other errors ignored. If all fail then the last error is reported. |
|
||||
| mfs (most free space) | Assuming the path is found to exist (ENOENT would not be returned) use the drive with the most free space available. |
|
||||
| epmfs (existing path, most free space) | If the path exists in multiple locations use the one with the most free space. Otherwise fall back to mfs. |
|
||||
| rand (random) | Pick a destination at random. Again the dirname of the full path must exist somewhere. |
|
||||
|
||||
#### statvfs ####
|
||||
|
||||
It normalizes the source drives based on the fragment size and sums the number of adjusted blocks and inodes. This means you will see the combined space of all sources. Total, used, and free. The sources however are dedupped based on the drive so multiple points on the same drive will not result in double counting it's space.
|
||||
|
||||
**NOTE:** create is really a search for existence and then create. The 'search' policy applies to the first part. If the [dirname](http://linux.die.net/man/3/dirname) of the full path is not found to exist [ENOENT](http://linux.die.net/man/3/errno) is returned.
|
||||
|
||||
# BUILDING
|
||||
|
||||
* Need to install FUSE development libraries (libfuse-dev).
|
||||
* Optionally need libattr1 (libattr1-dev).
|
||||
|
||||
|
||||
## Documentation
|
||||
```
|
||||
[trapexit:~/dev/mergerfs] $ make help
|
||||
usage: make
|
||||
make XATTR_AVAILABLE=0 - to build program without xattrs functionality (auto discovered otherwise)
|
||||
```
|
||||
|
||||
https://trapexit.github.io/mergerfs
|
||||
# Runtime Settings
|
||||
|
||||
#### /.mergerfs pseudo file ####
|
||||
```
|
||||
<mountpoint>/.mergerfs
|
||||
```
|
||||
|
||||
## Support
|
||||
There is a pseudo file available at the mountpoint which allows for the runtime modification of policies. The file will not show up in readdirs but can be stat'ed, read, and writen. Most other calls will fail with EPERM, EINVAL, or whatever may be appropriate for that call. Anything not understood while writing will result in EINVAL otherwise the number of bytes written will be returned.
|
||||
|
||||
https://trapexit.github.io/mergerfs/latest/support/
|
||||
Reading the file will result in a newline delimited list of current settings as followed:
|
||||
|
||||
```
|
||||
[trapexit:/tmp/mount] $ cat .mergerfs
|
||||
action=ff
|
||||
create=epmfs
|
||||
search=ff
|
||||
```
|
||||
|
||||
## Sponsorship and Donations
|
||||
Writing to the file is buffered and waits till a newline to process. Meaning echo works well.
|
||||
|
||||
[https://github.com/trapexit/support](https://github.com/trapexit/support)
|
||||
```
|
||||
[trapexit:/tmp/mount] $ echo "search=newest" >> .mergerfs
|
||||
[trapexit:/tmp/mount] $ cat .mergerfs
|
||||
action=ff
|
||||
create=epmfs
|
||||
search=newest
|
||||
```
|
||||
|
||||
Development and support of a project like mergerfs requires a
|
||||
significant amount of time and effort. The software is released under
|
||||
the very liberal [ISC](https://opensource.org/license/isc-license-txt)
|
||||
license and is therefore free to use for personal or commercial uses.
|
||||
*NOTE:* offset is not supported and ignored in both read and write. There is also a safety check which limits buffered + incoming length to a max of 1024 bytes.
|
||||
|
||||
If you are a non-commercial user and find mergerfs and its support valuable
|
||||
and would like to support the project financially it would be very
|
||||
much appreciated.
|
||||
#### xattrs ####
|
||||
|
||||
If you are using mergerfs commercially please consider sponsoring the
|
||||
project to ensure it continues to be maintained and receive
|
||||
updates. If custom features are needed feel free to [contact me
|
||||
directly](mailto:support@spawn.link).
|
||||
If xattrs has been enabled you can also use [{list,get,set}xattrs](http://linux.die.net/man/2/listxattr) on the pseudo file **.mergerfs** to modify the policies. The keys are **mergerfs.action**, **mergerfs.create**, and **mergerfs.search**.
|
||||
|
||||
```
|
||||
[trapexit:/tmp/mount] $ attr -l .mergerfs
|
||||
Attribute "mergerfs.action" has a 2 byte value for .mergerfs
|
||||
Attribute "mergerfs.create" has a 5 byte value for .mergerfs
|
||||
Attribute "mergerfs.search" has a 2 byte value for .mergerfs
|
||||
|
||||
[trapexit:/tmp/mount] $ attr -g mergerfs.action .mergerfs
|
||||
Attribute "mergerfs.action" had a 2 byte value for .mergerfs:
|
||||
ff
|
||||
|
||||
[trapexit:/tmp/mount] 1 $ attr -s mergerfs.action -V ffwp .mergerfs
|
||||
Attribute "mergerfs.action" set to a 4 byte value for .mergerfs:
|
||||
ffwp
|
||||
```
|
||||
|
|
|
|||
|
|
@ -1,16 +0,0 @@
|
|||
#!/bin/sh
|
||||
|
||||
GIT_BRANCH="${1:-master}"
|
||||
|
||||
set -x
|
||||
podman \
|
||||
build \
|
||||
--squash \
|
||||
--no-cache \
|
||||
--force-rm \
|
||||
--build-arg="GIT_REPO=file:///mnt" \
|
||||
--build-arg="BRANCH=${GIT_BRANCH}" \
|
||||
-v "${PWD}":/mnt:ro \
|
||||
-f "buildtools/containerimage/Containerfile" \
|
||||
--tag "ghcr.io/trapexit/mergerfs:${GIT_BRANCH}" \
|
||||
buildtools/
|
||||
|
|
@ -1,76 +0,0 @@
|
|||
#!/bin/sh
|
||||
set -e
|
||||
|
||||
if [ -e /tmp/build-env ]; then
|
||||
. /tmp/build-env
|
||||
fi
|
||||
|
||||
rpmbuild_flags()
|
||||
{
|
||||
case "$(command -v "${CXX:-g++}")" in
|
||||
/opt/rh/*)
|
||||
printf '%s' '--undefine _annotated_build'
|
||||
;;
|
||||
esac
|
||||
}
|
||||
|
||||
if [ $# -ne 2 ]; then
|
||||
echo "usage: $0 <branch> <git_repo>"
|
||||
echo "argv: $@"
|
||||
echo "argc: $#"
|
||||
exit 1
|
||||
fi
|
||||
|
||||
GIT_REPO="${1}"
|
||||
BRANCH="${2}"
|
||||
BUILDDIR="/tmp/build"
|
||||
|
||||
echo "Clone source from ${GIT_REPO} to ${BUILDDIR}"
|
||||
git \
|
||||
clone \
|
||||
--single-branch \
|
||||
--branch="${BRANCH}" \
|
||||
"${GIT_REPO}" \
|
||||
"${BUILDDIR}"
|
||||
|
||||
cd "${BUILDDIR}"
|
||||
|
||||
git log HEAD^1..
|
||||
|
||||
mkdir "/build"
|
||||
if [ -e /usr/bin/apt-get ]; then
|
||||
make deb
|
||||
elif [ -e /usr/bin/dnf ]; then
|
||||
make RPMBUILD_FLAGS="$(rpmbuild_flags)" rpm
|
||||
elif [ -e /usr/bin/yum ]; then
|
||||
make RPMBUILD_FLAGS="$(rpmbuild_flags)" rpm
|
||||
elif [ -e /sbin/apk ]; then
|
||||
echo "NOT YET SUPPORTED"
|
||||
exit 1
|
||||
elif [ -e /usr/sbin/pkg ]; then
|
||||
echo "NOT YET SUPPORTED"
|
||||
exit 1
|
||||
elif [ -e /usr/bin/pacman ]; then
|
||||
make tarball
|
||||
else
|
||||
echo "NOT YET SUPPORTED"
|
||||
exit 1
|
||||
fi
|
||||
|
||||
echo "Copy packages to host..."
|
||||
if [ -e /usr/bin/pacman ]; then
|
||||
find "${BUILDDIR}" \
|
||||
-type f \
|
||||
-name "*.tar.gz" \
|
||||
-exec cp -v {} /build/ \;
|
||||
else
|
||||
find "${BUILDDIR}" \
|
||||
-type f \
|
||||
\( -name "*.deb" -or -name "*.rpm" \) \
|
||||
-not -name "*dbgsym*" \
|
||||
-not -name "*sym*" \
|
||||
-not -name "*src.rpm" \
|
||||
-exec cp -v {} /build/ \;
|
||||
fi
|
||||
|
||||
exit 0
|
||||
|
|
@ -1,178 +0,0 @@
|
|||
#!/usr/bin/env python3
|
||||
|
||||
import subprocess
|
||||
import sys
|
||||
import os
|
||||
import argparse
|
||||
import time
|
||||
|
||||
|
||||
def build(git_repo,
|
||||
branch,
|
||||
containerfile,
|
||||
pkgdirpath):
|
||||
timestamp = time.time_ns()
|
||||
args = ['podman',
|
||||
'build',
|
||||
'--pull=always',
|
||||
'--force-rm']
|
||||
if os.path.exists(git_repo):
|
||||
git_repo = os.path.realpath(git_repo)
|
||||
args += ['-v',f'{git_repo}:/tmp/mergerfs.git:ro:z']
|
||||
args += ['--build-arg=GIT_REPO=file:///tmp/mergerfs.git']
|
||||
else:
|
||||
args += [f'--build-arg=GIT_REPO={git_repo}']
|
||||
os.makedirs(pkgdirpath,exist_ok=True)
|
||||
args += ['-v',f'{os.path.realpath(pkgdirpath)}:/pkgs:ro']
|
||||
args += ['-o',pkgdirpath,
|
||||
'-f',containerfile,
|
||||
f'--build-arg=BRANCH={branch}',
|
||||
f'--build-arg=BUILD_TIMESTAMP={timestamp}',
|
||||
'buildtools/']
|
||||
# TODO: Capture output and write to log
|
||||
print(args)
|
||||
rv = subprocess.run(args)
|
||||
report_filepath = os.path.join(pkgdirpath,"build-report.txt")
|
||||
with open(report_filepath,"a+") as f:
|
||||
build = os.path.basename(containerfile)
|
||||
f.write(build + ": ")
|
||||
f.write(f"branch={branch}; ")
|
||||
f.write(f"timestamp={timestamp}; ")
|
||||
f.write("rv=")
|
||||
if rv.returncode == 0:
|
||||
f.write("success;")
|
||||
else:
|
||||
f.write("fail;")
|
||||
f.write("\n")
|
||||
|
||||
|
||||
def setup():
|
||||
# Detect distribution
|
||||
if os.path.exists('/usr/bin/apt-get'):
|
||||
args = ['sudo',
|
||||
'apt-get',
|
||||
'install',
|
||||
'-fy',
|
||||
'qemu-user-static',
|
||||
'qemu-user-binfmt']
|
||||
elif os.path.exists('/usr/bin/pacman'):
|
||||
args = ['sudo',
|
||||
'pacman',
|
||||
'-Sy',
|
||||
'--noconfirm',
|
||||
'qemu-user-static']
|
||||
else:
|
||||
print("Unsupported distribution for setup")
|
||||
sys.exit(1)
|
||||
print(args)
|
||||
subprocess.run(args)
|
||||
setup_binfmt()
|
||||
|
||||
|
||||
def setup_binfmt():
|
||||
"""Register qemu-user-static binfmt handlers (needed on Arch, etc.)."""
|
||||
binfmt_conf = """\
|
||||
:qemu-aarch64:M::\\x7fELF\\x02\\x01\\x01\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x02\\x00\\xb7\\x00:\\xff\\xff\\xff\\xff\\xff\\xff\\xff\\x00\\xff\\xff\\xff\\xff\\xff\\xff\\xff\\xff\\xfe\\xff\\xff\\xff:/usr/bin/qemu-aarch64-static:F
|
||||
:qemu-arm:M::\\x7fELF\\x01\\x01\\x01\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x02\\x00\\x28\\x00:\\xff\\xff\\xff\\xff\\xff\\xff\\xff\\x00\\xff\\xff\\xff\\xff\\xff\\xff\\xff\\xff\\xfe\\xff\\xff\\xff:/usr/bin/qemu-arm-static:F
|
||||
:qemu-armeb:M::\\x7fELF\\x01\\x02\\x01\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x02\\x00\\x28:\\xff\\xff\\xff\\xff\\xff\\xff\\xff\\x00\\xff\\xff\\xff\\xff\\xff\\xff\\xff\\xff\\xff\\xfe\\xff\\xff:/usr/bin/qemu-armeb-static:F
|
||||
:qemu-riscv64:M::\\x7fELF\\x02\\x01\\x01\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x02\\x00\\xf3\\x00:\\xff\\xff\\xff\\xff\\xff\\xff\\xff\\x00\\xff\\xff\\xff\\xff\\xff\\xff\\xff\\xff\\xfe\\xff\\xff\\xff:/usr/bin/qemu-riscv64-static:F
|
||||
:qemu-i386:M::\\x7fELF\\x01\\x01\\x01\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x02\\x00\\x03\\x00:\\xff\\xff\\xff\\xff\\xff\\xff\\xff\\x00\\xff\\xff\\xff\\xff\\xff\\xff\\xff\\xff\\xfe\\xff\\xff\\xff:/usr/bin/qemu-i386-static:F
|
||||
"""
|
||||
conf_path = '/etc/binfmt.d/qemu-static.conf'
|
||||
print(f"Writing {conf_path}")
|
||||
try:
|
||||
with open(conf_path, 'w') as f:
|
||||
f.write(binfmt_conf)
|
||||
except PermissionError:
|
||||
print(f"Permission denied writing {conf_path}; trying with sudo...")
|
||||
subprocess.run(['sudo', 'tee', conf_path],
|
||||
input=binfmt_conf.encode(), check=True)
|
||||
|
||||
print("Restarting systemd-binfmt...")
|
||||
subprocess.run(['sudo', 'systemctl', 'restart', 'systemd-binfmt'])
|
||||
|
||||
|
||||
def podman_cleanup():
|
||||
args = ['podman',
|
||||
'system',
|
||||
'prune',
|
||||
'-af']
|
||||
print(args)
|
||||
subprocess.run(args)
|
||||
|
||||
|
||||
def parse_args():
|
||||
p = argparse.ArgumentParser()
|
||||
p.add_argument('--target',
|
||||
default='debian:13.amd64')
|
||||
p.add_argument('--pkgdirpath',
|
||||
default='build/pkgs/')
|
||||
p.add_argument('--branch',
|
||||
default='master')
|
||||
p.add_argument('--setup',
|
||||
required=False,
|
||||
action='store_true')
|
||||
p.add_argument('--cleanup',
|
||||
required=False,
|
||||
action='store_true')
|
||||
p.add_argument('--git-repo',
|
||||
required=False,
|
||||
type=str)
|
||||
|
||||
return p.parse_args()
|
||||
|
||||
|
||||
def should_skip(filename):
|
||||
if filename.endswith('~'):
|
||||
return True
|
||||
return False
|
||||
|
||||
|
||||
def main():
|
||||
args = parse_args()
|
||||
print(args)
|
||||
|
||||
if args.setup:
|
||||
setup()
|
||||
sys.exit(0)
|
||||
|
||||
if not args.git_repo:
|
||||
print("error: the following arguments are required: --git-repo")
|
||||
sys.exit(1)
|
||||
|
||||
containerfiles = []
|
||||
basepath = 'buildtools/containerfiles'
|
||||
if os.path.exists(f'{basepath}/{args.target}'):
|
||||
containerfiles.append(f'{basepath}/{args.target}')
|
||||
elif args.target == 'all':
|
||||
for root,dirnames,filenames in os.walk(basepath):
|
||||
filenames.sort()
|
||||
for filename in filenames:
|
||||
if should_skip(filename):
|
||||
continue
|
||||
containerfile = f'buildtools/containerfiles/{filename}'
|
||||
containerfiles.append(containerfile)
|
||||
else:
|
||||
for root,dirnames,filenames in os.walk(basepath):
|
||||
filenames.sort()
|
||||
for filename in filenames:
|
||||
if args.target not in filename:
|
||||
continue
|
||||
if should_skip(filename):
|
||||
continue
|
||||
containerfile = f'buildtools/containerfiles/{filename}'
|
||||
containerfiles.append(containerfile)
|
||||
|
||||
for containerfile in containerfiles:
|
||||
if args.cleanup:
|
||||
podman_cleanup()
|
||||
build(git_repo=args.git_repo,
|
||||
branch=args.branch,
|
||||
containerfile=containerfile,
|
||||
pkgdirpath=args.pkgdirpath)
|
||||
|
||||
sys.exit(0)
|
||||
|
||||
|
||||
if __name__ == '__main__':
|
||||
main()
|
||||
|
|
@ -1,10 +0,0 @@
|
|||
FROM --platform=amd64 debian:11 as build
|
||||
COPY install-build-pkgs build-mergerfs /tmp/
|
||||
RUN /tmp/install-build-pkgs
|
||||
ARG BUILD_TIMESTAMP=0
|
||||
ARG GIT_REPO
|
||||
ARG BRANCH
|
||||
RUN /tmp/build-mergerfs $GIT_REPO $BRANCH
|
||||
|
||||
FROM scratch
|
||||
COPY --from=build /build/ /
|
||||
|
|
@ -1,10 +0,0 @@
|
|||
FROM --platform=arm64 debian:11 as build
|
||||
COPY install-build-pkgs build-mergerfs /tmp/
|
||||
RUN /tmp/install-build-pkgs
|
||||
ARG BUILD_TIMESTAMP=0
|
||||
ARG GIT_REPO
|
||||
ARG BRANCH
|
||||
RUN /tmp/build-mergerfs $GIT_REPO $BRANCH
|
||||
|
||||
FROM scratch
|
||||
COPY --from=build /build/ /
|
||||
|
|
@ -1,10 +0,0 @@
|
|||
FROM --platform=armhf debian:11 as build
|
||||
COPY install-build-pkgs build-mergerfs /tmp/
|
||||
RUN /tmp/install-build-pkgs
|
||||
ARG BUILD_TIMESTAMP=0
|
||||
ARG GIT_REPO
|
||||
ARG BRANCH
|
||||
RUN /tmp/build-mergerfs $GIT_REPO $BRANCH
|
||||
|
||||
FROM scratch
|
||||
COPY --from=build /build/ /
|
||||
|
|
@ -1,10 +0,0 @@
|
|||
FROM --platform=i386 debian:11 as build
|
||||
COPY install-build-pkgs build-mergerfs /tmp/
|
||||
RUN /tmp/install-build-pkgs
|
||||
ARG BUILD_TIMESTAMP=0
|
||||
ARG GIT_REPO
|
||||
ARG BRANCH
|
||||
RUN /tmp/build-mergerfs $GIT_REPO $BRANCH
|
||||
|
||||
FROM scratch
|
||||
COPY --from=build /build/ /
|
||||
|
|
@ -1,10 +0,0 @@
|
|||
FROM --platform=amd64 debian:12 as build
|
||||
COPY install-build-pkgs build-mergerfs /tmp/
|
||||
RUN /tmp/install-build-pkgs
|
||||
ARG BUILD_TIMESTAMP=0
|
||||
ARG GIT_REPO
|
||||
ARG BRANCH
|
||||
RUN /tmp/build-mergerfs $GIT_REPO $BRANCH
|
||||
|
||||
FROM scratch
|
||||
COPY --from=build /build/ /
|
||||
|
|
@ -1,10 +0,0 @@
|
|||
FROM --platform=arm64 debian:12 as build
|
||||
COPY install-build-pkgs build-mergerfs /tmp/
|
||||
RUN /tmp/install-build-pkgs
|
||||
ARG BUILD_TIMESTAMP=0
|
||||
ARG GIT_REPO
|
||||
ARG BRANCH
|
||||
RUN /tmp/build-mergerfs $GIT_REPO $BRANCH
|
||||
|
||||
FROM scratch
|
||||
COPY --from=build /build/ /
|
||||
|
|
@ -1,10 +0,0 @@
|
|||
FROM --platform=armhf debian:12 as build
|
||||
COPY install-build-pkgs build-mergerfs /tmp/
|
||||
RUN /tmp/install-build-pkgs
|
||||
ARG BUILD_TIMESTAMP=0
|
||||
ARG GIT_REPO
|
||||
ARG BRANCH
|
||||
RUN /tmp/build-mergerfs $GIT_REPO $BRANCH
|
||||
|
||||
FROM scratch
|
||||
COPY --from=build /build/ /
|
||||
|
|
@ -1,10 +0,0 @@
|
|||
FROM --platform=i386 debian:12 as build
|
||||
COPY install-build-pkgs build-mergerfs /tmp/
|
||||
RUN /tmp/install-build-pkgs
|
||||
ARG BUILD_TIMESTAMP=0
|
||||
ARG GIT_REPO
|
||||
ARG BRANCH
|
||||
RUN /tmp/build-mergerfs $GIT_REPO $BRANCH
|
||||
|
||||
FROM scratch
|
||||
COPY --from=build /build/ /
|
||||
|
|
@ -1,10 +0,0 @@
|
|||
FROM --platform=amd64 debian:13 as build
|
||||
COPY install-build-pkgs build-mergerfs /tmp/
|
||||
RUN /tmp/install-build-pkgs
|
||||
ARG BUILD_TIMESTAMP=0
|
||||
ARG GIT_REPO
|
||||
ARG BRANCH
|
||||
RUN /tmp/build-mergerfs $GIT_REPO $BRANCH
|
||||
|
||||
FROM scratch
|
||||
COPY --from=build /build/ /
|
||||
|
|
@ -1,10 +0,0 @@
|
|||
FROM --platform=arm64 debian:13 as build
|
||||
COPY install-build-pkgs build-mergerfs /tmp/
|
||||
RUN /tmp/install-build-pkgs
|
||||
ARG BUILD_TIMESTAMP=0
|
||||
ARG GIT_REPO
|
||||
ARG BRANCH
|
||||
RUN /tmp/build-mergerfs $GIT_REPO $BRANCH
|
||||
|
||||
FROM scratch
|
||||
COPY --from=build /build/ /
|
||||
|
|
@ -1,10 +0,0 @@
|
|||
FROM --platform=armhf debian:13 as build
|
||||
COPY install-build-pkgs build-mergerfs /tmp/
|
||||
RUN /tmp/install-build-pkgs
|
||||
ARG BUILD_TIMESTAMP=0
|
||||
ARG GIT_REPO
|
||||
ARG BRANCH
|
||||
RUN /tmp/build-mergerfs $GIT_REPO $BRANCH
|
||||
|
||||
FROM scratch
|
||||
COPY --from=build /build/ /
|
||||
|
|
@ -1,10 +0,0 @@
|
|||
FROM --platform=i386 debian:13 as build
|
||||
COPY install-build-pkgs build-mergerfs /tmp/
|
||||
RUN /tmp/install-build-pkgs
|
||||
ARG BUILD_TIMESTAMP=0
|
||||
ARG GIT_REPO
|
||||
ARG BRANCH
|
||||
RUN /tmp/build-mergerfs $GIT_REPO $BRANCH
|
||||
|
||||
FROM scratch
|
||||
COPY --from=build /build/ /
|
||||
|
|
@ -1,10 +0,0 @@
|
|||
FROM --platform=riscv64 debian:13 as build
|
||||
COPY install-build-pkgs build-mergerfs /tmp/
|
||||
RUN /tmp/install-build-pkgs
|
||||
ARG BUILD_TIMESTAMP=0
|
||||
ARG GIT_REPO
|
||||
ARG BRANCH
|
||||
RUN /tmp/build-mergerfs $GIT_REPO $BRANCH
|
||||
|
||||
FROM scratch
|
||||
COPY --from=build /build/ /
|
||||
|
|
@ -1,10 +0,0 @@
|
|||
FROM --platform=amd64 fedora:40 as build
|
||||
COPY install-build-pkgs build-mergerfs /tmp/
|
||||
RUN /tmp/install-build-pkgs
|
||||
ARG BUILD_TIMESTAMP=0
|
||||
ARG GIT_REPO
|
||||
ARG BRANCH
|
||||
RUN /tmp/build-mergerfs $GIT_REPO $BRANCH
|
||||
|
||||
FROM scratch
|
||||
COPY --from=build /build/ /
|
||||
|
|
@ -1,10 +0,0 @@
|
|||
FROM --platform=arm64 fedora:40 as build
|
||||
COPY install-build-pkgs build-mergerfs /tmp/
|
||||
RUN /tmp/install-build-pkgs
|
||||
ARG BUILD_TIMESTAMP=0
|
||||
ARG GIT_REPO
|
||||
ARG BRANCH
|
||||
RUN /tmp/build-mergerfs $GIT_REPO $BRANCH
|
||||
|
||||
FROM scratch
|
||||
COPY --from=build /build/ /
|
||||
|
|
@ -1,10 +0,0 @@
|
|||
FROM --platform=amd64 fedora:41 as build
|
||||
COPY install-build-pkgs build-mergerfs /tmp/
|
||||
RUN /tmp/install-build-pkgs
|
||||
ARG BUILD_TIMESTAMP=0
|
||||
ARG GIT_REPO
|
||||
ARG BRANCH
|
||||
RUN /tmp/build-mergerfs $GIT_REPO $BRANCH
|
||||
|
||||
FROM scratch
|
||||
COPY --from=build /build/ /
|
||||
|
|
@ -1,10 +0,0 @@
|
|||
FROM --platform=arm64 fedora:41 as build
|
||||
COPY install-build-pkgs build-mergerfs /tmp/
|
||||
RUN /tmp/install-build-pkgs
|
||||
ARG BUILD_TIMESTAMP=0
|
||||
ARG GIT_REPO
|
||||
ARG BRANCH
|
||||
RUN /tmp/build-mergerfs $GIT_REPO $BRANCH
|
||||
|
||||
FROM scratch
|
||||
COPY --from=build /build/ /
|
||||
|
|
@ -1,10 +0,0 @@
|
|||
FROM --platform=amd64 fedora:42 as build
|
||||
COPY install-build-pkgs build-mergerfs /tmp/
|
||||
RUN /tmp/install-build-pkgs
|
||||
ARG BUILD_TIMESTAMP=0
|
||||
ARG GIT_REPO
|
||||
ARG BRANCH
|
||||
RUN /tmp/build-mergerfs $GIT_REPO $BRANCH
|
||||
|
||||
FROM scratch
|
||||
COPY --from=build /build/ /
|
||||
|
|
@ -1,10 +0,0 @@
|
|||
FROM --platform=arm64 fedora:42 as build
|
||||
COPY install-build-pkgs build-mergerfs /tmp/
|
||||
RUN /tmp/install-build-pkgs
|
||||
ARG BUILD_TIMESTAMP=0
|
||||
ARG GIT_REPO
|
||||
ARG BRANCH
|
||||
RUN /tmp/build-mergerfs $GIT_REPO $BRANCH
|
||||
|
||||
FROM scratch
|
||||
COPY --from=build /build/ /
|
||||
|
|
@ -1,10 +0,0 @@
|
|||
FROM --platform=amd64 fedora:43 as build
|
||||
COPY install-build-pkgs build-mergerfs /tmp/
|
||||
RUN /tmp/install-build-pkgs
|
||||
ARG BUILD_TIMESTAMP=0
|
||||
ARG GIT_REPO
|
||||
ARG BRANCH
|
||||
RUN /tmp/build-mergerfs $GIT_REPO $BRANCH
|
||||
|
||||
FROM scratch
|
||||
COPY --from=build /build/ /
|
||||
|
|
@ -1,10 +0,0 @@
|
|||
FROM --platform=arm64 fedora:43 as build
|
||||
COPY install-build-pkgs build-mergerfs /tmp/
|
||||
RUN /tmp/install-build-pkgs
|
||||
ARG BUILD_TIMESTAMP=0
|
||||
ARG GIT_REPO
|
||||
ARG BRANCH
|
||||
RUN /tmp/build-mergerfs $GIT_REPO $BRANCH
|
||||
|
||||
FROM scratch
|
||||
COPY --from=build /build/ /
|
||||
|
|
@ -1,10 +0,0 @@
|
|||
FROM --platform=amd64 fedora:44 as build
|
||||
COPY install-build-pkgs build-mergerfs /tmp/
|
||||
RUN /tmp/install-build-pkgs
|
||||
ARG BUILD_TIMESTAMP=0
|
||||
ARG GIT_REPO
|
||||
ARG BRANCH
|
||||
RUN /tmp/build-mergerfs $GIT_REPO $BRANCH
|
||||
|
||||
FROM scratch
|
||||
COPY --from=build /build/ /
|
||||
|
|
@ -1,10 +0,0 @@
|
|||
FROM --platform=arm64 fedora:44 as build
|
||||
COPY install-build-pkgs build-mergerfs /tmp/
|
||||
RUN /tmp/install-build-pkgs
|
||||
ARG BUILD_TIMESTAMP=0
|
||||
ARG GIT_REPO
|
||||
ARG BRANCH
|
||||
RUN /tmp/build-mergerfs $GIT_REPO $BRANCH
|
||||
|
||||
FROM scratch
|
||||
COPY --from=build /build/ /
|
||||
|
|
@ -1,10 +0,0 @@
|
|||
FROM --platform=amd64 quay.io/rockylinux/rockylinux:10 as build
|
||||
COPY install-build-pkgs build-mergerfs /tmp/
|
||||
RUN /tmp/install-build-pkgs
|
||||
ARG BUILD_TIMESTAMP=0
|
||||
ARG GIT_REPO
|
||||
ARG BRANCH
|
||||
RUN /tmp/build-mergerfs $GIT_REPO $BRANCH
|
||||
|
||||
FROM scratch
|
||||
COPY --from=build /build/ /
|
||||
|
|
@ -1,10 +0,0 @@
|
|||
FROM --platform=arm64 quay.io/rockylinux/rockylinux:10 as build
|
||||
COPY install-build-pkgs build-mergerfs /tmp/
|
||||
RUN /tmp/install-build-pkgs
|
||||
ARG BUILD_TIMESTAMP=0
|
||||
ARG GIT_REPO
|
||||
ARG BRANCH
|
||||
RUN /tmp/build-mergerfs $GIT_REPO $BRANCH
|
||||
|
||||
FROM scratch
|
||||
COPY --from=build /build/ /
|
||||
|
|
@ -1,10 +0,0 @@
|
|||
FROM --platform=amd64 quay.io/rockylinux/rockylinux:8 as build
|
||||
COPY install-build-pkgs build-mergerfs /tmp/
|
||||
RUN /tmp/install-build-pkgs
|
||||
ARG BUILD_TIMESTAMP=0
|
||||
ARG GIT_REPO
|
||||
ARG BRANCH
|
||||
RUN /tmp/build-mergerfs $GIT_REPO $BRANCH
|
||||
|
||||
FROM scratch
|
||||
COPY --from=build /build/ /
|
||||
|
|
@ -1,10 +0,0 @@
|
|||
FROM --platform=arm64 quay.io/rockylinux/rockylinux:8 as build
|
||||
COPY install-build-pkgs build-mergerfs /tmp/
|
||||
RUN /tmp/install-build-pkgs
|
||||
ARG BUILD_TIMESTAMP=0
|
||||
ARG GIT_REPO
|
||||
ARG BRANCH
|
||||
RUN /tmp/build-mergerfs $GIT_REPO $BRANCH
|
||||
|
||||
FROM scratch
|
||||
COPY --from=build /build/ /
|
||||
|
|
@ -1,10 +0,0 @@
|
|||
FROM --platform=amd64 quay.io/rockylinux/rockylinux:9 as build
|
||||
COPY install-build-pkgs build-mergerfs /tmp/
|
||||
RUN /tmp/install-build-pkgs
|
||||
ARG BUILD_TIMESTAMP=0
|
||||
ARG GIT_REPO
|
||||
ARG BRANCH
|
||||
RUN /tmp/build-mergerfs $GIT_REPO $BRANCH
|
||||
|
||||
FROM scratch
|
||||
COPY --from=build /build/ /
|
||||
|
|
@ -1,10 +0,0 @@
|
|||
FROM --platform=arm64 quay.io/rockylinux/rockylinux:9 as build
|
||||
COPY install-build-pkgs build-mergerfs /tmp/
|
||||
RUN /tmp/install-build-pkgs
|
||||
ARG BUILD_TIMESTAMP=0
|
||||
ARG GIT_REPO
|
||||
ARG BRANCH
|
||||
RUN /tmp/build-mergerfs $GIT_REPO $BRANCH
|
||||
|
||||
FROM scratch
|
||||
COPY --from=build /build/ /
|
||||
|
|
@ -1,15 +0,0 @@
|
|||
FROM --platform=linux/amd64 alpine:latest as build
|
||||
COPY install-build-pkgs /tmp/
|
||||
RUN /tmp/install-build-pkgs
|
||||
ARG BUILD_TIMESTAMP=0
|
||||
ARG GIT_REPO
|
||||
ARG BRANCH
|
||||
RUN git clone --single-branch --branch="${BRANCH}" ${GIT_REPO} /tmp/mergerfs
|
||||
WORKDIR /tmp/mergerfs
|
||||
RUN make NDEBUG=1 LTO=1 STATIC=1 DESTDIR=/tmp -j$(nproc) install
|
||||
RUN mkdir /build
|
||||
RUN /tmp/usr/local/bin/mergerfs --version | head -n1 | cut -c 11- | tee /tmp/version
|
||||
RUN tar cvfz /build/mergerfs-$(cat /tmp/version)-static-linux_amd64.tar.gz --directory=/tmp usr sbin
|
||||
|
||||
FROM scratch
|
||||
COPY --from=build /build/ /
|
||||
|
|
@ -1,15 +0,0 @@
|
|||
FROM --platform=linux/arm64 alpine:latest as build
|
||||
COPY install-build-pkgs /tmp/
|
||||
RUN /tmp/install-build-pkgs
|
||||
ARG BUILD_TIMESTAMP=0
|
||||
ARG GIT_REPO
|
||||
ARG BRANCH
|
||||
RUN git clone --single-branch --branch="${BRANCH}" ${GIT_REPO} /tmp/mergerfs
|
||||
WORKDIR /tmp/mergerfs
|
||||
RUN make NDEBUG=1 LTO=1 STATIC=1 DESTDIR=/tmp -j$(nproc) install
|
||||
RUN mkdir /build
|
||||
RUN /tmp/usr/local/bin/mergerfs --version | head -n1 | cut -c 11- | tee /tmp/version
|
||||
RUN tar cvfz /build/mergerfs-$(cat /tmp/version)-static-linux_arm64.tar.gz --directory=/tmp usr sbin
|
||||
|
||||
FROM scratch
|
||||
COPY --from=build /build/ /
|
||||
|
|
@ -1,15 +0,0 @@
|
|||
FROM --platform=linux/armhf alpine:latest as build
|
||||
COPY install-build-pkgs /tmp/
|
||||
RUN /tmp/install-build-pkgs
|
||||
ARG BUILD_TIMESTAMP=0
|
||||
ARG GIT_REPO
|
||||
ARG BRANCH
|
||||
RUN git clone --single-branch --branch="${BRANCH}" ${GIT_REPO} /tmp/mergerfs
|
||||
WORKDIR /tmp/mergerfs
|
||||
RUN make NDEBUG=1 LTO=1 STATIC=1 DESTDIR=/tmp -j$(nproc) install
|
||||
RUN mkdir /build
|
||||
RUN /tmp/usr/local/bin/mergerfs --version | head -n1 | cut -c 11- | tee /tmp/version
|
||||
RUN tar cvfz /build/mergerfs-$(cat /tmp/version)-static-linux_armhf.tar.gz --directory=/tmp usr sbin
|
||||
|
||||
FROM scratch
|
||||
COPY --from=build /build/ /
|
||||
|
|
@ -1,15 +0,0 @@
|
|||
FROM --platform=linux/i386 alpine:latest as build
|
||||
COPY install-build-pkgs /tmp/
|
||||
RUN /tmp/install-build-pkgs
|
||||
ARG BUILD_TIMESTAMP=0
|
||||
ARG GIT_REPO
|
||||
ARG BRANCH
|
||||
RUN git clone --single-branch --branch="${BRANCH}" ${GIT_REPO} /tmp/mergerfs
|
||||
WORKDIR /tmp/mergerfs
|
||||
RUN make NDEBUG=1 LTO=1 STATIC=1 DESTDIR=/tmp -j$(nproc) install
|
||||
RUN mkdir /build
|
||||
RUN /tmp/usr/local/bin/mergerfs --version | head -n1 | cut -c 11- | tee /tmp/version
|
||||
RUN tar cvfz /build/mergerfs-$(cat /tmp/version)-static-linux_i386.tar.gz --directory=/tmp usr sbin
|
||||
|
||||
FROM scratch
|
||||
COPY --from=build /build/ /
|
||||
|
|
@ -1,14 +0,0 @@
|
|||
ARG BUILD_TIMESTAMP=0
|
||||
FROM --platform=linux/amd64 ubuntu:latest as build
|
||||
COPY install-build-pkgs /tmp/
|
||||
RUN /tmp/install-build-pkgs && \
|
||||
DEBIAN_FRONTEND=noninteractive apt-get -qy --no-install-suggests --no-install-recommends install \
|
||||
makeself
|
||||
RUN mkdir -p /tmp/mergerfs/
|
||||
RUN ls -lhd /pkgs
|
||||
RUN cp -v /pkgs/*static*.gz /tmp/mergerfs/
|
||||
RUN mkdir -p /build
|
||||
RUN makeself /tmp/mergerfs/ /build/mergerfs-installer.run "mergerfs" echo "foo"
|
||||
|
||||
FROM scratch
|
||||
COPY --from=build /build/ /
|
||||
|
|
@ -1,15 +0,0 @@
|
|||
FROM --platform=linux/riscv64 alpine:latest as build
|
||||
COPY install-build-pkgs /tmp/
|
||||
RUN /tmp/install-build-pkgs
|
||||
ARG BUILD_TIMESTAMP=0
|
||||
ARG GIT_REPO
|
||||
ARG BRANCH
|
||||
RUN git clone --single-branch --branch="${BRANCH}" ${GIT_REPO} /tmp/mergerfs
|
||||
WORKDIR /tmp/mergerfs
|
||||
RUN make NDEBUG=1 LTO=1 STATIC=1 DESTDIR=/tmp -j$(nproc) install
|
||||
RUN mkdir /build
|
||||
RUN /tmp/usr/local/bin/mergerfs --version | head -n1 | cut -c 11- | tee /tmp/version
|
||||
RUN tar cvfz /build/mergerfs-$(cat /tmp/version)-static-linux_riscv64.tar.gz --directory=/tmp usr sbin
|
||||
|
||||
FROM scratch
|
||||
COPY --from=build /build/ /
|
||||
|
|
@ -1,17 +0,0 @@
|
|||
FROM alpine:latest as tarball
|
||||
|
||||
COPY install-build-pkgs /tmp/
|
||||
RUN /tmp/install-build-pkgs
|
||||
|
||||
ARG BUILD_TIMESTAMP=0
|
||||
ARG BRANCH=master
|
||||
ARG GIT_REPO
|
||||
|
||||
RUN git clone --single-branch "${GIT_REPO}" /tmp/mergerfs --branch="${BRANCH}"
|
||||
RUN cd /tmp/mergerfs && make version tarball
|
||||
|
||||
RUN mkdir /build
|
||||
RUN cp -v /tmp/mergerfs/*.tar.gz /build/
|
||||
|
||||
FROM scratch
|
||||
COPY --from=tarball /build/ /
|
||||
|
|
@ -1,10 +0,0 @@
|
|||
FROM --platform=amd64 ubuntu:22.04 as build
|
||||
COPY install-build-pkgs build-mergerfs /tmp/
|
||||
RUN /tmp/install-build-pkgs
|
||||
ARG BUILD_TIMESTAMP=0
|
||||
ARG GIT_REPO
|
||||
ARG BRANCH
|
||||
RUN /tmp/build-mergerfs $GIT_REPO $BRANCH
|
||||
|
||||
FROM scratch
|
||||
COPY --from=build /build/ /
|
||||
|
|
@ -1,10 +0,0 @@
|
|||
FROM --platform=arm64 ubuntu:22.04 as build
|
||||
COPY install-build-pkgs build-mergerfs /tmp/
|
||||
RUN /tmp/install-build-pkgs
|
||||
ARG BUILD_TIMESTAMP=0
|
||||
ARG GIT_REPO
|
||||
ARG BRANCH
|
||||
RUN /tmp/build-mergerfs $GIT_REPO $BRANCH
|
||||
|
||||
FROM scratch
|
||||
COPY --from=build /build/ /
|
||||
|
|
@ -1,10 +0,0 @@
|
|||
FROM --platform=armhf ubuntu:22.04 as build
|
||||
COPY install-build-pkgs build-mergerfs /tmp/
|
||||
RUN /tmp/install-build-pkgs
|
||||
ARG BUILD_TIMESTAMP=0
|
||||
ARG GIT_REPO
|
||||
ARG BRANCH
|
||||
RUN /tmp/build-mergerfs $GIT_REPO $BRANCH
|
||||
|
||||
FROM scratch
|
||||
COPY --from=build /build/ /
|
||||
|
|
@ -1,10 +0,0 @@
|
|||
FROM --platform=riscv64 ubuntu:22.04 as build
|
||||
COPY install-build-pkgs build-mergerfs /tmp/
|
||||
RUN /tmp/install-build-pkgs
|
||||
ARG BUILD_TIMESTAMP=0
|
||||
ARG GIT_REPO
|
||||
ARG BRANCH
|
||||
RUN /tmp/build-mergerfs $GIT_REPO $BRANCH
|
||||
|
||||
FROM scratch
|
||||
COPY --from=build /build/ /
|
||||
|
|
@ -1,10 +0,0 @@
|
|||
FROM --platform=amd64 ubuntu:24.04 as build
|
||||
COPY install-build-pkgs build-mergerfs /tmp/
|
||||
RUN /tmp/install-build-pkgs
|
||||
ARG BUILD_TIMESTAMP=0
|
||||
ARG GIT_REPO
|
||||
ARG BRANCH
|
||||
RUN /tmp/build-mergerfs $GIT_REPO $BRANCH
|
||||
|
||||
FROM scratch
|
||||
COPY --from=build /build/ /
|
||||
|
|
@ -1,10 +0,0 @@
|
|||
FROM --platform=arm64 ubuntu:24.04 as build
|
||||
COPY install-build-pkgs build-mergerfs /tmp/
|
||||
RUN /tmp/install-build-pkgs
|
||||
ARG BUILD_TIMESTAMP=0
|
||||
ARG GIT_REPO
|
||||
ARG BRANCH
|
||||
RUN /tmp/build-mergerfs $GIT_REPO $BRANCH
|
||||
|
||||
FROM scratch
|
||||
COPY --from=build /build/ /
|
||||
|
|
@ -1,10 +0,0 @@
|
|||
FROM --platform=armhf ubuntu:24.04 as build
|
||||
COPY install-build-pkgs build-mergerfs /tmp/
|
||||
RUN /tmp/install-build-pkgs
|
||||
ARG BUILD_TIMESTAMP=0
|
||||
ARG GIT_REPO
|
||||
ARG BRANCH
|
||||
RUN /tmp/build-mergerfs $GIT_REPO $BRANCH
|
||||
|
||||
FROM scratch
|
||||
COPY --from=build /build/ /
|
||||
|
|
@ -1,10 +0,0 @@
|
|||
FROM --platform=riscv64 ubuntu:24.04 as build
|
||||
COPY install-build-pkgs build-mergerfs /tmp/
|
||||
RUN /tmp/install-build-pkgs
|
||||
ARG BUILD_TIMESTAMP=0
|
||||
ARG GIT_REPO
|
||||
ARG BRANCH
|
||||
RUN /tmp/build-mergerfs $GIT_REPO $BRANCH
|
||||
|
||||
FROM scratch
|
||||
COPY --from=build /build/ /
|
||||
|
|
@ -1,10 +0,0 @@
|
|||
FROM --platform=amd64 ubuntu:25.04 as build
|
||||
COPY install-build-pkgs build-mergerfs /tmp/
|
||||
RUN /tmp/install-build-pkgs
|
||||
ARG BUILD_TIMESTAMP=0
|
||||
ARG GIT_REPO
|
||||
ARG BRANCH
|
||||
RUN /tmp/build-mergerfs $GIT_REPO $BRANCH
|
||||
|
||||
FROM scratch
|
||||
COPY --from=build /build/ /
|
||||
|
|
@ -1,10 +0,0 @@
|
|||
FROM --platform=arm64 ubuntu:25.04 as build
|
||||
COPY install-build-pkgs build-mergerfs /tmp/
|
||||
RUN /tmp/install-build-pkgs
|
||||
ARG BUILD_TIMESTAMP=0
|
||||
ARG GIT_REPO
|
||||
ARG BRANCH
|
||||
RUN /tmp/build-mergerfs $GIT_REPO $BRANCH
|
||||
|
||||
FROM scratch
|
||||
COPY --from=build /build/ /
|
||||
|
|
@ -1,10 +0,0 @@
|
|||
FROM --platform=armhf ubuntu:25.04 as build
|
||||
COPY install-build-pkgs build-mergerfs /tmp/
|
||||
RUN /tmp/install-build-pkgs
|
||||
ARG BUILD_TIMESTAMP=0
|
||||
ARG GIT_REPO
|
||||
ARG BRANCH
|
||||
RUN /tmp/build-mergerfs $GIT_REPO $BRANCH
|
||||
|
||||
FROM scratch
|
||||
COPY --from=build /build/ /
|
||||
|
|
@ -1,10 +0,0 @@
|
|||
FROM --platform=riscv64 ubuntu:25.04 as build
|
||||
COPY install-build-pkgs build-mergerfs /tmp/
|
||||
RUN /tmp/install-build-pkgs
|
||||
ARG BUILD_TIMESTAMP=0
|
||||
ARG GIT_REPO
|
||||
ARG BRANCH
|
||||
RUN /tmp/build-mergerfs $GIT_REPO $BRANCH
|
||||
|
||||
FROM scratch
|
||||
COPY --from=build /build/ /
|
||||
|
|
@ -1,10 +0,0 @@
|
|||
FROM --platform=amd64 ubuntu:25.10 as build
|
||||
COPY install-build-pkgs build-mergerfs /tmp/
|
||||
RUN /tmp/install-build-pkgs
|
||||
ARG BUILD_TIMESTAMP=0
|
||||
ARG GIT_REPO
|
||||
ARG BRANCH
|
||||
RUN /tmp/build-mergerfs $GIT_REPO $BRANCH
|
||||
|
||||
FROM scratch
|
||||
COPY --from=build /build/ /
|
||||
|
|
@ -1,10 +0,0 @@
|
|||
FROM --platform=arm64 ubuntu:25.10 as build
|
||||
COPY install-build-pkgs build-mergerfs /tmp/
|
||||
RUN /tmp/install-build-pkgs
|
||||
ARG BUILD_TIMESTAMP=0
|
||||
ARG GIT_REPO
|
||||
ARG BRANCH
|
||||
RUN /tmp/build-mergerfs $GIT_REPO $BRANCH
|
||||
|
||||
FROM scratch
|
||||
COPY --from=build /build/ /
|
||||
|
|
@ -1,10 +0,0 @@
|
|||
FROM --platform=armhf ubuntu:25.10 as build
|
||||
COPY install-build-pkgs build-mergerfs /tmp/
|
||||
RUN /tmp/install-build-pkgs
|
||||
ARG BUILD_TIMESTAMP=0
|
||||
ARG GIT_REPO
|
||||
ARG BRANCH
|
||||
RUN /tmp/build-mergerfs $GIT_REPO $BRANCH
|
||||
|
||||
FROM scratch
|
||||
COPY --from=build /build/ /
|
||||
|
|
@ -1,10 +0,0 @@
|
|||
FROM --platform=riscv64 ubuntu:25.10 as build
|
||||
COPY install-build-pkgs build-mergerfs /tmp/
|
||||
RUN /tmp/install-build-pkgs
|
||||
ARG BUILD_TIMESTAMP=0
|
||||
ARG GIT_REPO
|
||||
ARG BRANCH
|
||||
RUN /tmp/build-mergerfs $GIT_REPO $BRANCH
|
||||
|
||||
FROM scratch
|
||||
COPY --from=build /build/ /
|
||||
|
|
@ -1,10 +0,0 @@
|
|||
FROM --platform=amd64 ubuntu:26.04 as build
|
||||
COPY install-build-pkgs build-mergerfs /tmp/
|
||||
RUN /tmp/install-build-pkgs
|
||||
ARG BUILD_TIMESTAMP=0
|
||||
ARG GIT_REPO
|
||||
ARG BRANCH
|
||||
RUN /tmp/build-mergerfs $GIT_REPO $BRANCH
|
||||
|
||||
FROM scratch
|
||||
COPY --from=build /build/ /
|
||||
|
|
@ -1,10 +0,0 @@
|
|||
FROM --platform=arm64 ubuntu:26.04 as build
|
||||
COPY install-build-pkgs build-mergerfs /tmp/
|
||||
RUN /tmp/install-build-pkgs
|
||||
ARG BUILD_TIMESTAMP=0
|
||||
ARG GIT_REPO
|
||||
ARG BRANCH
|
||||
RUN /tmp/build-mergerfs $GIT_REPO $BRANCH
|
||||
|
||||
FROM scratch
|
||||
COPY --from=build /build/ /
|
||||
|
|
@ -1,10 +0,0 @@
|
|||
FROM --platform=armhf ubuntu:26.04 as build
|
||||
COPY install-build-pkgs build-mergerfs /tmp/
|
||||
RUN /tmp/install-build-pkgs
|
||||
ARG BUILD_TIMESTAMP=0
|
||||
ARG GIT_REPO
|
||||
ARG BRANCH
|
||||
RUN /tmp/build-mergerfs $GIT_REPO $BRANCH
|
||||
|
||||
FROM scratch
|
||||
COPY --from=build /build/ /
|
||||
|
|
@ -1,10 +0,0 @@
|
|||
FROM --platform=riscv64 ubuntu:26.04 as build
|
||||
COPY install-build-pkgs build-mergerfs /tmp/
|
||||
RUN /tmp/install-build-pkgs
|
||||
ARG BUILD_TIMESTAMP=0
|
||||
ARG GIT_REPO
|
||||
ARG BRANCH
|
||||
RUN /tmp/build-mergerfs $GIT_REPO $BRANCH
|
||||
|
||||
FROM scratch
|
||||
COPY --from=build /build/ /
|
||||
|
|
@ -1,19 +0,0 @@
|
|||
FROM --platform=linux/amd64 alpine:latest as build
|
||||
|
||||
ARG GIT_REPO
|
||||
ARG BRANCH
|
||||
|
||||
RUN <<EOF
|
||||
apk add git gcc g++ make linux-headers
|
||||
git clone --single-branch --branch="${BRANCH}" "${GIT_REPO}" /tmp/mergerfs
|
||||
cd /tmp/mergerfs
|
||||
make NDEBUG=1 LTO=1 STATIC=1 DESTDIR=/tmp/install -j$(nproc) install
|
||||
tar cvfz /tmp/mergerfs.tgz --directory=/tmp/install usr sbin
|
||||
EOF
|
||||
|
||||
FROM --platform=linux/amd64 alpine:latest
|
||||
RUN --mount=from=build,source=/tmp/mergerfs.tgz,target=/tmp/mergerfs.tgz <<EOF
|
||||
tar xvf /tmp/mergerfs.tgz -C /
|
||||
EOF
|
||||
ENTRYPOINT ["/usr/local/bin/mergerfs"]
|
||||
CMD ["--help"]
|
||||
|
|
@ -1,14 +0,0 @@
|
|||
#!/bin/sh
|
||||
set -x
|
||||
|
||||
BASEPATH="/tmp"
|
||||
|
||||
for x in $(seq -w 2)
|
||||
do
|
||||
FILEPATH="${BASEPATH}/mergerfs-${x}.img"
|
||||
MOUNTPOINT="${BASEPATH}/mergerfs-${x}"
|
||||
truncate -s 1G "${FILEPATH}"
|
||||
mkdir -p "${MOUNTPOINT}"
|
||||
mkfs.ext4 -m0 -L "mergerfs${x}" "${FILEPATH}"
|
||||
sudo mount "${FILEPATH}" "${MOUNTPOINT}"
|
||||
done
|
||||
|
|
@ -1,119 +0,0 @@
|
|||
#!/bin/bash
|
||||
# Detects local OS and maps to containerfiles target
|
||||
# Returns target if found in buildtools/containerfiles/, empty string otherwise
|
||||
|
||||
set -euo pipefail
|
||||
|
||||
CONTAINERFILES_DIR="buildtools/containerfiles"
|
||||
|
||||
# Detect system architecture and map to containerfiles format
|
||||
detect_arch() {
|
||||
local arch
|
||||
arch=$(uname -m)
|
||||
case "$arch" in
|
||||
x86_64) echo "amd64" ;;
|
||||
aarch64) echo "arm64" ;;
|
||||
armv7l) echo "armhf" ;;
|
||||
i686|i386) echo "i386" ;;
|
||||
riscv64) echo "riscv64" ;;
|
||||
*) echo "" ;;
|
||||
esac
|
||||
}
|
||||
|
||||
# Detect distro and version from os-release
|
||||
detect_distro() {
|
||||
local os_release="${1:-}"
|
||||
if [[ ! -r "$os_release" ]]; then
|
||||
return 1
|
||||
fi
|
||||
|
||||
# Source the file safely - ID and VERSION_ID are environment variables
|
||||
. "$os_release"
|
||||
|
||||
local distro="$ID"
|
||||
local version="$VERSION_ID"
|
||||
|
||||
# Map distro ID to containerfiles naming convention
|
||||
case "$distro" in
|
||||
ubuntu|debian|fedora) ;;
|
||||
rocky|rhel|centos|almalinux) distro="rockylinux" ;;
|
||||
*)
|
||||
# Fallback to ID_LIKE for derivatives
|
||||
if [[ -n "${ID_LIKE:-}" ]]; then
|
||||
case "$ID_LIKE" in
|
||||
*ubuntu*) distro="ubuntu" ;;
|
||||
*debian*) distro="debian" ;;
|
||||
*fedora*) distro="fedora" ;;
|
||||
esac
|
||||
fi
|
||||
;;
|
||||
esac
|
||||
|
||||
# Only return if we have valid distro and version
|
||||
if [[ -n "$distro" && -n "$version" ]]; then
|
||||
echo "$distro:$version"
|
||||
else
|
||||
return 1
|
||||
fi
|
||||
}
|
||||
|
||||
# Try to find a matching containerfile target
|
||||
find_containerfile() {
|
||||
local target="$1"
|
||||
local containerfile="${CONTAINERFILES_DIR}/${target}"
|
||||
|
||||
# Direct match
|
||||
if [[ -d "$containerfile" || -f "$containerfile" ]]; then
|
||||
echo "$target"
|
||||
return 0
|
||||
fi
|
||||
|
||||
return 1
|
||||
}
|
||||
|
||||
# Main detection logic
|
||||
main() {
|
||||
local arch
|
||||
|
||||
# Detect architecture
|
||||
arch=$(detect_arch)
|
||||
if [[ -z "$arch" ]]; then
|
||||
echo "WARNING: Could not detect system architecture" >&2
|
||||
exit 1
|
||||
fi
|
||||
|
||||
# Try to detect distro from multiple locations
|
||||
local distro_ver
|
||||
distro_ver=$(detect_distro "/etc/os-release") || \
|
||||
distro_ver=$(detect_distro "/usr/lib/os-release") || {
|
||||
echo "WARNING: Could not detect distro from os-release" >&2
|
||||
exit 1
|
||||
}
|
||||
|
||||
# Build target string
|
||||
local target="${distro_ver}.${arch}"
|
||||
|
||||
# Try to find matching containerfile
|
||||
if find_containerfile "$target"; then
|
||||
exit 0
|
||||
fi
|
||||
|
||||
# Try with partial version matching (e.g., 25.10 → 25)
|
||||
local distro="${distro_ver%%:*}"
|
||||
local version="${distro_ver##*:}"
|
||||
|
||||
# Strip last minor version number and try again
|
||||
local short_version="${version%.*}"
|
||||
if [[ -n "$short_version" ]]; then
|
||||
target="${distro}:${short_version}.${arch}"
|
||||
if find_containerfile "$target"; then
|
||||
exit 0
|
||||
fi
|
||||
fi
|
||||
|
||||
# No match found
|
||||
echo "WARNING: No containerfile found for target: ${distro_ver}.${arch}" >&2
|
||||
exit 1
|
||||
}
|
||||
|
||||
main "$@"
|
||||
|
|
@ -1,27 +0,0 @@
|
|||
#!/bin/sh
|
||||
|
||||
input_file="${1}"
|
||||
|
||||
while IFS= read -r line || [ -n "$line" ]; do
|
||||
# Skip empty lines
|
||||
if [ -z "$line" ]; then
|
||||
continue
|
||||
fi
|
||||
|
||||
image=$(echo "$line" | cut -d' ' -f1)
|
||||
platform=$(echo "$line" | cut -d' ' -f2-)
|
||||
|
||||
echo "${image} ${platform}"
|
||||
cat <<EOF > "containerfiles/${image##*/}.${platform}"
|
||||
FROM --platform=${platform} ${image} as build
|
||||
COPY install-build-pkgs build-mergerfs /tmp/
|
||||
RUN /tmp/install-build-pkgs
|
||||
ARG BUILD_TIMESTAMP=0
|
||||
ARG GIT_REPO
|
||||
ARG BRANCH
|
||||
RUN /tmp/build-mergerfs \$GIT_REPO \$BRANCH
|
||||
|
||||
FROM scratch
|
||||
COPY --from=build /build/ /
|
||||
EOF
|
||||
done < "$input_file"
|
||||
|
|
@ -1,202 +0,0 @@
|
|||
#!/bin/sh
|
||||
|
||||
# Copyright (c) 2016, Antonio SJ Musumeci <trapexit@spawn.link>
|
||||
|
||||
# Permission to use, copy, modify, and/or distribute this software for any
|
||||
# purpose with or without fee is hereby granted, provided that the above
|
||||
# copyright notice and this permission notice appear in all copies.
|
||||
|
||||
# THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES
|
||||
# WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF
|
||||
# MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR
|
||||
# ANY SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES
|
||||
# WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN
|
||||
# ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF
|
||||
# OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE.
|
||||
|
||||
# Default values
|
||||
NAME=""
|
||||
VERSION="::guess::"
|
||||
DISTRO="::guess::"
|
||||
CODENAME="::guess::"
|
||||
URGENCY="medium"
|
||||
|
||||
# Function to display usage
|
||||
usage() {
|
||||
cat << EOF
|
||||
Usage: $0 --name NAME [OPTIONS]
|
||||
|
||||
Generated debian/changelog from git log
|
||||
|
||||
Options:
|
||||
--name NAME Name of package (required)
|
||||
--version VERSION Place in git history to include upto (default: ::guess::)
|
||||
--distro DISTRO Distribution name (default: ::guess::)
|
||||
--codename NAME Distribution codename (default: ::guess::)
|
||||
--urgency URGENCY Urgency (default: medium)
|
||||
-h, --help Show this help message
|
||||
EOF
|
||||
exit 1
|
||||
}
|
||||
|
||||
# Parse command line arguments
|
||||
while [ $# -gt 0 ]; do
|
||||
case "$1" in
|
||||
--name)
|
||||
NAME="$2"
|
||||
shift 2
|
||||
;;
|
||||
--version)
|
||||
VERSION="$2"
|
||||
shift 2
|
||||
;;
|
||||
--distro)
|
||||
DISTRO="$2"
|
||||
shift 2
|
||||
;;
|
||||
--codename)
|
||||
CODENAME="$2"
|
||||
shift 2
|
||||
;;
|
||||
--urgency)
|
||||
URGENCY="$2"
|
||||
shift 2
|
||||
;;
|
||||
-h|--help)
|
||||
usage
|
||||
;;
|
||||
*)
|
||||
echo "Unknown option: $1" >&2
|
||||
usage
|
||||
;;
|
||||
esac
|
||||
done
|
||||
|
||||
# Check required arguments
|
||||
if [ -z "$NAME" ]; then
|
||||
echo "Error: --name is required" >&2
|
||||
usage
|
||||
fi
|
||||
|
||||
# Function to get git tags sorted in reverse order
|
||||
git_tags() {
|
||||
git tag -l | sort -rV
|
||||
}
|
||||
|
||||
# Function to get git log between two tags
|
||||
git_log() {
|
||||
local fromtag="$1"
|
||||
local totag="$2"
|
||||
git log --no-merges --oneline "${fromtag}...${totag}" 2>/dev/null
|
||||
}
|
||||
|
||||
# Function to get author and time for a tag
|
||||
git_author_and_time() {
|
||||
local tag="$1"
|
||||
git log -1 --format="-- %an <%ae> %cD" "$tag"
|
||||
}
|
||||
|
||||
# Function to get git version
|
||||
git_version() {
|
||||
git describe --always --tags --dirty 2>/dev/null || echo "unknown"
|
||||
}
|
||||
|
||||
# Function to guess distribution
|
||||
guess_distro() {
|
||||
if command -v lsb_release >/dev/null 2>&1; then
|
||||
lsb_release -i -s 2>/dev/null | tr '[:upper:]' '[:lower:]'
|
||||
else
|
||||
echo "unknown"
|
||||
fi
|
||||
}
|
||||
|
||||
# Function to guess codename
|
||||
guess_codename() {
|
||||
if command -v lsb_release >/dev/null 2>&1; then
|
||||
lsb_release -c -s 2>/dev/null | tr '[:upper:]' '[:lower:]'
|
||||
else
|
||||
echo "unknown"
|
||||
fi
|
||||
}
|
||||
|
||||
# Handle guessing of values
|
||||
if [ "$DISTRO" = "::guess::" ]; then
|
||||
DISTRO=$(guess_distro)
|
||||
fi
|
||||
|
||||
if [ "$CODENAME" = "::guess::" ]; then
|
||||
CODENAME=$(guess_codename)
|
||||
fi
|
||||
|
||||
if [ "$VERSION" = "::guess::" ]; then
|
||||
VERSION=$(git_version)
|
||||
fi
|
||||
|
||||
# Version suffix
|
||||
VERSUFFIX="~${DISTRO}-${CODENAME}"
|
||||
|
||||
# Get all tags
|
||||
TAGS=$(git_tags)
|
||||
|
||||
# Create temporary file for tag pairs
|
||||
TMPFILE=$(mktemp)
|
||||
trap 'rm -f "$TMPFILE"' EXIT
|
||||
|
||||
# Check if version is in tags
|
||||
if echo "$TAGS" | grep -q "^${VERSION}$"; then
|
||||
# If version is a tag, start from that tag
|
||||
echo "$TAGS" | awk -v ver="$VERSION" 'BEGIN{found=0} {if($0==ver)found=1; if(found)print}' > "$TMPFILE"
|
||||
else
|
||||
# If version is not a tag, use HEAD as starting point
|
||||
echo "$VERSION HEAD" > "$TMPFILE"
|
||||
echo "$TAGS" >> "$TMPFILE"
|
||||
fi
|
||||
|
||||
# Process tags and generate changelog
|
||||
PREV_TAG=""
|
||||
PREV_REF=""
|
||||
FIRST=1
|
||||
|
||||
while IFS= read -r line; do
|
||||
if [ -z "$line" ]; then
|
||||
continue
|
||||
fi
|
||||
|
||||
# Handle the special case of VERSION HEAD
|
||||
if [ "$line" = "$VERSION HEAD" ]; then
|
||||
TAG="$VERSION"
|
||||
REF="HEAD"
|
||||
else
|
||||
TAG="$line"
|
||||
REF="$line"
|
||||
fi
|
||||
|
||||
if [ -n "$PREV_TAG" ]; then
|
||||
# Get log entries between current and previous tag
|
||||
LOG_ENTRIES=$(git_log "$REF" "$PREV_REF")
|
||||
|
||||
# Skip if no changes
|
||||
if [ -z "$LOG_ENTRIES" ] || [ "$LOG_ENTRIES" = "" ]; then
|
||||
PREV_TAG="$TAG"
|
||||
PREV_REF="$REF"
|
||||
continue
|
||||
fi
|
||||
|
||||
# Print package header
|
||||
printf "%s (%s%s) %s; urgency=%s\n\n" "$NAME" "$PREV_TAG" "$VERSUFFIX" "$CODENAME" "$URGENCY"
|
||||
|
||||
# Print each log entry
|
||||
echo "$LOG_ENTRIES" | while IFS= read -r entry; do
|
||||
if [ -n "$entry" ]; then
|
||||
printf " * %s\n" "$entry"
|
||||
fi
|
||||
done
|
||||
|
||||
# Print author and time
|
||||
AUTHOR_TIME=$(git_author_and_time "$PREV_REF")
|
||||
printf "\n %s\n\n" "$AUTHOR_TIME"
|
||||
fi
|
||||
|
||||
PREV_TAG="$TAG"
|
||||
PREV_REF="$REF"
|
||||
done < "$TMPFILE"
|
||||
|
|
@ -1,162 +0,0 @@
|
|||
#!/bin/sh
|
||||
|
||||
set -e
|
||||
|
||||
BUILD_ENV=/tmp/build-env
|
||||
|
||||
write_gcc_env()
|
||||
{
|
||||
{
|
||||
printf 'export CC=%s\n' "$1"
|
||||
printf 'export CXX=%s\n' "$2"
|
||||
} > "$BUILD_ENV"
|
||||
}
|
||||
|
||||
write_toolset_env()
|
||||
{
|
||||
{
|
||||
printf '. %s\n' "$1"
|
||||
printf 'export CC=gcc\n'
|
||||
printf 'export CXX=g++\n'
|
||||
} > "$BUILD_ENV"
|
||||
}
|
||||
|
||||
latest_apt_gxx()
|
||||
{
|
||||
apt-cache --names-only search '^g[+][+]-[0-9]' \
|
||||
| sed -n 's/^\(g++-[0-9][0-9]*\) .*/\1/p' \
|
||||
| sort -t- -k2,2V \
|
||||
| tail -n 1
|
||||
}
|
||||
|
||||
latest_dnf_toolset()
|
||||
{
|
||||
dnf -q list --showduplicates 'gcc-toolset-*-gcc-c++' 2>/dev/null \
|
||||
| sed -n 's/^gcc-toolset-\([0-9][0-9]*\)-gcc-c++\..*/\1/p' \
|
||||
| sort -V \
|
||||
| tail -n 1
|
||||
}
|
||||
|
||||
latest_yum_toolset()
|
||||
{
|
||||
yum -q list available 'devtoolset-*-gcc-c++' 2>/dev/null \
|
||||
| sed -n 's/^devtoolset-\([0-9][0-9]*\)-gcc-c++\..*/\1/p' \
|
||||
| sort -V \
|
||||
| tail -n 1
|
||||
}
|
||||
|
||||
version_gt()
|
||||
{
|
||||
highest="$(printf '%s\n%s\n' "$1" "$2" | sort -V | tail -n 1)"
|
||||
[ "$highest" = "$1" ] && [ "$1" != "$2" ]
|
||||
}
|
||||
|
||||
check_cxx20()
|
||||
{
|
||||
cxx="${CXX:-g++}"
|
||||
test_src=/tmp/mergerfs-cxx20-check.cpp
|
||||
test_bin=/tmp/mergerfs-cxx20-check
|
||||
|
||||
cat > "$test_src" <<'EOF'
|
||||
template <typename T>
|
||||
concept Incrementable = requires(T value) { value + 1; };
|
||||
|
||||
static_assert(Incrementable<int>);
|
||||
|
||||
int main()
|
||||
{
|
||||
return 0;
|
||||
}
|
||||
EOF
|
||||
|
||||
if ! "$cxx" -std=c++20 "$test_src" -o "$test_bin" >/dev/null 2>&1; then
|
||||
echo "error: $cxx does not support C++20" >&2
|
||||
exit 1
|
||||
fi
|
||||
|
||||
rm -f "$test_src" "$test_bin"
|
||||
}
|
||||
|
||||
: > "$BUILD_ENV"
|
||||
|
||||
if [ -e /usr/bin/apt-get ]; then
|
||||
export DEBIAN_FRONTEND=noninteractive
|
||||
apt-get -qy update
|
||||
cxx_pkg="$(latest_apt_gxx)"
|
||||
if [ -n "$cxx_pkg" ]; then
|
||||
gcc_pkg="gcc-${cxx_pkg#g++-}"
|
||||
compiler_pkgs="$gcc_pkg $cxx_pkg"
|
||||
write_gcc_env "$gcc_pkg" "$cxx_pkg"
|
||||
else
|
||||
compiler_pkgs="g++"
|
||||
write_gcc_env gcc g++
|
||||
fi
|
||||
apt-get -qy --force-yes --no-install-suggests --no-install-recommends \
|
||||
install \
|
||||
ca-certificates \
|
||||
build-essential \
|
||||
dpkg-dev \
|
||||
git \
|
||||
$compiler_pkgs \
|
||||
debhelper \
|
||||
lsb-release \
|
||||
fakeroot
|
||||
. "$BUILD_ENV"
|
||||
check_cxx20
|
||||
elif [ -e /usr/bin/dnf ]; then
|
||||
dnf -y update
|
||||
toolset="$(latest_dnf_toolset)"
|
||||
dnf -y install \
|
||||
git rpm-build gcc-c++ make which libatomic
|
||||
default_gxx_major="$(g++ -dumpversion | cut -f1 -d.)"
|
||||
if [ -n "$toolset" ] && version_gt "$toolset" "$default_gxx_major"; then
|
||||
dnf -y install \
|
||||
"gcc-toolset-${toolset}-gcc" \
|
||||
"gcc-toolset-${toolset}-gcc-c++"
|
||||
write_toolset_env "/opt/rh/gcc-toolset-${toolset}/enable"
|
||||
else
|
||||
write_gcc_env gcc g++
|
||||
fi
|
||||
. "$BUILD_ENV"
|
||||
check_cxx20
|
||||
elif [ -e /usr/bin/yum ]; then
|
||||
yum -y update
|
||||
yum -y install \
|
||||
git rpm-build gcc-c++ make which
|
||||
yum -y install centos-release-scl || true
|
||||
yum -y update
|
||||
toolset="$(latest_yum_toolset)"
|
||||
default_gxx_major="$(g++ -dumpversion | cut -f1 -d.)"
|
||||
if [ -n "$toolset" ] && version_gt "$toolset" "$default_gxx_major"; then
|
||||
yum -y install \
|
||||
"devtoolset-${toolset}-gcc" \
|
||||
"devtoolset-${toolset}-gcc-c++"
|
||||
write_toolset_env "/opt/rh/devtoolset-${toolset}/enable"
|
||||
else
|
||||
write_gcc_env gcc g++
|
||||
fi
|
||||
. "$BUILD_ENV"
|
||||
check_cxx20
|
||||
elif [ -e /usr/bin/zypper ]; then
|
||||
zypper update -y
|
||||
zypper install -y \
|
||||
git rpm-build gcc-c++ make which
|
||||
write_gcc_env gcc g++
|
||||
. "$BUILD_ENV"
|
||||
check_cxx20
|
||||
elif [ -e /sbin/apk ]; then
|
||||
apk add \
|
||||
abuild git gcc g++ make \
|
||||
linux-headers
|
||||
write_gcc_env gcc g++
|
||||
. "$BUILD_ENV"
|
||||
check_cxx20
|
||||
elif [ -e /usr/sbin/pkg ]; then
|
||||
pkg install \
|
||||
git gmake gcc
|
||||
elif [ -e /usr/bin/pacman ]; then
|
||||
pacman -Sy --noconfirm base-devel git fakeroot
|
||||
write_gcc_env gcc g++
|
||||
. "$BUILD_ENV"
|
||||
check_cxx20
|
||||
fi
|
||||
|
|
@ -1,32 +0,0 @@
|
|||
#!/bin/sh
|
||||
|
||||
if [ "$(id -u)" -eq 0 ]; then
|
||||
SUDO=
|
||||
elif command -v sudo >/dev/null 2>&1; then
|
||||
SUDO=sudo
|
||||
elif command -v doas >/dev/null 2>&1; then
|
||||
SUDO=doas
|
||||
else
|
||||
printf '%s\n' "error: need root privileges; run as root or install sudo/doas" >&2
|
||||
exit 1
|
||||
fi
|
||||
|
||||
if [ -e /usr/bin/apt-get ]; then
|
||||
export DEBIAN_FRONTEND=noninteractive
|
||||
${SUDO} apt-get -qy update
|
||||
${SUDO} apt-get -qy --no-install-suggests --no-install-recommends install \
|
||||
qemu-user-static \
|
||||
podman
|
||||
elif [ -e /usr/bin/dnf ]; then
|
||||
${SUDO} dnf -y update
|
||||
${SUDO} dnf -y install \
|
||||
qemu-user-static \
|
||||
podman
|
||||
elif [ -e /sbin/apk ]; then
|
||||
${SUDO} apk add \
|
||||
python3 \
|
||||
qemu-user-static \
|
||||
podman
|
||||
elif [ -e /usr/bin/pacman ]; then
|
||||
${SUDO} pacman -Sy --noconfirm qemu-user-static podman
|
||||
fi
|
||||
|
|
@ -1,39 +0,0 @@
|
|||
debian:11 amd64
|
||||
debian:11 arm64
|
||||
debian:11 armhf
|
||||
debian:11 i386
|
||||
debian:12 amd64
|
||||
debian:12 arm64
|
||||
debian:12 armhf
|
||||
debian:12 i386
|
||||
debian:13 amd64
|
||||
debian:13 arm64
|
||||
debian:13 armhf
|
||||
debian:13 i386
|
||||
debian:13 riscv64
|
||||
fedora:40 amd64
|
||||
fedora:40 arm64
|
||||
fedora:41 amd64
|
||||
fedora:41 arm64
|
||||
fedora:42 amd64
|
||||
fedora:42 arm64
|
||||
fedora:43 amd64
|
||||
fedora:43 arm64
|
||||
quay.io/rockylinux/rockylinux:10 amd64
|
||||
quay.io/rockylinux/rockylinux:10 arm64
|
||||
quay.io/rockylinux/rockylinux:8 amd64
|
||||
quay.io/rockylinux/rockylinux:8 arm64
|
||||
quay.io/rockylinux/rockylinux:9 amd64
|
||||
quay.io/rockylinux/rockylinux:9 arm64
|
||||
ubuntu:22.04 amd64
|
||||
ubuntu:22.04 arm64
|
||||
ubuntu:22.04 armhf
|
||||
ubuntu:22.04 riscv64
|
||||
ubuntu:24.04 amd64
|
||||
ubuntu:24.04 arm64
|
||||
ubuntu:24.04 armhf
|
||||
ubuntu:24.04 riscv64
|
||||
ubuntu:25.04 amd64
|
||||
ubuntu:25.04 arm64
|
||||
ubuntu:25.04 armhf
|
||||
ubuntu:25.04 riscv64
|
||||
|
|
@ -1,20 +0,0 @@
|
|||
#!/bin/sh
|
||||
|
||||
VERSION=$(git describe --always --tags --dirty)
|
||||
if [ $? -ne 0 ]; then
|
||||
VERSION=$(cat VERSION)
|
||||
fi
|
||||
|
||||
if [ "${VERSION}" = "" ]; then
|
||||
VERSION="unknown"
|
||||
fi
|
||||
|
||||
echo "version = ${VERSION}"
|
||||
echo -n "${VERSION}" > VERSION
|
||||
grep -sq \"${VERSION}\" src/version.hpp
|
||||
RV=$?
|
||||
if [ $RV -ne 0 ]; then
|
||||
echo "Writing 'src/version.hpp'"
|
||||
echo "#pragma once" > src/version.hpp
|
||||
echo "static const char MERGERFS_VERSION[] = \"${VERSION}\";" >> src/version.hpp
|
||||
fi
|
||||
27
debian/changelog
vendored
Normal file
27
debian/changelog
vendored
Normal file
|
|
@ -0,0 +1,27 @@
|
|||
mergerfs (1.1.0) trusty; urgency=medium
|
||||
|
||||
* initial code commit
|
||||
* fix building without libattr
|
||||
* update build instructions in readme
|
||||
* use {get,list,set}xattr to modify runtime
|
||||
* use long instead of int to limit possibility of overflow in switch, closes #1
|
||||
* add FS_IOC_{GET}VERSION to ioctl
|
||||
* remove statfs policy
|
||||
* fs::make_path should check for forward slashes, add if missing
|
||||
* remove stat'ing of files in readdir. closes #3
|
||||
* use eaccess to determine permissions for ffwp. closes #2
|
||||
* rework policy code
|
||||
* add fgetattr. closes #5
|
||||
* when readdir's filler returns non-zero return ENOMEM. closes #7
|
||||
* enhance Makefile
|
||||
* further Makefile enhancements
|
||||
* fix free space calculations. closes #8
|
||||
* add debian package building
|
||||
|
||||
-- Antonio SJ Musumeci <trapexit@spawn.link> Fri, 30 May 2014 21:32:19 -0400
|
||||
|
||||
mergerfs (1.0.0) trusty; urgency=medium
|
||||
|
||||
* Initial commit
|
||||
|
||||
-- Antonio SJ Musumeci <trapexit@spawn.link> Fri, 30 May 2014 21:28:09 -0400
|
||||
1
debian/compat
vendored
Normal file
1
debian/compat
vendored
Normal file
|
|
@ -0,0 +1 @@
|
|||
9
|
||||
13
debian/control
vendored
13
debian/control
vendored
|
|
@ -2,7 +2,10 @@ Source: mergerfs
|
|||
Section: utils
|
||||
Priority: optional
|
||||
Maintainer: Antonio SJ Musumeci <trapexit@spawn.link>
|
||||
Build-Depends: debhelper-compat (= 10)
|
||||
Build-Depends: debhelper (>= 8.0.0),
|
||||
libfuse-dev,
|
||||
libattr1-dev,
|
||||
pkg-config
|
||||
Standards-Version: 3.9.4
|
||||
Homepage: http://github.com/trapexit/mergerfs
|
||||
|
||||
|
|
@ -10,8 +13,8 @@ Package: mergerfs
|
|||
Architecture: any
|
||||
Depends: ${shlibs:Depends},
|
||||
${misc:Depends},
|
||||
fuse [linux-any] | fuse4bsd [kfreebsd-any]
|
||||
libfuse2
|
||||
Description: another FUSE union filesystem
|
||||
mergerfs is a union filesystem geared towards simplifying storage and
|
||||
management of files across numerous commodity storage devices. It is
|
||||
similar to mhddfs, unionfs, and aufs.
|
||||
mergerfs is similar to mhddfs, unionfs, and aufs. Like mhddfs in that it too
|
||||
uses FUSE. Like aufs in that it provides multiple policies for how to handle
|
||||
behavior.
|
||||
|
|
|
|||
36
debian/copyright
vendored
36
debian/copyright
vendored
|
|
@ -4,22 +4,28 @@ Source: http://github.com/trapexit/mergerfs
|
|||
|
||||
Files: *
|
||||
Copyright: 2014 Antonio SJ Musumeci <trapexit@spawn.link>
|
||||
License: ISC
|
||||
License: MIT
|
||||
|
||||
Files: debian/*
|
||||
Copyright: 2014 Antonio SJ Musumeci <trapexit@spawn.link>
|
||||
License: ISC
|
||||
License: MIT
|
||||
|
||||
License: ISC
|
||||
Permission to use, copy, modify, and/or distribute this software for any
|
||||
purpose with or without fee is hereby granted, provided that the above
|
||||
copyright notice and this permission notice appear in all copies.
|
||||
.
|
||||
THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES
|
||||
WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF
|
||||
MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR
|
||||
ANY SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES
|
||||
WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN
|
||||
ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF
|
||||
OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE.
|
||||
n
|
||||
License: MIT
|
||||
Permission is hereby granted, free of charge, to any person obtaining a
|
||||
copy of this software and associated documentation files (the "Software"),
|
||||
to deal in the Software without restriction, including without limitation
|
||||
the rights to use, copy, modify, merge, publish, distribute, sublicense,
|
||||
and/or sell copies of the Software, and to permit persons to whom the
|
||||
Software is furnished to do so, subject to the following conditions:
|
||||
.
|
||||
The above copyright notice and this permission notice shall be included
|
||||
in all copies or substantial portions of the Software.
|
||||
.
|
||||
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS
|
||||
OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
|
||||
MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT.
|
||||
IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY
|
||||
CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT,
|
||||
TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE
|
||||
SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
|
||||
n
|
||||
1
debian/files
vendored
Normal file
1
debian/files
vendored
Normal file
|
|
@ -0,0 +1 @@
|
|||
mergerfs_1.1.0_amd64.deb utils optional
|
||||
9
debian/mergerfs.postinst
vendored
9
debian/mergerfs.postinst
vendored
|
|
@ -1,9 +0,0 @@
|
|||
#!/bin/sh
|
||||
|
||||
set -e
|
||||
|
||||
case "${1}" in
|
||||
configure)
|
||||
chmod 4755 /usr/bin/mergerfs-fusermount
|
||||
;;
|
||||
esac
|
||||
12
debian/rules
vendored
12
debian/rules
vendored
|
|
@ -1,16 +1,8 @@
|
|||
#!/usr/bin/make -f
|
||||
# -*- makefile -*-
|
||||
|
||||
export DH_VERBOSE=1
|
||||
# Uncomment this to turn on verbose mode.
|
||||
#export DH_VERBOSE=1
|
||||
|
||||
%:
|
||||
dh $@
|
||||
|
||||
override_dh_auto_build:
|
||||
$(MAKE) release
|
||||
|
||||
override_dh_auto_install:
|
||||
$(MAKE) \
|
||||
DESTDIR=$(CURDIR)/debian/mergerfs \
|
||||
PREFIX=/usr \
|
||||
install
|
||||
120
man/mergerfs.1
120
man/mergerfs.1
|
|
@ -1,120 +0,0 @@
|
|||
.TH MERGERFS 1 "May 2025" "mergerfs" "User Commands"
|
||||
.SH NAME
|
||||
mergerfs \- a featureful union filesystem
|
||||
.SH SYNOPSIS
|
||||
.B mergerfs
|
||||
[options] <branches> <mountpoint>
|
||||
.SH DESCRIPTION
|
||||
.B mergerfs
|
||||
is a FUSE-based union filesystem geared towards simplifying storage and management of files across numerous commodity storage devices. It is similar to
|
||||
.BR mhddfs ,
|
||||
.BR unionfs ,
|
||||
and
|
||||
.BR aufs .
|
||||
.SH QUICKSTART
|
||||
.UR https://trapexit.github.io/mergerfs/latest/quickstart
|
||||
.LI https://trapexit.github.io/mergerfs/latest/quickstart
|
||||
.UE
|
||||
.SH DOCUMENTATION
|
||||
.UR https://trapexit.github.io/mergerfs
|
||||
.LI https://trapexit.github.io/mergerfs
|
||||
.UE
|
||||
.SH SUPPORT
|
||||
.UR https://trapexit.github.io/mergerfs/latest/support
|
||||
.LI https://trapexit.github.io/mergerfs/latest/support
|
||||
.UE
|
||||
.SH FEATURES
|
||||
.IP \[bu] 2
|
||||
Logically combine numerous filesystems/paths into a single mount point
|
||||
.IP \[bu]
|
||||
Combine paths of the same or different filesystems
|
||||
.IP \[bu]
|
||||
Ability to add or remove filesystems/paths without impacting the rest of the data
|
||||
.IP \[bu]
|
||||
Unaffected by individual filesystem failure
|
||||
.IP \[bu]
|
||||
Configurable file selection and creation placement
|
||||
.IP \[bu]
|
||||
Works with filesystems of any size
|
||||
.IP \[bu]
|
||||
Works with filesystems of almost any type
|
||||
.IP \[bu]
|
||||
Ignore read-only filesystems when creating files
|
||||
.IP \[bu]
|
||||
Hard link copy-on-write / CoW
|
||||
.IP \[bu]
|
||||
Runtime configurable
|
||||
.IP \[bu]
|
||||
Support for extended attributes (xattrs)
|
||||
.IP \[bu]
|
||||
Support for file attributes (chattr)
|
||||
.IP \[bu]
|
||||
Support for POSIX ACLs
|
||||
.SH NON-FEATURES
|
||||
.IP \[bu] 2
|
||||
Read/write overlay on top of read-only filesystem like OverlayFS
|
||||
.IP \[bu]
|
||||
File whiteout
|
||||
.IP \[bu]
|
||||
RAID-like parity calculation
|
||||
.IP \[bu]
|
||||
Redundancy
|
||||
.IP \[bu]
|
||||
Splitting of files across branches
|
||||
.SH HOW IT WORKS
|
||||
.B mergerfs
|
||||
logically merges multiple filesystem paths together. It acts as a proxy to the underlying filesystem paths, combining the behaviors of some functions and being a selector for others.
|
||||
|
||||
When the contents of a directory are requested,
|
||||
.B mergerfs
|
||||
combines the list of files from each directory, deduplicating entries, and returns that list.
|
||||
|
||||
When a file or directory is created, a policy is first run to determine which branch will be selected for the creation.
|
||||
|
||||
For functions which change attributes or remove the file, the behavior may be applied to all instances found.
|
||||
|
||||
Read more about policies here:
|
||||
.UR https://trapexit.github.io/mergerfs/latest/config/functions_categories_policies
|
||||
.LI https://trapexit.github.io/mergerfs/latest/config/functions_categories_policies
|
||||
.UE
|
||||
.SH VISUALIZATION
|
||||
.nf
|
||||
A + B = C
|
||||
/disk1 /disk2 /merged
|
||||
| | |
|
||||
+-- /dir1 +-- /dir1 +-- /dir1
|
||||
| | | | | |
|
||||
| +-- file1 | +-- file2 | +-- file1
|
||||
| | +-- file3 | +-- file2
|
||||
+-- /dir2 | | +-- file3
|
||||
| | +-- /dir3 |
|
||||
| +-- file4 | +-- /dir2
|
||||
| +-- file5 | |
|
||||
+-- file6 | +-- file4
|
||||
|
|
||||
+-- /dir3
|
||||
| |
|
||||
| +-- file5
|
||||
|
|
||||
+-- file6
|
||||
.fi
|
||||
.SH SPONSORSHIP AND DONATIONS
|
||||
Development and support of a project like
|
||||
.B mergerfs
|
||||
require a significant amount of time and effort. The software is released under the very liberal ISC license and is therefore free to use for personal or commercial uses.
|
||||
|
||||
If you are a non-commercial user and find
|
||||
.B mergerfs
|
||||
and its support valuable and would like to support the project financially, it would be very much appreciated.
|
||||
|
||||
If you are using
|
||||
.B mergerfs
|
||||
commercially, please consider sponsoring the project to ensure it continues to be maintained and receive updates. If custom features are needed, feel free to contact the maintainer directly.
|
||||
|
||||
.UR https://github.com/trapexit/support
|
||||
.LI https://github.com/trapexit/support
|
||||
.UE
|
||||
.SH LICENSE
|
||||
ISC License
|
||||
.SH AUTHOR
|
||||
Antonio SJ Musumeci (aka trapexit)
|
||||
|
|
@ -1,59 +0,0 @@
|
|||
Name: mergerfs
|
||||
Version: __VERSION__
|
||||
Release: 1%{?dist}
|
||||
Summary: A featureful FUSE based union filesystem
|
||||
|
||||
Group: Applications/System
|
||||
License: ISC
|
||||
URL: https://github.com/trapexit/mergerfs
|
||||
|
||||
Source: mergerfs-%{version}.tar.gz
|
||||
|
||||
BuildRequires: gcc-c++
|
||||
# rpmbuild driven by the Makefile uses git to generate a version number
|
||||
BuildRequires: git
|
||||
|
||||
%global debug_package %{nil}
|
||||
%undefine _debuginfo_subpackages
|
||||
%global _enable_debug_packages 0
|
||||
|
||||
%prep
|
||||
%setup -q
|
||||
|
||||
%description
|
||||
mergerfs is a union filesystem geared towards simplifying storage and
|
||||
management of files across numerous commodity storage devices. It is
|
||||
similar to mhddfs, unionfs, and aufs.
|
||||
|
||||
%build
|
||||
make %{?_smp_mflags} CFLAGS="%{optflags}" CXXFLAGS="%{optflags}" LDFLAGS="%{__global_ldflags}"
|
||||
|
||||
%install
|
||||
make install PREFIX=%{_prefix} DESTDIR=%{buildroot}
|
||||
|
||||
%files
|
||||
/usr/bin/mergerfs
|
||||
/usr/bin/mergerfs-fusermount
|
||||
/usr/bin/fsck.mergerfs
|
||||
/usr/bin/mergerfs.collect-info
|
||||
/sbin/mount.mergerfs
|
||||
/usr/lib/mergerfs/preload.so
|
||||
%doc %{_mandir}/*
|
||||
|
||||
%changelog
|
||||
* Fri Apr 26 2019 Antonio SJ Musumeci <trapexit@spawn.link>
|
||||
- Update description
|
||||
|
||||
* Mon Jan 25 2016 Antonio SJ Musumeci <trapexit@spawn.link>
|
||||
- Remove sbin files
|
||||
|
||||
* Sat Sep 05 2015 Antonio SJ Musumeci <trapexit@spawn.link>
|
||||
- Include PREFIX to install
|
||||
|
||||
* Mon Dec 29 2014 Joe Lawrence <joe.lawrence@stratus.com>
|
||||
- Tweak rpmbuild to archive current git HEAD into a tarball, then (re)build in
|
||||
the rpmbuild directory -- more complicated but seemingly better suited to
|
||||
generate source and debug rpms.
|
||||
|
||||
* Fri Jun 20 2014 Joe Lawrence <joe.lawrence@stratus.com>
|
||||
- Initial rpm spec file.
|
||||
|
|
@ -1,23 +0,0 @@
|
|||
# mkdocs
|
||||
|
||||
## Getting started
|
||||
|
||||
```bash
|
||||
python3 -m venv venv
|
||||
source venv/bin/activate
|
||||
pip3 install --no-cache-dir mkdocs mkdocs-material pymdown-extensions mike
|
||||
mike serve
|
||||
```
|
||||
|
||||
## Deploy
|
||||
|
||||
```bash
|
||||
mike deploy --push <version_name> --allow-empty
|
||||
```
|
||||
|
||||
|
||||
## References
|
||||
|
||||
* https://www.mkdocs.org/
|
||||
* https://squidfunk.github.io/mkdocs-material/
|
||||
|
||||
|
|
@ -1,7 +0,0 @@
|
|||
#!/bin/sh
|
||||
set -x
|
||||
set -e
|
||||
|
||||
python3 -m venv venv
|
||||
. ./venv/bin/activate
|
||||
pip3 install --no-cache-dir 'mkdocs<2.0' mkdocs-material pymdown-extensions mike
|
||||
|
|
@ -1,111 +0,0 @@
|
|||
# Benchmarking
|
||||
|
||||
Filesystems are complicated. They do many things and many of those are
|
||||
interconnected. Additionally, the OS, drivers, hardware, etc. can all
|
||||
impact performance. Therefore, when benchmarking, it is **necessary**
|
||||
that the test focuses as narrowly as possible.
|
||||
|
||||
For most throughput is the key benchmark. To test throughput `dd` is
|
||||
useful but **must** be used with the correct settings in order to
|
||||
ensure the filesystem or device is actually being tested. The OS can
|
||||
and will cache data. Without forcing synchronous reads and writes
|
||||
and/or disabling caching the values returned will not be
|
||||
representative of the device's true performance.
|
||||
|
||||
When benchmarking through mergerfs ensure you only use 1 branch to
|
||||
remove any possibility of the policies or differences in underlying
|
||||
filesystems complicating the situation. Benchmark the underlying
|
||||
filesystem first and then mount mergerfs with that same filesystem and
|
||||
that. Throughput **will be** lower but if you are experiencing speeds
|
||||
below your expectation you will need to narrow down precisely which
|
||||
component is leading to the slowdown. Preferably test the following in
|
||||
the order listed (but not combined).
|
||||
|
||||
1. Enable `nullrw` mode with `nullrw=true`. This will effectively make
|
||||
reads and writes no-ops. Removing the underlying device /
|
||||
filesystem from the equation. This will give us the top theoretical
|
||||
speeds.
|
||||
2. Configure mergerfs to use a `tmpfs` branch. `tmpfs` is a RAM
|
||||
disk. Extremely high speed and very low latency. This is a more
|
||||
realistic best case scenario. Example: `mount -t tmpfs -o size=2G
|
||||
tmpfs /tmp/tmpfs`
|
||||
3. Configure mergerfs to use a local device filesystem branch. NVMe,
|
||||
SSD, HDD, etc. Test them individually. If you have different
|
||||
interconnects / controllers use the same storage device when
|
||||
testing to ensure consistency.
|
||||
4. Configure mergerfs to use any network filesystems you plan to use
|
||||
one at a time. It may also be worth trying a different network
|
||||
filesystem. `NFS` vs `CIFS/SMB/Samba` vs `sshfs`, etc.
|
||||
|
||||
Once you find the component which has the performance issue you can do
|
||||
further testing with different options to see if they impact
|
||||
performance. If you find a specific system, device, filesystem,
|
||||
controller, etc. that performs poorly contact the author so it can be
|
||||
investigated further.
|
||||
|
||||
Sometimes the problem is really the application accessing or writing
|
||||
data through mergerfs. Some software use small buffer sizes which can
|
||||
lead to more requests and therefore greater overhead. You can test
|
||||
this out yourself by replacing `bs=1M` in the examples below with
|
||||
`ibs` or `obs` and using a size of `512` instead of `1M`. In one
|
||||
example test using `nullrw` the write speed dropped from 4.9GB/s to
|
||||
69.7MB/s when moving from `1M` to `512`. Similar results were had when
|
||||
testing reads. Small writes overhead may be improved by leveraging a
|
||||
write cache but in casual tests little gain was found. If you have an
|
||||
app that appears slow with mergerfs it could be due to this. `strace`
|
||||
can be used with the app in question or mergerfs to see the size of
|
||||
read/writes. Contact the software author or worse case the mergerfs
|
||||
author so it may be investigated further.
|
||||
|
||||
|
||||
### Write Benchmark
|
||||
|
||||
```
|
||||
$ dd if=/dev/zero of=/mnt/mergerfs/16GB.file bs=1M count=16384 oflag=nocache conv=fdatasync status=progress
|
||||
```
|
||||
|
||||
### Read Benchmark
|
||||
|
||||
```
|
||||
$ dd if=/mnt/mergerfs/16GB.file of=/dev/null bs=1M iflag=nocache conv=fdatasync status=progress
|
||||
```
|
||||
|
||||
|
||||
### fio
|
||||
|
||||
`fio` is a popular and flexible benchmarking tool.
|
||||
|
||||
Below are some basic tests but also try setting `--direct=0`, changing
|
||||
the `ioengine`, or playing with other options.
|
||||
|
||||
|
||||
#### Write
|
||||
|
||||
```
|
||||
$ fio --name=writetest --filename=/mnt/mergerfs/16GB.file --size=16Gb --rw=write --bs=1M --direct=1 --numjobs=1 --iodepth=8 --group_reporting --runtime=60 --startdelay=0 --ioengine=psync
|
||||
```
|
||||
|
||||
|
||||
#### Read
|
||||
|
||||
```
|
||||
$ fio --name=readtest --filename=/mnt/mergerfs/16GB.file --size=16Gb --rw=read --bs=1M --direct=1 --numjobs=1 --iodepth=8 --group_reporting --runtime=60 --startdelay=0 --ioengine=psync
|
||||
```
|
||||
|
||||
|
||||
### Other Benchmarks
|
||||
|
||||
If you are attempting to benchmark other behaviors you must ensure you
|
||||
clear kernel caches before runs. In fact it would be a good deal to
|
||||
run before the read and write benchmarks as well just in case.
|
||||
|
||||
```
|
||||
sync
|
||||
echo 3 | sudo tee /proc/sys/vm/drop_caches
|
||||
```
|
||||
|
||||
## Additional Reading
|
||||
|
||||
* [Tweaking Performance](performance.md)
|
||||
* [Options](config/options.md)
|
||||
* [Tips and Notes](tips_notes.md)
|
||||
|
|
@ -1,89 +0,0 @@
|
|||
# branches-mount-timeout
|
||||
|
||||
* type: `UINT` (seconds)
|
||||
* default: `0`
|
||||
|
||||
mergerfs is compatible with any path, whether it be a mount point or a
|
||||
directory on your [root
|
||||
filesystem.](https://en.wikipedia.org/wiki/Root_directory) It doesn’t
|
||||
require branch paths to be mounted or to match a specific filesystem
|
||||
at startup, and it operates effectively without needing details about
|
||||
the intended filesystem. This flexibility eliminates the need to
|
||||
manage mount ordering, which is particularly advantageous on modern
|
||||
systems where filesystems are mounted asynchronously, resulting in
|
||||
unpredictable mount sequences. While
|
||||
[systemd](https://www.freedesktop.org/software/systemd/man/latest/systemd.mount.html)
|
||||
offers a way to enforce mount dependencies using the
|
||||
[x-systemd.requires=PATH](https://www.freedesktop.org/software/systemd/man/latest/systemd.mount.html#x-systemd.requires=)
|
||||
option in /etc/fstab, applying this to every branch path is both
|
||||
cumbersome and susceptible to errors.
|
||||
|
||||
`branches-mount-timeout` will cause mergerfs to first attempt to mount
|
||||
each branch if the branch appears to be on the same filesystem as the
|
||||
`mergerfs` mount point and then wait for all branches to become
|
||||
"mounted." The logic to determine "mounted" is as follows.
|
||||
|
||||
1. It is mounted if the branch directory has the extended attribute
|
||||
`user.mergerfs.branch`
|
||||
2. It is mounted if the branch directory contains a file named
|
||||
`.mergerfs.branch`
|
||||
3. It is **not** mounted if the branch directory has the extended
|
||||
attribute `user.mergerfs.branch_mounts_here`
|
||||
4. It is **not** mounted if the branch directory contains
|
||||
a file named `.mergerfs.branch_mounts_here`
|
||||
5. It is mounted if the `st_dev` value of the mergerfs mount point is
|
||||
different from the branch path.
|
||||
|
||||
These files and xattrs are used to help `mergerfs` know whether or not
|
||||
a directory happens to be a branch or is the underlying directory
|
||||
mount point.
|
||||
|
||||
## Paths which appear after startup
|
||||
|
||||
The [`branches` resolution process](branches.md#resolution) runs before
|
||||
this option starts waiting. A missing literal path is retained as a
|
||||
branch, so a configuration such as `/mnt/hdd/disk0/foobar` can work
|
||||
when `/mnt/hdd/disk0` is mounted asynchronously and `foobar` only
|
||||
becomes visible afterward, provided the path then meets one of the
|
||||
"mounted" conditions above.
|
||||
|
||||
For each branch which is not ready, mergerfs first runs `mount` once
|
||||
with the full branch path as the target and then polls that same path
|
||||
until the timeout expires. It does not create the path, try to mount
|
||||
its parent, or rerun `mount`. In the example above, the initial
|
||||
`mount /mnt/hdd/disk0/foobar` may fail while the path is absent. If
|
||||
another service mounts `/mnt/hdd/disk0` and
|
||||
`/mnt/hdd/disk0/foobar` appears and meets one of the "mounted"
|
||||
conditions before the deadline, mergerfs detects it during polling
|
||||
and continues.
|
||||
|
||||
Globs are different. If `/mnt/hdd/disk0/*` has no matches during
|
||||
branch resolution, the literal path `/mnt/hdd/disk0/*` is retained.
|
||||
The mount wait does not expand it again after `/mnt/hdd/disk0`
|
||||
becomes available. Ensure the parent filesystem is mounted before
|
||||
mergerfs, configure a literal branch path, or update `branches`
|
||||
through the runtime API once the paths exist.
|
||||
|
||||
|
||||
**NOTE:** If on a `systemd` based system and using `fstab` it is a
|
||||
good idea to set the mount option
|
||||
[x-systemd.mount-timeout](https://www.freedesktop.org/software/systemd/man/latest/systemd.mount.html#x-systemd.mount-timeout=)
|
||||
to some value longer than `branches-mount-timeout` otherwise `systemd`
|
||||
might kill `mergerfs` before it finishes waiting.
|
||||
|
||||
**NOTE:** The way `mergerfs` attempts to mount the branch is by
|
||||
calling out to the system command `mount` with the branch as the
|
||||
target. If that path is configured in `/etc/fstab` as a mount point
|
||||
then `mount` should attempt to mount it with the settings defined in
|
||||
`fstab`.
|
||||
|
||||
|
||||
## branches-mount-timeout-fail
|
||||
|
||||
Default: `false`
|
||||
|
||||
When set to `true` mergerfs will fail entirely after
|
||||
`branches-mount-timeout` expires without all branches being
|
||||
mounted. If set to `false` it will simply ignore the mount status of
|
||||
the branches and continue on. The details will be
|
||||
[logged](../error_handling_and_logging.md).
|
||||
|
|
@ -1,237 +0,0 @@
|
|||
# branches
|
||||
|
||||
The `branches` argument is a colon (':') delimited list of paths to be
|
||||
pooled together. It does not matter if the paths are on the same or
|
||||
different filesystems nor does it matter [the filesystem
|
||||
type](../faq/compatibility_and_integration.md/#what-filesystems-can-be-used-as-branches)
|
||||
(within reason). Used and available space metrics will not be
|
||||
duplicated for paths on the same filesystem and any features which
|
||||
aren't supported by the underlying filesystem (such as file attributes
|
||||
or extended attributes) will return the appropriate errors.
|
||||
|
||||
Branches currently have two options which can be set. A
|
||||
[mode](#branch-mode) which impacts whether or not the branch is
|
||||
included in a policy calculation and a individual
|
||||
[minfreespace](#minfreespace) value. The values are set by
|
||||
prepending an `=` at the end of a branch designation and using commas
|
||||
as delimiters. Example: `/mnt/hdd/disk0=RW,1234`
|
||||
|
||||
|
||||
### branch mode
|
||||
|
||||
- RW: (read/write) - Default behavior. Will be eligible in all policy
|
||||
categories.
|
||||
- RO: (read-only) - Will be excluded from `create` and `action`
|
||||
policies. Same as a read-only mounted filesystem would be (though
|
||||
faster to process).
|
||||
- NC: (no-create) - Will be excluded from `create` policies. You can't
|
||||
create on that branch but you can change or delete.
|
||||
|
||||
|
||||
### minfreespace
|
||||
|
||||
Same purpose and syntax as the [global option](minfreespace.md) but
|
||||
specific to the branch. Defaults to the global value.
|
||||
|
||||
|
||||
### resolution
|
||||
|
||||
Each colon-delimited branch designation is resolved independently when
|
||||
the `branches` option is parsed, both at startup and when it is updated
|
||||
through the runtime API:
|
||||
|
||||
1. The path portion is expanded as a glob.
|
||||
2. If the glob matches one or more directories, each match becomes a
|
||||
branch.
|
||||
3. If the glob has no matches, mergerfs logs a notice and keeps the
|
||||
configured path unchanged as a single, literal branch.
|
||||
4. mergerfs resolves each resulting path with `realpath`. If that
|
||||
succeeds, the canonical path is stored. If it fails, typically
|
||||
because the path does not exist yet, the unchanged path is stored.
|
||||
5. A path which exists but is not a directory is logged and skipped. A
|
||||
path which does not exist is logged but retained.
|
||||
|
||||
Consequently, a literal branch such as `/mnt/hdd/disk0/foobar` may be
|
||||
configured before it exists and can become usable after
|
||||
`/mnt/hdd/disk0` is mounted. Branch resolution is not repeated in the
|
||||
background, however. If `/mnt/hdd/disk0/*` has no matches when it is
|
||||
resolved, mergerfs stores the literal path `/mnt/hdd/disk0/*`;
|
||||
directories which match it later are not discovered automatically.
|
||||
Update `branches` through the runtime API or restart mergerfs to
|
||||
resolve the glob again.
|
||||
|
||||
|
||||
### globbing
|
||||
|
||||
To make it easier to include multiple branches mergerfs supports
|
||||
[globbing](http://linux.die.net/man/7/glob). **The globbing tokens
|
||||
MUST be escaped when using via the shell else the shell itself will
|
||||
apply the glob itself.**
|
||||
|
||||
```
|
||||
# mergerfs /mnt/hdd/\*:/mnt/ssd/\* /media
|
||||
```
|
||||
|
||||
The above line will use all directories within `/mnt/hdd` and
|
||||
`/mnt/ssd`.
|
||||
|
||||
To have the pool mounted at boot or otherwise accessible from related
|
||||
tools use `/etc/fstab`.
|
||||
|
||||
```
|
||||
# <file system> <mount point> <type> <options> <dump> <pass>
|
||||
/mnt/hdd/*:/mnt/ssd/* /media mergerfs minfreespace=16G 0 0
|
||||
```
|
||||
|
||||
**NOTE:** The globbing is done at mount or when updated using the
|
||||
runtime API. If a new directory is added matching the glob after the
|
||||
fact it will not be automatically included.
|
||||
|
||||
A convenient way to configure branches is to use
|
||||
[symlinks](../quickstart.md/#etcfstab-w-config-file).
|
||||
|
||||
|
||||
## branch setup
|
||||
|
||||
`mergerfs` does **not** require any special setup of branch paths in
|
||||
order to be used however here are some suggestions.
|
||||
|
||||
|
||||
### layout
|
||||
|
||||
When you have a large collection of storage devices, types of storage,
|
||||
and types of interconnects it can be useful to have a verbose naming
|
||||
convention. My preference is to create a directory within `/mnt/` for
|
||||
each major type of storage device and/or interconnect. `hdd` for
|
||||
traditional hard drives; `ssd` for slower SATA based SSDs; `nvme` for
|
||||
M.2 or U.2 NVME SSDs; `remote` for NFS, SMB, sshfs, etc.
|
||||
|
||||
The mount points within those directories are named after the size of
|
||||
the storage and its serial number in the case of physical storage and
|
||||
the hostname and remote filesystem for remote.
|
||||
|
||||
```
|
||||
$ ls -lh /mnt/
|
||||
total 16K
|
||||
drwxr-xr-x 8 root root 4.0K Aug 18 2024 hdd
|
||||
drwxr-xr-x 6 root root 4.0K Oct 8 2024 nvme
|
||||
drwxr-xr-x 3 root root 4.0K Aug 24 2024 remote
|
||||
drwxr-xr-x 3 root root 4.0K Jul 14 2024 ssd
|
||||
|
||||
$ ls -lh /mnt/hdd/
|
||||
total 8K
|
||||
d--------- 2 root root 4.0K Apr 14 15:58 10T-01234567
|
||||
d--------- 2 root root 4.0K Apr 12 20:51 20T-12345678
|
||||
|
||||
$ ls -lh /mnt/nvme/
|
||||
total 8K
|
||||
d--------- 2 root root 4.0K Apr 14 16:00 1T-ABCDEFGH
|
||||
d--------- 2 root root 4.0K Apr 14 23:24 1T-BCDEFGHI
|
||||
|
||||
$ ls -lh /mnt/remote/
|
||||
total 8K
|
||||
d--------- 2 root root 4.0K Apr 12 20:23 foo-sshfs
|
||||
d--------- 2 root root 4.0K Apr 12 20:24 bar-nfs
|
||||
|
||||
# You can find the serial number of a drive using lsblk
|
||||
$ lsblk -d -o NAME,PATH,SIZE,SERIAL
|
||||
NAME PATH SIZE SERIAL
|
||||
sda /dev/sda 9.1T 01234567
|
||||
sdb /dev/sdb 18.2T 12345678
|
||||
nvme0n1 /dev/nvme0n1 953.9G ABCDEFGH
|
||||
nvme1n1 /dev/nvme1n1 953.9G BCDEFGHI
|
||||
```
|
||||
|
||||
|
||||
### permissions and mode
|
||||
|
||||
#### mount points
|
||||
|
||||
To ensure the directory is **only** used as a point to mount another
|
||||
filesystem it is good to lock it down as much as possible. Be sure to
|
||||
do this **before** mounting a filesystem to it.
|
||||
|
||||
```
|
||||
$ sudo chown root:root /mnt/hdd/10T-XYZ
|
||||
$ sudo chmod 0000 /mnt/hdd/10T-XYZ
|
||||
$ sudo setfattr -n user.mergerfs.branch_mounts_here
|
||||
$ sudo chattr +i /mnt/hdd/10T-XYZ
|
||||
```
|
||||
|
||||
The extended attribute `user.mergerfs.branch_mounts_here` is used by
|
||||
the [branches-mount-timeout](branches-mount-timeout.md) option to
|
||||
recognize whether or not a mergerfs branch path points to the intended
|
||||
filesystem.
|
||||
|
||||
The `chattr` is likely to only work on EXT{2,3,4} filesystems but will
|
||||
restrict even `root` from modifying the directory or its content but
|
||||
is still able to be a mount point.
|
||||
|
||||
|
||||
#### mounted filesystems
|
||||
|
||||
For those new to Linux, intending to be the primary individual logged
|
||||
into the system, or simply want to simplify permissions it is
|
||||
recommended to set the root of mounted filesystems like `/tmp/` is set
|
||||
to. Owned by `root`, `ugo+rwx` and [sticky
|
||||
bit](https://en.wikipedia.org/wiki/Sticky_bit) set.
|
||||
|
||||
This must be done **after** mounting the filesystem to the target
|
||||
mount point.
|
||||
|
||||
```
|
||||
$ sudo chown root:root /mnt/hdd/10T-SERIALNUM
|
||||
$ sudo chmod 1777 /mnt/hdd/10T-SERIALNUM
|
||||
$ sudo setfattr -n user.mergerfs.branch /mnt/hdd/10T-SERIALNUM
|
||||
```
|
||||
|
||||
|
||||
### formatting
|
||||
|
||||
While even less relevant to mergerfs than the details above the topic
|
||||
does com up regularly with mergerfs users. When it comes to
|
||||
partitioning and formatting it is suggested to keep things
|
||||
simple. Most users of mergerfs will be using the whole drive capacity
|
||||
and as such have a singular partition and filesystem on that
|
||||
partition. Something many people don't realize is that the partition
|
||||
is not necessary and actually can become problematic.
|
||||
|
||||
Rather than creating paritions just format the block device. Let's use
|
||||
`/dev/sda` as an example.
|
||||
|
||||
```
|
||||
$ lsblk -d -o NAME,PATH,SERIAL /dev/sda
|
||||
NAME PATH SERIAL
|
||||
sda /dev/sda 01234567
|
||||
|
||||
$ sudo mkfs.ext4 -m0 -L 01234567 /dev/sda
|
||||
mke2fs 1.47.0 (5-Feb-2023)
|
||||
Discarding device blocks: done
|
||||
Creating filesystem with 262144 4k blocks and 65536 inodes
|
||||
Filesystem UUID: ede8bf96-9aff-464e-a4fb-a4ab8a197cd7
|
||||
Superblock backups stored on blocks:
|
||||
32768, 98304, 163840, 229376
|
||||
|
||||
Allocating group tables: done
|
||||
Writing inode tables: done
|
||||
Creating journal (8192 blocks): done
|
||||
Writing superblocks and filesystem accounting information: done
|
||||
```
|
||||
|
||||
You can then use the serial number as the identifier in `fstab`.
|
||||
|
||||
```
|
||||
# <file system> <mount point> <type> <options> <dump> <pass>
|
||||
LABEL=01234567 /mnt/hdd/10TB-01234567 auto nofail 0 2
|
||||
```
|
||||
|
||||
Benefits of doing it this way?
|
||||
|
||||
* One less thing (partitions) to configure and worry about.
|
||||
* Guarantees alignment of blocks
|
||||
* Makes it easier to use the drive with different enclosures. Some
|
||||
SATA to USB adapters use 4K blocks while the drive itself is using
|
||||
512b. If you create partitions with one block size and move the
|
||||
drive to a controller that uses the other the offset on the device
|
||||
where the filesystem starts will be misinterpreted. It is possible
|
||||
to manually fix this but it isn't well documented and avoidable.
|
||||
|
|
@ -1,193 +0,0 @@
|
|||
# caching
|
||||
|
||||
## cache.files
|
||||
|
||||
Controls how [page caching](https://en.wikipedia.org/wiki/Page_cache)
|
||||
works for mergerfs itself. Not the underlying filesystems.
|
||||
|
||||
* `cache.files=off`: Disables page caching for mergerfs.
|
||||
* `cache.files=partial`: Enables page caching. Files are cached
|
||||
while open.
|
||||
* `cache.files=full`: Enables page caching. Files are cached across
|
||||
opens.
|
||||
* `cache.files=auto-full`: Enables page caching. Files are cached
|
||||
across opens if mtime and size are unchanged since previous
|
||||
open. Cache is dropped if mtime or size change on open.
|
||||
* `cache.files=per-process`: Enable page caching (equivalent to
|
||||
`cache.files=partial`) only for processes whose 'comm' name matches
|
||||
one of the values defined in cache.files.process-names. If the name
|
||||
does not match the file open is equivalent to `cache.files=off`.
|
||||
|
||||
|
||||
Generally, enabling the page cache actually *harms*
|
||||
performance[^1]. In part because it can lead to buffer bloat due to
|
||||
the kernel caching both the underlying filesystem's file content as
|
||||
well as the file through mergerfs. However, if you want to confirm
|
||||
performance differences it is recommended that you perform some
|
||||
benchmark to confirm which option works best for your setup.
|
||||
|
||||
Why then would you want to enable page caching if it consumes ~2x the
|
||||
RAM as normal and is on average slower? Because it is the only way to
|
||||
support
|
||||
[mmap](https://man7.org/linux/man-pages/man2/mmap.2.html). `mmap` is a
|
||||
way for programs to treat a file as if it is a contiguous RAM buffer
|
||||
which is regularly used by a number of programs such as those that
|
||||
leverage **sqlite3**. Despite `mmap` not being supported by all
|
||||
filesystems it is unfortunately common for software to not have an
|
||||
option to use regular file IO instead of `mmap`.
|
||||
|
||||
The good thing is that in Linux v6.6[^2] and above FUSE can now
|
||||
transparently enable page caching when mmap is requested. This means
|
||||
it should be safe to set `cache.files=off`. However, on Linux v6.5 and
|
||||
below you will need to configure `cache.files` as you need.
|
||||
|
||||
If [passthrough.io](passthrough.md) is enabled so must page
|
||||
caching. mergerfs will set `cache.files=auto-full` if `passthrough.io`
|
||||
is enabled. And when using `passthrough.io` the there is no double
|
||||
page caching since it is in fact passing through the IO.
|
||||
|
||||
|
||||
[^1]: This is not unique to mergerfs and affects all FUSE
|
||||
filesystems. It is something that the FUSE community hopes to
|
||||
investigate at some point but as of early 2025 there are a number
|
||||
of major reworking going on with FUSE which needs to be finished
|
||||
first.
|
||||
[^2]: [https://kernelnewbies.org/Linux_6.6#FUSE](https://kernelnewbies.org/Linux_6.6#FUSE)
|
||||
|
||||
|
||||
## cache.entry
|
||||
|
||||
* `cache.entry=UINT`: Sets the number of seconds to cache
|
||||
entry queries. Defaults to `1`.
|
||||
|
||||
The kernel must ask mergerfs about the existence of files. The entry
|
||||
cache caches that those details which limits the number of requests
|
||||
sent to mergerfs.
|
||||
|
||||
The risk of setting this value, as with most any cache, is related to
|
||||
[out-of-band](https://en.wikipedia.org/wiki/Out-of-band) changes. If
|
||||
the filesystems are changed outside mergerfs there is a risk of files
|
||||
which have been removed continuing to show as available. It will fail
|
||||
gracefully if a phantom file is actioned on in some way so there is
|
||||
little risk in setting the value much higher. Especially if there are
|
||||
no out-of-band changes.
|
||||
|
||||
|
||||
## cache.negative-entry
|
||||
|
||||
* `cache.negative-entry=UINT`: Sets the number of seconds to cache
|
||||
negative entry queries. Defaults to `1`.
|
||||
|
||||
This is a cache for negative entry query responses. Such as when a
|
||||
file which does not exist is referenced.
|
||||
|
||||
The risk of setting this value, as with most any cache, is related to
|
||||
[out-of-band](https://en.wikipedia.org/wiki/Out-of-band) changes. If
|
||||
the filesystems are changed outside mergerfs there is a risk of files
|
||||
which have been added outside mergerfs not appearing correctly till
|
||||
the cache entry times out if there had been a request for the same
|
||||
name within mergerfs which didn't exist. This is mostly an
|
||||
inconvenience.
|
||||
|
||||
|
||||
## cache.attr
|
||||
|
||||
* `cache.attr=UINT`: Sets the number of seconds to cache file
|
||||
attributes. Defaults to `1`.
|
||||
|
||||
This is a cache for file attributes and metadata such as that which is
|
||||
collected by the
|
||||
[stat](https://man7.org/linux/man-pages/man2/stat.2.html) system call
|
||||
which is used when you run commands such as `find` or `ls -lh`.
|
||||
|
||||
As with other caches the risk of enabling the attribute cache is if
|
||||
changes are made to the file out-of-band there could be
|
||||
inconsistencies between the actual file and the cached details which
|
||||
could result in different issues depending on how the data is used. If
|
||||
the simultaneous writing of a file from inside and outside is unlikely
|
||||
then you should be safe. That said any simultaneous, uncoordinated
|
||||
manipulation of a file can lead to unexpected results.
|
||||
|
||||
|
||||
## cache.statfs
|
||||
|
||||
* `cache.statfs=UINT`: Sets the number of seconds to cache `statfs`
|
||||
calls used by policies. Defaults to `0`.
|
||||
|
||||
A number of policies require looking up the available space of the
|
||||
branches being considered. This is accomplished by calling
|
||||
[statfs](https://man7.org/linux/man-pages/man2/statfs.2.html). This
|
||||
call however is a bit expensive so this cache reduces the overhead by
|
||||
limiting how often the calls are actually made.
|
||||
|
||||
This will mean that if the available space of branches changed
|
||||
somewhat rapidly there is a risk of `create` or `mkdir` calls made
|
||||
within the timeout period ending up on the same branch. This however
|
||||
should even itself out over time.
|
||||
|
||||
|
||||
## cache.symlinks
|
||||
|
||||
* `cache.symlinks=BOOL`: Enable kernel caching of symlink
|
||||
values. Defaults to `false`.
|
||||
|
||||
As of Linux v4.20 there is an ability to cache the value of symlinks
|
||||
so that the kernel does not need to make a request to mergerfs every
|
||||
single time a
|
||||
[readlink](https://man7.org/linux/man-pages/man2/readlink.2.html)
|
||||
request is made. While not a common usage pattern, if software very
|
||||
regularly queries symlink values, the use of this cache could
|
||||
significantly improve performance.
|
||||
|
||||
mergerfs will not error if the kernel used does not support symlink
|
||||
caching.
|
||||
|
||||
As with other caches the main risk in enabling it is if you are
|
||||
manipulating symlinks from both within and without the mergerfs
|
||||
mount. Should the value be changed outside of mergerfs then it will
|
||||
not be reflected in the mergerfs mount till the cached value is
|
||||
invalidated.
|
||||
|
||||
|
||||
## cache.readdir
|
||||
|
||||
* `cache.readdir=BOOL`: Enable kernel caching of readdir
|
||||
results. Defaults to `false`.
|
||||
|
||||
As of Linux v4.20 it supports readdir caching. This can have a
|
||||
significant impact on directory traversal. Especially when combined
|
||||
with entry (`cache.entry`) and attribute (`cache.attr`) caching. If
|
||||
the kernel doesn't support readdir caching setting the option to true
|
||||
has no effect. This option is configurable at runtime via xattr
|
||||
user.mergerfs.cache.readdir.
|
||||
|
||||
## cache.writeback
|
||||
|
||||
* `cache.writeback=BOOL`: Enable writeback cache. Defaults to `false`.
|
||||
|
||||
When `cache.files` is enabled the default is for it to perform
|
||||
writethrough caching. This behavior won't help improve performance as
|
||||
each write still goes one for one through the filesystem. By enabling
|
||||
the FUSE writeback cache small writes *may* be aggregated by the
|
||||
kernel and then sent to mergerfs as one larger request. This can
|
||||
greatly improve the throughput for apps which write to files
|
||||
inefficiently. The amount the kernel can aggregate is limited by the
|
||||
size of a FUSE message. Read the [fuse-msg-size](fuse-msg-size.md)
|
||||
section for more details.
|
||||
|
||||
There is a side effect as a result of enabling writeback
|
||||
caching. Underlying files won't ever be opened with O_APPEND or
|
||||
O_WRONLY. The former because the kernel then manages append mode and
|
||||
the latter because the kernel may request file data from mergerfs to
|
||||
populate the write cache. The O_APPEND change means that if a file is
|
||||
changed outside of mergerfs it could lead to corruption as the kernel
|
||||
won't know the end of the file has changed. That said any time you use
|
||||
caching you should keep from writing to the same file outside of
|
||||
mergerfs at the same time.
|
||||
|
||||
Note that if an application is properly sizing writes then writeback
|
||||
caching will have little or no effect. It will only help with writes
|
||||
of sizes below the FUSE message size (128K on older kernels, 1M on
|
||||
newer). Even then its effectiveness might not be great. Given the side
|
||||
effects of enabling this feature it is recommended that its benefits
|
||||
be proved out with benchmarks.
|
||||
|
|
@ -1,26 +0,0 @@
|
|||
# Deprecated Options
|
||||
|
||||
These are old, deprecated options which may no longer have any
|
||||
function or have been replaced. **They should not be used.**
|
||||
|
||||
* **allow_other**: mergerfs v2.35.0 and above sets this FUSE option
|
||||
automatically if running as root.
|
||||
* **async_read**: Use `async-read=true`.
|
||||
* **atomic_o_trunc**: Does nothing.
|
||||
* **attr_timeout**: Use [cache.attr](cache.md).
|
||||
* **auto_cache**: Use [cache.files=auto-full](cache.md).
|
||||
* **big_writes**: Does nothing.
|
||||
* **direct_io**: Use [cache.files=off](cache.md).
|
||||
* **entry_timeout**: Use [cache.entry](cache.md).
|
||||
* **hard_remove**: Does nothing.
|
||||
* **kernel_cache**: Use [cache.files=full](cache.md).
|
||||
* **negative_entry**: Use [cache.negative-entry](cache.md).
|
||||
* **remember**: Use `never-forget-nodes`.
|
||||
* **remember-nodes**: Use `never-forget-nodes`.
|
||||
* **noforget**: Use `never-forget-nodes`.
|
||||
* **nonempty**: Does nothing.
|
||||
* **splice_move**: Does nothing.
|
||||
* **splice_read**: Does nothing.
|
||||
* **splice_write**: Does nothing.
|
||||
* **sync_read**: Use `async-read=false`.
|
||||
* **use_ino**: Use [inodecalc](inodecalc.md).
|
||||
|
|
@ -1,15 +0,0 @@
|
|||
# export-support
|
||||
|
||||
* `export-support=true|false`
|
||||
* Default: `true`
|
||||
|
||||
|
||||
In theory, this flag should not be exposed to the end user. It is a
|
||||
low-level FUSE flag which indicates whether or not the kernel can send
|
||||
certain kinds of messages to it for the purposes of using it with
|
||||
NFS. mergerfs does support these messages but due to bugs and quirks
|
||||
found in the kernel and mergerfs this option is provided just in case
|
||||
it is needed for debugging.
|
||||
|
||||
Given that this flag is set when the FUSE connection is first
|
||||
initiated it is not possible to change during run time.
|
||||
|
|
@ -1,21 +0,0 @@
|
|||
# flush-on-close
|
||||
|
||||
By default, FUSE would issue a flush before the release of a file
|
||||
descriptor. This was considered a bit aggressive and a feature added
|
||||
to give the FUSE server the ability to choose when that happens.
|
||||
|
||||
* `flush-on-close=always`
|
||||
* `flush-on-close=never`
|
||||
* `flush-on-close=opened-for-write`
|
||||
* Defaults to `opened-for-write`.
|
||||
|
||||
For now it defaults to `opened-for-write` which is less aggressive
|
||||
than the behavior before this feature was added. It should not be a
|
||||
problem because the flush is really only relevant when a file is
|
||||
written to. Given flush is irrelevant for many filesystems in the
|
||||
future a branch specific flag may be added so only files opened on a
|
||||
specific branch would be flushed on close.
|
||||
|
||||
## References
|
||||
|
||||
* [https://lkml.kernel.org/linux-fsdevel/20211024132607.1636952-1-amir73il@gmail.com/T/](https://lkml.kernel.org/linux-fsdevel/20211024132607.1636952-1-amir73il@gmail.com/T/)
|
||||
|
|
@ -1,23 +0,0 @@
|
|||
# follow-symlinks
|
||||
|
||||
This feature, when enabled, will cause symlinks to be interpreted by
|
||||
mergerfs as their target.
|
||||
|
||||
When there is a getattr/stat request for a file mergerfs will check if
|
||||
the file is a symlink and depending on the `follow-symlinks` setting
|
||||
will replace the information about the symlink with that of that which
|
||||
it points to.
|
||||
|
||||
When unlink'ing or rmdir'ing the followed symlink it will remove the
|
||||
symlink itself and not that which it points to.
|
||||
|
||||
* `follow-symlinks=never`: Behave as normal. Symlinks are treated as such.
|
||||
* `follow-symlinks=directory`: Resolve symlinks only which point to directories.
|
||||
* `follow-symlinks=regular`: Resolve symlinks only which point to regular files.
|
||||
* `follow-symlinks=all`: Resolve all symlinks to that which they point
|
||||
to. Symlinks which do not point to anything are left as is.
|
||||
* Defaults to `never`.
|
||||
|
||||
**WARNING:** This feature should be considered experimental. There
|
||||
might be edge cases yet found. If you find any odd behaviors please
|
||||
file a ticket on [github](https://github.com/trapexit/mergerfs/issues/new?assignees=&labels=bug%2C+investigating&projects=&template=bug_report.md&title=).
|
||||
|
|
@ -1,43 +0,0 @@
|
|||
# func.readdir
|
||||
|
||||
* type: `STR`
|
||||
* default: `seq`
|
||||
* examples: `func.readdir=seq`, `func.readdir=cor:4`
|
||||
|
||||
`readdir` has policies to control how to read directory content.
|
||||
|
||||
| Policy | Description |
|
||||
| ------ | ----------- |
|
||||
| seq | "sequential" : Iterate sequentially over branches in the order defined in `branches`. This is the default and traditional behavior found prior to the readdir policy introduction. This will be increasingly slower as more branches are added to the pool. Especially if needing to wait for drives to spin up or network filesystems to respond. |
|
||||
| cosr:N:M | "concurrent open, sequential read" : Concurrently open branch directories using a thread pool and process them in the order defined in `branches`. This keeps memory and CPU usage low while also reducing the time spent waiting on branches to respond. `N` is the number of threads. If negative it will be the core count divided by `abs(N)`. `M` is the queue depth. If either value is `0` it will be decided based on system configuration. |
|
||||
| cosr:N | cosr:N:M with M = 0 |
|
||||
| cosr | cosr:N:M with N = 0 and M = 0 |
|
||||
| cor:N:M | "concurrent open and read" : Concurrently open branch directories and immediately start reading their contents using a thread pool. This will result in slightly higher memory and CPU usage but reduced latency. Particularly when using higher latency / slower speed network filesystem branches. Unlike `seq` and `cosr` the order of files could change due the async nature of the thread pool. This should not be a problem since the order of files returned in not guaranteed. `N` is the number of threads. If negative it will be the core count divided by `abs(N)`. `M` is the queue depth. If either value is `0` it will be decided based on system configuration. |
|
||||
| cor:N | cosr:N:M with M = 0 |
|
||||
| cor | cosr:N:M with N = 0 and M = 0 |
|
||||
|
||||
Keep in mind that `readdir` mostly just provides a list of file names
|
||||
in a directory and possibly some basic metadata about said files. To
|
||||
know details about the files, as one would see from commands like
|
||||
`find` or `ls`, it is required to call `stat` on the file which is
|
||||
controlled by `fuse.getattr`.
|
||||
|
||||
Until Linux v6.16, despite being able to change the size of FUSE
|
||||
messages, `readdir` requests were limited to 1 page (4KiB) in size per
|
||||
message. This limitation limited throughput and therefore performance
|
||||
of the `readdir` call. In v6.16 that was changed to be able to grow to
|
||||
the [fuse-msg-size](fuse-msg-size.md) which allows a lot more data to
|
||||
be sent per request and therefore much better performance in certain
|
||||
situations.
|
||||
|
||||
|
||||
## Technical Details
|
||||
|
||||
* On Linux
|
||||
[getdents](https://man7.org/linux/man-pages/man2/getdents.2.html) is
|
||||
used instead of
|
||||
[readdir](https://man7.org/linux/man-pages/man2/readdir.2.html) in
|
||||
order to leverage larger buffers for better performance on larger
|
||||
directory reads as well as have more control in concurrent policies.
|
||||
* [readdir](https://man7.org/linux/man-pages/man2/readdir.2.html) is
|
||||
used on FreeBSD.
|
||||
|
|
@ -1,150 +0,0 @@
|
|||
# functions, categories and policies
|
||||
|
||||
The POSIX filesystem API is made up of a number of functions. `creat`,
|
||||
`stat`, `chown`, etc. For ease of configuration in mergerfs, most of
|
||||
the core functions are grouped into 3 categories: `action`, `create`,
|
||||
and `search`. These functions and categories can be assigned a policy
|
||||
which dictates which branch is chosen when performing that
|
||||
function. IE. When an application requests that a file or directory is
|
||||
created how should mergerfs decide which branch to choose to attempt
|
||||
the create? That is the policy.
|
||||
|
||||
Some functions, listed in the category `N/A` below, can not be
|
||||
assigned the normal policies because they are directly related to a
|
||||
file which has already been opened.
|
||||
|
||||
|
||||
## Functions and their Category classifications
|
||||
|
||||
| Category | Functions |
|
||||
| -------- | --------- |
|
||||
| action | chmod, chown, link, removexattr, rename, rmdir, setxattr, truncate, unlink, utimens |
|
||||
| create | create, mkdir, mknod, symlink |
|
||||
| search | access, getattr, getxattr, ioctl (directories), listxattr, open, readlink |
|
||||
| N/A | fchmod, fchown, futimens, ftruncate, fallocate, fgetattr, fsync, ioctl (files), read, readdir, release, statfs, write, copy_file_range |
|
||||
|
||||
In cases where something may be searched for (such as a path to clone)
|
||||
`getattr` will usually be used.
|
||||
|
||||
|
||||
## Policies
|
||||
|
||||
See below for [available policies and their descriptions](#policy-descriptions).
|
||||
|
||||
A policy is an algorithm designed to select one or more branches for a
|
||||
function to operate on.
|
||||
|
||||
Policies do not actually manage the filesystem or layout. They are
|
||||
strictly responsible for deciding which files or branches will be
|
||||
worked on in relation to the function being performed. Once the branch
|
||||
is chosen other parts of the system does what is necessary to
|
||||
accomplish the function. Such as the cloning of directories between
|
||||
branches.
|
||||
|
||||
When using policies which are based on a branch's available space the
|
||||
branch base path provided is used. Not the full path to the file or
|
||||
directory in question. Meaning that mounts within the branch will not
|
||||
be considered in the space calculations.
|
||||
|
||||
**NOTE:** While any policy can be assigned to a function or category,
|
||||
some may not be very useful in practice. For instance: `rand` (random)
|
||||
may be useful for file creation but could lead to very odd behavior if
|
||||
used for `chmod` if there were more than one copy of the file. Unless
|
||||
users find this flexibility useful it will likely be removed in the
|
||||
future.
|
||||
|
||||
|
||||
## Filtering
|
||||
|
||||
Most policies search branches and create a list of files / paths for
|
||||
functions to work on. The policy is responsible for filtering and
|
||||
sorting the branches. Filters include [minfreespace](minfreespace.md),
|
||||
whether or not a branch is mounted read-only, and the branch mode
|
||||
(RO,NC,RW). These filters are applied across most policies.
|
||||
|
||||
- No `search` function policies filter.
|
||||
- All `action` function policies filter out branches which are
|
||||
mounted **read-only** or mode is **RO (read-only)**.
|
||||
- All `create` function policies filter out branches which are
|
||||
mounted **read-only**, mode **RO (read-only)** or **NC (no
|
||||
create)**, or has available space less than
|
||||
[minfreespace](minfreespace.md).
|
||||
|
||||
Policies may have their own additional filtering such as those that
|
||||
require relative paths to be present.
|
||||
|
||||
If all branches are filtered an error will be returned. Typically
|
||||
`EROFS` (read-only filesystem) or `ENOSPC` (no space left on
|
||||
device) depending on the most recent reason for filtering a
|
||||
branch.
|
||||
|
||||
If `create`, `mkdir`, `mknod`, or `symlink` fail with `EROFS`
|
||||
or other fundamental errors then mergerfs will mark any branch found
|
||||
to be read-only as such (IE will set the mode `RO`) and will rerun the
|
||||
policy and try again. This is mostly for `ext4` filesystems that can
|
||||
suddenly become read-only when it encounters an error.
|
||||
|
||||
|
||||
## Path Preservation
|
||||
|
||||
Policies, as described below, are of two basic classifications. `path
|
||||
preserving` and `non-path preserving`.
|
||||
|
||||
All policies which start with `ep` (`epff`, `eplfs`, `eplus`, `epmfs`,
|
||||
`eprand`) are `path preserving`. `ep` stands for `existing path`.
|
||||
|
||||
A path preserving policy will only consider branches where the relative
|
||||
path being accessed already exists.
|
||||
|
||||
With the `msp` or `most shared path` policies they are defined as
|
||||
`path preserving` for the purpose of controlling `link` and `rename`'s
|
||||
behaviors since `ignorepponrename` is available to disable that
|
||||
behavior.
|
||||
|
||||
|
||||
## Policy descriptions
|
||||
|
||||
A policy's behavior differs, as mentioned above, based on the function
|
||||
it is used with. Sometimes it really might not make sense to even
|
||||
offer certain policies because they are literally the same as others
|
||||
but it makes things a bit more uniform.
|
||||
|
||||
| Policy | Description |
|
||||
| --------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
|
||||
| pfrd (percentage free random distribution) | Selects a branch randomly, with the probability of selection proportional to its available space relative to the total available space across all branches. For instance, if Branch A has 100 GB free and Branch B has 50 GB free, Branch A is twice as likely to be chosen. |
|
||||
| rand (random) | Calls **all** and then randomizes. Returns 1 branch. |
|
||||
| mfs (most free space) | Pick the branch with the most available free space. |
|
||||
| ff (first found) | Given the order of the branches, as defined at mount time or configured at runtime, act on the first one found. |
|
||||
| lfs (least free space) | Pick the branch with the least available free space. |
|
||||
| lup (least used percent) | Pick the branch with the least used percentage of space |
|
||||
| lus (least used space) | Pick the branch with the least used space. |
|
||||
| all | Search: For **mkdir**, **mknod**, and **symlink** it will apply to all branches. **create** works like **ff**. |
|
||||
| msppfrd (most shared path, percentage free random distribution) | Like **eppfrd** but if it fails to find a branch it will try again with the parent directory. Continues this pattern till finding one. |
|
||||
| mspmfs (most shared path, most free space) | Like **epmfs** but if it fails to find a branch it will try again with the parent directory. Continues this pattern till finding one. |
|
||||
| msplfs (most shared path, least free space) | Like **eplfs** but if it fails to find a branch it will try again with the parent directory. Continues this pattern till finding one. |
|
||||
| msplus (most shared path, least used space) | Like **eplus** but if it fails to find a branch it will try again with the parent directory. Continues this pattern till finding one. |
|
||||
| eppfrd (existing path, percentage free random distribution) | Like **pfrd** but limited to existing paths. |
|
||||
| epmfs (existing path, most free space) | Of all the branches on which the relative path exists choose the branch with the most free space. |
|
||||
| eprand (existing path, random) | Calls **epall** and then randomizes. Returns 1. |
|
||||
| epff (existing path, first found) | Given the order of the branches, as defined at mount time or configured at runtime, act on the first one found where the relative path exists. |
|
||||
| eplfs (existing path, least free space) | Of all the branches on which the relative path exists choose the branch with the least free space. |
|
||||
| eplus (existing path, least used space) | Of all the branches on which the relative path exists choose the branch with the least used space. |
|
||||
| epall (existing path, all) | For **mkdir**, **mknod**, and **symlink** it will apply to all found. **create** works like **epff** (but more expensive because it doesn't stop after finding a valid branch). |
|
||||
| newest | Pick the file / directory with the largest mtime. |
|
||||
|
||||
**NOTE:** If you are using an underlying filesystem that reserves
|
||||
blocks such as ext2, ext3, or ext4 be aware that mergerfs respects the
|
||||
reservation by using `f_bavail` (number of free blocks for
|
||||
unprivileged users) rather than `f_bfree` (number of free blocks) in
|
||||
policy calculations. **df** does NOT use `f_bavail`, it uses
|
||||
`f_bfree`, so direct comparisons between **df** output and mergerfs'
|
||||
policies is not appropriate.
|
||||
|
||||
|
||||
## Defaults
|
||||
|
||||
| Category | Policy |
|
||||
| -------- | ------ |
|
||||
| action | epall |
|
||||
| create | pfrd |
|
||||
| search | ff |
|
||||
|
|
@ -1,50 +0,0 @@
|
|||
# fuse-msg-size
|
||||
|
||||
* type: `UINT|SIZE`
|
||||
* default: `1M`
|
||||
* example: `fuse-msg-size=1M`
|
||||
* If the value is a `SIZE` in the form of `xB`, `xK`, `xM`, `xG`, or
|
||||
`xT` then it will choose the closest multiple of the system page
|
||||
size. If the value is just an integer without a size multipler suffix
|
||||
then it is treated as a page count
|
||||
|
||||
FUSE applications communicate with the kernel over a special character
|
||||
device: `/dev/fuse`. A large portion of the overhead associated with
|
||||
FUSE is the cost of going back and forth between user space and kernel
|
||||
space over that device. Generally speaking, the fewer trips needed the
|
||||
better the performance will be. Reducing the number of trips can be
|
||||
done a number of ways. Kernel level caching and increasing message
|
||||
sizes being two significant ones. When it comes to reads and writes if
|
||||
the message size is doubled the number of trips are approximately
|
||||
halved.
|
||||
|
||||
In Linux v4.20 a new feature was added allowing the negotiation of the
|
||||
max message size. Since the size is in multiples of
|
||||
[pages](https://en.wikipedia.org/wiki/Page_(computer_memory)) the
|
||||
feature is called `max_pages`. In versions of Linux prior to v6.13
|
||||
there is a maximum `max_pages` value of 256 (1MiB) and minimum of 1
|
||||
(4KiB). In [Linux
|
||||
v6.13](https://web.git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=2b3933b1e0a0a4b758fbc164bb31db0c113a7e2c)
|
||||
and above the max value supported by the kernel can range from 1
|
||||
(4KiB) to 65535 (~256MiB) (assuming a page size of 4KiB.) The default
|
||||
used by Linux >= 4.20, and hard coded value used before 4.20, is 32
|
||||
(128KiB). In mergerfs it is referred to as `fuse-msg-size` to make it
|
||||
clear what it impacts and provide some abstraction.
|
||||
|
||||
If the `fuse-msg-size` value provided is more than the system wide
|
||||
maximum mergerfs will attempt to increase the system wide value to keep
|
||||
the user from needing to set the value using `sysctl`,
|
||||
`/etc/sysctl.conf`, or `/proc/sys/fs/fuse/max_pages_limit`.
|
||||
|
||||
The main downside to increasing the value is that memory usage will
|
||||
increase approximately relative to the number of [processing
|
||||
threads](threads.md) configured. Keep this in mind for systems with
|
||||
lower amounts of memory like most SBCs. Performance improvements seem
|
||||
to peak around 4MiB.
|
||||
|
||||
On kernels before v4.20 the option has no effect. On kernels between
|
||||
v4.20 and v6.13 the max value is 256. On kernels >= v6.13 the maximum
|
||||
value is 65535.
|
||||
|
||||
NOTE: If you intend to enable `cache.files` you should also set
|
||||
[readahead](readahead.md) to match `fuse-msg-size`.
|
||||
|
|
@ -1,80 +0,0 @@
|
|||
# inodecalc
|
||||
|
||||
[Inodes](https://en.wikipedia.org/wiki/Inode) (`st_ino`) are unique
|
||||
identifiers for files within a filesystem. Each mounted filesystem
|
||||
also has a (typically) unique device ID (`st_dev`) as well and
|
||||
together they can uniquely identify a file on the whole of the
|
||||
system. Entries on the same device with the same inode are (generally)
|
||||
in fact references to the same underlying file. It is a many to one
|
||||
relationship between names and an inode. Directories, however, do not
|
||||
have multiple links on most systems due to the complexity that would
|
||||
add.
|
||||
|
||||
FUSE allows the server (in this case `mergerfs`) to set inode values
|
||||
but not device IDs. Creating an inode value is somewhat complex in
|
||||
mergerfs' case as files aren't really in its control. If a policy
|
||||
changes what directory or file is to be selected or changes are made
|
||||
to branches out of band it becomes unclear what value should be
|
||||
used. Most software does not to care what the values are but those
|
||||
that do often break if a value changes unexpectedly. For instance: the
|
||||
tool `find` will abort a directory walk if it sees a directory inode
|
||||
change. NFS can return stale handle errors if the inode changes
|
||||
out-of-band. File dedup tools will usually leverage device ids and
|
||||
inodes as a shortcut in searching for duplicate files and would resort
|
||||
to full file comparisons should it find different inode values.
|
||||
|
||||
mergerfs offers multiple ways to calculate the inode in hopes of
|
||||
covering different usecases.
|
||||
|
||||
* `passthrough`: Passes through the underlying inode value. Mostly
|
||||
intended for testing as using this does not address any of the
|
||||
problems mentioned above and could confuse file deduplication
|
||||
software as inodes from different filesystems can be the same.
|
||||
* `path-hash`: Hashes the relative path of the entry in question. The
|
||||
underlying file's values are completely ignored. This means the
|
||||
inode value will always be the same for that file path. This is
|
||||
useful when using NFS and you make changes out of band such as copy
|
||||
data between branches. This also means that entries that do point to
|
||||
the same file will not be recognizable via inodes. That does not
|
||||
mean hard links don't work. They will.
|
||||
* `path-hash32`: 32bit version of path-hash.
|
||||
* `devino-hash`: Hashes the branch path and inode of the underlying
|
||||
entry. This won't prevent issues with NFS should the policy pick a
|
||||
different file or files move out of band but will present the same
|
||||
inode for underlying files that share the same inode. Previous
|
||||
releases of mergerfs used the device ID but that changes on
|
||||
reboot. The downside to using the branch path is inodes for files on
|
||||
the same underlying filesystem will be different.
|
||||
* `devino-hash32`: 32bit version of devino-hash.
|
||||
* `hybrid-hash`: Performs path-hash on directories and devino-hash on
|
||||
other file types. Since directories can't have hard links the static
|
||||
value won't make a difference and the files will get values useful
|
||||
for finding duplicates. Probably the best to use if not using
|
||||
NFS. As such it is the default.
|
||||
* `hybrid-hash32`: 32bit version of hybrid-hash.
|
||||
|
||||
32bit versions are provided as there is some software which does not
|
||||
handle 64bit inodes well.
|
||||
|
||||
While there is a risk of hash collision in tests of a couple of
|
||||
million entries there were zero collisions. Unlike a typical
|
||||
filesystem FUSE filesystems can reuse inodes and not refer to the same
|
||||
entry. The internal identifier used to reference a file in FUSE is
|
||||
different from the inode value presented. The former is the nodeid and
|
||||
is actually a tuple of 2 64bit values: nodeid and generation. This
|
||||
tuple is not client facing. The inode that is presented to the client
|
||||
is passed through the kernel uninterpreted.
|
||||
|
||||
From FUSE docs for `use_ino`:
|
||||
|
||||
> Honor the st_ino field in the functions getattr() and
|
||||
> fill_dir(). This value is used to fill in the st_ino field
|
||||
> in the stat(2), lstat(2), fstat(2) functions and the d_ino
|
||||
> field in the readdir(2) function. The filesystem does not
|
||||
> have to guarantee uniqueness, however some applications
|
||||
> rely on this value being unique for the whole filesystem.
|
||||
> Note that this does *not* affect the inode that libfuse
|
||||
> and the kernel use internally (also called the "nodeid").
|
||||
|
||||
**NOTE:** As of version 2.35.0 the use_ino option has been
|
||||
removed. mergerfs should always be managing inode values.
|
||||
Some files were not shown because too many files have changed in this diff Show more
Loading…
Add table
Add a link
Reference in a new issue