mirror of
https://github.com/trapexit/mergerfs.git
synced 2026-10-26 19:01:56 -04:00
Replace std::filesystem::path with custom fs::relpath
std::filesystem::path is wrong for mergerfs's hot path:
- `branch->path / fusepath_` heap-allocates per FUSE op (3 allocs each
on libstdc++; the SSO buffer is 15 bytes and any joined path exceeds it).
- The implicit `operator string_type()` adds another std::string when
passing to `fs::*` wrappers that take `const std::string&`.
- Per-branch loops rebuild fullpath from scratch every iteration.
The new fs::relpath is a fixed 8192-byte (2 * PATH_MAX) inline buffer,
NUL-terminated, never heap-allocates. The rel data is anchored at offset
REL_OFFSET = 4096; the 4096 bytes before the anchor form the "prefix
area" so a long base/prefix can be written there in place via
set_prefix() without disturbing the rel. The remaining ~4096 bytes
after the rel are available for trailing append (filenames in
readdir-style loops, etc.).
set_prefix's cost is O(prefix size) -- only the actually-used bytes
are touched -- so the buffer size is a stack-footprint question, not a
hot-path cost. FUSE worker threads use the pthread default stack
(~8 MB on Linux); a 4-relpath frame in fuse_link/fuse_rename is 32 KB,
~0.4% of the stack budget.
The per-branch hot loop:
fs::relpath fullpath = fusepath_; // one copy of the rel
for (auto branch : branches) {
fullpath.set_prefix(branch->path); // writes only [start, 4096)
fs::syscall(fullpath);
}
set_prefix only rewrites the prefix area; the rel stays anchored. A
cached _rel_size means no recomputation across iterations.
Microbenchmark (8 branches, 30-char fusepath, -O3, 4 M path builds):
variant ns/build allocs/build
-------------------------------------- -------- ------------
origin/master (std::filesystem::path) 158 3.00
fs::relpath assign_concat (single-shot) 9.1 0
fs::relpath set_prefix (loop-optimized) 4.5 0
35x faster, zero allocations in the hot path.
Contract: paths flow through fs::relpath in canonical form -- prefix
without trailing '/', rel without leading '/'. No sanitization;
violations yield double slashes. Mergerfs satisfies this by
construction:
- libfuse's try_get_path() walks leaf-to-root via prepend(), then
calls consolidate_to_rel(strip=true) which drops the leading '/'
that prepend produces.
- branch paths are realpath-canonicalized at config load so they
never have a trailing slash.
Mergerfs has exactly one path-manipulation type: a relative fusepath.
Branch paths and other absolute paths are stored as std::string -- they
are spliced into the prefix area of an fs::relpath at syscall time but
are never themselves mutated as path objects.
Major changes:
- vendored/libfuse/{include/fs_path.hpp,lib/fs_path.cpp}: new
fs::relpath class (8 KB inline, REL anchor at 4096, set_prefix /
clear_prefix / consolidate_to_rel, _rel_size cache, self-aliasing-
safe assign_concat, sticky ENAMETOOLONG via errnum()).
- 30+ FUSE-op call sites converted from `fullpath = branch / fusepath`
to the `set_prefix` loop pattern (fuse_unlink, fuse_chmod,
fuse_chown, fuse_listxattr, fuse_truncate, fuse_utimens,
fuse_setxattr, fuse_rmdir, fuse_removexattr, fuse_link's
preserve-path and create-path loops, fuse_rename's create-path
and preserve-path loops, fuse_statfs FULL mode, fs::findallfiles,
fs::findonfs).
- fs::* syscall wrappers that previously only took
`const std::string&` got `const char*` overloads so fs::relpath's
implicit `operator const char*()` lands cleanly (fs::link,
fs::lutimens, fs::umount_lazy, fs::open_dir_ro, fs::readlink,
fs::lremovexattr, fs::lstatvfs, fs::info, fs::statvfs_cache).
- fs::exists gained a (fs::relpath&, basepath, ...) overload that
calls set_prefix internally so per-branch policy fan-out loops
share a single fullpath, rewriting only the prefix area on each
iteration.
- All policies (epall, epff, eplfs, eplus, epmfs, eppfrd, ff, lup,
msplfs, msplus, mspmfs, msppfrd, newest) refactored to hoist the
fs::relpath fullpath construction out of the per-branch loop and
use the set_prefix-based fs::exists overload.
- fs::inode::ReaddirCalc::_fusepath migrated from std::string to
fs::relpath with the same resize/append API.
- std::filesystem removed entirely from the project: branches.cpp
and option_parser.cpp's exists/is_directory/equivalent calls now
use direct ::stat(); mergerfs_fsck.cpp's recursive_directory_iterator
was replaced with an opendir-based walk.
Tests (tests/tests.cpp): 27 new fs::relpath test cases covering
construction, set_prefix in/out of REL anchor, prefix()/rel()
accessors, clear_prefix, concat canonical form, self-aliasing
(`p /= "x"`), operator/=, resize zero-fill, swap, copy/move,
equality, implicit conversions, lexically_relative, remove_filename,
_rel_size invariant across every mutator, capacity bounds.
Full suite: 213/213 pass.
This commit is contained in:
Antonio S. Musumeci
parent
7ec6f8e2f2
commit
947024914a
187 changed files with 3596 additions and 1510 deletions
|
|
@ -289,11 +289,11 @@ fs::inode::get_algo(void)
|
|||
return {};
|
||||
}
|
||||
|
||||
fs::inode::ReaddirCalc::ReaddirCalc(const fs::path &branch_path_,
|
||||
const fs::path &dirpath_)
|
||||
fs::inode::ReaddirCalc::ReaddirCalc(const std::string &branch_path_,
|
||||
const fs::relpath &dirpath_)
|
||||
: _algo(::_algo_from_func(g_func.load())),
|
||||
_branch_seed(::_branch_seed(branch_path_.native())),
|
||||
_fusepath((dirpath_ / "__mergerfs__").native()),
|
||||
_branch_seed(::_branch_seed(branch_path_)),
|
||||
_fusepath(fs::relpath::concat(dirpath_,"__mergerfs__")),
|
||||
_filename_offset(_fusepath.size() - std::string_view("__mergerfs__").size())
|
||||
{
|
||||
}
|
||||
|
|
@ -337,23 +337,11 @@ fs::inode::ReaddirCalc::calc(const char *name_,
|
|||
|
||||
u64
|
||||
fs::inode::calc(const std::string &branch_path_,
|
||||
const std::string &fusepath_,
|
||||
const mode_t mode_,
|
||||
const ino_t ino_)
|
||||
{
|
||||
return g_func.load()(branch_path_,
|
||||
fusepath_,
|
||||
mode_,
|
||||
ino_);
|
||||
}
|
||||
|
||||
u64
|
||||
fs::inode::calc(const fs::path &branch_path_,
|
||||
const fs::path &fusepath_,
|
||||
const fs::relpath &fusepath_,
|
||||
const mode_t mode_,
|
||||
const ino_t ino_)
|
||||
{
|
||||
return g_func.load()(branch_path_.native(),
|
||||
return g_func.load()(branch_path_,
|
||||
fusepath_.native(),
|
||||
mode_,
|
||||
ino_);
|
||||
|
|
@ -361,18 +349,7 @@ fs::inode::calc(const fs::path &branch_path_,
|
|||
|
||||
void
|
||||
fs::inode::calc(const std::string &branch_path_,
|
||||
const std::string &fusepath_,
|
||||
struct stat *st_)
|
||||
{
|
||||
st_->st_ino = calc(branch_path_,
|
||||
fusepath_,
|
||||
st_->st_mode,
|
||||
st_->st_ino);
|
||||
}
|
||||
|
||||
void
|
||||
fs::inode::calc(const fs::path &branch_path_,
|
||||
const fs::path &fusepath_,
|
||||
const fs::relpath &fusepath_,
|
||||
struct stat *st_)
|
||||
{
|
||||
st_->st_ino = calc(branch_path_,
|
||||
|
|
@ -382,19 +359,8 @@ fs::inode::calc(const fs::path &branch_path_,
|
|||
}
|
||||
|
||||
void
|
||||
fs::inode::calc(const std::string &branch_path_,
|
||||
const std::string &fusepath_,
|
||||
struct fuse_statx *st_)
|
||||
{
|
||||
st_->ino = calc(branch_path_,
|
||||
fusepath_,
|
||||
st_->mode,
|
||||
st_->ino);
|
||||
}
|
||||
|
||||
void
|
||||
fs::inode::calc(const fs::path &branch_path_,
|
||||
const fs::path &fusepath_,
|
||||
fs::inode::calc(const std::string &branch_path_,
|
||||
const fs::relpath &fusepath_,
|
||||
struct fuse_statx *st_)
|
||||
{
|
||||
st_->ino = calc(branch_path_,
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue