mirror of
https://github.com/trapexit/mergerfs.git
synced 2026-10-26 19:01:56 -04:00
Replace std::filesystem::path with custom fs::relpath
std::filesystem::path is wrong for mergerfs's hot path:
- `branch->path / fusepath_` heap-allocates per FUSE op (3 allocs each
on libstdc++; the SSO buffer is 15 bytes and any joined path exceeds it).
- The implicit `operator string_type()` adds another std::string when
passing to `fs::*` wrappers that take `const std::string&`.
- Per-branch loops rebuild fullpath from scratch every iteration.
The new fs::relpath is a fixed 8192-byte (2 * PATH_MAX) inline buffer,
NUL-terminated, never heap-allocates. The rel data is anchored at offset
REL_OFFSET = 4096; the 4096 bytes before the anchor form the "prefix
area" so a long base/prefix can be written there in place via
set_prefix() without disturbing the rel. The remaining ~4096 bytes
after the rel are available for trailing append (filenames in
readdir-style loops, etc.).
set_prefix's cost is O(prefix size) -- only the actually-used bytes
are touched -- so the buffer size is a stack-footprint question, not a
hot-path cost. FUSE worker threads use the pthread default stack
(~8 MB on Linux); a 4-relpath frame in fuse_link/fuse_rename is 32 KB,
~0.4% of the stack budget.
The per-branch hot loop:
fs::relpath fullpath = fusepath_; // one copy of the rel
for (auto branch : branches) {
fullpath.set_prefix(branch->path); // writes only [start, 4096)
fs::syscall(fullpath);
}
set_prefix only rewrites the prefix area; the rel stays anchored. A
cached _rel_size means no recomputation across iterations.
Microbenchmark (8 branches, 30-char fusepath, -O3, 4 M path builds):
variant ns/build allocs/build
-------------------------------------- -------- ------------
origin/master (std::filesystem::path) 158 3.00
fs::relpath assign_concat (single-shot) 9.1 0
fs::relpath set_prefix (loop-optimized) 4.5 0
35x faster, zero allocations in the hot path.
Contract: paths flow through fs::relpath in canonical form -- prefix
without trailing '/', rel without leading '/'. No sanitization;
violations yield double slashes. Mergerfs satisfies this by
construction:
- libfuse's try_get_path() walks leaf-to-root via prepend(), then
calls consolidate_to_rel(strip=true) which drops the leading '/'
that prepend produces.
- branch paths are realpath-canonicalized at config load so they
never have a trailing slash.
Mergerfs has exactly one path-manipulation type: a relative fusepath.
Branch paths and other absolute paths are stored as std::string -- they
are spliced into the prefix area of an fs::relpath at syscall time but
are never themselves mutated as path objects.
Major changes:
- vendored/libfuse/{include/fs_path.hpp,lib/fs_path.cpp}: new
fs::relpath class (8 KB inline, REL anchor at 4096, set_prefix /
clear_prefix / consolidate_to_rel, _rel_size cache, self-aliasing-
safe assign_concat, sticky ENAMETOOLONG via errnum()).
- 30+ FUSE-op call sites converted from `fullpath = branch / fusepath`
to the `set_prefix` loop pattern (fuse_unlink, fuse_chmod,
fuse_chown, fuse_listxattr, fuse_truncate, fuse_utimens,
fuse_setxattr, fuse_rmdir, fuse_removexattr, fuse_link's
preserve-path and create-path loops, fuse_rename's create-path
and preserve-path loops, fuse_statfs FULL mode, fs::findallfiles,
fs::findonfs).
- fs::* syscall wrappers that previously only took
`const std::string&` got `const char*` overloads so fs::relpath's
implicit `operator const char*()` lands cleanly (fs::link,
fs::lutimens, fs::umount_lazy, fs::open_dir_ro, fs::readlink,
fs::lremovexattr, fs::lstatvfs, fs::info, fs::statvfs_cache).
- fs::exists gained a (fs::relpath&, basepath, ...) overload that
calls set_prefix internally so per-branch policy fan-out loops
share a single fullpath, rewriting only the prefix area on each
iteration.
- All policies (epall, epff, eplfs, eplus, epmfs, eppfrd, ff, lup,
msplfs, msplus, mspmfs, msppfrd, newest) refactored to hoist the
fs::relpath fullpath construction out of the per-branch loop and
use the set_prefix-based fs::exists overload.
- fs::inode::ReaddirCalc::_fusepath migrated from std::string to
fs::relpath with the same resize/append API.
- std::filesystem removed entirely from the project: branches.cpp
and option_parser.cpp's exists/is_directory/equivalent calls now
use direct ::stat(); mergerfs_fsck.cpp's recursive_directory_iterator
was replaced with an opendir-based walk.
Tests (tests/tests.cpp): 27 new fs::relpath test cases covering
construction, set_prefix in/out of REL anchor, prefix()/rel()
accessors, clear_prefix, concat canonical form, self-aliasing
(`p /= "x"`), operator/=, resize zero-fill, swap, copy/move,
equality, implicit conversions, lexically_relative, remove_filename,
_rel_size invariant across every mutator, capacity bounds.
Full suite: 213/213 pass.
This commit is contained in:
parent
7ec6f8e2f2
commit
947024914a
187 changed files with 3596 additions and 1510 deletions
|
|
@ -24,29 +24,26 @@
|
|||
#include "fs_path.hpp"
|
||||
#include "fs_statvfs_cache.hpp"
|
||||
#include "policy.hpp"
|
||||
#include "policy_epall.hpp"
|
||||
#include "policy_error.hpp"
|
||||
#include "strvec.hpp"
|
||||
|
||||
#include <string>
|
||||
|
||||
|
||||
static
|
||||
int
|
||||
_create(const Branches::Ptr &branches_,
|
||||
const fs::path &fusepath_,
|
||||
const fs::relpath &fusepath_,
|
||||
std::vector<Branch*> &paths_)
|
||||
{
|
||||
int rv;
|
||||
int error;
|
||||
fs::info_t info;
|
||||
fs::relpath fullpath(fusepath_);
|
||||
|
||||
error = ENOENT;
|
||||
for(auto &branch : *branches_)
|
||||
{
|
||||
if(branch.ro_or_nc())
|
||||
error_and_continue(error,EROFS);
|
||||
if(!fs::exists(branch.path,fusepath_))
|
||||
if(!fs::exists(fullpath,branch.path))
|
||||
error_and_continue(error,ENOENT);
|
||||
rv = fs::info(branch.path,&info);
|
||||
if(rv < 0)
|
||||
|
|
@ -68,19 +65,20 @@ _create(const Branches::Ptr &branches_,
|
|||
static
|
||||
int
|
||||
_action(const Branches::Ptr &branches_,
|
||||
const fs::path &fusepath_,
|
||||
const fs::relpath &fusepath_,
|
||||
std::vector<Branch*> &paths_)
|
||||
{
|
||||
int rv;
|
||||
int error;
|
||||
bool readonly;
|
||||
fs::relpath fullpath(fusepath_);
|
||||
|
||||
error = ENOENT;
|
||||
for(auto &branch : *branches_)
|
||||
{
|
||||
if(branch.ro())
|
||||
error_and_continue(error,EROFS);
|
||||
if(!fs::exists(branch.path,fusepath_))
|
||||
if(!fs::exists(fullpath,branch.path))
|
||||
error_and_continue(error,ENOENT);
|
||||
rv = fs::statvfs_cache_readonly(branch.path,&readonly);
|
||||
if(rv < 0)
|
||||
|
|
@ -100,12 +98,14 @@ _action(const Branches::Ptr &branches_,
|
|||
static
|
||||
int
|
||||
_search(const Branches::Ptr &branches_,
|
||||
const fs::path &fusepath_,
|
||||
const fs::relpath &fusepath_,
|
||||
std::vector<Branch*> &paths_)
|
||||
{
|
||||
fs::relpath fullpath(fusepath_);
|
||||
|
||||
for(auto &branch : *branches_)
|
||||
{
|
||||
if(!fs::exists(branch.path,fusepath_))
|
||||
if(!fs::exists(fullpath,branch.path))
|
||||
continue;
|
||||
|
||||
paths_.emplace_back(&branch);
|
||||
|
|
@ -119,7 +119,7 @@ _search(const Branches::Ptr &branches_,
|
|||
|
||||
int
|
||||
Policy::EPAll::Action::operator()(const Branches::Ptr &branches_,
|
||||
const fs::path &fusepath_,
|
||||
const fs::relpath &fusepath_,
|
||||
std::vector<Branch*> &paths_) const
|
||||
{
|
||||
return ::_action(branches_,fusepath_,paths_);
|
||||
|
|
@ -127,7 +127,7 @@ Policy::EPAll::Action::operator()(const Branches::Ptr &branches_,
|
|||
|
||||
int
|
||||
Policy::EPAll::Create::operator()(const Branches::Ptr &branches_,
|
||||
const fs::path &fusepath_,
|
||||
const fs::relpath &fusepath_,
|
||||
std::vector<Branch*> &paths_) const
|
||||
{
|
||||
return ::_create(branches_,fusepath_,paths_);
|
||||
|
|
@ -135,7 +135,7 @@ Policy::EPAll::Create::operator()(const Branches::Ptr &branches_,
|
|||
|
||||
int
|
||||
Policy::EPAll::Search::operator()(const Branches::Ptr &branches_,
|
||||
const fs::path &fusepath_,
|
||||
const fs::relpath &fusepath_,
|
||||
std::vector<Branch*> &paths_) const
|
||||
{
|
||||
return ::_search(branches_,fusepath_,paths_);
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue