Commit graph

1528 commits

Author SHA1 Message Date
Antonio S. Musumeci
38fee4dd5f Fix use-after-free in start_routine_autoscale idle self-exit
After _try_remove_self_if_above detaches the thread, the destructor
will not join it. If the destructor runs before the detached thread
reaches the syslog call, btp->_name is a dangling reference.

Copy _name into a local string at thread start so the syslog after
self-removal uses stack-local data that outlives the detach.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-09 20:59:31 -04:00
Antonio S. Musumeci
9222a4ff62 Move _tp_now_usecs into ThreadPool as private static _now_usecs
The file-scope function _tp_now_usecs had a leading underscore
followed by a lowercase letter, which is reserved at file scope
by the C++ standard. Move it to a private static member function
ThreadPool::_now_usecs where leading underscores on member names
are permitted.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-09 20:46:42 -04:00
Antonio S. Musumeci
e87acf0433 Add sane defaults to ScalingConfig members
ScalingConfig was value-initialized to all zeros when no config
is provided. While the non-autoscale path gates all access, a
zero idle_threshold_usecs would cause try_wait_dequeue_for to
return immediately if the autoscale path were ever reached by
mistake.

Add default member initializers: min_threads=2, max_threads=64,
sample_interval=100ms, cooldown=200ms, idle_threshold=5s. These
match the values recommended in the scaling policy design and
provide safe behavior even if a caller constructs ScalingConfig
without setting all fields.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-09 20:43:21 -04:00
Antonio S. Musumeci
73b6bb3cdf Move _tasks_enqueued increment to after successful enqueue
The counter was incremented before the enqueue call, which can
block indefinitely for bounded queues. If tasks_pending() is used
as a drain-complete signal (pending == 0), the pre-increment
would overcount while a producer is blocked waiting for a slot.

Move the increment to after the enqueue returns in all paths:
enqueue_work, enqueue_work_autoscale, and enqueue_task. The
try_enqueue variants already incremented after success.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-09 20:39:32 -04:00
Antonio S. Musumeci
3bcf66cdaa Fix monitor and idle-exit racing past min_threads
The hill-climb monitor and idle self-exit paths could both shrink
the pool past min_threads. The monitor read thread count, released
the lock, then called remove_thread - but between those steps an
idle worker could also self-exit, dropping below the threshold.

Three changes:
- _remove_thread_locked now checks min_threads (from ScalingConfig)
  under the same lock as the size check, refusing removal if at or
  below the bound.
- Idle self-exit replaced with _try_remove_self_if_above which
  atomically checks thread count and erases under a single lock
  acquisition, closing the TOCTOU window.
- The previous _remove_self is retained for the non-scaling paths
  (poison pill exit during set_threads/remove_thread).

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-09 20:36:48 -04:00
Antonio S. Musumeci
6a52d7b1a2 Fix semaphore drift from enqueue_unbounded in BoundedQueue
enqueue_unbounded bypasses the semaphore so control messages
(poison pills, thread removal tasks) can never block on a full
queue. But wait_dequeue always signals a slot after consuming,
so each unbounded item that gets dequeued inflated the semaphore
by +1, permanently raising the effective max queue depth.

Fix by calling tryWait() in enqueue_unbounded before enqueueing.
When the queue has room (common case), this consumes a slot and
the dequeue-side signal restores balance perfectly. When the queue
is genuinely full (tryWait fails), the item still enqueues but
temporarily exceeds max_depth by one until consumed - a transient
overshoot bounded by control messages in flight.

A CAS-based approach that absorbed extra signals on the dequeue
side was also considered but rejected: it can starve blocked
bounded producers by swallowing the signal they are waiting for.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-09 20:21:36 -04:00
Antonio S. Musumeci
cb6f444e45 Fix double std::forward in enqueue_work_autoscale
The try-enqueue and fallback block-enqueue both called
std::forward<FuncType>(func_) on the same parameter. If
try_enqueue's internals ever change to move from the argument
before failing, the second forward would operate on a moved-from
object.

Fix by materializing into a Func up front and using std::move
for both attempts. Also extract the duplicated scaling logic into
_maybe_grow_on_pressure(), eliminating the copy-paste between the
PToken and non-PToken overloads.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-09 19:39:18 -04:00
Antonio S. Musumeci
c0b4ea3fae Fix destructor race with _remove_self during shutdown
The destructor and autoscale idle-exit path could race: a worker
calling _remove_self would erase itself from _threads and detach
while the destructor was iterating _threads to enqueue poison pills
and join. This could cause too few poison pills (deadlocking
remaining workers) or pthread_join on a detached thread (UB).

Two changes:
- _remove_self now checks _stop and returns immediately during
  shutdown, ensuring no thread detaches after destruction begins.
- Destructor snapshots _threads under the mutex before enqueueing
  poison pills and joining, so it operates on a stable copy.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-09 19:35:35 -04:00
Antonio S. Musumeci
9ae145b7e9 checkpoint 2026-04-09 19:31:47 -04:00
Antonio S. Musumeci
9dc461c96d checkpoint 2026-04-09 19:13:58 -04:00
Antonio SJ Musumeci
3905355522 Detect OS for release-sample make target
Some checks failed
mkdocs / deploy (push) Has been cancelled
2026-04-08 20:48:02 -05:00
Antonio SJ Musumeci
9e2e1afafa Update docs on limiting drive spinup
Some checks failed
mkdocs / deploy (push) Has been cancelled
2026-04-06 14:01:20 -05:00
Antonio SJ Musumeci
6d218fa87f Misc updates and fixes
Some checks failed
mkdocs / deploy (push) Has been cancelled
2026-04-04 02:46:24 -05:00
Antonio SJ Musumeci
3b10406f95 Add more details to project comparisons 2026-04-04 02:17:06 -05:00
Antonio SJ Musumeci
f2c8e5d9eb Add FUSE request_timeout
Some checks failed
mkdocs / deploy (push) Has been cancelled
Not yet hooked up to config parser to set as need to evaluate usage.
2026-04-01 23:20:05 -05:00
Antonio SJ Musumeci
b581c5d5e7 Add support for FUSE_COPY_FILE_RANGE_64 2026-04-01 23:20:05 -05:00
Antonio SJ Musumeci
1ebc8cfa5f Enable statx with file handle
Some checks are pending
mkdocs / deploy (push) Waiting to run
2026-03-31 19:53:17 -04:00
Antonio SJ Musumeci
acfa797503 Misc formatting changes 2026-03-31 19:53:17 -04:00
Antonio SJ Musumeci
7fc20457ca Rework open file resource management 2026-03-31 19:53:17 -04:00
Antonio SJ Musumeci
ecee84b323 Improve proj comparisons + new SnapRAID details 2026-03-31 19:51:28 -04:00
Antonio SJ Musumeci
b8776d205d Update docs; add nofs, policyfs and greyhole
Some checks failed
mkdocs / deploy (push) Has been cancelled
2026-03-27 22:54:39 -05:00
Antonio SJ Musumeci
06433a0548 Fallback to fusermount3 where available
Some checks failed
mkdocs / deploy (push) Has been cancelled
Also general cleanup of mount_generic.
2026-03-24 21:13:34 -04:00
Antonio SJ Musumeci
f4b602340b Add more integration tests 2026-03-24 21:13:34 -04:00
Antonio SJ Musumeci
a78aef36e0 Additional check for ENOTCONN on mount 2026-03-24 21:13:34 -04:00
Antonio SJ Musumeci
0d05523dfd Move maintainence thread func take u64 rather than int
signed rollover is UB
2026-03-24 21:13:34 -04:00
Antonio SJ Musumeci
0cfab60f37 replace raw C-strings with std::string in converted libfuse files
util/mount.mergerfs.cpp:
- Remove xstrdup/xrealloc wrappers entirely
- Convert type, source, options, command, setuid to std::string
- Replace add_arg() pointer-rewriting with shell_quote() returning std::string
- Replace add_option() with simple std::string += concatenation
- Fix const_cast UB on string literal assignment

lib/helper.cpp:
- Convert add_default_subtype local subtype_opt: malloc+snprintf+free -> std::string construction

lib/mount_generic.h:
- Convert fuse_mount_sys locals source/type: malloc+strcpy+strcat -> std::string
- Remove 4 unnecessary free() calls (RAII cleanup)
- .c_str() at mount() and fuse_mnt_add_mount() boundaries

lib/mount_util.h:
- Rewrite fuse_mnt_resolve_path internals using std::string: rfind/substr
  instead of manual pointer arithmetic, strrchr, strlen chains
2026-03-24 21:13:34 -04:00
Antonio SJ Musumeci
21966ade97 Convert vendored/libfuse C sources to C++17
Convert all C source files in vendored/libfuse/ to C++17:
- lib/crc32b.c    -> crc32b.cpp  (const void* cast)
- lib/fuse_opt.c -> fuse_opt.cpp (designated init, void* arithmetic)
- lib/fuse_signals.c -> fuse_signals.cpp
- lib/helper.c   -> helper.cpp  (void* data cast)
- lib/mount.c    -> mount.cpp  (thin wrapper)
- lib/mount_generic.c -> mount_generic.h (static inline, #pragma once)
- lib/mount_bsd.c -> mount_bsd.h    (static inline, #pragma once)
- lib/mount_util.c -> mount_util.h   (static inline, #pragma once)
- util/fusermount.c -> fusermount.cpp
- util/mount.mergerfs.c -> mount.mergerfs.cpp

Key changes:
- Replace C99 designated initializer in fuse_opt_parse with field-by-field init
- Fix void* pointer arithmetic via char* base + offset pattern
- All C-style (T*)voidptr casts replaced with static_cast/reinterpret_cast
- Remove redundant #define _GNU_SOURCE (already in FUSE_FLAGS)
- Wrap __clone2 declaration in extern "C" for ia64 path
- Update Makefile: empty SRC_C, build utils with $(CXX) instead of $(CC)
- Remove stale .c files now replaced by .cpp/.h counterparts

All binaries build cleanly: mergerfs, mergerfs-fusermount, mount.mergerfs
2026-03-24 21:13:34 -04:00
Antonio SJ Musumeci
96eb51eecb Move fuse_session to C++ and make exit flag atomic 2026-03-24 21:13:34 -04:00
Antonio SJ Musumeci
881147daa2 Fix null ptr handling issue in ioctl processing 2026-03-24 21:13:34 -04:00
Antonio SJ Musumeci
0f1a8e9b69 Fix error condition in fs_copyfile 2026-03-24 21:13:34 -04:00
Antonio SJ Musumeci
08f9cc6932 Fix readlink implementation, add more tests 2026-03-24 21:13:34 -04:00
Antonio SJ Musumeci
3099b15db1 More aggressive error handing with pagesize queries 2026-03-24 21:13:34 -04:00
Antonio SJ Musumeci
07b1e6bdba Auto-commit fuse_msgbuf.cpp 2026-03-24 21:13:34 -04:00
Antonio SJ Musumeci
7763c7384a Use fatal::abort() rather than assert in places 2026-03-24 21:13:34 -04:00
Antonio SJ Musumeci
61365d226d Move scope_guard to vendored/ 2026-03-24 21:13:34 -04:00
Antonio SJ Musumeci
f33e78bc24 Cleanup procfs and abort on critical issues 2026-03-24 21:13:34 -04:00
Antonio SJ Musumeci
7f52623536 Cleanup branches and add more unit tests 2026-03-24 21:13:34 -04:00
Antonio SJ Musumeci
a30a4df9c0 Cleanup config and add tests 2026-03-24 21:13:34 -04:00
Antonio SJ Musumeci
1e0d5aa91d fs_wait_for_mount cleanup 2026-03-24 21:13:34 -04:00
Antonio SJ Musumeci
d09fadbc28 fs_inode cleanup 2026-03-24 21:13:34 -04:00
Antonio SJ Musumeci
21ce889449 Fixup listxattr error handling 2026-03-24 21:13:34 -04:00
Antonio SJ Musumeci
9d1a4696e9 Small improvements to fs::statvfs_cache 2026-03-24 21:13:34 -04:00
Antonio SJ Musumeci
ed0cbcf064 Cleanup fuse_readlink 2026-03-24 21:13:34 -04:00
Antonio SJ Musumeci
076e359891 Improve thread pool 2026-03-24 21:13:34 -04:00
Antonio SJ Musumeci
bf98d3f3a6 Fix possible error handling issue with errno 2026-03-24 21:13:34 -04:00
Antonio SJ Musumeci
c45c5ce920 Make cfgfile depth calculation thread friendly 2026-03-24 21:13:34 -04:00
Antonio SJ Musumeci
e0a892f0d0 Improve error handling when querying pagesize 2026-03-24 21:13:34 -04:00
Antonio SJ Musumeci
164b7f73c9 Misc fixes and improvements to thread pool 2026-03-24 21:13:34 -04:00
Antonio SJ Musumeci
3a45be4cd8 Rework string utils, more test cases 2026-03-24 21:13:34 -04:00
Antonio SJ Musumeci
107a3acf95 Better error handling in pagesize config type 2026-03-24 21:13:34 -04:00