rt/mergerfs
1
0
Fork 0
mirror of https://github.com/trapexit/mergerfs.git synced 2026-10-26 19:01:56 -04:00
Code Issues Projects Releases Packages Wiki Activity Actions

Detect external scaling in hill-climb to avoid misattribution

Browse source 
The hill-climb monitor samples throughput, perturbs thread count by
+/-1, and checks whether throughput improved.  But between samples,
other mechanisms (fast-path pressure grow, idle self-exit) can also
change the thread count.  The monitor had no way to distinguish its
own perturbation's effect from these external changes, which could
cause it to reinforce or reverse direction based on unrelated scaling.

Track the expected thread count after each perturbation.  At the next
sample, compare the actual count to the expected one.  If they differ,
external scaling occurred — skip the throughput attribution, reset the
decline streak, and re-baseline from the current count.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
This commit is contained in:
Antonio S. Musumeci 2026-04-10 01:13:32 -04:00
parent 20acdfce19
commit 11fafba6ae
1 changed files with 26 additions and 5 deletions
Download patch file Download diff file Expand all files Collapse all files

31
vendored/libfuse/include/thread_pool.hpp
View file

@ -487,6 +487,12 @@ ThreadPool::monitor_routine(void *arg_)
constexpr int decline_threshold = 2;
int warmup_samples = 0;
// Track expected thread count after the last perturbation so we
// can detect external scaling (fast-path grow, idle self-exit)
// and avoid misattributing their throughput effects to our action.
std::size_t expected_count = btp->thread_count();
bool acted_last_cycle = false;
const struct timespec sleep_ts = {
static_cast<time_t>(interval / 1000000),
static_cast<long>((interval % 1000000) * 1000)
@ -534,6 +540,20 @@ ThreadPool::monitor_routine(void *arg_)
continue;
}
std::size_t count = btp->thread_count();
// If external scaling changed the thread count since our last
// perturbation, we can't attribute the throughput delta to our
// action. Reset the streak and skip this cycle.
if(acted_last_cycle && count != expected_count)
{
decline_streak = 0;
acted_last_cycle = false;
expected_count = count;
continue;
}
acted_last_cycle = false;
// Hill-climb: did smoothed throughput improve since last sample?
if(ema_throughput >= prev_ema)
{
@ -562,19 +582,18 @@ ThreadPool::monitor_routine(void *arg_)
continue;
}
// Read count, act, then re-read for an accurate log message.
std::size_t count = btp->thread_count();
if(direction > 0 && count < btp->_scaling_config.max_threads)
{
btp->add_thread();
btp->_last_scale_time_usecs.store(_now_usecs(),
std::memory_order_relaxed);
acted_last_cycle = true;
expected_count = btp->thread_count();
syslog(LOG_DEBUG,
"threadpool (%s): hill-climb grow to %zu threads "
"(ema %.1f -> %.1f)",
btp->_name.c_str(),
btp->thread_count(),
expected_count,
prev_ema,
ema_throughput);
}
@ -583,11 +602,13 @@ ThreadPool::monitor_routine(void *arg_)
btp->remove_thread();
btp->_last_scale_time_usecs.store(_now_usecs(),
std::memory_order_relaxed);
acted_last_cycle = true;
expected_count = btp->thread_count();
syslog(LOG_DEBUG,
"threadpool (%s): hill-climb shrink to %zu threads "
"(ema %.1f -> %.1f)",
btp->_name.c_str(),
btp->thread_count(),
expected_count,
prev_ema,
ema_throughput);
}