mirror of
https://github.com/trapexit/mergerfs.git
synced 2026-10-26 19:01:56 -04:00
Rework thread pool for increased stability + config and doc updates (#1453)
Also updates fmt to 11.2.0 and replaced syslog usage with a wrapper leveraging fmt.
This commit is contained in:
parent
455a81290b
commit
344014aa81
58 changed files with 9097 additions and 24020 deletions
|
|
@ -1,14 +1,20 @@
|
|||
# func.readdir
|
||||
|
||||
examples: `func.readdir=seq`, `func.readdir=cor:4`
|
||||
Defaults to `seq`
|
||||
|
||||
Examples: `func.readdir=seq`, `func.readdir=cor:4`
|
||||
|
||||
`readdir` has policies to control how it reads directory content.
|
||||
|
||||
| Policy | Description |
|
||||
| ------ | ----------- |
|
||||
| seq | "sequential" : Iterate sequentially over branches in the order defined in `branches`. This is the default and traditional behavior found prior to the readdir policy introduction. This will be increasingly slower as more branches are added to the pool. Especially if needing to wait for drives to spin up or network filesystems to respond. |
|
||||
| cosr | "concurrent open, sequential read" : Concurrently open branch directories using a thread pool and process them in the order defined in `branches`. This keeps memory and CPU usage low while also reducing the time spent waiting on branches to respond. Number of threads defaults to the number of logical cores. Can be overwritten via the syntax `func.readdir=cosr:N` where `N` is the number of threads. |
|
||||
| cor | "concurrent open and read" : Concurrently open branch directories and immediately start reading their contents using a thread pool. This will result in slightly higher memory and CPU usage but reduced latency. Particularly when using higher latency / slower speed network filesystem branches. Unlike `seq` and `cosr` the order of files could change due the async nature of the thread pool. This should not be a problem since the order of files listed in not guaranteed. Number of threads defaults to the number of logical cores. Can be overwritten via the syntax `func.readdir=cor:N` where `N` is the number of threads. |
|
||||
| seq | "sequential" : Iterate sequentially over branches in the order defined in `branches`. This is the default and traditional behavior found prior to the readdir policy introduction. This will be increasingly slower as more branches are added to the pool. Especially if needing to wait for drives to spin up or network filesystems to respond. |
|
||||
| cosr:N:M | "concurrent open, sequential read" : Concurrently open branch directories using a thread pool and process them in the order defined in `branches`. This keeps memory and CPU usage low while also reducing the time spent waiting on branches to respond. `N` is the number of threads. If negative it will be the core count divided by `abs(N)`. `M` is the queue depth. If either value is `0` it will be decided based on system configuration. |
|
||||
| cosr:N | cosr:N:M with M = 0 |
|
||||
| cosr | cosr:N:M with N = 0 and M = 0 |
|
||||
| cor:N:M | "concurrent open and read" : Concurrently open branch directories and immediately start reading their contents using a thread pool. This will result in slightly higher memory and CPU usage but reduced latency. Particularly when using higher latency / slower speed network filesystem branches. Unlike `seq` and `cosr` the order of files could change due the async nature of the thread pool. This should not be a problem since the order of files listed in not guaranteed. `N` is the number of threads. If negative it will be the core count divided by `abs(N)`. `M` is the queue depth. If either value is `0` it will be decided based on system configuration. |
|
||||
| cor:N | cosr:N:M with M = 0 |
|
||||
| cor | cosr:N:M with N = 0 and M = 0 |
|
||||
|
||||
Keep in mind that `readdir` mostly just provides a list of file names
|
||||
in a directory and possibly some basic metadata about said files. To
|
||||
|
|
|
|||
|
|
@ -6,48 +6,56 @@ parallel behaviors.
|
|||
|
||||
## read-thread-count
|
||||
|
||||
The number of threads used to read (and possibly process) messages
|
||||
from the kernel.
|
||||
Defaults to `0`
|
||||
|
||||
* `read-thread-count=0`: Create a thread pool sized to the number of
|
||||
logical CPUs.
|
||||
* `read-thread-count=N` where `N>0`: Create a thread pool of `N` threads.
|
||||
* `read-thread-count=N` where `N<0`: Create a thread pool of `CPUCount /
|
||||
-N` threads.
|
||||
* `read-thread-count=-1` where `process-thread-count=-1`: Creates `2`
|
||||
read threads and `max(2,CPUCount-2)` process threads.
|
||||
* Defaults to `0`.
|
||||
The number of threads used to read messages from the kernel. If used
|
||||
alone processing will be done on the same thread. If
|
||||
`process-thread-count` is enabled then this thread pool will read
|
||||
messages and the process thread pool will do the work. This can
|
||||
increase how much mergerfs can process at one time but may reduce
|
||||
throughput.
|
||||
|
||||
When `process-thread-count=-1` (the default) this option sets the
|
||||
number of threads which read and then process requests from the
|
||||
kernel.
|
||||
|
||||
When `process-thread-count` is set to anything else mergerfs will
|
||||
create two thread pools. A "read" thread pool which just reads from
|
||||
the kernel and hands off requests to the "process" thread pool.
|
||||
|
||||
Generally, only 1 or 2 "read" threads are necessary.
|
||||
* `read-thread-count=0` and `process-thread-count=-1`: Create `1`
|
||||
read+process thread per logical CPU core upto `8`.
|
||||
* `read-thread-count=N` where `N>0` and `process-thread-count=-1`:
|
||||
Create a thread pool of `N` read+process threads.
|
||||
* `read-thread-count=N` where `N<0` and `process-thread-count=-1`:
|
||||
Create a read+process thread pool of `CPUCount / -N`
|
||||
threads. Minimum of `1`.
|
||||
* `read-thread-count=0` and `process-thread-count=0`: Create `2` read
|
||||
threads and a process thread per logical CPU core upto `8`.
|
||||
* `read-thread-count=0` and `process-thread-count!=-1`: Creates `2`
|
||||
read threads and number of process threads as defined below.
|
||||
|
||||
|
||||
## process-thread-count
|
||||
|
||||
Defaults to `-1`
|
||||
|
||||
When enabled this sets the number of threads in the message processing pool.
|
||||
|
||||
* `process-thread-count=-1`: Process thread pool is disabled.
|
||||
* `process-thread-count=0`: Create a thread pool sized to the number
|
||||
of logical CPUs.
|
||||
* `process-thread-count=0`: Create `1` thread process thread per
|
||||
logical CPU core upto `8`.
|
||||
* `process-thread-count=N` where `N>0`: Create a thread pool of `N` threads.
|
||||
* `process-thread-count=N` where `N<-1`: Create a thread pool of `CPUCount /
|
||||
-N` threads.
|
||||
* Defaults to `-1`.
|
||||
-N` threads. Minimum of 1.
|
||||
|
||||
|
||||
## process-thread-queue-depth
|
||||
|
||||
* `process-thread-queue-depth=N` where `N>0`: Sets the number of outstanding
|
||||
requests that a process thread can have to N. If requests come in
|
||||
faster than can be processed and the max queue depth hit then
|
||||
queuing the request will block in order to limit memory growth.
|
||||
* `process-thread-queue-depth=0`: Sets the queue depth to the thread
|
||||
pool count.
|
||||
* Defaults to `0`.
|
||||
Defaults to `2`
|
||||
|
||||
Sets the depth queue for the processing thread queue per
|
||||
thread. Meaning if the read threads are getting requests faster than
|
||||
can be processed they will be queued up upto the queue depth. Despite
|
||||
the calculation being per thread the queue depth is shared across all
|
||||
in the pool.
|
||||
|
||||
* `process-thread-queue-depth=N` where `N>0`: Sets the number of
|
||||
outstanding requests that the process thread pool can have to `N *
|
||||
process-thread-count`. If requests come in faster than can be
|
||||
processed and the max queue depth hit then queuing the request will
|
||||
block in order to limit memory growth.
|
||||
* `process-thread-queue-depth<=0`: Sets the queue depth to 2. May be
|
||||
used in the future to set dynamically.
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue