Skip to content

Commit

Permalink
PS-9449 Update the Thread pool topic in 8.0
Browse files Browse the repository at this point in the history
	modified:   docs/threadpool.md
  • Loading branch information
patrickbirch committed Oct 2, 2024
1 parent 81edcbd commit 04d67e3
Showing 1 changed file with 37 additions and 31 deletions.
68 changes: 37 additions & 31 deletions docs/threadpool.md
Original file line number Diff line number Diff line change
@@ -1,21 +1,40 @@
# Thread pool

You can use thread pooling for MySQL databases to improve performance and scalability. Thread pooling is a technique that reuses a fixed number of threads to handle multiple client connections and execute statements. Thread pooling reduces the overhead of creating and destroying threads, and avoids the contention and context switching that can occur when there are too many threads.
Thread pooling can improve performance and scalability for MySQL databases. This technique reuses a fixed number of threads to handle multiple client connections and execute statements. It reduces the overhead of creating and destroying threads and avoids the contention and context switching that can occur when there are too many threads.

MySQL supports thread pooling through the thread pool plugin, which replaces the default one-thread-per-connection model. When a statement arrives, the thread group either begins executing it immediately or queues it for later execution in a round-robin fashion. The high-priority queue consists of a number of thread groups, each of which manages a set of client connections. Each thread group has a listener thread that listens for incoming statements from the connections assigned to the group. The thread pool exposes several system variables that can be used to configure its operation, such as thread_pool_size, thread_pool_algorithm, thread_pool_stall_limit and others.
MySQL supports thread pooling through the thread pool plugin, which replaces the default one-thread-per-connection model. When a statement arrives, the thread group either begins executing it immediately or queues it for later execution in a round-robin fashion. The high-priority queue consists of several thread groups, each managing client connections. Each thread group has a listener thread that listens for incoming statements from the connections assigned to the group. The thread pool exposes several system variables that can be used to configure its operation, such as thread_pool_size, thread_pool_algorithm, thread_pool_stall_limit, and others.

The thread pool plugin consists of a number of thread groups, each of which manages a set of client connections. As connections are established, the thread pool assigns them to thread groups in round-robin fashion. For more details, see .
The thread pool plugin consists of several thread groups, each of which manages a set of client connections. As connections are established, the thread pool assigns them to thread groups using the round-robin method. This method assigns threads fairly and efficiently. Here's how it works:

1. The thread pool starts with a set number of thread groups.

2. When a new task arrives, the pool needs to assign it to a group.

3. It does this by going through the groups in order, one by one.

4. Let's say you have four thread groups. The assignment would work like this:
- Task 1 goes to Group 1
- Task 2 goes to Group 2
- Task 3 goes to Group 3
- Task 4 goes to Group 4
- Task 5 goes back to Group 1

5. This pattern continues, always moving to the next group and starting over when it reaches the end.

6. Each group handles its assigned tasks using its available threads.

This round-robin approach spreads work evenly across all groups. It prevents any single group from getting overloaded while others sit idle. This method helps maintain balanced performance across the system.

MySQL executes statements using one thread per client connection. When the number of connections increases past a specific point, performance degrades.
This feature introduces a dynamic thread pool, which enables the server to keep the top performance even with a large number of client connections. The server decreases the number of threads using the thread pool and reduces the context switching and hot locks contentions. The thread pool is most effective with `OLTP` workloads (relatively short CPU-bound queries).
This feature introduces a dynamic thread pool, which enables the server to maintain top performance even with a large number of client connections. The server decreases the number of threads using the thread pool and reduces the context switching and hot lock contentions. The thread pool is most effective with `OLTP` workloads (relatively short CPU-bound queries).

Set the thread pool variable [thread_handling](#thread_handling) to `pool-of-threads` by adding the following line to `my.cnf`:

```text
thread_handling=pool-of-threads
```

Although the default values for the thread pool should provide good performance, perform additional tuning with the dynamic system variables. The goal is to minimize the number of open transactions on the server. Short-running transactions commit faster and deallocate server resources and locks.
Although the default values for the thread pool should provide good performance, additional tuning should be performed with the dynamic system variables. The goal is to minimize the number of open transactions on the server. Short-running transactions commit faster and deallocate server resources and locks.

Due to the following differences, this implementation is not compatible with upstream:

Expand All @@ -25,7 +44,7 @@ Due to the following differences, this implementation is not compatible with ups

Priority Queue:

This is a queue that assigns a priority to each data element and processes them according to their priority. The data element with the highest priority is served first, regardless of its order in the queue. A priority queue can be implemented using an array, a linked list, a heap or a binary search tree. A priority queue can also be ascending or descending, meaning that the highest priority is either the smallest or the largest value.
A queue that assigns a priority to each data element and processes them according to their priority. The data element with the highest priority is served first, regardless of its order in the queue. A priority queue can be implemented using an array, a linked list, a heap, or a binary search tree. It can also be ascending or descending, meaning that the highest priority is either the smallest or the largest value.

## Version specific information

Expand All @@ -45,27 +64,24 @@ The thread pool adds the connection to the high-priority queue and decrements th

Otherwise, the variable adds the connection to the low-priority queue with the initial value.

The thread pool checks the high-priority queue for the next connection each time. The thread pool picks connections from the low-priority queue when the high-priority queue is empty. The default behavior is to put events from already started transactions into the high-priority queue.
Each time, the thread pool checks the high-priority queue for the next connection. When the high-priority queue is empty, the thread pool picks connections from the low-priority queue. The default behavior is to put events from already started transactions into the high-priority queue.

If the value equals `0`, all connections are put into the low-priority queue. Each connection could be put into a high-priority queue if the value exceeds zero.
If the value equals `0`, all connections are put into the low-priority queue. If the value exceeds zero, each connection could be put into a high-priority queue.

The [thread_pool_high_prio_mode](#thread_pool_high_prio_mode) variable prioritizes all statements for a connection or assigns connections to the low-priority queue. To implement this new [thread_pool_high_prio_mode](#thread_pool_high_prio_mode) variable


## Low-priority queue throttling

One case that can limit thread pool performance and even lead to deadlocks under high concurrency is when thread groups are oversubscribed due to active threads reaching the oversubscribe limit. Still, all/most worker threads are waiting on locks currently held by a transaction from another connection not currently in the thread pool.
One case that can limit thread pool performance and even lead to deadlocks under high concurrency is when thread groups are oversubscribed due to active threads reaching the oversubscribe limit. Still, all/most worker threads are waiting on locks currently held by a transaction from another connection that is not currently in the thread pool.

In this case, the oversubscribe limit does not account for those threads in the pool that marked themselves inactive. As a result, the number of threads (both active and waiting) in the pool grows until it hits the [`thread_pool_max_threads`](#thread_pool_max_threads) value. If the connection executing the transaction holding the lock has managed to enter the thread pool by then, we get a large (depending on the [`thread_pool_max_threads`](#thread_pool_max_threads) value) number of concurrently running threads and, thus, suboptimal performance. Otherwise, we get a deadlock as no more threads can be created to process those transaction(s) and release the lock(s).

Such situations are prevented by throttling the low-priority queue when the total number of worker threads (both active and waiting ones) reaches the oversubscribe limit. If there are too many worker threads, do not start new transactions and create new threads until queued events from the already started transactions are processed.
Such situations are prevented by throttling the low-priority queue when the total number of worker threads (both active and waiting ones) reaches the oversubscribe limit. If there are too many worker threads, do not start new transactions; create new threads until queued events from the already-started transactions are processed.

## Handling long network waits

Specific workloads (large result sets, BLOBs, slow clients) can wait longer on network I/O (socket reads and writes). Whenever the server waits, this should be communicated to the thread pool so it can start a new query by either waking a waiting thread or sometimes creating a new one. This implementation has been ported from *MariaDB* patch MDEV-156.



## System variables

### `thread_handling`
Expand Down Expand Up @@ -111,7 +127,7 @@ The following values are allowed:

* `statements`. In this mode, all individual statements go into the high-priority queue, regardless of the transactional state and the number of available high-priority tickets. Use this value to prioritize `AUTOCOMMIT` transactions or other statements, such as administrative ones. Setting this value globally essentially disables high-priority scheduling. All connections use the high-priority queue.

* `none`. This mode disables the priority queue for a connection. Certain types of connections, for example, monitoring, are insensitive to execution latency and do not allocate the server resources that would impact the performance of other connections. These types of connections do not require high-priority scheduling. Setting this value globally essentially disables high-priority scheduling. All connections use the low-priority queue.
* `none`. This mode disables the priority queue for a connection. Certain types of connections, such as monitoring, are insensitive to execution latency and do not allocate the server resources that would impact the performance of other connections. These types of connections do not require high-priority scheduling. Setting this value globally essentially disables high-priority scheduling. All connections use the low-priority queue.

### `thread_pool_high_prio_tickets`

Expand All @@ -137,7 +153,7 @@ This variable controls the high-priority queue policy. Assigns the selected numb
| Data type: | Numeric |
| Default value: | 100000 |

This variable can limit the maximum number of threads in the pool. The server does not create new threads when the limit is reached.
This variable can limit the maximum number of threads in the pool. When the limit is reached, the server does not create new threads.

### `thread_pool_oversubscribe`

Expand Down Expand Up @@ -176,19 +192,14 @@ Defines the number of threads that can use the CPU simultaneously.
| Data type: | Numeric |
| Default value: | 500 (ms) |

Defines the number of milliseconds before a running thread is considered stalled. The thread pool will wake up or create another thread when this limit is reached. This variable prevents a long-running query from monopolizing the pool.

### Upgrading from a version before 8.0.14 to 8.0.14 or higher


Defines the number of milliseconds before a running thread is considered stalled. When this limit is reached, the thread pool will wake up or create another thread. This variable prevents a long-running query from monopolizing the pool.

### `extra_port`

The variable was removed in [Percona Server for MySQL 8.0.14](release-notes/Percona-Server-8.0.14.md).

Specifies an additional port that Percona Server for MySQL listens on. This can be used in case no new connections can be established
due to all worker threads being busy or being locked when `pool-of-threads`
feature is enabled.
It specifies an additional port that Percona Server for MySQL listens to. This port can be used in case no new connections can be established
due to all worker threads being busy or being locked when the `pool-of-threads` feature is enabled.

The following command connects to the extra port:

Expand All @@ -207,11 +218,11 @@ mysql --port='extra-port-number' --protocol=tcp
| Data type: | Numeric |
| Default value: | 1 |

The varible was removed in [Percona Server for MySQL 8.0.14](release-notes/Percona-Server-8.0.14.md). This variable can be used to specify the maximum allowed number of connections
plus one extra `SUPER` users connection on the extra_port. This
The variable was removed in [Percona Server for MySQL 8.0.14](release-notes/Percona-Server-8.0.14.md). This variable can be used to specify the maximum allowed number of connections
plus one extra `SUPER` user connection on the extra_port. This
can be used with the extra_port variable to access the server in
case no new connections can be established due to all worker threads being busy
or being locked when `pool-of-threads` feature is enabled.
or being locked when the `pool-of-threads` feature is enabled.

## Status variables

Expand All @@ -233,8 +244,3 @@ This status variable shows the number of idle threads in the pool.

This status variable shows the number of threads in the pool.

## Other reading

* [Thread pool in MariaDB 5.5](https://kb.askmonty.org/en/threadpool-in-55/)

* [Thread pool implementation in Oracle MySQL](https://mikaelronstrom.blogspot.com/2011_10_01_archive.html)

0 comments on commit 04d67e3

Please sign in to comment.