Skip to content

Commit

Permalink
Change status of intra broker balancing from beta to GA (#855)
Browse files Browse the repository at this point in the history
  • Loading branch information
kbatuigas authored Nov 19, 2024
1 parent 0cf4b28 commit ac16003
Show file tree
Hide file tree
Showing 2 changed files with 1 addition and 16 deletions.
13 changes: 1 addition & 12 deletions modules/manage/pages/cluster-maintenance/cluster-balancing.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -118,20 +118,9 @@ This mode is not recommended for production clusters. Only set to `off` if you n

== Intra-broker partition balancing

[NOTE]
====
This is a beta feature for v24.2 and is not recommended for use for production clusters. The feature is disabled by default. To unlock this feature, run this command with a superuser:
```bash
curl -X PUT -d '{"state": "active"}' http://127.0.0.1:9644/v1/features/node_local_core_assignment
```
====

In Redpanda, every partition replica is assigned to a CPU core on a broker. While Redpanda's default <<partition-replica-balancing,partition balancing>> monitors cluster-level events, such as the addition of new brokers or broker failure to balance partition assignments, it does not account for the distribution of partitions _within_ an individual broker.

Prior to Redpanda version 24.2, this meant that some cores on a broker could inadvertently host many partitions of heavily-used topics and cause the CPU to be xref:manage:monitoring.adoc#cpu-usage[overburdened]. Additionally, when the partition rebalance moved some partitions away from a broker, the remaining partitions did not necessarily rebalance across the broker's cores. Or, if a broker's core count was increased, Redpanda did not assign any partitions to the new cores until new partitions were created or old partitions were moved out.

Starting in v24.2, topic-aware intra-broker partition balancing allows for dynamically reassigning partitions within a broker. Redpanda prioritizes an even distribution of a topic's partition replicas across all cores in a broker. If a broker's core count changes, when the broker starts back up, Redpanda can check partition assignments across the broker's cores and reassign partitions, so that a balanced assignment is maintained across all cores. Redpanda can also check partition assignments when partitions are added to or removed from a broker, and rebalance the remaining partitions between cores.
Topic-aware intra-broker partition balancing allows for dynamically reassigning partitions within a broker. Redpanda prioritizes an even distribution of a topic's partition replicas across all cores in a broker. If a broker's core count changes, when the broker starts back up, Redpanda can check partition assignments across the broker's cores and reassign partitions, so that a balanced assignment is maintained across all cores. Redpanda can also check partition assignments when partitions are added to or removed from a broker, and rebalance the remaining partitions between cores.

To determine when to use intra-broker partition balancing, use the public metrics for CPU usage described in the xref:manage:monitoring.adoc#cpu-usage[Monitoring] guide, as well as the xref:reference:internal-metrics-reference.adoc#vectorized_reactor_utilization[`vectorized_reactor_utilization`] internal metric.

Expand Down
4 changes: 0 additions & 4 deletions modules/manage/pages/kubernetes/k-manage-resources.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -24,10 +24,6 @@ kubectl describe nodes

- <<memory, Enable memory locking>>. This configuration prevents the operating system from paging out Redpanda's memory to disk, which can significantly impact performance.

== Limitations

Redpanda does not support decreasing the CPU cores for brokers in an existing cluster.

[[memory]]
== Configure memory resources

Expand Down

0 comments on commit ac16003

Please sign in to comment.