This page describes how Managed Service for Apache Kafka provisions brokers
for your Kafka cluster.

## Overview

A Kafka cluster is made up of one or more *brokers* . Producer applications
send messages to brokers, and consumer applications connect to brokers to
read messages. Multiple consumers can read messages in parallel, typically using
a [consumer group](https://docs.cloud.google.com/managed-service-for-apache-kafka/docs/consumer-groups.md.txt).
Each partition in a topic is assigned a leader broker. The leader is responsible
for replicating the partition data that it receives to other brokers.

Managed Service for Apache Kafka automatically creates brokers for you.
Although you can't directly configure the number of brokers, you can scale
up your cluster to add brokers.

## Broker provisioning

When you create a Managed Service for Apache Kafka cluster, the service
provisions at least one broker in each of three zones. The number of brokers is
calculated as follows:

`number of brokers = max(3, ceiling(vCPUs / 15))`

For example, a cluster with 75 vCPUs starts with 5 brokers.

Brokers are distributed as evenly across zones as possible, and all brokers
have the same number of vCPUs, which might be a fractional value. Each broker
has a minimum of one vCPU.

The service automatically provisions disk capacity for the brokers. Optionally,
you can configure the disk capacity per broker when you create the cluster and
increase it later. For more information, see
[Configure broker disk size](https://docs.cloud.google.com/managed-service-for-apache-kafka/docs/brokers/configure-disk-size.md.txt).

## Scaling the cluster

As you increase the size of the cluster, additional vCPUs are distributed among
the existing brokers, up to a limit of 15 vCPUs per broker. If cluster size
exceeds 15 vCPUs per broker, the service provisions a new broker. The brokers
are equally distributed among the three zones, with a maximum difference of one.

If you decrease the cluster size, vCPUs are distributed across the existing
brokers, down to a minimum of one vCPU per broker. Brokers are never deleted
during the lifetime of a cluster. Once a new broker is provisioned, it can be
scaled down to one vCPU, but is never deleted.

## Rebalancing broker partitions

Often, the reason for scaling up a cluster is to reduce load on existing
brokers, by reassigning some partitions to the new broker. This process is
called *rebalancing*. Rebalancing can be performed automatically or manually.

- **Automatic rebalancing** . Automatic rebalancing is a cluster-wide setting.
  When a new broker is provisioned, if automatic rebalancing is enabled, the
  service automatically rebalances the partitions from existing brokers. The
  rebalancing algorithm is based on the partition count, and aims to keep
  approximately the same number of partitions per broker. It doesn't take into
  account the actual traffic served by each partition. For more information, see
  [Rebalancing configuration](https://docs.cloud.google.com/managed-service-for-apache-kafka/docs/create-cluster.md.txt#rebalancing-config).

- **Manual rebalancing** . You can manually rebalance partitions by changing the
  partition assignment. You might do this because automatic rebalancing is
  disabled, or because a partition receives significantly more traffic than
  other partitions (called a *hot partition* ). For more information, see
  [Change the partition assignment](https://docs.cloud.google.com/managed-service-for-apache-kafka/docs/brokers/assign-partitions.md.txt)

## What's next

- [View the brokers in a cluster](https://docs.cloud.google.com/managed-service-for-apache-kafka/docs/brokers/view-brokers.md.txt)
- [Configure broker disk size](https://docs.cloud.google.com/managed-service-for-apache-kafka/docs/brokers/configure-disk-size.md.txt)
- [Change a partition assignment](https://docs.cloud.google.com/managed-service-for-apache-kafka/docs/brokers/assign-partitions.md.txt)
- [Use quotas for load isolation](https://docs.cloud.google.com/managed-service-for-apache-kafka/docs/brokers/kafka-quotas.md.txt)

*Apache Kafka® is a registered
trademark of The Apache Software Foundation or its affiliates in the United
States and/or other countries.*