What is an index?
An index is more than just a container. It bundles together:- A mapping that defines the schema of the documents it holds.
- Settings that control how data is stored, replicated, and analyzed.
- A collection of shards that physically store the data on disk.
Primary and replica shards
Elasticsearch splits each index into one or more shards. There are two kinds:Primary shards
Hold the original copy of the data. Every write operation targets a primary shard first. The number of primary shards is fixed at index creation time and cannot be changed afterward without reindexing.
Replica shards
Hold identical copies of a primary shard. Replicas provide redundancy against node failure and increase read throughput because search requests can be served by either the primary or any of its replicas. The replica count can be changed at any time.
How sharding works
When you create an index withN primary shards and M replicas, Elasticsearch stores N × (M + 1) shard copies across the cluster nodes. Each document is assigned to exactly one primary shard using a deterministic hash of its _id:
Replica shards are never allocated on the same node as their corresponding primary shard. A cluster with only one node will show a yellow health status when replicas are configured, because there is nowhere to place them.
Creating an index with shard settings
Use thePUT /<index> API to create an index and specify shard counts in the settings block:
Shard sizing best practices
Target 10–50 GB per shard
Target 10–50 GB per shard
Shards below 1 GB impose unnecessary overhead on the cluster. Shards above 50 GB become slow to recover after a node restart and difficult to rebalance. Aim for a shard size in the 10–50 GB range for most workloads. Use
GET /_cat/shards?v&s=store:desc to check current shard sizes.Avoid over-sharding
Avoid over-sharding
Each shard is a Lucene index that consumes file handles, heap memory, and CPU. A common mistake is creating too many small primary shards. As a rule of thumb, keep the number of shards per GB of heap to around 20 or fewer. For a node with 30 GB of heap, that means no more than ~600 shards total.
Use one primary shard for small indices
Use one primary shard for small indices
For indices that will hold only a small amount of data — for example, a configuration index or a low-volume log stream — a single primary shard is often the right choice. It eliminates cross-shard scatter-gather overhead and simplifies operations.
Scale writes with more primary shards
Scale writes with more primary shards
If a single primary shard becomes an indexing bottleneck, increasing the primary shard count (on a new index) distributes write load. Pair this with the Rollover API for time-series data to automate index rotation.
Checking shard allocation
You can inspect how shards are distributed across nodes at any time:p indicates a primary shard and r a replica. Each primary and its replica are on different nodes.