This article was completely rewritten in October 2026 and updated with the latest changes to the service.
When it comes to storing billions of rows of data with single-digit-millisecond latency, Bigtable is one of the first options that comes to data engineers’ minds. Bigtable started as an internal Google storage system on which services such as Search, Google Earth and Google Analytics were built; today the same technology is available to everyone as Cloud Bigtable, a fully managed service on Google Cloud.
In this guide we look at what exactly Bigtable is, how its architecture works, what impact it has had on open-source NoSQL databases, when you should (and shouldn’t) use it, and which new capabilities have been added over the last two years.
Bigtable is a NoSQL database of the wide-column and key-value kind. Google’s original paper defines it as “a sparse, distributed, persistent multi-dimensional sorted map”. Every value in Bigtable is addressed by three components — a row key, a column key and a timestamp — and the value itself is just an array of bytes whose interpretation is left to the application.
Bigtable is not relational: tables have no fixed schema, foreign keys or JOINs in the way traditional SQL databases do. Instead, it is designed for very large volumes: billions of rows, thousands of columns and terabytes to petabytes of data, with high read and write throughput and low latency.
Understanding the data model is the most important step in working with Bigtable, because good key design directly affects performance.
The classic example from the Bigtable paper is storing a copy of the web: the row key is the reversed URL of the page (for example com.cnn.www), one column holds the page contents, and several versions with different timestamps show what the page looked like at different crawl times.
In Cloud Bigtable, all client requests first reach a frontend server and are then routed to a node (called a tablet server in the original paper). Nodes are organized into a cluster, and a cluster belongs to an instance.
The key point of the architecture is that data is never stored on the nodes themselves. Tablets are kept in the SSTable format on Colossus (the successor of the Google File System), and each node merely points to a set of tablets. An SSTable is a persistent, ordered, immutable map from keys to values. Writes are recorded in Colossus’s shared log as soon as they are acknowledged. As a result:
Bigtable uses a pattern that later became known as the LSM tree (log-structured merge): writes are first recorded in a log and in memory (the memtable), then turned into immutable SSTable files that are merged over time (compaction). The same idea was later used in LevelDB, RocksDB, HBase and Cassandra. In the original design, the Chubby distributed lock service was used to manage the master, discover tablet servers and store schema information.
A management process in each zone automatically splits busy or large tablets, merges small ones and moves them between nodes. Even so, row key design remains critical: keys that increase monotonically over time (such as a timestamp at the start of the key) send all writes to a single tablet and create a hotspot. The official recommendation is to spread writes as evenly as possible while keeping related rows next to each other.
The Bigtable paper, together with the GFS and MapReduce papers, was one of the foundations of the Big Data movement:
Interestingly, the road now runs the other way: in recent years Bigtable has offered tools for migrating Cassandra applications directly to it (details in the latest-changes section).
Cloud Bigtable is the same engine Google uses internally, with the infrastructure managed by Google Cloud. Its most important features, according to the official documentation:
According to the editions comparison table, each SSD node supports up to roughly 17,000 reads and 14,000 writes per second with latency below 5 ms.
Bigtable’s cost has three main components:
Data Boost and backups are billed separately, and one- and three-year committed use discounts (CUDs) are available. Prices vary by region; for exact figures see the official pricing page or the Pricing Calculator.
Over the last two years Bigtable has evolved from a pure key-value database into a platform with SQL, continuous views and AI capabilities. The items below are taken from the official release notes and the Google Cloud blog; dates are those announced in the source.
EXPORT DATA statement.UNPACK for reading time-series data in tabular form; continuous materialized views and logical views were introduced in preview.With the cbt command-line tool, after a simple setup you can create a table, add a column family and write and read data in just a few commands:
# point cbt at your project and instance (once)
echo project = my-project >> ~/.cbtrc
echo instance = my-instance >> ~/.cbtrc
# create a table and a column family
cbt createtable my-table
cbt createfamily my-table cf1
# write two versions of the same cell, then read them back
cbt set my-table r1 cf1:c1=test-value1
cbt set my-table r1 cf1:c1=test-value2
cbt read my-tableAnd with GoogleSQL you can query the same data. In SQL, the special _key column is the row key, and each column family appears as a MAP. The example below returns the state of the sensors at a specific moment in the past:
SELECT _key AS location,
alarmType['carbonMonoxide'] AS co_sensor
FROM alarms(as_of => TIMESTAMP('2023-04-01T09:00:00.000Z'))
WHERE _key LIKE 'building1%'
AND alarmType['smoke'] = 'on';| Feature | Bigtable | BigQuery | Spanner | Firestore |
|---|---|---|---|---|
| Type | Wide-column and key-value NoSQL | Analytical data warehouse | Globally distributed relational | Document-oriented NoSQL |
| Main workload | High-throughput operational, time series, real-time analytics | Analytics (OLAP) on huge data volumes | Transactional (OLTP) with strong consistency | Web and mobile apps |
| Query language | Key-value API, GoogleSQL, HBase and CQL | Full GoogleSQL | GoogleSQL and PostgreSQL | SDKs and document queries |
| Transactions | Atomic at the single-row level | Suited to analytical work, not OLTP | Full, distributed ACID | ACID transactions across multiple documents |
| Latency | A few milliseconds; sub-millisecond with the in-memory tier | Seconds to minutes | Milliseconds | Milliseconds |
| Scalability | Add nodes, up to petabytes | Serverless | Horizontal and global | Serverless |
| Best for | IoT, ad tech, finance, personalization | Reporting, BI and machine learning | Global financial, inventory and booking systems | Small to medium apps with real-time sync |
Bigtable has grown from an internal Google system — whose paper inspired HBase and Cassandra — into one of the most complete managed NoSQL databases. Its backbone is still the same simple yet powerful set of ideas: sorted row keys, column families, timestamp-based versioning, distributed tablets and the separation of compute from storage on Colossus.
But today’s Bigtable is no longer just a key-value store: GoogleSQL, continuous materialized views, Data Boost, Cassandra compatibility, tiered storage and the in-memory tier, vector search and tools built for AI agents make it ready for a new generation of real-time and AI-driven applications. If you work with large volumes of key-based data, Bigtable remains one of the most serious options — provided you take row key design seriously.
A complete, practical guide to managing secrets with HashiCorp Vault: history, the move to BSL, OpenBao and IBM’s acquisition of HashiCorp, seal/unseal and Raft, tokens and policies, AppRole and Kubernetes, dynamic secrets, PKI and Transit, a Docker Compose setup with TLS, Spring Boot integration and a production checklist.
A complete, plain-language guide to proxy servers: forward, reverse and transparent proxies, HTTP and SOCKS, load-balancing algorithms, and a full Nginx tutorial covering installation, configuration structure, reverse proxying, SSL with Let's Encrypt, WebSocket, caching, rate limiting, security headers and troubleshooting 502 and 504 errors.
A complete, practical guide to WSO2: the company and its products, the architecture of API Manager 4.7, installation with Docker and Docker Compose, configuring deployment.toml, SSL with Let's Encrypt and Nginx, production deployment tips and a comparison with competitors.
Comments No comments yet
To leave a comment, open this page in a web browser.