Why Is One Storage Key Doing All the Work?

Botie groups every maintenance record under its tenant. That makes a tenant’s records easy to locate, but north is a much busier customer than the others. A key can be convenient to read and still gather more work than one part of the system can handle.

Before adding machines, let’s find out what is being grouped together. We will compare logical keys using a measured request trace, then ask what a different grouping makes harder to read.

Connect the request trace and key fields before comparing the accumulated traffic with the teaching capacity.

[
  {
    "key": [
      "north"
    ],
    "load": 10,
    "overloaded": true
  }
]

The technician opens one asset’s recent readings. That request knows the tenant and asset identity. A key containing both can locate the asset’s records without asking every partition to search. A report across every asset has different needs and may require another view.

Write down those access patterns before selecting fields. A partition key is part of the query design because it decides what can be located together. Keep tenant identity in the authenticated request as well; a key containing a tenant name does not enforce authorisation.

A convenient lookup and an acceptable traffic distribution both matter.

A logical key is the value the application supplies. The database maps key ranges or hashes onto physical storage partitions, often moving them as the service grows. Several logical keys can share physical storage.

Our workshop groups traffic by logical key. It does not reproduce a vendor’s partition placement, adaptive capacity or hot-key mitigation. If one logical key dominates the trace, that is evidence to investigate. It is not a measurement of a physical server’s utilisation.

A random-looking key can distribute many independent items while leaving one popular item hot.

The Python trace is a list of dictionaries. Each dictionary contains the candidate key fields and load, an invented nonnegative work count measured over the same interval. A tuple gives us an unambiguous compound key. Concatenating text without boundaries can accidentally give different field combinations the same address.

Accumulate load for each tuple, then compare it with the supplied capacity. Preserve the groups in first-seen order so we can relate the output to the trace.

The trace measures work, not merely the number of stored records.

rows = [{"tenant":"north","asset":"P7"},
        {"tenant":"north","asset":"P8"},
        {"tenant":"north","asset":"P7"}]
keys = [(r["tenant"], r["asset"]) for r in rows]
print(keys)
print(len(set(keys)))

[('north', 'P7'), ('north', 'P8'), ('north', 'P7')]
2

The changed workshop trace sends most work to P7. Adding asset to the key no longer resolves the concentration. Time buckets can bound a growing history; a write shard can spread independent events from the same asset.

These choices move work into the reader. A day-spanning query must visit several buckets, and a sharded history may require parallel queries followed by a merge. A latest-value item that every request updates cannot be spread by renaming the same item several ways without defining how its values will be reconciled.

Measure both the writes you distribute and the extra reads your design creates.

Two request groups with equal counts may consume different work because their items have different sizes or their queries return different amounts of data. Measure the relevant service units and latency under representative load. Review secondary indexes too; they have their own keys and can concentrate writes differently.

A tenant can become busy after a successful rollout. Watch per-key concentration and throttling rather than relying only on an average across the table. Retain the access-pattern assumptions beside the design so that a new workload prompts a review.

The workshop limit is intentionally simple. Production capacity comes from the chosen database and measured operation sizes.

Changing a key can change every address the readers know. A migration may need new records, backfill, compatibility reads and a controlled cutover. Decide how concurrent updates reach the new representation while the copy runs.

Keep a way to reconcile counts and identities before retiring the old view. If the old and new stores are independently writable, record which one owns the truth during each phase. Otherwise a traffic optimisation can introduce conflicting copies of the maintenance history.

The next lesson makes a bounded history bucket concrete and shows how its ordering supports the actual reading request.