01 — Overview and Native Storage

Crate: kinetic-storage Stage: 4 of 10 Reading time: ~8 minutes Depends on: kinetic-core Source files: kinetic-storage/src/lib.rs (Lines 1 to 172)

What Is This?

kinetic-storage is the persistent Key-Value database engine for the Kinetic network. It provides a unified, cross-platform storage interface that full nodes use to save the blockchain state, DIDs, and network routing tables to disk.

Why Kinetic Needs This

A decentralized network generates massive amounts of data that must be persisted across reboots. However, Kinetic is designed to run anywhere — from high-powered Linux servers to lightweight browser extensions.

Standard databases like PostgreSQL or SQLite are too heavy or require complex setups. Kinetic needs an embedded database. Furthermore, browser extensions cannot write to a real filesystem.

This crate solves both problems by providing a single storage API that automatically swaps its underlying engine based on where it is compiled.

How It Works

The crate consists of a single file (lib.rs) with 308 lines. It implements a trait called StorageEngine (defined in kinetic-core). This trait guarantees four fundamental operations:

1. put(key, value)
2. get(key)
3. delete(key)
4. scan_prefix(prefix)

The Native Engine (sled)

#![allow(unused)]
fn main() {
-> See: `kinetic-storage/src/lib.rs` — Lines 18 to 172
}

When you compile Kinetic for Linux, Mac, or Windows, the compiler activates the native module. This module wraps Sled, which is a pure-Rust, high-performance embedded database.

Opening the Database:

Warning

When SledStorage::new(path) is called, it attempts to acquire a lock on the directory. If two Kinetic nodes try to open the same folder, Sled will throw an error. The code catches WouldBlock or PermissionDenied errors and gracefully translates them to StorageError::DatabaseLocked.

Corruption Handling (The Safety Net):

#![allow(unused)]
fn main() {
-> See: `kinetic-storage/src/lib.rs` — Lines 55 to 91
}

Important

If a node loses power while writing, the database can corrupt. Sled detects this upon startup. Instead of just crashing and destroying the user’s data forever, the code intercepts the Corruption error, automatically renames the broken database folder to a backup file (e.g., .corrupt.1623910.bak), and alerts the operator.

Scanning Prefixes:

#![allow(unused)]
fn main() {
-> See: `kinetic-storage/src/lib.rs` — Lines 110 to 133
}

In a Key-Value store, you don’t have SQL SELECT * WHERE. Instead, you design your keys cleverly (e.g., did:kin:123, did:kin:456). The scan_prefix method takes a prefix (did:kin:) and an optional limit. It asks Sled to return an iterator starting exactly at that prefix, pulling records out extremely efficiently.

Key Pieces

SledStorage

• What it does: The struct holding the raw sled::Db connection.
• Location: lib.rs:24
• Why it matters: It manages the lifetime of the database connection. As long as this struct exists in memory, the database is open and locked to that process.

CORRUPT_COUNTER

• Location: lib.rs:56

Note

-> See RUST_CONCEPTS.md for an explanation of AtomicU64. It safely ensures multiple concurrent threads don’t overwrite the same corrupted backup file.

How This Connects to the Rest of Kinetic

• FORWARD DEPENDENCY: The kinetic-daemon crate will instantiate this storage engine when the node boots up.
• CROSS-CRATE: It uses StorageError and StorageEngine which are defined in kinetic-core. (Note: kinetic-core is technically Stage 7, so for now, treat those trait definitions as a black box.)

Quick Reference

Open Questions / Things to Revisit

• Async Trait: The current StorageEngine trait uses synchronous (blocking) calls. In a highly concurrent P2P network using tokio, blocking file I/O operations can stall the async executor. We may need to wrap these Sled calls in tokio::task::spawn_blocking at the daemon layer, or upgrade the trait to be async fn.