Keyboard shortcuts

Press ← or → to navigate between chapters

Press S or / to search in the book

Press ? to show this help

Press Esc to hide this help

Background Network Services

Crate: kinetic-daemon Stage: 9 Reading time: 15 minutes Depends on: kinetic-network, kinetic-core

What Is This?

This topic covers the background networking workers that run persistently as long as the Kinetic daemon is alive. Specifically, it documents two critical autonomous loops inside kinetic-daemon/src/services/network.rs. The first is the Proof of Work (PoW) Sybil resistance miner. The second is the Distributed Hash Table (DHT) name republisher.

Inside the Kinetic network, your node cannot simply connect to peers and sit idle. The network imposes strict, time-based rules to prevent spam (Sybil attacks). It also has rules to keep the global routing table clean of dead or obsolete data. These background services act as the life support systems of your node. They run in the background, out of sight of the user. They ensure that your node remains a valid, legally participating citizen of the Kinetic network. They are automatically spawned when the daemon starts and run until the daemon process is killed.

Without these background workers, your node would inevitably degrade in the network’s eyes. First, it would get blacklisted by peers as its cryptographic identity expires. Second, the network would slowly forget your claimed names as your DHT records rot away. This file explains how the daemon prevents both scenarios automatically.

Why Kinetic Needs This

The Kinetic daemon must solve two fundamental problems related to time and decay in decentralized networks:

1. Identity Expiration and Sybil Resistance

The Kinetic network uses Proof of Work (PoW) to prevent malicious actors from spinning up thousands of fake nodes. If a node could mine a valid PoW once and use it forever, an attacker could slowly build up a massive botnet over months. To prevent this, Kinetic ties the validity of a node’s PoW to the current “epoch”. This epoch shifts regularly based on Drand randomness.

When an epoch changes, your node’s PeerId and its associated PoW are no longer valid. If you do nothing, other nodes will reject your messages. They will drop active connections with you because your identity is cryptographically stale. Therefore, the daemon needs a mechanism to detect when its identity is expiring. It must mine a new identity before it gets disconnected. Crucially, it must transition to this new identity without interrupting the user’s experience or dropping pending requests. This transition must happen behind the scenes, abstracting the complexity away from the user.

2. DHT Record Rot

The Distributed Hash Table (DHT) is not a permanent, reliable database. It is a volatile memory space shared across thousands of transient nodes. Nodes go offline, drop records to save space, or simply forget things after a timeout. When you register a name in Kinetic, you publish a Commitment and a Reveal to this DHT.

If you just publish these records once and walk away, the network will eventually forget you own that name. The daemon must act as a heartbeat for your data. It must constantly wake up to remind the network, “I am still here, and I still own this name.” Without this constant refreshing, names would silently expire and become claimable by others.

How It Works

The PoW Miner Loop: Seamless Hot-Swapping

The start_pow_miner_loop function is responsible for keeping the node’s identity valid.

1. Listening to Drand: The loop is powered by a tokio::sync::watch::Receiver. This channel receives the latest Drand “kyn” (round number) as soon as it is fetched by the Drand service. Because Drand kyns are the heartbeat clock of the Kinetic network, every new kyn triggers an evaluation. For every new kyn, the loop evaluates the current state of the world to see if action is needed. -> See: kinetic-daemon/src/services/network.rs — Lines 28 to 36

2. Evaluating the Epoch: It calculates the “staggered epoch” for the current PeerId. The staggered epoch ensures that not all nodes expire at the exact same time. If everyone expired simultaneously, it would cause massive network-wide disconnect events. After calculating the epoch, it checks if the PoW for this identity is still valid. -> See: kinetic-daemon/src/services/network.rs — Lines 39 to 57

3. Offloaded Mining: If the epoch has shifted and the PoW is no longer valid, the daemon must mine a new one. Because mining is a CPU-intensive hashing operation, it cannot run directly in the async loop. If it did, it would freeze the entire tokio runtime, stopping all other networking and the REST API. To solve this, the daemon uses tokio::task::spawn_blocking. This sends the mining job to a dedicated OS thread pool specifically designed for heavy computation. -> See: kinetic-daemon/src/services/network.rs — Lines 61 to 68

4. The Hot-Swap Maneuver: Once the new keypair is mined, the daemon performs a dangerous but necessary maneuver: the hot-swap.

  • First, it forcefully aborts the currently running NetworkEventLoop (the active libp2p backend).
  • Second, it loops and creates a new NetworkEventLoop.
  • It uses the newly mined keypair for this new loop.
  • It injects the exact same shared states: storage, configuration, and drand receivers.
  • It contains a retry loop with exponential backoff (up to 10 times) in case the OS hasn’t released the network port yet.
  • Third, it calls hc_client.update_backend(...).
  • This tells the frontend NetworkClient (which the REST API is actively holding) to point its internal channels to the new backend.
  • The rest of the daemon has no idea the underlying identity just changed.
  • The frontend steering wheel stayed the same, but the engine was swapped while driving. -> See: kinetic-daemon/src/services/network.rs — Lines 71 to 108

The DHT Republisher: Preserving Ownership

The start_republisher function ensures your names don’t fade from the network’s memory.

1. The 12-Hour Heartbeat Timer: It sets up a tokio::time::interval timer. This timer wakes the task up every 12 hours. The 12-hour value is defined by the TIMEOUTS_HEARTBEAT_AGE_WARNING_SECONDS constant. -> See: kinetic-daemon/src/services/network.rs — Lines 124 to 126

2. Reading from Local Storage: When the timer ticks, it reads the list of all owned names from the local storage database. It uses the DB_PREFIX_OWNED_NAMES prefix to find this array. For each name found in the array, it fetches the corresponding Reveal payload from storage. -> See: kinetic-daemon/src/services/network.rs — Lines 129 to 139

3. Anti-Spam Staggering: The republisher spawns a new async Tokio task for each individual name it owns. To prevent flooding the local libp2p node and overwhelming the network with hundreds of concurrent DHT requests, it staggers these tasks. It multiplies the loop index i of the name by 100 milliseconds and sleeps for that duration before starting the republish sequence. This ensures a smooth, staggered broadcast instead of a massive sudden spike. -> See: kinetic-daemon/src/services/network.rs — Lines 144 to 148

4. Reconstructing the Commitment on the Fly: The daemon does not store the Commitment directly in the database. Because a Commitment is just a SHA-256 hash of the Reveal components, the daemon reconstructs it. It creates a new sha2::Sha256 hasher and updates it in a very specific order. It hashes the name as bytes, then the salt, then the decoded Drand signature, and finally the public key. This specific order must exactly match the original commitment logic used during name registration. This saves storage space and ensures the commitment always matches the reveal data. -> See: kinetic-daemon/src/services/network.rs — Lines 150 to 159

5. The Two-Step Publication and the Maturity Gate: First, it publishes the reconstructed Commitment to the DHT. -> See: kinetic-daemon/src/services/network.rs — Lines 167 to 169 Then, it hits a critical, hardcoded delay: it sleeps for exactly 36 seconds. -> See: kinetic-daemon/src/services/network.rs — Lines 172 to 173 Why 36 seconds? Because Kinetic has a “maturity gate.” A reveal is only considered valid by the network if its corresponding commitment was published at least 10 Drand rounds prior. 10 Drand rounds is exactly 30 seconds. The 36-second sleep provides a 6-second safety buffer. If the daemon published both instantly, new nodes on the network would reject the reveal. They would reject it because they haven’t seen the commitment age properly. After the 36-second wait, it safely publishes the actual Reveal to the DHT. -> See: kinetic-daemon/src/services/network.rs — Lines 175 to 185

Analogy for Republishing: Think of the DHT as a massive bulletin board in a windy town square. People routinely tear down old posters to make room for new ones. If you claim a name, you pin your poster to the board. Because the town square forgets over time, you have to hire a worker (this background task). The worker comes back every 12 hours. They pin a fresh copy of your initial poster (the commitment). They wait exactly 36 seconds for the glue to dry and for people to notice it (the maturity gate). Finally, they pin the final document (the reveal) right next to it.

Key Pieces

  • start_pow_miner_loop

    • Where it lives: kinetic-daemon/src/services/network.rs — Line 8
    • What it does: The autonomous worker that monitors Drand rounds. It detects epoch expiration, mines new libp2p identities, and hot-swaps them into the running application.
    • Why it matters: It keeps the node alive on the network without requiring restarts from the user. It prevents Sybil decay.

  • tokio::task::spawn_blocking

    • Where it lives: kinetic-daemon/src/services/network.rs — Line 49 and Line 61
    • What it does: Sends CPU-heavy functions like is_valid_sybil_pow and mine_sybil_keypair to a separate operating system thread.
    • Why it matters: It prevents the async event loop from freezing. If it ran on the main thread, all incoming network requests would timeout while the CPU churns through hashes.

  • hc_client.update_backend(...)

    • Where it lives: kinetic-daemon/src/services/network.rs — Line 104
    • What it does: The critical mechanism that connects the old frontend client handles to the newly spawned libp2p backend loop.
    • Why it matters: It allows the REST API and other external services to maintain their reference to the NetworkClient even after the underlying P2P node has been destroyed and recreated.

  • start_republisher

    • Where it lives: kinetic-daemon/src/services/network.rs — Line 119
    • What it does: Periodically iterates over owned names in local storage and re-broadcasts them to the DHT in two steps.
    • Why it matters: It prevents name ownership from silently expiring in the volatile DHT memory space. It keeps the node’s claimed assets alive.

How This Connects to the Rest of Kinetic

  • CROSS-CRATE: The concept of staggered epochs, Drand kyns, and PoW Sybil resistance is fundamentally defined and implemented in kinetic-network (Stage 8).
  • CROSS-CRATE: The Commitment and Reveal flow, including the hashing logic and the 10-round maturity requirement, is part of the naming system defined in kinetic-core (Stage 7).
  • This file acts as the operational glue. It takes the theoretical rules defined in core and network and actually executes them persistently in a live daemon.
  • It leverages the StorageEngine trait from kinetic-core to read the locally saved names that need republishing.

Quick Reference

  • Drand Trigger: The miner loop evaluates identity validity on every new Drand round received over the watch channel.
  • Validation: Uses spawn_blocking to check is_valid_sybil_pow before deciding to mine.
  • Mining Strategy: Offloaded to a blocking thread pool to preserve async responsiveness using mine_sybil_keypair.
  • Backend Hot-Swap: When an epoch expires, the old libp2p loop is aborted via its JoinHandle.
  • Port Retries: The hot-swap includes up to 10 retries with exponential backoff if the port is in use.
  • Republish Interval: Runs unconditionally every 12 hours.
  • Staggering: Names are delayed by 100ms per index before being republished.
  • On-the-fly Hashing: Commitments are rebuilt from Reveals rather than stored raw.
  • Maturity Wait: Sleeps 36 seconds (12 Drand rounds) between publishing a commitment and publishing the corresponding reveal to satisfy network maturity rules.

Open Questions / Things to Revisit

  • Hot-Swap Port Contention: The hot-swap logic uses an exponential backoff retry loop (up to 10 retries) if the network port is still in use when spinning up the new backend. If this retry loop takes several seconds to complete, could incoming proxy requests from the REST API be dropped during the transition window?
  • Reconstructed Commitments Consistency: The republisher reconstructs the commitment from the Reveal struct (name, salt, signature, pubkey). If a future update changes how commitments are hashed, the daemon will need to ensure this background loop mirrors the original registration hashing logic, or else the republisher will accidentally publish invalid commitments.
  • Memory Growth in Republisher: The republisher spawns a new async task for every single name it owns, staggered by 100ms. If a node owns thousands of names, this could spawn thousands of overlapping sleep-and-publish tasks every 12 hours. Is there a risk of hitting resource limits here?
  • Missing Error Handling: If hc_client.update_backend fails or the network loop crashes immediately after being spawned, the network_loop_handle might be pointing to a dead task, leaving the daemon disconnected from the network without recovering.