Keyboard shortcuts

Press ← or → to navigate between chapters

Press S or / to search in the book

Press ? to show this help

Press Esc to hide this help

VDF API Processing — Part 2: Renewals & Task Management

Crate: kinetic-daemon Stage: 9 Reading time: 20 minutes Depends on: 04_api_vdf_1.md (VDF Generation), docs/learn/verify/01_overview.md, docs/learn/core/01_overview.md

What Is This?

This file completes our deep dive into the VDF API layer of the Kinetic Daemon by documenting the second half of kinetic-daemon/src/api/vdf.rs. While the first half focused exclusively on generating new identities and names from scratch, this half focuses on the lifecycle of existing names. Specifically, it covers the logic required to renew a Kinetic name, which involves reading old cryptographic proofs from the local storage database and linking them to new, freshly computed proofs.

Additionally, because VDF operations (both generation and renewal) are intentionally slow, blocking operations that utilize the CPU, they cannot be processed synchronously within a standard HTTP request-response cycle. If the daemon tried to compute a 10-minute VDF while the HTTP client waited, the network connection would time out and the HTTP worker thread would be stalled. Instead, the daemon uses an asynchronous background task architecture. The endpoints documented in this file include the mechanisms the daemon uses to expose the status of those background tasks to the outside world, allowing user interfaces to poll for progress, handle errors gracefully, and clean up memory when tasks are complete.

Why Kinetic Needs This

The Kinetic network is designed around the principle of continuous proof of work. In traditional centralized domain name systems (like DNS), you maintain ownership of a name by paying a central registrar a yearly fiat fee. In Kinetic, there is no central registrar, and there are no ongoing fiat or token fees for standard names. Instead, you pay with computational time. To maintain ownership of a standard name, you must periodically prove that you are still willing to dedicate CPU resources to it by generating a new Verifiable Delay Function (VDF) proof. If you fail to do this before your previous proof expires, the network considers the name abandoned, and anyone else can claim it.

The VDF utilized here (the Chia VDF Engine) guarantees that time has passed because it is sequential; it cannot be parallelized across multiple cores or GPUs. This levels the playing field so that massive data centers do not have an unfair advantage over standard consumer hardware when claiming and renewing names.

However, a critical architectural decision was made regarding how renewals work. If a user has owned a name for five years, they have built up a massive chain of VDF proofs over time. If they were forced to generate an new proof from scratch every time they renewed, the computational burden would be staggering and punishing to long-term participants. To solve this, Kinetic implements a renewal discount. When you renew a name, you reference your previous proof. Because the network can verify the link between the old proof and the new one, it grants an 80% discount on the required VDF iterations. You only have to do 20% of the work that a brand new registration would require.

To make this discount a reality, the daemon needs a specialized API endpoint (handle_vdf_renew). This endpoint must act as an orchestrator between the local storage database (where the old proof lives), the network (to fetch new randomness), the local CPU (to calculate the new 20% proof), and the DHT (to broadcast the linked proofs).

Furthermore, because these operations are dispatched to background threads, the daemon needs a robust way to communicate across thread boundaries. If a background thread encounters a network error while fetching Drand, the HTTP client needs to know immediately. If the thread successfully finishes the 20% computation, the client needs the final transaction hash. The task management endpoints (handle_vdf_status and handle_vdf_status_delete) provide the necessary visibility into the black box of Tokio background tasks without exposing the underlying memory structures.

How It Works

The back half of vdf.rs revolves around the handle_vdf_renew function and its supporting task management infrastructure. Let us break down the exact sequence of events when a renewal request hits the daemon.

Phase 1: Request Validation and Queue Management

When an HTTP POST request arrives at the renewal endpoint, it carries a JSON payload matching the NameRenewRequest struct. The daemon immediately performs several synchronous checks before dedicating any background resources to the request.

Authorization via Axum Extensions

-> See: kinetic-daemon/src/api/vdf.rs — Lines 378 to 397

The handler function signature makes heavy use of Axum extractors:

#![allow(unused)]
fn main() {
pub async fn handle_vdf_renew(
    Extension(role): Extension<Role>,
    State(state): State<ApiState>,
    Json(req): Json<NameRenewRequest>,
)
}

The Extension extractor pulls data that was inserted earlier in the request lifecycle by an authentication middleware. The daemon uses this to retrieve the Role of the API caller. If the caller does not possess the can_vdf privilege (usually reserved exclusively for the Node Admin), the request is instantly rejected with a 403 FORBIDDEN status code.

If authorized, the daemon extracts the target name from the JSON payload and passes it through kinetic_core::types::normalize_name. This ensures the name is converted to lowercase and stripped of any invalid or hidden characters. The daemon then verifies that the name is a valid apex name (e.g., alice, not alice.bob). If the format is invalid, a 400 BAD REQUEST is returned.

Iteration Bounds and Task Pruning

-> See: kinetic-daemon/src/api/vdf.rs — Lines 398 to 415

The user can optionally request a specific number of iterations (useful if they want to over-collateralize their name against faster CPUs in the future). To prevent denial-of-service attacks where a user requests an astronomically large number of iterations (which would tie up the node’s CPU indefinitely), the daemon enforces a hard limit of MAX_USER_ITERATIONS, set to 10,000,000.

Next, the daemon accesses the shared vdf_tasks state map. This map holds the status of all currently running or recently completed tasks. Before adding a new task, the daemon performs maintenance using the retain method:

#![allow(unused)]
fn main() {
tasks.retain(|_, t| t.progress < 100 && t.error.is_none());
}

This line iterates over the map and drops any task that has reached 100% progress or encountered an error. If, after this cleanup, there are still 50 or more tasks actively running, the daemon rejects the new request with a 429 TOO MANY REQUESTS error. This is a critical self-preservation mechanism ensuring the node does not exhaust its memory or CPU threads.

Task Initialization

-> See: kinetic-daemon/src/api/vdf.rs — Lines 417 to 425

If there is space in the queue, the daemon generates a unique UUID (Version 4) to identify the task. It creates a new VdfTaskStatus object, initialized with 0% progress and a status message of “Starting Renewal…”. It inserts this into the map, drops the Mutex lock to free up the shared state for other API requests, and returns the UUID to the HTTP client. Everything from this point forward happens asynchronously.

Phase 2: The Asynchronous Renewal Pipeline

The daemon uses tokio::spawn to launch a green thread that handles the heavy lifting. This thread moves sequentially through several stages, updating the task status map along the way.

Step 1: Retrieving the Previous Proof from Sled

-> See: kinetic-daemon/src/api/vdf.rs — Lines 434 to 470

To qualify for the renewal discount, the daemon must present the exact cryptographic state of the name as it currently exists on the network. It attempts to load this state from the local Sled database via the storage_clone. It constructs the database key by concatenating kinetic_core::constants::DB_PREFIX_REVEAL and the normalized FQDN. Sled operates on raw bytes (IVec). If the raw bytes are found, the daemon uses serde_json::from_slice to deserialize them into a strongly-typed NameRecord.

A critical architectural distinction is made here based on the enum variant of the NameRecord:

  • If the record is a Standard name, the daemon extracts the previous reveal data and proceeds.
  • If the record is a Premium name, the daemon halts and logs an error to the task status. Premium names are purchased with KYN tokens and are permanently exempt from VDF resquaring. Attempting to run a VDF renewal on a premium name is a logical error, as the network consensus rules do not require it.

Step 2: Fetching Fresh Randomness

-> See: kinetic-daemon/src/api/vdf.rs — Lines 472 to 481

The daemon instantiates a DrandClient and fetches the latest beacon from the Drand network. This ensures that the new VDF computation is seeded with cryptographic unpredictability that did not exist when the previous proof was generated. This prevents users from pre-computing their renewals years in advance. If the fetch fails, the task aborts and updates the error state.

Step 3: Crafting the Private Commitment

-> See: kinetic-daemon/src/api/vdf.rs — Lines 483 to 521

The daemon generates a new 32-byte secure random salt using the getrandom crate. It then constructs a SHA-256 hash that binds together:

  1. The FQDN of the name being renewed
  2. The newly generated salt
  3. The cryptographic randomness from the latest Drand beacon
  4. The node’s ML-DSA public identity key

This hash becomes the Commitment. Crucially, the daemon does not broadcast this commitment to the network yet. In early versions of Kinetic, commitments were broadcast before the VDF started. This led to a vulnerability where an attacker could observe a commitment on the DHT and start computing a parallel VDF to steal the name. By generating the commitment privately and keeping it hidden until the VDF is complete, the daemon protects the user’s computational effort.

Step 4: The Discounted VDF Evaluation (Blocking Thread)

-> See: kinetic-daemon/src/api/vdf.rs — Lines 523 to 568

The daemon consults the ConsensusParams to determine the baseline number of iterations required for the name. It then applies the 80% renewal discount:

#![allow(unused)]
fn main() {
let discounted_iters = (required_iters as f64 * 0.2) as u64;
}

The daemon will execute whichever is larger: the discounted minimum, or the iterations requested by the user.

Because VDF evaluation is purely CPU-bound and can take several minutes, it cannot be run on standard Tokio async threads, which are meant for fast, non-blocking I/O. If a VDF was run on a standard async thread, it would block the executor from processing other network events or HTTP requests, stalling the entire node. Instead, the daemon requests a permit from the vdf_semaphore (to ensure the node isn’t running too many VDFs simultaneously). Once granted, it uses tokio::task::spawn_blocking to move the Chia VDF engine execution to a dedicated OS thread pool specifically designed for heavy, long-running computational workloads.

Step 5: Broadcast and Mandatory Maturation Delay

-> See: kinetic-daemon/src/api/vdf.rs — Lines 570 to 603

Once the blocking thread returns the completed VDF proof, the daemon finally broadcasts the Commitment to the DHT.

However, the network consensus rules state that a Reveal is only valid if its corresponding Commitment has been visible on the network for a minimum amount of time (CONSENSUS_MINIMUM_COMMIT_AGE_KYNS). Therefore, the daemon must intentionally pause its workflow. It calculates the required wait time in seconds (usually spanning a few Drand epochs plus a small buffer) and calls tokio::time::sleep. This suspends the background task entirely, yielding the thread back to the Tokio executor to do other work, until the commitment has aged sufficiently.

Step 6: Assembling and Publishing the Reveal

-> See: kinetic-daemon/src/api/vdf.rs — Lines 605 to 665

After waking up from the sleep, the daemon constructs the final Reveal struct. It creates a PreviousProof struct containing the salt, drand signature, iterations, vdf proof, and ML-DSA signature from the old reveal. It attaches this PreviousProof to the new Reveal. This is what allows the network nodes to verify the chain of custody and authorize the 80% discount. It also ensures that any payload data (like DNS records) attached to the old name is preserved in the new Reveal.

Next, it applies the ML-DSA post-quantum signature. Because these signatures are quite large and computationally intensive, they are applied at the very end of the process to the signable_bytes of the Reveal. Finally, it serializes the Reveal to JSON, broadcasts it to the DHT via publish_redundant_payload (which ensures the payload reaches multiple distinct nodes on the Kademlia network), and saves the updated record to the local Sled database. The task progress is set to 100%, and the background thread terminates successfully.

Phase 3: Task Status and Visibility

While the background thread is grinding through the steps above, the HTTP client needs a way to check in. The API provides endpoints specifically for this purpose.

Status Updates via handle_vdf_status

-> See: kinetic-daemon/src/api/vdf.rs — Lines 700 to 718

This endpoint uses the Path<String> Axum extractor to grab the UUID from the request URL. It locks the Arc<Mutex<HashMap>> holding the tasks, looks up the UUID, and returns the VdfTaskStatus object. The status object contains a progress integer (0-100), a status string explaining the current step (e.g., “Computing Renewal VDF… (this may take a while)”), and an optional error string.

Notice the use of .unwrap_or_else(|e| e.into_inner()) when locking the Mutex:

#![allow(unused)]
fn main() {
let tasks = state.vdf_tasks.lock().unwrap_or_else(|e| e.into_inner());
}

This handles Mutex Poisoning. If another thread panics while holding the lock, the Mutex becomes “poisoned”, meaning subsequent lock attempts will return an error to prevent the use of potentially corrupted state. However, in this API, the state is just a HashMap of task statuses. If a thread panics, the map might be incomplete, but it is not dangerous to read. into_inner() allows the daemon to safely bypass the poison error and read the map anyway, preventing a single panic from taking down the entire API layer.

Memory Management via handle_vdf_status_delete

-> See: kinetic-daemon/src/api/vdf.rs — Lines 720 to 734

Although the daemon automatically prunes completed tasks when the queue hits 50, well-behaved clients should clean up after themselves. Once a UI client sees that a task has reached 100% progress or hit a fatal error, it should issue a DELETE request to this endpoint. The endpoint locks the map, calls .remove(&task_id), freeing up memory and keeping the queue lean.

Key Pieces

handle_vdf_renew

  • What it does: The primary API handler for initiating the renewal of an existing Kinetic name.
  • Where it lives: kinetic-daemon/src/api/vdf.rs — Lines 378 to 671
  • Why it matters: This function implements the vital 80% renewal discount logic. Without it, maintaining long-term identities on the Kinetic network would be computationally prohibitive for average users.

update_task_status & update_task_error

  • What they do: Internal utility functions used by the background threads to safely mutate the shared task map.
  • Where they live: kinetic-daemon/src/api/vdf.rs — Lines 673 to 698
  • Why they matter: They encapsulate the lock acquisition logic, ensuring that updating a progress bar does not introduce data races or complex locking boilerplate directly inside the main asynchronous flow.

handle_vdf_status

  • What it does: An HTTP GET endpoint that returns the current state of a running task.
  • Where it lives: kinetic-daemon/src/api/vdf.rs — Lines 700 to 718
  • Why it matters: It bridges the gap between long-running asynchronous cryptography and instantaneous HTTP requests, allowing user interfaces to remain responsive while waiting for VDF proofs.

handle_vdf_status_delete

  • What it does: An HTTP DELETE endpoint to manually remove a task from the node’s memory.
  • Where it lives: kinetic-daemon/src/api/vdf.rs — Lines 720 to 734
  • Why it matters: It provides a mechanism for proactive memory management, preventing the vdf_tasks HashMap from slowly leaking memory over the lifetime of the daemon process.

How This Connects to the Rest of Kinetic

  • Storage Subsystem: The renewal process relies on kinetic_storage (wrapped in the ApiState) to locate the previous Reveal data using the DB_PREFIX_REVEAL constant.
  • Core Types: The task relies on the Reveal, NameRecord, and PreviousProof structures defined in the core crate. (CROSS-CRATE: NameRecord — defined and explained in docs/learn/types/06_name_records.md).
  • Consensus Math: The core logic for calculating the baseline iterations and enforcing the delay lives in kinetic_core::consensus_math and kinetic_core::constants.
  • Network Layer: The final output of the VDF process is handed off to kinetic-network via the publish_redundant_payload method, pushing the new identity state into the global DHT across multiple peers.

Quick Reference

  • Renewal Discount: Renewals only require 20% of the baseline iterations (an 80% discount).
  • Premium Exemption: Premium names (purchased with KYN tokens) do not require VDF resquaring and will trigger an error if submitted to this endpoint.
  • Task Queue Limit: The daemon will reject new requests with a 429 status code if there are 50 or more active tasks in memory.
  • Commitment Privacy: Commitments are generated privately and only broadcast after the VDF proof is computed, mitigating front-running attacks.
  • Mandatory Sleep: The daemon will intentionally suspend execution for CONSENSUS_MINIMUM_COMMIT_AGE_KYNS epochs to satisfy network maturity rules before publishing the Reveal.
  • Mutex Poisoning: Handled gracefully on status reads using unwrap_or_else(|e| e.into_inner()) to prevent cascading failures.
  • Background Tasks: Handled asynchronously using tokio::spawn and tokio::task::spawn_blocking.

Open Questions / Things to Revisit

  • Hardcoded Discount: The 80% discount is currently hardcoded directly in the API handler as (required_iters as f64 * 0.2). From an architectural perspective, this specific modifier should probably live inside kinetic_core::consensus_math alongside the baseline calculations. If the network ever votes to change the discount to 75%, having it buried in the daemon API layer could lead to mismatched consensus rules between different implementations.
  • Passive Memory Leaks: The automatic pruning in handle_vdf_renew only triggers when a new request is received. If a user generates a few VDFs and then never calls the API again, those completed task records will sit in RAM indefinitely. A lightweight background ticker that sweeps the map every hour might provide more robust garbage collection.
  • Error Granularity: When a task fails, the error string is populated with a plain text message. Moving to a structured error enum for VDF tasks could allow frontend UIs to respond more intelligently (e.g., automatically retrying network timeouts vs. aborting on cryptographic failures).