Keyboard shortcuts

Press ← or → to navigate between chapters

Press S or / to search in the book

Press ? to show this help

Press Esc to hide this help

API Tests (Daemon API Testing Suite)

Crate: kinetic-daemon Stage: 9 Reading time: 30 minutes Depends on: docs/learn/network/01_overview.md, docs/learn/storage/01_overview.md, docs/learn/core/01_overview.md

What Is This?

This file (kinetic-daemon/src/api/api_tests.rs) contains the comprehensive internal testing suite for the Kinetic Daemon’s HTTP API.

It is designed to rigorously validate that all the public-facing HTTP endpoints exposed by the daemon behave correctly. These endpoints include critical paths such as:

  • /commit
  • /publish
  • /resolve
  • /vdf/register

These are isolated unit-level API tests. They do not require spinning up an entire local Kinetic node. They do not require connecting it to a live P2P network swarm. They do not require binding the HTTP server to a real localhost TCP port.

Instead, they construct the axum web router in memory. They inject mocked dependencies to isolate the API logic. These mocked dependencies include:

  • A dummy network client.
  • A temporary, ephemeral storage engine.

The tests then fire simulated HTTP requests directly at the router’s core logic.

The test suite thoroughly covers several distinct categories of API behavior:

  • Authentication Enforcement: Ensuring that protected endpoints demand valid Bearer tokens. Instantly rejecting unauthenticated requests with HTTP 401.
  • Payload Validation: Checking that malformed JSON bodies are caught. Validating that invalid domain names are rejected. Ensuring structurally incorrect cryptographic proofs are blocked. These checks prevent bad data from polluting the internal database.
  • Complex Business Logic Constraints: Validating advanced temporal rules. Validating cryptographic rules. Verifying that VDF proofs are cryptographically anchored to a recent Drand randomness round.
  • Concurrency Control: Ensuring multiple concurrent HTTP requests are handled safely. Validating that internal locks (Mutexes and Semaphores) successfully serialize the work. Preventing race conditions. Preventing duplicate CPU-heavy tasks.
  • Graceful Fallback Mechanisms: Verifying behavior when the primary network (the DHT) fails to find a record. Ensuring the daemon gracefully falls back to querying its own local database cache. Confirming this happens without crashing or surfacing confusing internal errors.

By mocking the external world, this suite allows the developers to instantly simulate extreme edge cases. These edge cases would be difficult, flaky, or nearly impossible to reliably trigger on a live, distributed network.

Why Kinetic Needs This

The daemon API is the absolute primary surface area for the Kinetic node. External applications interact with the node here. CLI tools interact with the node here. Frontends and end users interact with the node here.

It acts as the critical bridge between the chaotic, untrusted external world (the HTTP client) and the structured, cryptographic internal world of the Kinetic network. If this API fails, misbehaves, or contains logic vulnerabilities, the entire node is effectively compromised. It could even be rendered useless.

Kinetic requires this rigorous, isolated API testing suite for several non-negotiable reasons:

  1. Security and Access Control Guarantees: The API handles privileged operations. The API handles potentially destructive operations. Endpoints like /commit and /publish alter the local database. They broadcast messages to the global P2P network. We must be certain that endpoints requiring administrative access are secured. Endpoints requiring specific capabilities (governance, VDF registration) cannot be accessed by unauthorized HTTP requests. This test suite guarantees that the authentication middleware acts as an impenetrable shield.

  2. Deterministic Edge-Case Simulation: Testing advanced cryptographic business logic on a live testnet is difficult. It is flaky and time-consuming. Testing “stale Drand randomness” normally requires manually manipulating a live Drand feed. Or, it requires literally waiting hours for real time to pass just to test an expiry threshold. By mocking the database state in these unit tests, we bypass this. We instantly and deterministically simulate a scenario where a user submits a proof based on an ancient Drand round. We verify that the API immediately detects it and rejects it.

  3. Concurrency and Race Condition Prevention: The daemon utilizes asynchronous tasks. It relies on shared memory locks (like the vdf_tasks concurrent map). When two users attempt to register the exact same identity at the exact same millisecond, the system must hold up. The system must not crash. The system must not panic. The system must not create duplicate, CPU-heavy VDF tasks that would starve the machine. These in-memory API tests allow us to programmatically fire concurrent requests. We fire overlapping requests and prove that our concurrency primitives work. Our tokio::sync::Mutex and tokio::sync::Semaphore implementations must hold up under maximum, coordinated pressure.

  4. Instant Developer Feedback Loop: Standing up a full Kinetic node is slow. Establishing a DHT swarm takes time. Syncing blocks takes time. Booting a storage backend takes time. Doing all this just to see if /commit correctly returns a 400 Bad Request is inefficient. It disrupts the developer flow. These in-memory API tests run in mere milliseconds. They bypass the actual TCP networking stack. They bypass the actual libp2p network. They provide instant, reliable feedback during the software development process.

  5. API Contract Adherence: The HTTP status codes returned by the API constitute a binding contract. The error messages returned by the API are also part of this contract. Downstream clients (like the CLI or web wallets) rely on this contract. If a domain name is invalid, the client expects exactly a 400 Bad Request. They expect a specific error string along with it. If a cryptographic signature is invalid, it expects exactly a 422 Unprocessable Entity. These tests ensure that the API contract remains stable. It guarantees the contract does not accidentally regress when deep internal core logic is refactored.

Without this comprehensive suite, any minor change could silently break the daemon’s external interface. A change to the core validation logic could break it. An accidental misconfiguration of the HTTP routing tree could break it. This would leave users unable to interact with the broader network.

How It Works

The architectural magic and speed of this test suite relies on two core software engineering patterns. These patterns are Dependency Injection and Headless HTTP Testing. By combining these two techniques, the tests achieve total environmental isolation.

1. Dependency Injection for Isolated State Setup

When the actual compiled Kinetic daemon runs, it initializes a heavy environment. It initializes a real, persistent Sled database on the physical disk. It connects to real libp2p peers over TCP sockets. It spawns real gossipsub networking tasks. In these tests, we must bypass all of that heavy machinery. This is necessary to maintain extreme speed and determinism.

Every single test in this file begins by executing the setup_test_app() asynchronous helper function. This function constructs a isolated, fake universe for the API to live within. It creates a disposable universe.

The Ephemeral Storage Engine: It utilizes the powerful tempfile::tempdir() function from the Rust ecosystem. This creates a temporary directory directly on the operating system’s filesystem. It then initializes a real SledStorage engine inside this temporary directory. When the test finishes execution, the directory variable goes out of scope. At this moment, the directory is immediately and automatically deleted by Rust’s Drop implementation. This guarantees zero state contamination between different tests.

The Mock Network Client: It initializes the network interface using NetworkClient::new_mock(cmd_tx). It does not attempt to dial out to a real P2P network. This mocked network client simply takes internal network requests (like resolving a name). It sends them as serialized enums into a local MPSC (Multi-Producer, Single-Consumer) channel. The test itself can then safely await on the receiver end of this channel. This allows the test to inspect exactly what commands the API attempted to send. It even allows the test to manually inject fake responses back into the API. This simulates complex network behavior.

2. Anatomy of the Mocked ApiState

The Axum router requires an ApiState object to function. In production, this state is built dynamically during daemon startup. In testing, setup_test_app() constructs a specific dummy version:

  • network: Passed the NetworkClient::new_mock().
  • gossip_tx: Given a fresh tokio::sync::broadcast::channel. Since we don’t care about testing the gossip broadcast listener here, we just give it a channel that drops broadcast messages into the void.
  • storage: Passed the Arc<SledStorage> pointing to the ephemeral tempdir.
  • tokens: Hardcoded with strings like "test-token-123" and "publish-token". This allows tests to hardcode the Authorization: Bearer test-token-123 header easily. It prevents the tests from needing to dynamically read a config file.
  • vdf_tasks: Initialized as an empty Arc<Mutex<HashMap>>. This is the memory space what the concurrency test will intentionally pound against.
  • vdf_semaphore: Initialized with Arc::new(tokio::sync::Semaphore::new(1)). This restricts the mock state to only allowing one single VDF task at a time. It mimics a constrained CPU environment.
  • atlas_tlds: Initialized as an empty Arc<RwLock<HashSet>>.

This meticulously controlled state injection is what makes the tests fast, deterministic, and impenetrable. It removes all outside environmental flakiness.

3. Headless HTTP Testing via Tower Service Ext

The Kinetic API is constructed using the axum web framework. Axum is fundamentally built on top of the robust tower middleware ecosystem. In a normal production deployment, you bind an axum::Router to a real TCP port (like 127.0.0.1:8080). The operating system then routes incoming network packets to it.

In these specific tests, we never bind to a port. We bypass the OS network stack entirely. We utilize the tower::ServiceExt::oneshot method. Because an axum::Router implements the tower::Service trait, it can be treated uniquely. It can be conceptually treated as a simple, asynchronous Rust function. This function takes an http::Request object and returns an http::Response object. We manually construct these request objects in memory using a builder pattern. We pass them directly into the core router using the oneshot method. We simply await the HTTP response without ever touching a socket.

4. Reading the Response Body

Axum HTTP responses are asynchronous streams of bytes. The test must actively consume the stream body to read the response error message. It does this using the http_body_util::BodyExt extension trait. It calls .into_body().collect().await.unwrap().to_bytes(). This takes the asynchronous stream of HTTP body chunks. It collects them into a single contiguous byte array in memory. It allows the test to convert it into a UTF-8 string for exact string-matching assertions.

5. The Strict Execution Flow of a Standard Test

Every test in this suite follows a rigorous, three-step methodology. This is the Arrange-Act-Assert pattern:

  1. Arrange: The test calls setup_test_app(). It obtains the router, the network command receiver, and a pointer to the storage database. It intentionally prepopulates the mock storage database with fake data. For example, it might inject a specific historical Drand round. It might inject a fake Reveal record. This ensures the API inherently believes it is in a specific edge-case state.

  2. Act: The test constructs an http::Request manually. It uses the HTTP builder pattern. It serializes a JSON payload into the request body using serde_json::json!. It sets required headers. The most important header is usually the Authorization: Bearer token. It sets the URI and HTTP method. It then fires this request into the router using .oneshot(request).await.

  3. Assert: The test inspects the returned http::Response. It first verifies that the HTTP StatusCode exactly matches expectations. For example, it might assert StatusCode::BAD_REQUEST. It then extracts the raw bytes of the response body. It converts them to a UTF-8 string. It rigorously asserts that the string contains the precise error message. It checks the payload expected by the client application.

Key Pieces

The file contains numerous specific, tightly scoped tests. Here is a detailed breakdown of every critical piece. We explore what it tests, how it executes, and why it deeply matters to the daemon’s stability.

The Foundation: The setup_test_app Helper

-> See: crates/kinetic-daemon/src/api/api_tests.rs — Lines 21 to 50

This internal function is the bedrock of the entire testing suite. It takes zero arguments and returns a tuple containing three vital pieces of infrastructure:

  1. The axum::Router, which is the fully constructed API, loaded with the mocked state.
  2. An mpsc::Receiver<Command>, which is the listener for the mock network. The test can await on this receiver to see exactly what commands the API attempted to send to the P2P network.
  3. An Arc<SledStorage>, which is a direct, thread-safe pointer to the temporary database. The test uses this pointer to directly inject data into the database behind the API’s back. This sets up complex internal state scenarios without needing to run real network sync mechanisms.

The Security Check: Authorization Enforcement

-> See: crates/kinetic-daemon/src/api/api_tests.rs — Lines 52 to 64

This test is mechanically simple but architecturally critical. It constructs a raw POST request aimed at the /commit endpoint. It intentionally omits the Authorization: Bearer <token> HTTP header. It then verifies that the server rejects the request. It checks that it responds with a strict 401 UNAUTHORIZED status code. This definitively proves that the require_auth middleware is successfully wrapping the protected routes. It ensures no malicious external actor can bypass the token checks and interact with the node.

The First Line of Defense: Input Structure Validation

-> See: crates/kinetic-daemon/src/api/api_tests.rs — Lines 66 to 116

These tests ensure that the API does not blindly accept well-formatted JSON. It checks if the actual cryptographic contents logically violate network rules. For example, a commitment hash cannot logically be all zeros, as that represents an unhashed state. If a user submits a payload with hash: vec![0; 32], the API must deeply inspect the contents. It must reject it with a 400 BAD_REQUEST. Similarly, if a user attempts to commit a name like sub.saifmukhtar.kin, it fails. This is technically a subdomain, not a root apex domain, which violates current kinetic rules. The API instantly rejects it. These tests conclusively prove that the HTTP layer correctly invokes the core domain validators. These validators live inside kinetic-core. They are invoked before proceeding with any local state mutations.

The Domain Integrity Check: Structural Validation

-> See: crates/kinetic-daemon/src/api/api_tests.rs — Lines 154 to 189

When a user attempts to publish an identity, they must provide a massive, complex JSON payload. This payload contains cryptographic proofs, public keys, Ed25519 signatures, and protocol versions. If they provide an invalid protocol version, the API must reject it. For instance, purposefully providing 0 instead of the expected version 2 must fail. The test verifies that the body of the HTTP response mentions “Invalid Reveal”. This proves that the core structural validation logic is propagating its specific error messages correctly. It proves they travel correctly up through the HTTP boundary to the end user.

The Temporal Security Check: Drand Staleness Verification

-> See: crates/kinetic-daemon/src/api/api_tests.rs — Lines 191 to 249

This is arguably one of the most sophisticated and vital tests in the suite. It tests the temporal security mechanisms of the Kinetic network. When you publish a name, your expensive VDF proof must be firmly anchored to a recent Drand randomness round. If you utilize an ancient round, it implies you had too much time to precompute the proof. This is a massive security violation of the protocol designed to prevent front-running. The test purposefully mocks the storage database to logically believe the current Drand round (kyn) is 10_000_000. It then constructs and submits a publish HTTP request where the drand_kyn utilized for the proof is 100. This is functionally millions of rounds in the past. The API successfully detects this massive temporal gap. It calculates that it vastly exceeds the RESQUARING_EPOCH_KYNS safety threshold. It rejects the request with a 400 BAD_REQUEST. It notes “VDF kyn” in the error string to guide the user.

The System Redundancy Check: Network Fallback Simulation

-> See: crates/kinetic-daemon/src/api/api_tests.rs — Lines 251 to 306

When a user asks the daemon to resolve an identity name via the API, the daemon first attempts to query the global DHT (the P2P network). If the DHT fails to find the record (perhaps due to network partition), the daemon must recover. It is programmed to automatically fall back and check its own local storage database cache. This test beautifully simulates this complex, asynchronous two-step process:

  1. It manually inserts a valid mock_reveal into the temporary Sled storage.
  2. It spawns an async background Tokio task to actively listen on the mock network command receiver. When the API asks the network to resolve the name, this background task intercepts the command. It intentionally replies over a oneshot channel with an Error::ResolutionError::NotFound. This simulates a total network failure.
  3. The API, receiving this error from the mock network, silently and gracefully falls back. It queries the local SledStorage. It finds the manually injected record and successfully returns a 200 OK to the HTTP client containing the record. This test conclusively proves that the fallback mechanism is robust. It proves it is fully operational. It proves it is transparent to the end API user.

The Missing Dependency Check: Zone Publishing Rules

-> See: crates/kinetic-daemon/src/api/api_tests.rs — Lines 308 to 321

Kinetic enforces a strict structural hierarchy for zones. You cannot publish DNS zone records for a name unless that identity name has already been registered. It must be committed to the local database first. This test systematically verifies this strict constraint. It constructs an HTTP POST request targeting /zone/validname.kin/publish. The underlying mock storage database is empty. We did not manually inject a registration record for validname.kin. Because of this, the daemon’s internal lookup fundamentally fails. The API layer correctly intercepts this internal core failure. It translates it and returns a 404 NOT_FOUND HTTP status code. This proves that the API will block dangling or orphaned zone records from entering the network layer.

The Operational Race Condition Check: Concurrency Lock Verification

-> See: crates/kinetic-daemon/src/api/api_tests.rs — Lines 323 to 360

In Kinetic, initializing and running a VDF task is expensive in terms of sustained CPU cycles. If a user maliciously or accidentally spams the “register” button on a frontend client, the daemon could panic. It might inadvertently spawn two identical VDF tasks for the same name. This would burn unnecessary CPU resources and potentially crash the node through starvation. To prevent this, the daemon utilizes an in-memory concurrent lock map called vdf_tasks. This test conclusively proves that the lock works under heavy load. It utilizes the powerful tokio::join! macro to fire two identical HTTP requests at the exact same microsecond. Because they run concurrently on the Tokio async runtime, they will both hit the route handler simultaneously. The test verifies that the locking mechanism successfully serializes them. Exactly one request definitively wins the race and gets a 200 OK. The other request hits the locked Mutex and is rejected. It recognizes the task is already actively running for that specific name. It correctly returns a 409 CONFLICT. This is a mission-critical test for overall daemon stability.

The Identity Forgery Check: Cryptographic Signature Verification

-> See: crates/kinetic-daemon/src/api/api_tests.rs — Lines 362 to 385

This test guarantees that the /publish-kid endpoint enforces cryptographic signature checks before processing. It submits a JSON payload containing a purposefully fake public key. It also contains an intentionally invalid, garbage signature string. Kinetic fundamentally relies on Ed25519 signatures to cryptographically prove ownership of decentralized identities (KIDs). Therefore, the daemon must catch this blatant forgery instantly. The test confirms that the API immediately detects the invalid cryptographic structure. It returns a 422 UNPROCESSABLE_ENTITY. This permanently blocks the forged identity from ever touching the internal network queues. It prevents it from polluting the local persistent storage.

How This Connects to the Rest of Kinetic

This file acts as the ultimate integration testing point for almost all other core Kinetic crates. Because the API must touch every sub-system to serve a user request, the API tests inherently validate the inter-crate contracts:

  • CROSS-CRATE: SledStorage Defined and explained in docs/learn/storage/01_overview.md. The tests rely on instantiating an isolated Sled database. They use this to simulate the daemon’s local memory and inject complex, adversarial test states.
  • CROSS-CRATE: NetworkClient Defined and explained in docs/learn/network/01_overview.md. The tests utilize NetworkClient::new_mock(). They use this to intercept outgoing DHT requests. They use it to actively inject fake P2P responses back into the API logic.
  • CROSS-CRATE: Core Types Defined and explained in docs/learn/core/01_overview.md. The tests manually construct core structures. They construct Reveal, RawKyn, and VdfProof objects. They feed these into the mock API to trigger specific, deep validation execution paths.

The API routes act as thin wrappers around the deep core business logic. Therefore, these tests implicitly validate that the API layer is correctly translating standard HTTP concepts (headers, JSON bodies, status codes) into internal kinetic-core concepts. They also validate that it seamlessly translates deep kinetic-network errors back into understandable, standard HTTP status codes.

Quick Reference

When reviewing, debugging, or extending the API test suite, keep these core Rust mechanics in mind:

  • The Test Framework: Uses the standard #[tokio::test] attribute macro to run async tests on a local executor.
  • Headless HTTP Routing: Uses tower::ServiceExt::oneshot(request) to bypass the OS TCP network stack. This allows testing the axum router directly in memory.
  • Disposable Mock Storage: Uses tempfile::tempdir() to create a disposable storage instance. This ensures zero-contamination SledStorage instances for each individual test run.
  • Intercepted Mock Network: Intercepts outgoing network commands (like Command::ResolveRedundant) via an MPSC receiver channel. This is used to manually inject fake DHT responses.
  • Aggressive Concurrency Testing: Uses tokio::join! to fire parallel HTTP requests simultaneously. This verifies Mutex locking and prevents state race conditions.
  • Payload Construction: Uses the serde_json::json! macro to easily and construct complex request bodies. This handles nested JSON directly inline within the test functions.
  • Body Extraction: Uses BodyExt::collect().await.unwrap().to_bytes() to convert the asynchronous Axum response stream. It converts it into a readable string for assertions.

Open Questions / Things to Revisit

  • Missing Token Scope Tests: Right now, the authorization test just checks if any token is required. It does not thoroughly verify token scoping logic. For example, it doesn’t currently verify that a user holding only the vdf token is correctly denied access to the admin token routes. This is a critical gap in the API test coverage that needs to be addressed to ensure strict least-privilege enforcement.

  • Mock Network Completeness: Currently, the mock network client only actively handles a few specific commands (primarily ResolveRedundant). As the daemon API continues to expand in feature scope, the mock client will need to simulate far more complex P2P interactions. This includes Gossipsub message publishing and direct DHT put operations. If the mock client does not evolve alongside the API, future advanced tests will be impossible to write.

  • Database State Instantiation Speed: Right now, every single test spins up a brand new SledStorage instance in a newly created temporary directory on the physical disk. While this guarantees test isolation, it might become slow. If the test suite grows to hundreds of tests, it will introduce major disk I/O bottlenecks. We might eventually need to engineer a way to quickly wipe and reset a single in-memory database. This would be much faster than requesting a new filesystem directory from the OS for every single test execution.

  • Complete Error Code Coverage: While the critical happy paths and primary rejection paths are tested (commits, publishes, validation), there are blind spots. There may be deeply nested edge cases around error propagation. For example, what exactly happens if the database panics or corrupts during a read request? These complex failure modes are not currently simulated or caught by this specific suite.