Client Core: The Network Client Handle (Part 1)

Crate: kinetic-network Stage: 8 Reading time: 15 minutes Depends on: Stage 7 (kinetic-core), Stage 1 (kinetic-types)

What Is This?

This file introduces the NetworkClient. This is one of the most critical structural components in the entire Kinetic codebase. It serves as the primary, exclusive bridge between the rest of the Kinetic application and the background peer-to-peer (P2P) network engine.

When any part of Kinetic—whether it is a background daemon process, a data resolver, a web interface proxy, or a storage component—wants to send a message over the network or fetch a payload from the DHT (Distributed Hash Table), it does not touch the network sockets directly. It does not open TCP streams. It does not manage libp2p states. Instead, it talks solely to the NetworkClient.

Think of the actual P2P network engine as a secure, chaotic vault where a million things are happening at once: connections are opening, peers are dropping, streams are multiplexing. The NetworkClient is the calm bank teller standing at the window. You hand your request to the teller, the teller takes it into the vault, and eventually brings you back an answer.

The NetworkClient acts as a thread-safe, easily cloneable handle. It does not run the network event loop itself. Instead, it takes requests (like “publish this payload” or “send this proxy request”), packages them into an internal Command enum, and ships them over an asynchronous channel to the actual background task that manages the network.

Because it is just a lightweight handle wrapped around a channel sender, you can easily clone the NetworkClient and hand copies of it to a hundred different asynchronous tasks inside Kinetic. Every clone points back to the exact same background network engine, allowing the entire application to interact with the network simultaneously without fighting over locks.

Why Kinetic Needs This

Networking in Rust, especially advanced P2P networking using libraries like libp2p, is inherently stateful and complex. The network requires a central, monolithic event loop (often called a “swarm”) that must continuously poll for incoming connections, handle stream multiplexing, manage peer disconnections, and route complex DHT queries.

If every part of Kinetic tried to borrow or lock this network state directly to send a message, the entire application architecture would collapse under its own weight.

Here is exactly what would happen without the NetworkClient abstraction:

• Lock Contention: If the network state was hidden behind a standard Mutex, every time a component wanted to send a message, it would lock the entire network. If multiple components tried to talk at once, the system would grind to a halt waiting for access.
• Protocol Freezes: If a long-running process held a lock on the network state while processing a heavy piece of data, the core network event loop would stop polling. Incoming network events would be blocked, causing Kinetic to drop active connections, miss heartbeats, or fail strict protocol timeouts.
• Architectural Entanglement: The core daemon would have to know exactly how to drive the libp2p state machine, making the code impossible to test in isolation, difficult to scale, and a nightmare to maintain.

Kinetic avoids this disaster by separating the handle (what the app sees) from the engine (what does the work).

The NetworkClient solves three massive architectural problems for Kinetic:

1. Concurrency Without Locking: By using message passing (Rust channels) instead of shared state (mutexes on the swarm), any thread can issue network commands concurrently. The background network engine processes these commands one by one from a queue, ensuring internal state safety without freezing the rest of the application.
2. Hot Swapping (The update_backend trick): Networks drop. Connections fail. Sometimes, Kinetic needs to reboot its P2P swarm from scratch. If every component held a direct reference to the swarm, a reboot would require tracking down and updating every reference across the codebase. Because NetworkClient wraps its sender in an Arc<RwLock>, Kinetic can quietly hot-swap the internal channel without the rest of the application even noticing. The handles stay valid, but they instantly start pointing to the new network engine.
3. Asynchronous Response Routing: P2P commands are inherently asynchronous. When Kinetic asks to resolve a payload from the DHT, it might take several seconds as the query hops across multiple peers. The NetworkClient implements a clever “oneshot channel” trick. It creates a temporary, single-use return channel, bundles it with the command, and waits. The background task does the hard work, finds the payload, and sends it back through that custom channel directly to the task that asked for it.

Without this specific file and its patterns, the entire Kinetic daemon would be a tangled mess of libp2p internals.

How It Works

The NetworkClient is fundamentally a wrapper around a channel sender, but it has several advanced Rust patterns built into it to handle the realities of a distributed network. Let’s break down exactly how it operates, step by step, using the source code as our map.

1. The Core Structure and the RwLock Advantage

-> See: kinetic-network/src/client/core.rs — Lines 10 to 15

The struct definition sets up the entire paradigm for network communication in Kinetic:

• It holds sender: std::sync::Arc<std::sync::RwLock<mpsc::Sender<Command>>>
• It conditionally holds stream_control (unless compiling for WASM).

Wait, mpsc::Sender is already cloneable in Rust. The standard way to share a sender across multiple tasks is just to call .clone() on it. So why did Saif wrap it in an Arc<RwLock<...>>?

In standard Rust, if you just clone the mpsc::Sender, every clone holds a hardcoded link to that specific receiver. But Saif built Kinetic with a specific resilience feature in mind: total network resets.

If the Kinetic node loses connection entirely, or encounters a fatal protocol error, it needs to rebuild its libp2p swarm from scratch. When that happens, the old background task dies, and the old mpsc receiver is destroyed. If we had just cloned the sender directly, every task in Kinetic would now hold a dead sender, and the node would be functionally lobotomized.

By wrapping the sender in an Arc<RwLock>, we create a shared pointer (Arc) to a mutable memory slot (RwLock).

Why an RwLock (Read-Write Lock) instead of a standard Mutex?

• Read-heavy workload: 99.99% of the time, Kinetic just wants to read the sender so it can clone a copy of it to send a message. An RwLock allows an infinite number of simultaneous readers. A Mutex would block all other readers, introducing the exact contention we are trying to avoid.
• Write-rare workload: The only time Kinetic needs to write to this lock is during a total network restart, which happens very rarely.

When the network reboots, Kinetic calls the update_backend method.

2. The Hot-Swap Mechanism for Network Resilience

-> See: kinetic-network/src/client/core.rs — Lines 45 to 57

The update_backend function is the secret sauce for Kinetic’s node resilience. When the network restarts, this function takes a write lock on the RwLock:

#![allow(unused)]
fn main() {
if let Ok(mut s) = self.sender.write() {
    *s = sender;
}
}

It overwrites the old sender with a brand new one. Every existing NetworkClient clone out there in the application (in the daemon, in the API server, in the storage engine) immediately starts routing its new commands to the new background task, with zero disruption to the higher-level logic. This is hot-swapping at its finest.

3. The Oneshot Channel Trick for Async Responses

When you ask the network to do something, you usually want an answer back. But you are communicating over an mpsc (Multi-Producer, Single-Consumer) channel, which is inherently a one-way street. The client sends a command to the background, but how does the background send the answer back to that specific client task out of the hundreds running?

-> See: kinetic-network/src/client/core.rs — Lines 91 to 111 (The send_proxy_request function)

Kinetic uses the “oneshot channel” pattern. This is analogous to ordering food at a busy restaurant and being handed a buzzing pager.

Here is the exact step-by-step flow inside send_proxy_request:

1. The NetworkClient creates a brand new oneshot::channel. This channel only ever carries exactly one message. It has a tx (transmitter) and an rx (receiver).
2. It packages the actual network request (the ProxyRequest), along with the tx half of the oneshot channel (the “pager”), into a Command::SendProxyRequest enum.
3. It sends this Command over the main mpsc channel to the background network engine.
4. The NetworkClient then immediately calls .await on the rx half of its oneshot channel. It goes to sleep, waiting for the pager to buzz.
5. In the background, the network engine processes the queue, sees the command, does the complex libp2p network I/O, gets a response from the remote peer, and pushes that response into the tx channel.
6. The NetworkClient wakes up from its .await, unwraps the response, and hands it back to the caller seamlessly.

This pattern transforms a decoupled, asynchronous, one-way message passing architecture into a clean, easy-to-use async function that looks like a normal RPC call to the rest of Kinetic.

Notice how errors are mapped here:

#![allow(unused)]
fn main() {
.map_err(|_| ProxyError::ChannelClosed)?;
}

If the background task crashes or is restarted while processing, the tx half is dropped. The client’s .await immediately resolves with an error, which we safely map to ProxyError::ChannelClosed. This prevents the client from hanging forever if the network engine dies.

4. Publishing Redundant Payloads and Size Limits

-> See: kinetic-network/src/client/core.rs — Lines 143 to 179

Kinetic relies on the DHT to store data redundantly across the network. The publish_redundant_payload function is the main gateway for data to enter the DHT.

Notice the explicit size limit check on line 152:

#![allow(unused)]
fn main() {
if payload_bytes.len() > 80_000 { ... }
}

The core Kinetic schema sets a strict semantic limit of 64 KB (65,536 bytes) for actual payloads. But here, the network client allows up to 80 KB. Why the discrepancy? Because data traveling over the P2P network is not just raw user data. It is wrapped in cryptographic proofs (like VDF outputs), signatures, and structural serialization overhead (like Protobuf or MessagePack framing).

If the network client enforced a 64 KB limit at the network edge, a valid 64 KB user payload would be rejected the moment a 500-byte signature was attached to it by the storage layer. The 80 KB limit provides necessary safety headroom while still fundamentally preventing malicious nodes from flooding the network with multi-megabyte spam payloads that could cause Out-Of-Memory (OOM) crashes on smaller nodes.

Once the size is validated, it uses the exact same oneshot trick described above, sending a Command::PublishRedundant to the background task and waiting for confirmation.

5. Heartbeats vs. Reveals

-> See: kinetic-network/src/client/core.rs — Lines 187 to 213

Kinetic has a separate publish_heartbeat function. Structurally, it looks nearly identical to publish_redundant_payload. It uses the exact same oneshot pattern, the exact same error mapping, and the exact same internal command flow. So why duplicate it in the client handle?

Because in the background (within the DHT routing logic), heartbeats are treated vastly differently than standard data (which are often called Reveals). Heartbeats go to a dedicated keyspace in the DHT and likely have much shorter Time-To-Live (TTL) values, ensuring the network isn’t clogged with stale node statuses.

By creating a distinct publish_heartbeat method on the NetworkClient, the architectural intent is crystallized directly into the type system and API surface. A developer cannot accidentally publish a massive Reveal payload into the heartbeat keyspace if they are forced to choose between these two explicit, well-named methods. It enforces correct usage at compile time.

6. Resolving Payloads with Strict Timeouts and WASM Support

-> See: kinetic-network/src/client/core.rs — Lines 220 to 260

Getting data out of the DHT uses resolve_redundant_payload.

A DHT lookup is not like a local database query. The background engine has to ask peer A, who might not know, so they ask peer B, who might ask peer C. This distributed search takes time. If peer C drops offline halfway through the query, the request could theoretically hang indefinitely.

To protect the node from resource exhaustion, the NetworkClient enforces a strict 10-second timeout on the oneshot receiver. However, because Kinetic is designed to run in web browsers as well as native servers, this logic requires complex conditional compilation.

-> See lines 241-247 for the non-WASM (native) timeout logic: It uses standard tokio::time::timeout. If 10 seconds pass without a response from the oneshot channel, it aborts the wait and returns a ResolutionError::Internal.

-> See lines 252-259 for the WASM-specific logic:

#![allow(unused)]
fn main() {
#[cfg(target_arch = "wasm32")]
}

Standard tokio timers do not work in WebAssembly because WASM lacks a native operating system timer interface. Instead, Saif uses futures::future::select combined with futures_timer::Delay to race the network response against a 10-second timer. Whichever future finishes first wins the race.

If the background task does not reply within 10 seconds, the NetworkClient aborts the wait. This prevents a slow P2P network from creating a backlog of blocked asynchronous tasks that would eventually exhaust the node’s memory.

7. Handling Poisoned Locks and Panics

When dealing with shared state across multiple asynchronous tasks, error handling becomes paramount. The NetworkClient employs several layers of defense against panics.

-> See: kinetic-network/src/client/core.rs — Lines 68 to 73

#![allow(unused)]
fn main() {
pub fn get_sender(&self) -> mpsc::Sender<Command> {
    self.sender
        .read()
        .unwrap_or_else(|e| e.into_inner())
        .clone()
}
}

What does unwrap_or_else(|e| e.into_inner()) do here? In Rust, if a thread panics while holding a lock (like an RwLock), the lock becomes “poisoned.” By default, future attempts to acquire the lock will return an Err, to prevent tasks from accessing potentially corrupted data.

However, the NetworkClient is just holding a cloneable mpsc::Sender inside this lock. The sender itself doesn’t become corrupted if a thread panics—it just sends messages to a channel. By using into_inner() on the lock error, Kinetic tells Rust: “I don’t care if the lock is poisoned. The data inside (the sender) is still safe to use. Extract it anyway and clone it.”

This ensures that a panic in one isolated part of the Kinetic application does not permanently brick the NetworkClient for the rest of the application. The network client remains resilient and continues functioning even in a degraded state.

8. Stream Control for Raw Connections

-> See: kinetic-network/src/client/core.rs — Lines 76 to 83

You will notice the stream_control field, which returns an Option<libp2p_stream::Control>. While most commands go through the mpsc::Sender, libp2p provides a dedicated Control handle for managing raw byte streams directly between peers. The NetworkClient safely holds this control handle alongside the sender.

Why is this wrapped in an Option and an RwLock?

• Option: Because mock clients (used in tests) and WASM clients do not always have access to the raw stream controller. It must be optional to allow compiling across different environments without crashing.
• RwLock: Just like the main sender, the stream controller might need to be hot-swapped during a total network reset, so it lives behind the same read-write locking paradigm.

When a client wants to open a direct stream to a peer (bypassing the command queue for raw data transfer), it calls stream_control(), gets a clone of the control handle, and uses it directly.

Key Pieces

• NetworkClient struct: (Lines 10-15) The thread-safe, cloneable handle that holds an Arc<RwLock> to the command sender. It is the only approved way for the application to command the network.
• new and new_mock: (Lines 18-42) Constructors. Notice how the mock version ignores stream control, allowing for isolated unit testing without spinning up a real libp2p swarm.
• update_backend: (Lines 45-57) The hot-swap mechanism. It locks the RwLock and safely replaces the sender, allowing the network to restart without invalidating existing client handles.
• get_sender: (Lines 68-73) Provides a quick clone of the underlying sender, gracefully handling poisoned lock states by using unwrap_or_else(|e| e.into_inner()). This ensures a panic in one thread doesn’t permanently brick the client.
• send_proxy_request: (Lines 91-111) Uses a oneshot channel to send a direct request to a specific peer and asynchronously await their direct response.
• send_proxy_response: (Lines 118-136) The reverse of the above. Allows Kinetic to reply to an incoming request using a provided response channel.
• publish_redundant_payload: (Lines 143-179) Pushes data to the DHT. Enforces an 80 KB size limit to account for 64 KB of raw data plus cryptographic and serialization overhead.
• publish_heartbeat: (Lines 187-213) Semantically distinct method for publishing node liveness signals to a specific DHT keyspace.
• resolve_redundant_payload: (Lines 220-260) Pulls data from the DHT. Crucially implements a 10-second timeout so the application never hangs indefinitely waiting for unresponsive peers. Contains complex WASM-specific timer fallbacks.

How This Connects to the Rest of Kinetic

• CROSS-CRATE: This module relies on kinetic_core::error types (like NetworkClientError, PublishError, and ResolutionError), which were established in Stage 7.
• CROSS-CRATE: It uses types from kinetic-types like ProxyRequest and ProxyResponse, defined in Stage 1.
• Inbound Connections: This file represents the client side of the internal API. When Kinetic wants to talk to the network, it calls methods here.
• Outbound Connections (The Backend): The Command enum sent over the channel is processed by the actual libp2p event loop. The event loop is the consumer of this channel, acting upon the requests.
• WASM Compatibility: The explicit conditional compilation attributes (#[cfg(target_arch = "wasm32")]) prove that Kinetic is structurally designed to run directly inside a browser tab, requiring alternative timer implementations.

Quick Reference

• The Goal: Provide a safe, concurrent way to interact with the single-threaded network event loop.
• The Delivery Mechanism: mpsc::Sender<Command> locked behind an Arc<RwLock> to allow for zero-downtime hot-swapping.
• The Response Mechanism: tokio::sync::oneshot::channel passed inside the command payload. Acts like a restaurant pager for async tasks.
• Max Payload Size Limit: 80,000 bytes (safely fits a 64 KB schema limit + crypto overhead without rejecting valid data).
• DHT Resolution Timeout: 10 seconds maximum. Uses tokio natively, futures_timer in WASM.
• Locking strategy: Read locks for sending commands (fast, concurrent), Write locks for restarting the network (rare, blocking).
• Mocking: Fully mockable via new_mock for fast unit tests.

Open Questions / Things to Revisit

• WASM Backend Updates: The update_backend function exists for WASM builds, but it intentionally drops the stream_control component. It would be worth verifying if WASM instances in Kinetic ever actually undergo hot-restarts in practice, or if the browser environment just drops the instance upon failure, rendering this hot-swap logic unnecessary overhead for the web build.
• Hardcoded 10-Second Timeout: The 10-second timeout in resolve_redundant_payload is hardcoded into the binary. In congested or geographically distant networks (e.g., cross-continental DHT routing), 10 seconds might be too aggressive and cause premature failures. Consider if this should be exposed as a configurable parameter via a node settings file or environment variable in the future.
• Timeout Memory Leaks and Panic Risks: When a resolution times out on the client side, the oneshot rx channel is immediately dropped. However, the background network task is still blindly running the DHT query. We need to ensure the background task handles dropped tx channels gracefully without panicking when it eventually tries to send the late response. If the background task calls .unwrap() on the send operation, a client-side timeout could crash the entire network event loop.