Keyboard shortcuts

Press ← or → to navigate between chapters

Press S or / to search in the book

Press ? to show this help

Press Esc to hide this help

Network Infrastructure & DNS Discovery

Crate: kinetic-network Stage: 8 Reading time: 20 minutes Depends on: docs/learn/types/01_overview.md

What Is This?

This document covers the structural glue and auxiliary protocols of the kinetic-network crate. Rather than focusing on a single continuous pipeline (like the event loop or the storage engine), this file examines the crucial connective tissue that makes the network functional. Specifically, it documents the crate entry point (lib.rs), the aggregate libp2p network behavior (behavior.rs), the module boundaries for the store and event loop (store/mod.rs, store/constants.rs, event_loop/mod.rs), and the custom DNS-based bootstrap discovery protocol (dns_tree.rs).

While files like the Kademlia store handle the heavy lifting, these structural files define exactly what the network is capable of doing and how a newly booted node finds its first connection to the outside world. The KineticBehavior struct acts as the grand manifest, dictating the exact set of P2P protocols a Kinetic node speaks. Meanwhile, the DNS tree protocol ensures a node does not have to rely on a hardcoded list of static IP addresses to join the network. Finally, the module files establish strict API boundaries so that the rest of the workspace can interact with the network cleanly.

Why Kinetic Needs This

In a decentralized network, several foundational problems must be solved before any actual domain data can be exchanged.

1. Protocol Composition and Identity (behavior.rs) The underlying networking framework, libp2p, is not a single unified protocol. It is a modular toolkit. You have to declare which pieces you want to use. Kinetic requires Kademlia for the DHT, Gossipsub for real-time broadcasts, AutoNAT for firewall traversal, and custom Request-Response protocols for proxying domain traffic. The behavior.rs file takes all these independent, isolated protocols and merges them into a single, cohesive state machine. Without this file, the node would have no defined network identity and would not know how to handle incoming connections.

2. Resilient Bootstrap Discovery (dns_tree.rs) When a Kinetic node boots up for the very first time, it faces the “first contact” problem: how does it know the IP address of another node? Hardcoding IP addresses directly into the source code is brittle—if those initial nodes go offline, new nodes can never join the network. Kinetic solves this elegantly using a custom DNS Tree protocol called kintree. By querying standard DNS TXT records for a known domain name, a node can discover a dynamically updating list of active bootstrap nodes. This allows network maintainers to update the entry points without requiring users to download new software binaries.

3. Architectural Isolation (lib.rs, mod.rs) The kinetic-network crate is vast and complex. lib.rs and the various mod.rs files act as the architectural blueprint. They define the public API boundaries so that downstream crates (like kinetic-daemon) do not have to understand the messy, asynchronous internals of libp2p swarms. They encapsulate the complexity, ensuring that the daemon only interacts with a clean message-passing interface.

4. Keyspace Separation (store/constants.rs) The Kademlia DHT is a massive, flat key-value store shared across the globe. Kinetic stores multiple different types of data in this DHT: reveals, heartbeats, and commitments. Without distinct namespaces, a malicious or malfunctioning node could publish a heartbeat using a key that accidentally overwrites a critical domain reveal. The constants ensure cryptographic separation within the DHT keyspace.

How It Works

Crate Architecture and API Surface (lib.rs)

The lib.rs file serves as the gateway to the kinetic-network crate. -> See: file:///home/saif/kinetic/kinetic-network/src/lib.rs — Lines 1 to 53

It begins with strict documentation linting rules (#![deny(missing_docs)]), enforcing that all public APIs are documented. The architecture is centralized around the NetworkEventLoop. The lib.rs file publicly exports only what is necessary: NetworkClient, NetworkConfig, NetworkMode, ProxyRequest, ProxyResponse, KineticStoreError, and NetworkEventLoop. By hiding the inner workings of the swarm and the Kademlia handlers, it forces a decoupled design. The external caller (the daemon) spawns the event loop and then solely communicates with it using the NetworkClient handle via an asynchronous channel. This prevents lock contention and keeps the swarm single-threaded.

The Aggregate Network Behavior (behavior.rs)

In libp2p, every distinct network feature (e.g., pinging a peer, searching the DHT) is implemented as a Behaviour. To create a functional node, you must combine them. -> See: file:///home/saif/kinetic/kinetic-network/src/behavior.rs — Lines 10 to 51

The KineticBehavior struct uses the #[derive(NetworkBehaviour)] macro. This macro is a powerful piece of Rust metaprogramming that automatically generates the necessary event-routing code to run multiple behaviors concurrently. The struct includes:

  • identify: The libp2p::identify::Behaviour protocol. When two nodes connect, they use this to exchange their supported protocols, public keys, and observed IP addresses. This is critical for nodes to discover if they can speak the same version of Kinetic.
  • ping: The libp2p::ping::Behaviour protocol. It periodically sends tiny messages to keep TCP/UDP connections alive through aggressive home NATs and measures network latency.
  • kademlia: The kad::Behaviour<KineticRecordStore>. This is the decentralized database. It uses Kinetic’s custom record store to hold domain state.
  • gossipsub: The gossipsub::Behaviour. This is the fast, real-time pub-sub network used for immediate propagation of reveals and heartbeats to thousands of nodes.
  • proxy: A custom request_response behavior that allows nodes to ask each other to proxy HTTP traffic for domain resolution.
  • cdn: Another custom request_response behavior specifically for serving DHT caches, speeding up data retrieval.
  • stream: A libp2p_stream::Behaviour used for passing raw traffic, essential for proxying actual data streams.
  • autonat: The libp2p::autonat::Behaviour. This asks peers to report back the IP address they see, allowing a node to figure out if it is behind a restrictive firewall.
  • relay_client & dcutr: Complex NAT traversal protocols. If a node is trapped behind a router, it uses a relay to punch a hole (DCUtR - Direct Connection Upgrade through Relay) to establish a direct connection eventually.
  • upnp: Uses the Internet Gateway Device (IGD) protocol to automatically ask home routers to forward ports.
  • mdns: Uses multicast DNS to discover other Kinetic nodes on the same local network (LAN) instantly without hitting the internet.

Several of these behaviors (like upnp and mdns) are wrapped in a Toggle. This allows the daemon to dynamically enable or disable them at runtime based on the node’s configuration (e.g., turning off mDNS in cloud servers).

DNS Tree Discovery (dns_tree.rs)

When a node needs to bootstrap, it relies on the resolve_dns_tree(domain) function. -> See: file:///home/saif/kinetic/kinetic-network/src/dns_tree.rs — Lines 11 to 91

This function performs a decentralized walk of DNS TXT records to find libp2p Multiaddrs. The step-by-step mechanism is:

  1. Initialize Resolver: It creates an asynchronous DNS resolver using the hickory_resolver crate, pulling configuration from the host OS.
  2. Fetch Root Records: It queries the base domain for TXT records.
  3. Parse and Identify Root: It scans the TXT strings. If it finds a raw Multiaddr (starting with /ip4/ or /ip6/), it saves it. If it finds the special kintree-root:v1 e=<hash> string, it extracts the hash to begin tree traversal.
  4. Iterative Branch Traversal: It uses a while loop to process branches.
    • It constructs a subdomain query: <branch_hash>.<domain>.
    • It queries TXT records for this specific subdomain.
    • It looks for kintree-branch:<hash1>,<hash2> records to discover deeper branches in the tree.
    • It looks for kintree-leaf:<multiaddr> records to extract actual peer connection strings.
  5. Safety Constraints: To prevent infinite loops caused by malicious or misconfigured DNS trees, the function enforces a maximum of 20 DNS lookups (max_lookups). It also stops immediately once it has collected 50 peer addresses. It utilizes a HashSet to track visited branch hashes, preventing cyclical traversal.

For WebAssembly targets (wasm32), this entire process is short-circuited. -> See: file:///home/saif/kinetic/kinetic-network/src/dns_tree.rs — Lines 93 to 97 Because web browsers do not permit raw UDP or TCP socket creation for custom DNS queries, the function simply returns an empty vector. WASM nodes must rely on alternative bootstrapping methods provided by their host environment.

Storage Constants (store/constants.rs)

The Kademlia DHT requires all keys to be byte arrays. To ensure that different subsystems do not collide, Kinetic uses namespace prefixes. -> See: file:///home/saif/kinetic/kinetic-network/src/store/constants.rs — Lines 3 to 9

  • KRS_REVEAL_PREFIX (b"krs_reveal:"): Prepended when a node publishes a domain reveal transaction.
  • KRS_HB_PREFIX (b"krs_hb:"): Prepended for heartbeat broadcasts indicating node liveness.
  • KRS_COMMIT_PREFIX (b"krs_cmt:"): Prepended for cryptographic commitments before a reveal.

By applying these prefixes before the key is hashed and placed into the DHT, Kinetic guarantees that a heartbeat for “Node A” will hash to an different topological location in the network than a reveal for “Node A”.

Module Boundaries (store/mod.rs & event_loop/mod.rs)

The mod.rs files establish the internal hierarchy. -> See: file:///home/saif/kinetic/kinetic-network/src/store/mod.rs — Lines 1 to 18 -> See: file:///home/saif/kinetic/kinetic-network/src/event_loop/mod.rs — Lines 1 to 19

Notice the extensive use of pub(crate). Files like handlers, verification, lightnode, and fullnode are marked pub(crate) because they are vital to the network’s internal machinery but should be invisible to the outside world. The only thing exported publicly from the store is the KineticRecordStore itself, and from the event loop, the NetworkEventLoop. This strict encapsulation is what makes the network layer maintainable.

Key Pieces

  • KineticBehavior

    • Where: kinetic-network/src/behavior.rs
    • What: A comprehensive struct aggregating every single libp2p protocol (DHT, Gossipsub, Request-Response, NAT traversal) used by the network.
    • Why it matters: It defines the exact network capabilities of a Kinetic node. It is the core state machine for peer-to-peer interactions.

  • resolve_dns_tree

    • Where: kinetic-network/src/dns_tree.rs
    • What: An asynchronous traversal function that resolves a domain name into a list of libp2p Multiaddr connection strings by walking a cryptographic tree of DNS TXT records.
    • Why it matters: It provides a resilient, decentralized, and updateable mechanism for new nodes to find the network entry points without relying on brittle hardcoded IP addresses.

  • Storage Prefixes (KRS_REVEAL_PREFIX, etc.)

    • Where: kinetic-network/src/store/constants.rs
    • What: Static byte arrays prepended to Kademlia DHT keys.
    • Why it matters: They namespace the decentralized database, physically separating different types of network state across the DHT topology.

  • NetworkClient & NetworkEventLoop Exports

    • Where: kinetic-network/src/lib.rs
    • What: The meticulously restricted public API of the crate.
    • Why it matters: It enforces the architectural law that external crates must communicate with the network via asynchronous message passing, preventing locking and concurrency bugs.

How This Connects to the Rest of Kinetic

  • Types: The proxy protocols in behavior.rs rely on ProxyRequest and ProxyResponse, which in turn utilize fundamental structs from kinetic-types (Stage 1).
  • CROSS-CRATE: The kinetic-types crate defines CdnRequest and CdnResponse, which are imported and integrated directly into the KineticBehavior here for decentralized content delivery.
  • Daemon Integration: The exports defined in lib.rs are the exact imports utilized by kinetic-daemon (Stage 9) to boot the node. The daemon executes resolve_dns_tree to find initial peers, configures the NetworkEventLoop using the KineticBehavior, and then takes operational control using the NetworkClient.

Quick Reference

  • To inspect all supported P2P protocols: Look at the fields of KineticBehavior in behavior.rs.
  • To understand how bootstrapping logic works: Read the while loop inside resolve_dns_tree in dns_tree.rs.
  • To find the Kademlia DHT key namespaces: Look at store/constants.rs.
  • To review the network public API: Check the pub use statements at the bottom of lib.rs.
  • WASM Network Target: DNS resolution is deliberately stubbed out for the wasm32 architecture due to browser I/O restrictions.
  • Toggle Protocols: mdns, relay_server, and upnp are wrapped in Toggle, meaning they can be instantiated in a disabled state based on configuration.

Open Questions / Things to Revisit

  • Hardcoded DNS Limits: The resolve_dns_tree function contains hardcoded values: max_lookups = 20 and a hard cap of 50 peer addresses. As the Kinetic network scales to thousands of nodes, these arbitrary limits might artificially restrict a node’s initial view of the network topology. Should these limits be moved to the NetworkConfig?
  • Silent DNS Failures: If the txt_lookup fails for a specific branch subdomain during tree traversal, the code silently ignores the error and continues. While this is resilient, it might mask severe network partitions or DNS misconfigurations. It would be beneficial to log these specific branch traversal failures for debugging purposes.
  • WASM Bootstrapping Gap: Currently, resolve_dns_tree returns an empty vector for WASM environments. If Kinetic intends to support in-browser light nodes, they will require an different bootstrapping mechanism (such as WebRTC signaling servers or HTTP-based bootstrap endpoints). This is a significant architectural gap that needs to be addressed for true browser compatibility.
  • Behavior Opacity: Several behaviors (like upnp and relay_server) use libp2p::swarm::behaviour::toggle::Toggle. This means they can be present in the KineticBehavior struct but functionally inactive. The exact conditions for when they are enabled are handled separately in the swarm builder, making the behavior.rs file slightly opaque regarding the true runtime state of the network protocols.