Keyboard shortcuts

Press ← or → to navigate between chapters

Press S or / to search in the book

Press ? to show this help

Press Esc to hide this help

Proxy Base, Tunneling, and Security

Crate: kinetic-daemon Stage: 9 Reading time: 30 minutes Depends on: 16_proxy_http.md

What Is This?

This file covers the foundational architecture of the local HTTP/HTTPS proxy server inside the kinetic-daemon crate. Specifically, it documents the proxy entry point (mod.rs), the TLS termination and upgrading mechanism (tunnel.rs), and the network security perimeters designed to prevent abuse (security.rs and proxy_tests.rs).

Inside the Kinetic network architecture, the daemon acts as the critical gateway between the user’s standard web browser (like Google Chrome, Mozilla Firefox, or Safari) and the decentralized .kin peer-to-peer network.

To achieve this seamless integration without requiring users to install custom browsers, the daemon runs a local HTTP proxy server that intercepts outgoing traffic directly from the browser’s network stack.

This documentation explains exactly how that server boots up, how it handles the deeply complex process of decrypting HTTPS traffic via Man-In-The-Middle (MITM) TLS termination, and how it filters out dangerous local network requests to prevent Server-Side Request Forgery (SSRF) attacks.

This is the system that takes raw TCP streams from a web browser, decrypts the TLS layer cryptographically on the fly, and prepares the inner HTTP requests for decentralized P2P routing.

It is the very edge of the Kinetic network on the local machine. It represents the boundary where legacy web protocols (HTTP/TCP) are translated into Kinetic’s modern cryptographic peer-to-peer overlay.

Why Kinetic Needs This

The local proxy is the mandatory bridge between legacy internet protocols and the Kinetic network. Without these modules, the browser would have no way to talk to .kin websites, because standard web browsers do not natively understand P2P networking protocols, Distributed Hash Tables (DHTs), or peer ID routing.

Here is why each individual component is necessary for the daemon to function:

The Server Loop (mod.rs)

We need a continuous listener bound to a local port (for example, 127.0.0.1:8080) that the user’s browser is configured to use as its proxy. This listener must be resilient. It must gracefully handle operating system quirks (like IPv6-only environments where IPv4 bindings silently fail, which is common in modern containerized deployments). It must also be concurrent, capable of handling thousands of parallel connections asynchronously using Tokio’s event loop without blocking the main thread. If this server loop crashes, the user loses all access to the .kin network.

TLS Termination (tunnel.rs)

When a browser navigates to a secure site like https://something.kin, it expects to negotiate a secure TLS connection. If the daemon simply routed those encrypted bytes blindly over the P2P network, the target node wouldn’t know what to do with them, because the target node expects Kinetic’s internal P2P encryption, not a browser’s TLS encryption. More importantly, the daemon itself wouldn’t be able to inspect the HTTP headers to determine the destination peer ID. The routing information is inside the HTTP Host header, which is locked inside the TLS payload. The daemon must decrypt the traffic locally. It does this by acting as a local Man-In-The-Middle (MITM). It dynamically generates a forged TLS certificate for something.kin, performs a full cryptographic TLS handshake with the browser, and extracts the plain HTTP request from the decrypted tunnel.

SSRF Protection (security.rs)

The daemon is a powerful agent. It operates with unrestricted network access privileges on the user’s local machine. If a malicious .kin website, or a compromised P2P peer, could trick the daemon into making requests to the user’s home router (e.g., 192.168.1.1) or local cloud metadata APIs (169.254.169.254), the attacker could steal sensitive data, read AWS/GCP administrative credentials, or pivot into the internal network. This is a classic Server-Side Request Forgery (SSRF) vulnerability. Strict SSRF rules ensure the daemon never routes traffic to private or restricted IP spaces, sandboxing the daemon’s network access to the public internet and the P2P overlay.

How It Works

This system operates in distinct, sequential phases: bootstrapping the server, intercepting the HTTP connection, upgrading to a TCP tunnel, terminating the TLS encryption, and verifying strict security boundaries.

1. Bootstrapping the Proxy Server (mod.rs)

The entry point for this entire subsystem is the start_proxy_server function. Its primary job is to bind a TCP listener to the host network and spawn asynchronous Tokio tasks for every incoming connection.

-> See: crates/kinetic-daemon/src/proxy/mod.rs — Lines 46 to 74

The Binding Loop and IPv6 Fallback: The function attempts to bind a tokio::net::TcpListener to the IP address specified in the Kinetic configuration (usually 127.0.0.1). However, network stacks can be temperamental. In some Docker environments or specific OS configurations (referred to in the code comments as “Case 198: IPv6 Only Network Support”), IPv4 loopback binding outright fails and throws an error.

To mitigate this race condition or stack limitation, the code uses a for loop that tries up to 10 times, with a 200ms delay between attempts. If binding to the configured IPv4 IP fails repeatedly, it immediately attempts to bind to [::1] (the standard IPv6 loopback address) on the same port. This intelligent fallback ensures the daemon can boot successfully even on strict IPv6-only machines without requiring manual user intervention or complex configuration.

Handling Connections with Tokio and Hyper: Once the TcpListener is active, it enters an infinite loop, passively waiting for a client connection via listener.accept().await. When a browser connects, it produces a raw TCP stream. This stream is wrapped in a TokioIo::new(stream) struct. This wrapper is necessary because Hyper 1.x (the HTTP library Kinetic uses) is runtime-agnostic. It requires IO streams to implement specific generic traits, and TokioIo bridges Tokio’s asynchronous IO traits with Hyper’s strict generic expectations.

A new Tokio task is spawned using tokio::task::spawn for every single connection. Inside this asynchronous task, http1::Builder::new().serve_connection(...) takes control. It uses a service_fn to pass the raw HTTP request directly to handle_proxy_request. Crucially, it chains .with_upgrades(). This function call signals to Hyper that this standard HTTP connection might ask to be upgraded to a raw TCP tunnel — which is exactly what happens when the browser sends a CONNECT request for HTTPS proxying.

2. TLS Termination and The Nested Server (tunnel.rs)

When the browser wants to talk HTTPS, it doesn’t immediately start sending TLS bytes. First, it sends an HTTP CONNECT request to the proxy, asking for a direct pipe to the target domain (e.g., CONNECT domain.kin:443 HTTP/1.1). Once the proxy agrees (sending a 200 OK), the connection is formally “upgraded”. It ceases to be an HTTP connection and becomes a raw, bidirectional byte pipe.

-> See: crates/kinetic-daemon/src/proxy/tunnel.rs — Lines 20 to 57

The handle_connect function manages this delicate state transition.

Step A: Minting the Certificate Before the daemon can talk TLS to the browser over this raw pipe, it needs an X.509 certificate for the requested domain. It locks the LeafCertCache and calls get_or_create(&raw_host, &root_ca). This dynamically generates a certificate signed by the daemon’s local Root CA.

Architectural Detail: Notice that the cache.lock().await call happens inside an explicit synchronous block { ... }.

#![allow(unused)]
fn main() {
let server_config = {
    let mut cache = leaf_cache.lock().await;
    cache.get_or_create(&raw_host, &root_ca)?
}; // Lock released here
}

This is a critical Rust concurrency pattern. Although Kinetic uses tokio::sync::Mutex (which technically can be held across an .await point without compiler errors), holding a lock across a slow network operation like a TLS handshake is a massive anti-pattern. If the lock was held during acceptor.accept(...).await, then no other browser connection could generate a certificate until this specific client finished its cryptographic handshake. It would stall the entire proxy server. By encapsulating the lock in a block, the lock is dropped immediately after the certificate is retrieved and before the asynchronous handshake yields.

Step B: The Cryptographic TLS Handshake With the server_config (which contains the newly forged certificate), the code creates a tokio_rustls::TlsAcceptor. It takes the raw upgraded TCP stream, wraps it again in TokioIo, and awaits acceptor.accept(...). This performs the actual cryptographic handshake with the browser, negotiating cipher suites, verifying the Root CA, and establishing the encrypted tunnel.

Step C: The Nested Hyper Server Once acceptor.accept resolves successfully, we have a tls_stream. This stream contains the decrypted, plaintext HTTP traffic. But how do we parse it? We don’t write a custom HTTP parser, which would be error-prone and dangerous. Instead, we literally spin up a second Hyper HTTP server inside the connection!

#![allow(unused)]
fn main() {
http1::Builder::new()
    .serve_connection(TokioIo::new(tls_stream), service)
    .await?;
}

This nested HTTP server uses its own inner service_fn to take the decrypted Request<Incoming> and pass it to forward_to_backend_direct. If the backend routing fails, it guarantees a response by returning a 502 Bad Gateway. Notice the use of std::convert::Infallible in the return type of the closure. This tells Rust’s type system that this specific closure will never panic or return an unhandled error; it will always successfully yield an HTTP Response, even if that response represents a hard failure.

3. Network Security and SSRF Protection (security.rs & proxy_tests.rs)

Because the daemon executes network requests on behalf of the user, it must rigorously verify that the destination IP address is safe to query.

-> See: crates/kinetic-daemon/src/proxy/security.rs — Lines 3 to 5

The is_ssrf_risk function simply delegates to kinetic_core::net::is_ssrf_safe and negates the result. This gatekeeper ensures that traffic is never routed to restricted internal network ranges.

Property-Based Testing with proptest! To guarantee the safety of the SSRF filter, security.rs utilizes property-based testing via the proptest! macro. Unlike a standard unit test that checks a single hardcoded IP (e.g., 127.0.0.1), a property test generates thousands of random inputs that conform to specific rules.

For example, to test that all loopback addresses are rejected, the test defines the generation ranges:

#![allow(unused)]
fn main() {
a in 127u8..=127, b in 0u8..=255, c in 0u8..=255, d in 0u8..=255
}

The fuzzer continuously generates random IP addresses matching the 127.*.*.* pattern and asserts that is_ssrf_risk returns true every single time. It repeats this logic for the 10.*.*.* internal range, and similarly ensures that public IPs (e.g., ranges starting with 1 to 9) are always allowed. This provides a much higher degree of confidence than standard unit tests, effectively fuzzing the security boundaries.

Static Boundary Testing In proxy_tests.rs, specific edge cases and boundaries are tested explicitly. -> See: crates/kinetic-daemon/src/proxy/proxy_tests.rs — Lines 21 to 30

This file verifies that Carrier-Grade NAT (CGNAT) addresses (100.64.0.1) are blocked. Why is this important? Because cloud providers like AWS and GCP often use CGNAT spaces to host sensitive instance metadata APIs. If a cloud-hosted daemon was subjected to an SSRF attack, the attacker could extract AWS IAM credentials by querying the metadata API. It also blocks Link-Local addresses (169.254.169.254 and fe80::1), ensuring complete coverage against standard local network attacks.

Key Pieces

  • start_proxy_server (in mod.rs) The primary boot sequence for the local proxy. It handles IP binding, IPv6 fallback loops, and spawns the primary asynchronous connection handlers using Tokio. It is the backbone of the local interceptor.

  • ProxyError (in mod.rs) A centralized thiserror enum that standardizes failure states across the proxy module. It covers DNS failures (NameNotFound), Hyper library errors, CA generation failures, and generic IO faults, providing a clean abstraction for error propagation.

  • handle_connect (in tunnel.rs) The core Man-In-The-Middle logic. It takes an upgraded TCP stream from a CONNECT request, mints a TLS certificate on the fly using the leaf cache, completes the cryptographic handshake, and spawns a nested Hyper server to handle the decrypted internal traffic.

  • is_ssrf_risk (in security.rs) A boolean check that validates IP addresses against known private, local, and restricted subnets to prevent malicious network pivoting and unauthorized internal data access.

  • proptest! fuzzers (in security.rs) Property-based tests that exhaustively generate thousands of IP combinations to prove the SSRF filters have no logical gaps or edge-case failures.

How This Connects to the Rest of Kinetic

  • CROSS-CRATE: kinetic_core::config::KineticConfig — Determines the IP and port the proxy server attempts to bind to during initialization.

  • CROSS-CRATE: kinetic_core::net::is_ssrf_safe — Provides the low-level logic for IP address categorization used by security.rs.

  • CROSS-CRATE: kinetic_network::NetworkClient — Passed through the proxy and tunneling layers so that decrypted HTTP requests can eventually be routed over the P2P network to the correct peer.

  • Same-Crate Reference: LeafCertCache and RootCa (from the ca module) These are used directly by tunnel.rs to dynamically forge the TLS certificates required for HTTPS interception without triggering browser security warnings.

Quick Reference

  • IPv6 Fallback: If binding to the configured IPv4 address fails during boot, the proxy loops and automatically falls back to [::1] (IPv6 loopback).

  • TokioIo Wrapper: Hyper 1.x requires IO streams to implement specific traits. TokioIo wraps standard Tokio streams to make them compatible with Hyper’s engine.

  • Nested Servers: HTTPS interception works by upgrading the HTTP connection to a raw TCP stream, doing a TLS handshake, and running a second Hyper HTTP server over the decrypted bytes.

  • Mutex Lock Scope: When fetching certificates in async contexts, the Mutex lock is constrained to a synchronous block { ... } so it is dropped before the .await on the TLS handshake, preventing deadlocks.

  • SSRF Ranges Blocked: 127.0.0.0/8, 10.0.0.0/8, 172.16.0.0/12, 192.168.0.0/16, 100.64.0.0/10 (CGNAT/Metadata), and 169.254.0.0/16 (Link-Local).

Open Questions / Things to Revisit

  • Performance of Nested Servers: Spawning a new, second Hyper serve_connection for every single HTTPS request is functionally brilliant and separates concerns, but it might incur minor memory and CPU overhead. Profiling under high load (thousands of concurrent .kin requests) would determine if this architecture is too heavy for low-end mobile devices.

  • HTTP/2 & HTTP/3 Support: The nested server in tunnel.rs uses http1::Builder. As the Kinetic network grows, upgrading the decrypted inner streams to multiplexed HTTP/2 might yield significant performance benefits for complex .kin websites that load many assets simultaneously.

  • Binding Retry Loop: The 10-iteration loop for port binding (with 200ms sleeps) in mod.rs is functional but somewhat arbitrary. A more robust state machine or explicit network interface query might be a cleaner architectural choice in the future, rather than relying on a blind retry mechanism.

4. Deep Dive: Why Mutexes and Awaits Don’t Mix

You might look at the lock scoping in tunnel.rs and wonder why it was written with such explicit blocks. This is a fundamental Rust concurrency concept.

In a normal synchronous program, you lock a mutex, do your work, and unlock it. In Tokio, tasks are multiplexed over a pool of operating system threads. When you call .await on a future (like a network request or a TLS handshake), Tokio might decide to park your current task and run something else on that thread. Later, when the network request finishes, Tokio might wake your task up on a different thread.

The tokio::sync::MutexGuard technically can be held across an .await point, unlike std::sync::MutexGuard. However, just because you can doesn’t mean you should.

If you write code like this:

#![allow(unused)]
fn main() {
let mut cache = leaf_cache.lock().await; let cert = cache.get_or_create(&raw_host, &root_ca)?; // ... keeping the lock alive ... let tls_stream = acceptor.accept(io).await?; // AWAIT POINT
}

The proxy will compile, but it will suffer performance degradation. The .await on the TLS handshake can take hundreds of milliseconds if the network is slow. If the lock on the global LeafCertCache is held during this time, every single other incoming HTTPS connection will be blocked waiting for the lock to be released.

To fix this, the developer must ensure the lock is dropped before the .await. This is achieved by wrapping the lock acquisition in its own lexical scope:

#![allow(unused)]
fn main() {
let server_config = {
    let mut cache = leaf_cache.lock().await;
    cache.get_or_create(&raw_host, &root_ca)?
}; // Lock is guaranteed to be dropped right here. // Now it's safe to await. let tls_stream = acceptor.accept(io).await?;
}

This pattern is utilized in tunnel.rs to allow the concurrent proxy to mint certificates without blocking the entire async executor pool or angering the borrow checker.