Keyboard shortcuts

Press ← or → to navigate between chapters

Press S or / to search in the book

Press ? to show this help

Press Esc to hide this help

kinetic-core/build.rs & High-Level Integration Tests

Crate: kinetic-core Stage: 19 Reading Time: ~45 minutes Depends On: kinetic-core/src/types.rs, kinetic-core/src/constants.rs, network.json

What Is This?

This documentation serves as a comprehensive guide to the build-time configuration engine. It covers the high-level security integration tests for the kinetic-core crate. The kinetic-core/build.rs file is a Cargo build script. In Rust, a build.rs file is compiled and executed before the main crate is built. It has the power to generate source code dynamically. It interacts with the host environment to detect variables and capabilities. It dictates exactly how the main crate will be compiled. You mentioned protobuf/capnp compilation logic in your prompt. It is important to note that as the codebase has evolved, this is no longer accurate. build.rs no longer handles .proto or .capnp schemas. Instead, its primary and exclusive responsibility is parsing the network.json configuration file. It translates this JSON file into static, optimized Rust constants. If Kinetic were to reintroduce Cap’n Proto or Protobuf for RPC or on-wire serialization, their build directives would be restored here. They would sit alongside the JSON parsing logic using prost-build or capnpc. In addition to the build script, this document analyzes the security-focused integration tests. These tests are located in the kinetic-core/tests/ directory. While unit tests are typically co-located with their source code inside src/, the tests/ directory is used for black-box integration testing. These tests interact with the crate exactly as an external consumer would. They verify critical security invariants. Invariant 1: Preventing Out-Of-Memory (OOM) attacks from bloated network payloads. Invariant 2: Preventing adversaries from hijacking subdomains to spoof identities. Invariant 3: Stopping protocol downgrade attacks, where an attacker tricks a node into using an older, vulnerable parsing standard.

Why Kinetic Needs This

1. The Necessity of Build-Time Configuration

Kinetic is designed to be a flexible, decentralized protocol. Different deployments require different sets of physical rules. The global mainnet needs strict sybil resistance. The global mainnet needs high redundancy. A local developer testnet needs fast consensus. A local developer testnet needs low VDF (Verifiable Delay Function) iterations. A local developer testnet needs reduced redundancy for rapid iteration. These rules are defined in a human-readable network.json file. However, parsing this JSON file at runtime presents several massive disadvantages. Performance Overhead: Parsing JSON at runtime requires memory allocation. It requires string matching and error handling every time a node boots. This slows down startup times, especially on embedded devices. Lost Optimization Opportunities: If configuration values are runtime variables, the Rust compiler cannot optimize them. It cannot inline them into the assembly. It cannot unroll loops based on them. It cannot optimize away dead branches. State Inconsistency: If the JSON file is modified while the node is running, it could cause issues. If a user accidentally deploys a node with a missing JSON file, the node crashes at runtime. This potentially leads to consensus failures across the network. Dependency Bloat: If we parsed JSON at runtime, the final compiled binary would need to bundle a full JSON parsing library. This would pull serde_json into the final runtime binary, increasing the binary size. By using build.rs to parse the JSON file at compile time, we solve all these problems simultaneously. Zero-Cost Abstractions: The configuration values become hardcoded, static const primitives. They are baked into the final binary. The compiler optimizes them into raw assembly values. Fail-Fast Compilation: If network.json is malformed, missing fields, or contains invalid types, the compiler panics. The build fails before a single line of runtime code is compiled. You can never accidentally deploy a misconfigured node. Security Floor Guarantees: The build script actively inspects the values. For example, if a developer sets the DHT redundancy level too low, the build script aborts the compilation. This enforces security policies at the compiler level. Network Isolation: The build script exposes the NETWORK_ID to the compiler environment. This ensures that different forks of the network cannot accidentally gossip with one another. They cannot share P2P topic strings.

2. The Necessity of Security Integration Tests

While the build script ensures the node is configured safely, the integration tests ensure that the core data structures behave securely. They must be tested against adversarial data. P2P networks operate in a zero-trust environment. A Kinetic node will receive bytes from unknown IP addresses all over the world. If the node blindly trusts those bytes, it will be destroyed. The integration tests simulate specific, known attack vectors. They test against the protocol’s core data structures, like Reveal. They ensure that our validate() and signable_bytes() methods are robust. They ensure malicious data is dropped before it reaches the consensus or storage engines. We do this in the tests/ directory rather than inside src/ because integration tests compile the crate exactly as an external user would. This guarantees that we aren’t relying on private internal functions to achieve security. The public API itself is secure by default.

How It Works: The Build Script

The build script operates in several distinct phases. It starts from locating the configuration file. It ends by finally emitting valid Rust source code.

Phase 1: Defining the Deserialization Schema

-> See: kinetic-core/build.rs — Lines 6 to 97

The script uses the serde framework to map the unstructured JSON data. It maps it into typed Rust structures. Every field in network.json has a corresponding struct definition here. Here is a breakdown of the specific structs: 1. SquatterMultipliers This struct defines the multiplier curves for the Verifiable Delay Function (VDF). Short domain names (like kin) are valuable. To prevent domain squatters from snapping them up instantly, Kinetic requires a multiplied VDF proof. This proof is required to register short names. This struct maps domain lengths to their specific VDF multipliers. It parses len_0_to_1 for 0-1 character domains. It parses len_2 for 2 character domains. It parses len_3 for 3 character domains. It parses len_4 for 4 character domains. It parses len_5 for 5 character domains. It parses len_6 for 6 character domains. It parses len_7 for 7 character domains. It parses len_8_to_10 for 8 to 10 character domains. It parses len_11_to_17 for 11 to 17 character domains. It parses len_18_to_20 for 18 to 20 character domains. 2. ConsensusConfig This defines the core consensus rules. It includes the minimum_commit_age_kyns. This is how many Drand epochs must pass between a Commit and a Reveal. This prevents front-running on the network. It includes the vdf_squatter_multipliers which wraps the previous struct. It includes vdf_discount_min_iterations. It includes vdf_discount_percentage. It includes the absolute iteration caps for the VDF (vdf_max_iterations). It includes the size limits for proofs (vdf_max_proof_bytes). 3. LimitsConfig This is crucial for node stability. It defines strict memory and sizing limits. It parses p2p_max_packet_size. It parses p2p_max_circuit_bytes. It parses proxy_max_body_bytes. It parses storage_max_value_bytes. It parses kid_max_public_key_bytes. It parses kid_max_location_bytes. It parses kid_max_endpoint_bytes. It parses drand_max_response_bytes. It parses lru_cache_size. By baking these limits into the binary at compile time, the node can pre-allocate memory safely. 4. TimeoutsConfig Defines network resilience parameters. It parses idle_timeout_seconds. This is how long a node can be idle before being dropped. It parses heartbeat_age_warning_seconds. It parses heartbeat_age_critical_seconds. It parses dns_cache_ttl_seconds. This is how long a DNS cache entry remains valid. It parses network_prune_interval_seconds. It parses host_route_max_age_seconds. 5. NetworkSection It parses tld (Top Level Domain). It parses base_domain. It parses network_id. It parses docs_url. It parses ipfs_gateway. It parses local_bind_ip. It parses bootstrap_nodes (a list of initial P2P contacts). 6. DrandSection It parses drand_genesis_time. It parses drand_period. It parses kinetic_genesis_drand_kyn. It parses drand_public_key. It parses drand_http_endpoints. 7. GovernanceSection It parses governance_model. It parses max_age_seconds. 8. AdvancedSection It parses benchmark_base_iterations. It parses benchmark_target_minutes. It parses steal_target_kyns. It parses m_redundancy. It parses dev_mode_iterations. It includes the LimitsConfig and TimeoutsConfig. 9. NetworkConfig Finally, this struct ties them all together as the root object. It implements Deserialize. If the network.json file contains a string "100" where the LimitsConfig expects an integer 100, the serde deserializer will immediately fail. It will halt the build. This provides immense peace of mind.

Phase 2: Dependency Tracking and File Resolution

-> See: kinetic-core/build.rs — Lines 100 to 116

Cargo is designed to be lazy. It caches build artifacts aggressively. It only recompiles when necessary. If build.rs simply read a file using standard I/O, Cargo wouldn’t know that the file was a dependency. Changing the JSON file wouldn’t trigger a rebuild of the Rust code. This would lead to stale configurations in the final binary. To fix this, the script issues special directives to standard output. println!("cargo:rerun-if-changed=../network.json"); println!("cargo:rerun-if-env-changed=KINETIC_NETWORK_JSON"); These directives tell Cargo to invalidate the cache if these targets change. Next, the script must locate the JSON file. It uses a strict fallback hierarchy. First, it checks Environment Override. It checks if KINETIC_NETWORK_JSON is set via std::env::var. This allows CI/CD pipelines to inject custom testing configurations dynamically. Second, it checks Workspace Root. It looks for ../network.json. This is the standard location for developers working within the repository. Third, it checks Bundled Fallback. It looks for default_network.json in the current directory as a last resort. If all three fail, it panics. It throws the error: "Failed to find network.json in any location."

Phase 3: Parsing and Code Generation Setup

-> See: kinetic-core/build.rs — Lines 118 to 127

The script reads the file contents into a string. It passes it to serde_json::from_str. Assuming this succeeds, it now holds a fully populated NetworkConfig object in memory. It then queries the environment for OUT_DIR. When Cargo runs a build script, it creates an isolated, temporary directory for generated files. It passes its path via this environment variable. The script constructs a path to OUT_DIR/network_constants.rs. It initializes a mutable String named out. The rest of the script is responsible for appending valid Rust source code to this string.

Phase 4: Emitting Constants

-> See: kinetic-core/build.rs — Lines 128 to 362

The script uses the format! macro to generate pub const definitions. It meticulously injects doc-comments (///). This ensures that the generated code is self-documenting. When you use constants::TLD elsewhere in the codebase, your IDE will display the documentation generated here. Here are the key constants generated: TLD and TLD_SUFFIX: Defines the top-level domain (e.g., kin). DID_PREFIX: Defines the Decentralized Identifier prefix (e.g., did:kin:). BASE_DOMAIN: Base infrastructure domain. NETWORK_ID: The unique identifier isolating P2P protocols. BASE_ITERATIONS: The hardware anchor for the VDF. The script documents that lowering this on mainnet is dangerous. TARGET_MINUTES: Time target for iterations. STEAL_TARGET_KYNS: The decay rate for name stealing. It determines how long a name must be inactive before it can be reclaimed. M_REDUNDANCY: The DHT replication factor. GOVERNANCE_MODEL: The swappable governance engine. LOCAL_BIND_IP: The local IP address where services bind. MAX_AGE_SECONDS: Governance proposal expiry. DEV_MODE_ITERATIONS: Iterations for dev/sim mode. CONSENSUS_MINIMUM_COMMIT_AGE_KYNS: Drand commit age rules. DRAND_GENESIS_TIME: Drand chain genesis timestamp. DRAND_PERIOD: Duration of Drand epochs. KINETIC_GENESIS_DRAND_KYN: Kinetic genesis epoch. KINETIC_GENESIS_TIME: Absolute kinetic genesis timestamp. DRAND_PUBLIC_KEY: LoE public key. DOCS_URL: Documentation URL. IPFS_GATEWAY: IPFS gateway URL. It also loops through arrays to generate slices. It generates DRAND_HTTP_ENDPOINTS as a static array slice &[&str]. It generates BOOTSTRAP_NODES as a static array slice &[&str]. It unwraps all the limits from the LimitsConfig. It writes them out as LIMITS_... constants. It unwraps all the timeouts from TimeoutsConfig. It writes them out as TIMEOUTS_... constants.

Phase 5: The Eclipse Attack Safety Floor

-> See: kinetic-core/build.rs — Lines 168 to 184

This is a masterclass in using build scripts for security engineering. The script reads config.advanced.m_redundancy. This variable controls the Kademlia DHT replication factor. It dictates how many distinct, independent nodes must store a piece of data. If an attacker surrounds a node with malicious peers, they can censor data. This is known as an Eclipse attack. High redundancy makes this statistically improbable. The canonical mainnet uses a redundancy of 32. The build script enforces a hard floor. If m_redundancy < 5, it deliberately panics. This is a compile-time security guarantee. It is physically impossible to produce a Kinetic binary that is catastrophically vulnerable to basic Eclipse attacks. The compiler itself refuses to build it. If a developer tries to compile with m_redundancy = 4, the build instantly fails with this panic message.

Phase 6: Compiler Environment Injection

-> See: kinetic-core/build.rs — Lines 236 to 245

While generating a file is useful, sometimes we need values directly injected into the compiler’s environment variables. The script uses the cargo:rustc-env directive. It exposes KINETIC_NETWORK_ID directly to the rustc compiler. It exposes KINETIC_NETWORK_ID_UPPER directly to the rustc compiler. Why do this? Because in cryptographic functions, we often use the concat! macro to build domain separators. For example: concat!(env!("KINETIC_NETWORK_ID"), "-reveal"). The concat! macro is processed at compile time. It only works with string literals and environment variables. It cannot read constants from a generated file. By exposing the network ID as an environment variable, we enable zero-cost string concatenation for cryptographic prefixes.

Phase 7: Writing to Disk

-> See: kinetic-core/build.rs — Lines 363 to 364

Finally, the script calls fs::write. It dumps the massive generated string into network_constants.rs. The kinetic-core crate will later use include!(concat!(env!("OUT_DIR"), "/network_constants.rs")). This pulls this code into its module tree, completing the build-time generation cycle.

How It Works: The Integration Tests

The tests in kinetic-core/tests/ are designed to attack the data structures generated and validated by the core protocol. They verify that they hold up under duress.

1. The OOM Payload Bomb Test

-> See: kinetic-core/tests/test_003_oom_payload_bomb.rs

The Vulnerability: In Rust, Vec<u8> is dynamically sized. When deserializing network data, a node reads a header. If that header claims the payload is 500MB, the default behavior of many serializers is to allocate 500MB of RAM immediately. If an attacker spams 100 of these headers, the node attempts to allocate 50GB of RAM. The operating system’s OOM killer will terminate the node process. This is a trivial remote Denial-of-Service (DoS) attack. The Test Execution: The test constructs an attack vector. let oversized_payload = vec![0u8; MAX_PAYLOAD_SIZE + 1]; It creates a payload exactly one byte larger than the legally permitted MAX_PAYLOAD_SIZE. It then embeds this payload inside a dummy Reveal struct. This mocks a network message a node might receive. When the test calls reveal.validate(), it asserts that the result is_err(). The Guarantee: This test proves that the validate() method correctly enforces the upper bounds on memory allocation. It does this before the payload can be forwarded to the storage layer. It does this before it reaches the consensus engine. It does this before it reaches the signature verifier. The malicious data is dropped instantly. This preserves node stability and prevents memory exhaustion.

2. The Subdomain Hijack Test

-> See: kinetic-core/tests/test_021_subdomain_hijack.rs

The Vulnerability: The Kinetic namespace is flat. Unlike the traditional DNS system where com owns google.com which owns mail.google.com, Kinetic only recognizes apex domains. If a user registers satoshi.kin, they have absolute control over it. If the protocol validation logic was sloppy and permitted internal periods (e.g., blog.satoshi.kin), an attacker could register that subdomain independently. Users seeing blog.satoshi.kin would naturally assume it belongs to the owner of satoshi.kin. This would destroy the trust model of the decentralized namespace. It would lead to widespread phishing and identity spoofing. The Test Execution: The test creates two Reveal scenarios to verify the string validation logic. The Attack: It constructs an invalid_reveal targeting blog.saifmukhtar.kin. Because this name contains a subdomain (indicated by the internal period before the TLD suffix), it is malformed according to network rules. It calls .validate(). It asserts that the function returns an error. The protocol must reject any attempt to register a subdomain. The Baseline: It constructs a valid_reveal targeting saifmukhtar.kin (the apex domain). It calls .validate(). It asserts that the function succeeds, returning Ok(). The Guarantee: This verifies that the string parsing logic embedded in Reveal::validate() is impenetrable to hierarchical domain spoofing. It ensures the mathematical flatness of the namespace is enforced at the earliest possible stage of packet processing.

3. The Protocol Downgrade Test

-> See: kinetic-core/tests/test_029_protocol_downgrade.rs

The Vulnerability: Cryptographic signatures guarantee that data was authorized by a specific private key. However, signatures are “dumb”. They only prove authorization for the exact bytes provided to the signing algorithm. They have no concept of context or intent. Imagine Kinetic launches Protocol Version 1. Later, a severe flaw is found in how V1 parses payloads. The network upgrades to Protocol Version 2. Users begin signing Version 2 transactions. If the signature process does not include the version number in the signed bytes, an attacker could intercept a valid V2 transaction on the wire. They could modify the unencrypted version byte to 0 (representing V1). They could broadcast it to nodes that haven’t updated yet. Because the signature remains valid for the rest of the payload, the downgrade attack succeeds. The attacker exploits the V1 flaw using a signature intended for V2. The Test Execution: This test ensures cryptographic agility and domain separation. It proves that signatures are irrevocably bound to their specific protocol version. First, it creates a legitimate V1 Reveal. It computes the exact byte array the user must sign using reveal_v1.signable_bytes(env!("KINETIC_NETWORK_ID")). Second, it simulates the attacker. It clones the Reveal. It manually mutates protocol_version to 0. It computes the new byte array bytes_v0. Third, it asserts assert_ne!(bytes_v1, bytes_v0). Because changing the version number fundamentally alters the bytes output by signable_bytes, the attacker’s forged V0 message will fail signature verification. The attacker does not possess the private key required to generate a valid signature for the newly computed bytes_v0 array. Furthermore, the test inspects the exact byte layout. let prefix = concat!(env!("KINETIC_NETWORK_ID"), "-vdf-reveal-v1").as_bytes(); assert_eq!(bytes_v1[prefix.len()], 1); assert_eq!(bytes_v0[prefix.len()], 0); It verifies that the network ID is acting as a Domain Separator. This prevents replay attacks between mainnet and testnet. It verifies that the version byte is injected into the prefix. Finally, it confirms that reveal_v0.validate() fails outright. The Guarantee: The network is immune to downgrade attacks. Every signature commits to the specific protocol version it was intended for. Replay attacks across different networks are impossible.

Key Pieces

NetworkConfig (and Sub-Structs)

What it is: The root deserialization schema mapping to network.json. Location: kinetic-core/build.rs — Lines 90 to 97 Why it matters: It acts as the canonical bridge between human-readable JSON configurations and machine-optimized Rust compilation. It provides strict type safety to the network parameters.

Eclipse Resistance Enforcement

What it is: The compile-time panic trigger for low DHT redundancy. Location: kinetic-core/build.rs — Lines 168 to 184 Why it matters: It demonstrates how build scripts can be used as active security policy enforcers. It overrides developer misconfigurations before they can compile.

cargo:rustc-env Injection

What it is: The mechanism exposing NETWORK_ID to the compiler. Location: kinetic-core/build.rs — Lines 236 to 245 Why it matters: Enables zero-cost, compile-time string concatenation for cryptographic domain separators using the concat! and env! macros. It avoids the need for runtime string allocations.

test_003_oom_payload_bomb()

What it is: The defensive integration test against unbounded memory allocation. Location: kinetic-core/tests/test_003_oom_payload_bomb.rs Why it matters: Proves that the Reveal struct’s validate() method successfully blocks the most prevalent class of P2P Denial-of-Service attacks. It stops memory exhaustion via malicious Vec sizing.

test_subdomain_hijack_validation()

What it is: The integration test verifying namespace flatness. Location: kinetic-core/tests/test_021_subdomain_hijack.rs Why it matters: Ensures the decentralized identity model cannot be undermined by hierarchical domain spoofing. It maintains the integrity of the naming system.

test_protocol_downgrade_prevention()

What it is: The cryptographic agility test ensuring signatures commit to protocol versions. Location: kinetic-core/tests/test_029_protocol_downgrade.rs Why it matters: Prevents replay attacks across different protocol versions and different network forks (mainnet vs testnet). It ensures forward security.

How This Connects to the Rest of Kinetic

CROSS-CRATE / FORWARD DEPENDENCIES: To kinetic-core/src/constants.rs: The build script writes network_constants.rs into the OUT_DIR. The constants.rs file within the main crate source tree uses the include! macro. It physically pulls this generated text into the module structure. This is the handoff point between build-time generation and runtime availability. To kinetic-core/src/types.rs: The integration tests act as the primary consumer and verifier of the logic defined in types.rs. The tests directly import RevealExt::validate() and RevealExt::signable_bytes(). This ensures that modifications to the core types do not break fundamental security invariants. To P2P Gossip (Future): The LIMITS_P2P_MAX_PACKET_SIZE generated by build.rs will be consumed by the networking crate. It will configure the Kademlia swarm and libp2p behavior. This ensures the physical transport layer drops oversized packets before they even reach the application layer.

Quick Reference

Build Script Goal: Parse network.json at compile-time to generate statically typed const variables. This guarantees zero-cost abstraction. It prevents invalid or insecure configurations from ever compiling. OUT_DIR: The temporary directory provided by Cargo where generated files are stored. This is where network_constants.rs is stored before being included in the main build. OOM Protection: Validation logic drops any payload > MAX_PAYLOAD_SIZE. Namespace Rules: The protocol is apex-only. Validation drops any name containing internal periods. Downgrade Protection: Altering a struct’s protocol version alters the output of signable_bytes(). This immediately invalidates any associated cryptographic signatures.

Open Questions / Things to Revisit

Missing Protobuf/Capnp Directives: As noted, your prompt mentioned protobuf compilation, but the current build.rs relies on Serde and JSON. If you are migrating back to Protobuf for wire efficiency, you need to be aware that kinetic-core currently does not compile them. If you maintain .proto schemas in a separate crate (like kinetic-rpc), they need to be compiled there. Re-adding them here would require importing prost-build. Scaling Integration Tests: The current integration tests focus on the Reveal struct. As the protocol expands to include Transfer, Update, and Revoke messages, we must ensure coverage scales. These same OOM and downgrade tests must be systematically applied to all new data structures to maintain uniform security. JSON Fallback Safety: The build.rs script defaults to default_network.json if a specific environment or workspace file is not found. While convenient for local development, this introduces a risk. A production builder might accidentally compile the fallback network instead of the intended mainnet parameters. We may want to add a strict mode flag to disable the fallback in production builds to enforce explicit configuration. KINETIC_NETWORK_ID in Tests: The downgrade test relies on env!("KINETIC_NETWORK_ID"). This works during standard cargo test runs because Cargo executes build.rs first and populates the environment variables. However, if a developer tries to run a single test manually through certain IDE debuggers without triggering the build script, it may fail. It may fail to find the environment variable. It is a minor friction point worth documenting for new contributors.