> For the complete documentation index, see [llms.txt](https://swappulse.gitbook.io/swappulse-docs/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://swappulse.gitbook.io/swappulse-docs/network-and-web3/full-node.md). # Full node and full observer Run and verify the experimental SwapPulse Madara full-observer node safely. A SwapPulse **full observer** maintains its own chain database, follows the network, verifies the state it receives and serves Starknet read RPC from locally verified data. It is the node role for operators who want stronger assurance than forwarding requests to somebody else's RPC. {% hint style="warning" %} Full-observer support is currently an engineering and node-lab capability. The live `SWAPPULSE_TESTNET` still uses the separate Shardlabs Starknet Devnet runtime. The tested `SWAPPULSE_NODELAB_1` observer has one sequencer and does not provide permissionless consensus or independent-operator decentralisation. {% endhint %} ## What this node does The current Madara observer can: * synchronise confirmed blocks and state from the node-lab sequencer; * maintain its own persistent database; * verify the configured chain ID and reproduce block hashes at a common confirmed height; * expose a local Starknet JSON-RPC interface for verification and application reads; * reproduce deployed V2 contract class hashes and application state; * restart from its preserved volume and catch up; * operate without Base44, the transaction relay or privileged application keys. It does **not**: * produce blocks merely because it runs in full mode; * participate in permissionless consensus or finality; * become a validator through the current `StakingPool`; * hold user keys, identity evidence or Base44 private records; * replace an archive indexer for rich historical search; * make its raw RPC safe for public exposure. ## Node roles compared | Role | Stores and verifies chain state | Produces blocks | Typical use | | ----------------- | ------------------------------- | ------------------------ | ---------------------------------------------------- | | Full observer | Yes | No | Independent reads, state verification and resilience | | Testing sequencer | Yes | Yes, in the isolated lab | Orders and produces node-lab blocks | | Archive/indexer | Yes, plus extended indexes | No | Explorer history, search and analytics | | Lite node | No complete state database | No | Low-resource checks across one or more RPC peers | ## Current tested implementations ### Madara Stage A `chain/node/full` follows public Starknet Sepolia to qualify the client and hardware. It answers whether a host can run Madara under a real synchronisation workload. It does not make the host a SwapPulse full node. The current guarded profile uses: * loopback RPC `127.0.0.1:19944`; * an immutable Madara image digest; * a persistent Docker volume; * a 3 GiB memory limit, two logical CPUs and a 512 PID limit; * no automatic restart during qualification. The reference Intel N95 mini-server passed the short restart and recovery test with Madara `v0.11.0-alpha.9`, pinned to digest `sha256:3c931fa515bbd3760fd5cbc0bcdceb557d3edbd44bec0231cdf52dd6abb475f6`. Extended Sepolia sync also revealed substantial storage I/O and write amplification, so this is not a production full-history support claim. ### SWAPPULSE\_NODELAB\_1 full observer `chain/node/nodelab` runs the first SwapPulse-specific two-node topology: | Component | Host endpoint | Purpose | | ----------------- | ------------------------------- | ----------------------------------------------------------------------- | | Testing sequencer | `127.0.0.1:19950` | Produces blocks for the isolated development network | | Full observer | `127.0.0.1:19951` | Synchronises into a separate database with no sequencer or deployer key | | Feeder gateway | Docker bridge only, port `8080` | Supplies state from sequencer to observer, never published to the host | The network identity is: ``` SWAPPULSE_NODELAB_1 0x5357415050554c53455f4e4f44454c41425f31 ``` On 4 September 2026, the observer reproduced the permanent V2 flag, identity assurance and staking state with the same block hash as the sequencer. The lite verifier then reached `multi-peer-agreement` across the two separate databases. Later fault tests stopped the observer and sequencer separately. In both cases, the lite verifier returned HTTP `503` from `/readyz` and `/rpc` rather than trusting the surviving peer alone. Agreement recovered automatically after the stopped node restarted from its preserved volume. Both nodes were on the same physical host, so these results prove separate state, fail-closed read behaviour and restart recovery. They do not prove separate operator control or permissionless consensus. ### Stage D: physically separate full observer Stage D first passed its two-host observer and canary gates on 6 September 2026. On 7 September, the tested topology was promoted to two restricted, reboot-managed lite services and passed a controlled primary-host reboot. The primary mini-server remains the only block-producing node-lab sequencer. A second machine synchronises as a keyless Madara full observer and preserves its own database. {% hint style="success" %} Stage D proves **physical-host state-source independence** for `SWAPPULSE_NODELAB_1`. It does not prove independent operators, permissionless consensus or validator decentralisation. Both tested machines were administered by the same operator, so `operator_independence` correctly remained `false`. {% endhint %} #### Verified topology | Component | Private or local endpoint | Role | | ------------------------- | ------------------------------ | --------------------------------------------------------------- | | Primary testing sequencer | `127.0.0.1:19950` | Produces blocks for the isolated node lab | | Primary feeder gateway | `:19952` | Supplies the remote observer over the private overlay | | Same-host full observer | `127.0.0.1:19951` | Retained as a tested fallback state source | | Remote full observer | `:19961` | Maintains a separate persistent database without signing keys | | Same-host lite fallback | `127.0.0.1:18101` | Reboot-managed verifier using the sequencer and local observer | | Cross-host lite verifier | `127.0.0.1:18102` | Reboot-managed verifier using the sequencer and remote observer | Neither the feeder gateway nor the remote observer RPC was exposed to the public Internet. Both were bound to reviewed Tailscale IPv4 addresses. #### Evidence that passed The guarded workflow passed all seven Stage D gates: * the remote observer returned the exact `SWAPPULSE_NODELAB_1` chain ID; * it reached confirmed state and reproduced primary checkpoint block `80536` with hash `0x5ca9365c6917f23caa48d7f5d6abf9dacc5f7d293752cc29bf6b93468d6169e`; * it reproduced the canonical permanent V2 deployment pins and `verification_v2_required=true`; * its container command and environment contained no sequencer, deployer, registry-owner, verifier or user private key; * its named volume survived container removal and restart; * after restart, block `81467` retained hash `0x18e87acf1ed780a06beb7b759a0bbde9c9a307cd203ddfb1313c60f740e08e`, while the observer continued to a later height; * the cross-host lite canary reached `multi-peer-agreement`, with two healthy peers, two contract-pin checks and matching block hashes. The initial canary advanced while retaining agreement. It was then packaged as the durable `18102` verifier, while `18101` became a restricted Docker-managed same-host fallback. A controlled reboot recovered the sequencer, both observers, both lite verifiers and the three live testnet services automatically. All seven recorded container identities were retained, and block `133443` with hash `0x60bf26c5e8d8bf1a14ea97f1d13b1d05187f3748331fb6bde5362f16f34cf99` remained available from all three full-node state sources after reboot. The scoped RPC verification fix is recorded in [`b6aba9ad`](https://github.com/beitmenotyou1/swappulse2/commit/b6aba9ade8829ef00933a61b51c4eb95b76a06f2), the opt-in Tailscale lite-peer policy in [`2d2254d9`](https://github.com/beitmenotyou1/swappulse2/commit/2d2254d90fe1e336106a5890ca11a352ef741059), the durable verifier package in [`475bb0c6`](https://github.com/beitmenotyou1/swappulse2/commit/475bb0c62216dccec40b7d0d2086e1ae87a7e548), and reboot support in [`eef1b5d7`](https://github.com/beitmenotyou1/swappulse2/commit/eef1b5d710d540521e191fab62fef2987f61aab7). #### Run the guarded workflow Use the scripts from a reviewed checkout. Replace placeholders with the two hosts' actual Tailscale IPv4 addresses. If the node-lab environment lives outside the checkout, set `NODELAB_DIR` to its real `chain/node/nodelab` directory before running the primary scripts. {% stepper %} {% step %} #### Preflight and enable the primary gateway From `chain/node/stage-d` on the primary host: ```bash export NODELAB_STAGE_D_TAILSCALE_IP= bash primary-gateway-preflight.sh NODELAB_CONFIRM_STAGE_D_GATEWAY=YES \ bash enable-primary-gateway.sh bash create-primary-checkpoint.sh ``` The enable script refuses non-Tailscale binds, checks the existing node lab, recreates only the node-lab sequencer when required, waits for same-host observer agreement to recover and rechecks the separate live services. {% endstep %} {% step %} #### Prepare the remote observer On the second host: ```bash cd swappulse2/chain/scripts/tooling npm ci cd ../../node/stage-d/remote-observer cp .env.example .env.remote ``` Edit only the public settings in `.env.remote`: the primary gateway URL, the remote Tailscale bind and the reviewed immutable Madara image. Do not copy `.env.local` or any authority material from the primary host. {% endstep %} {% step %} #### Start and verify the remote observer Copy the public `stage-d-primary-checkpoint.json` to the second host, then run: ```bash bash preflight.sh .env.remote bash start.sh .env.remote bash verify.sh \ /absolute/path/to/chain \ /absolute/path/to/stage-d-primary-checkpoint.json \ .env.remote ``` Success requires the chain ID, checkpoint hash, V2 manifest and key-separation checks to pass. A running container on its own is not sufficient evidence. {% endstep %} {% step %} #### Prove database persistence Stop only the remote observer, confirm that the named volume remains, restart it and repeat the full verification: ```bash bash stop.sh .env.remote docker volume ls | grep swappulse-nodelab-1-stage-d-remote_remote-observer-data bash start.sh .env.remote bash verify.sh \ /absolute/path/to/chain \ /absolute/path/to/stage-d-primary-checkpoint.json \ .env.remote ``` Also query at least one block captured before the restart and confirm that its hash is unchanged. {% endstep %} {% step %} #### Start the durable verifiers and prove reboot recovery Use [Stage D Multi-host Operations](/swappulse-docs/network-and-web3/stage-d-operations.md) to configure the reviewed `lite-service` package on `18102` and the same-host `reboot-support` fallback on `18101`. Both services must remain loopback-only, use `restart: unless-stopped`, pass their status scripts and disclose `operator_independence: false`. Before accepting the deployment, record a block checkpoint, reboot only the primary host, wait without manually starting services, then prove automatic container recovery and retention of the older block across all three full-node state sources. {% endstep %} {% endstepper %} {% hint style="warning" %} The successful test did not turn the remote observer into a validator or make the network decentralised. Preserve the same-host observer and managed fallback, keep both Stage D RPC surfaces private, and treat loss of the remote peer as a fail-closed event for the cross-host verifier. {% endhint %} ## Host requirements Use a dedicated or carefully resource-limited 64-bit Linux host. | Resource | Current guidance | | -------- | --------------------------------------------------------------------------------------------------------------------------------------------------------- | | CPU | Four cores recommended for the reference lab; the supplied node-lab caps each node at two CPUs | | Memory | 16 GB on the reference shared host; each node-lab container is capped at 2 GiB | | Storage | SSD or NVMe with monitored free space and write endurance; do not use a low-end SD card for sustained chain writes | | Software | Docker Engine, Docker Compose, Git, Bash and standard command-line tools | | Network | Stable outbound connectivity; same-host RPCs stay on loopback, while Stage D additionally requires a reviewed private-overlay route between the two hosts | | Time | Working system time synchronisation | Pi 4 and Pi 5 devices remain candidate hardware. Do not describe them as supported full observers until their restart, catch-up, storage, thermal and multi-day soak tests pass. ## Host the tested node-lab observer {% stepper %} {% step %} ### Obtain the current repository ```bash git clone https://github.com/beitmenotyou1/swappulse2.git cd swappulse2/chain/node/nodelab ``` If the repository already exists, use its normal authenticated update workflow and review the changes before starting containers. {% endstep %} {% step %} ### Prepare local configuration ```bash bash prepare-nodelab.sh bash preflight-nodelab.sh ``` Preparation creates `.env.image` and a mode-`0600` `.env.local`. They contain the immutable image selection and fresh node-lab-only keys. Never commit, print or reuse those keys on the live testnet. {% endstep %} {% step %} ### Start and verify the sequencer ```bash bash start-sequencer.sh bash verify-sequencer.sh ``` The verification must return the exact node-lab chain ID and at least one confirmed block before you start the observer. {% endstep %} {% step %} ### Start the full observer ```bash bash start-observer.sh bash verify-nodelab.sh ``` The final verifier checks both chain IDs, compares the block hash at the common confirmed height, confirms the observer has no sequencer private key, and verifies that the separate live SwapPulse services remain healthy. {% endstep %} {% step %} ### Inspect the local RPCs ```bash curl -fsS http://127.0.0.1:19950 \ -H 'content-type: application/json' \ --data '{"jsonrpc":"2.0","id":1,"method":"starknet_chainId","params":[]}' curl -fsS http://127.0.0.1:19951 \ -H 'content-type: application/json' \ --data '{"jsonrpc":"2.0","id":2,"method":"starknet_blockNumber","params":[]}' ``` For a meaningful comparison, use `verify-nodelab.sh`. Two different block heights alone do not prove disagreement because the observer may be catching up. {% endstep %} {% step %} ### Stop without deleting state ```bash bash stop-nodelab.sh ``` Normal stop preserves both named volumes. Do not add `-v` to a Compose down command unless you deliberately intend to destroy the lab databases and have already preserved the required evidence. {% endstep %} {% endstepper %} ## V2 contracts in the node lab The tested lab contains the same audited V2 contract suite as the application baseline, deployed with fresh lab-only authorities. Its canonical manifest is: ``` chain/deployments/swappulse-nodelab-1.json ``` Never import node-lab addresses into the live Base44 `ChainNetworkConfig`. The live and lab networks have different chain IDs, deployments, keys and operational purposes. The deployment, assurance exercise and irreversible V2 cut-over scripts are engineering harnesses. Run them only from a clean, tested `chain/` workspace and only when you deliberately intend to reproduce the development-network evidence. A failed cut-over command must be investigated on-chain before any retry because the one-way transaction may already have committed. ## Operating guidelines ### Keep the RPC private Raw Madara RPC should remain on `127.0.0.1` by default. Stage D has two narrow exceptions: the primary feeder gateway and remote observer RPC bind only to the hosts' reviewed Tailscale IPv4 addresses. Neither service should be exposed to the public Internet. If users need public reads, place the [read-only RPC gateway](/swappulse-docs/apis/read-only-rpc-gateway.md) in front of a reviewed upstream and publish only the gateway through HTTPS. ### Monitor the host Watch: * container state and restart count; * confirmed head and observer lag; * free disk space, database growth and I/O pressure; * memory availability and sustained swap activity; * CPU temperature and throttling on small hardware; * RPC latency and error rate; * successful restart and catch-up after maintenance. Useful commands: ```bash docker compose ps docker stats --no-stream df -h vmstat 1 5 bash verify-nodelab.sh ``` ### Preserve reproducibility * Pin the full container image by immutable digest. Never benchmark `latest`. * Preserve the public network and deployment manifests with any backup. * Keep sequencer and observer databases separate. * Test an upgrade against a copied or fresh volume before touching the only working state. * Record the image digest, chain ID, block height and verification result for each qualification. ## Security checklist Never place any of these on a community full observer: * relay bearer token; * registry-owner or verifier private key; * user smart-account private key; * Base44 secret or session credential; * private age-verification evidence; * AT Protocol app password; * Cloudflare account-wide credential. A full observer validates public protocol state. It must not depend on private Base44 data to decide whether a state transition is valid. ## Troubleshooting | Symptom | Check | Safe response | | ------------------------------------ | ------------------------------------------------------------- | --------------------------------------------------------------------------- | | Observer height trails the sequencer | Container logs, feeder-gateway reachability and disk pressure | Allow catch-up, then rerun `verify-nodelab.sh` | | Chain IDs differ | `.env.local`, chain override and image digest | Stop the lab and correct configuration. Do not merge or copy databases | | Common-height hashes differ | Exact common height, image version and database integrity | Treat as a verification failure and isolate the affected state | | RPC is unreachable | Loopback port ownership and `docker compose ps` | Fix the local bind or container health. Do not widen the bind to `0.0.0.0` | | Disk is nearly full | Database size and Docker volume location | Stop cleanly and expand or migrate storage before corruption risk increases | | New image fails against old data | Migration notes and clean-volume result | Preserve the old volume as evidence and retest with a fresh volume | ## Related pages * [Lite node](/swappulse-docs/network-and-web3/lite-node.md) * [Stage D Multi-host Operations](/swappulse-docs/network-and-web3/stage-d-operations.md) * [Read-only RPC gateway](/swappulse-docs/apis/read-only-rpc-gateway.md) * [Transaction relay](/swappulse-docs/apis/transaction-relay-api.md) * [SwapPulse Node Architecture Roadmap](/swappulse-docs/network-and-web3/node-architecture.md) * [Cairo and Starknet Chain Overview](/swappulse-docs/network-and-web3/chain-overview.md)