Skip to content

refactor(pkg/p2p): swap GossipSub by FloodSub #3263

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
julienrbrt merged 2 commits into from
Apr 17, 2026
Merged

refactor(pkg/p2p): swap GossipSub by FloodSub #3263

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
28ec4f9
refactor(pkg/p2p): swap GossipSub by FloodSub
julienrbrt Apr 17, 2026
c599936
docs + cl
julienrbrt Apr 17, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion AGENTS.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -65,7 +65,7 @@ The project uses a zero-dependency core package pattern:

### P2P Architecture

- Built on libp2p with GossipSub and Kademlia DHT
- Built on libp2p with FloodSub and Kademlia DHT
- Nodes advertise capabilities (full/light, DA layers)
- Automatic peer discovery with rendezvous points

Expand Down
1 change: 1 addition & 0 deletions CHANGELOG.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -11,6 +11,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0

### Changes

- Improve P2P gossiping by switching pubsub internals from `GossipSub` to `FloodSub` [#3263](https://github.com/evstack/ev-node/pull/3263)
- Add `sequencer_blocks_synchronized_total` Prometheus counter metric tracking blocks synced by source (DA/P2P) [#3259](https://github.com/evstack/ev-node/pull/3259)
- Make it easier to override `DefaultMaxBlobSize` by ldflags [#3235](https://github.com/evstack/ev-node/pull/3235)
- Add solo sequencer (simple in memory single sequencer without force inclusion) [#3235](https://github.com/evstack/ev-node/pull/3235)
Expand Down
20 changes: 10 additions & 10 deletions docs/adr/adr-003-peer-discovery.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -11,12 +11,12 @@ Libp2p provides multiple ways to discover peers (DHT, mDNS, PubSub peer exchange
## Proposed network architecture

1. There will be a set of well-known, application-agnostic seed nodes. Every Evolve client will be able to connect to such node, addresses will be saved in configuration.
- This does not limit applications as they can still create independent networks with separate set of seed nodes.
- This does not limit applications as they can still create independent networks with separate set of seed nodes.
2. Nodes in the network will serve DHT. It will be used for active peer discovery. Client of each optimistic network will be able to find other peers in this particular network.
- All nodes will cooperate on the same DHT.
- ChainID will be used to advertise that client participates in a particular optimistic network.
- All nodes will cooperate on the same DHT.
- ChainID will be used to advertise that client participates in a particular optimistic network.
3. Nodes from multiple networks will help with peer discovery (via single DHT).
4. After connecting to nodes found in DHT, GossipSub will handle peer lists for clients.
4. After connecting to nodes found in DHT, FloodSub will handle peer lists for clients.

### Pros

Expand All @@ -30,11 +30,11 @@ Libp2p provides multiple ways to discover peers (DHT, mDNS, PubSub peer exchange
## Alternatives

1. Joining public IPFS DHT for peer discovery.
- pros: large network - finding peers should be very easy
- cons: we may affect public IPFS network stability in case of misconfiguration, possibly lot of unrelated traffic
- pros: large network - finding peers should be very easy
- cons: we may affect public IPFS network stability in case of misconfiguration, possibly lot of unrelated traffic
2. Custom peer-exchange protocol.
- pros: full flexibility of implementation
- cons: need to create from scratch and test
- pros: full flexibility of implementation
- cons: need to create from scratch and test
3. Re-use of existing peer discovery mechanism like `discv5`
- pros: ready & battle-tested software
- cons: use different network stack, requires lot of integration
- pros: ready & battle-tested software
- cons: use different network stack, requires lot of integration
28 changes: 14 additions & 14 deletions docs/overview/architecture.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -41,13 +41,13 @@ Evolve uses a modular architecture where each component has a well-defined inter

The block package is the heart of ev-node. It's organized into specialized components:

| Component | Responsibility | Runs On |
|-----------|---------------|---------|
| **Executor** | Produces blocks by getting batches from sequencer and executing via execution layer | Aggregator only |
| **Reaper** | Scrapes transactions from execution layer mempool and submits to sequencer | Aggregator only |
| **Syncer** | Coordinates block sync from DA layer and P2P network | All nodes |
| **Submitter** | Submits blocks to DA layer and tracks inclusion | Aggregator only |
| **Cache** | Manages in-memory state for headers, data, and pending submissions | All nodes |
| Component | Responsibility | Runs On |
| ------------- | ----------------------------------------------------------------------------------- | --------------- |
| **Executor** | Produces blocks by getting batches from sequencer and executing via execution layer | Aggregator only |
| **Reaper** | Scrapes transactions from execution layer mempool and submits to sequencer | Aggregator only |
| **Syncer** | Coordinates block sync from DA layer and P2P network | All nodes |
| **Submitter** | Submits blocks to DA layer and tracks inclusion | Aggregator only |
| **Cache** | Manages in-memory state for headers, data, and pending submissions | All nodes |

### Component Interaction

Expand Down Expand Up @@ -81,12 +81,12 @@ The block package is the heart of ev-node. It's organized into specialized compo

Evolve supports several node configurations:

| Type | Block Production | Full Validation | DA Submission | Use Case |
|------|-----------------|-----------------|---------------|----------|
| **Aggregator** | Yes | Yes | Yes | Block producer (sequencer) |
| **Full Node** | No | Yes | No | RPC provider, validator |
| **Light Node** | No | Headers only | No | Mobile, embedded clients |
| **Attester** | No | Yes | No | Soft consensus participant |
| Type | Block Production | Full Validation | DA Submission | Use Case |
| -------------- | ---------------- | --------------- | ------------- | -------------------------- |
| **Aggregator** | Yes | Yes | Yes | Block producer (sequencer) |
| **Full Node** | No | Yes | No | RPC provider, validator |
| **Light Node** | No | Headers only | No | Mobile, embedded clients |
| **Attester** | No | Yes | No | Soft consensus participant |

### Aggregator

Expand Down Expand Up @@ -158,7 +158,7 @@ User Tx → Execution Layer Mempool

Built on libp2p with:

- **GossipSub** for transaction and block propagation
- **FloodSub** for transaction and block propagation
- **Kademlia DHT** for peer discovery
- **Topics**: `{chainID}-tx`, `{chainID}-header`, `{chainID}-data`

Expand Down
20 changes: 10 additions & 10 deletions pkg/p2p/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -51,12 +51,12 @@ type P2PConfig struct {

### Configuration Parameters

| Parameter | Description | Default | Example |
|-----------|-------------|---------|---------|
| ListenAddress | The address where the node listens for incoming P2P connections | `/ip4/0.0.0.0/tcp/7676` | `/ip4/0.0.0.0/tcp/7676` |
| Seeds | Comma-separated list of seed nodes (bootstrap nodes) | "" | `/ip4/1.2.3.4/tcp/7676/p2p/12D3KooWA8EXV3KjBxEU...,/ip4/5.6.7.8/tcp/7676/p2p/12D3KooWJN9ByvD...` |
| BlockedPeers | Comma-separated list of peer IDs to block | "" | `12D3KooWA8EXV3KjBxEU...,12D3KooWJN9ByvD...` |
| AllowedPeers | Comma-separated list of peer IDs to explicitly allow | "" | `12D3KooWA8EXV3KjBxEU...,12D3KooWJN9ByvD...` |
| Parameter | Description | Default | Example |
| ------------- | --------------------------------------------------------------- | ----------------------- | ------------------------------------------------------------------------------------------------ |
| ListenAddress | The address where the node listens for incoming P2P connections | `/ip4/0.0.0.0/tcp/7676` | `/ip4/0.0.0.0/tcp/7676` |
| Seeds | Comma-separated list of seed nodes (bootstrap nodes) | "" | `/ip4/1.2.3.4/tcp/7676/p2p/12D3KooWA8EXV3KjBxEU...,/ip4/5.6.7.8/tcp/7676/p2p/12D3KooWJN9ByvD...` |
| BlockedPeers | Comma-separated list of peer IDs to block | "" | `12D3KooWA8EXV3KjBxEU...,12D3KooWJN9ByvD...` |
| AllowedPeers | Comma-separated list of peer IDs to explicitly allow | "" | `12D3KooWA8EXV3KjBxEU...,12D3KooWJN9ByvD...` |

## libp2p Components

Expand All @@ -69,7 +69,7 @@ graph LR
Client --> DHT["Kademlia DHT"]
Client --> Discovery["Routing Discovery"]
Client --> Gater["Connection Gater"]
Client --> PubSub["GossipSub PubSub"]
Client --> PubSub["FloodSub PubSub"]

Host --> Transport["Transport Protocols"]
Host --> Security["Security Protocols"]
Expand All @@ -96,7 +96,7 @@ graph LR
- Used for finding other peers within the same network
- Bootstrapped with seed nodes defined in configuration

3. **GossipSub**: Publish-subscribe protocol for message dissemination
3. **FloodSub**: Publish-subscribe protocol for message dissemination
- Used for gossiping transactions, headers, and blocks
- Provides efficient message propagation with reduced bandwidth overhead
- Supports message validation through custom validators
Expand All @@ -118,7 +118,7 @@ sequenceDiagram
participant P as P2P Client
participant H as libp2p Host
participant D as DHT
participant G as GossipSub
participant G as FloodSub
participant N as Network/Other Peers

A->>P: NewClient(config, chainID, datastore, logger, metrics)
Expand Down Expand Up @@ -163,7 +163,7 @@ The P2P clients in full and light nodes handle transaction validation differentl

## Message Gossiping

Messages (transactions, blocks, etc.) are gossiped through the network using GossipSub topics. The topic format is:
Messages (transactions, blocks, etc.) are gossiped through the network using FloodSub topics. The topic format is:

`<chainID>+<topicSuffix>`

Expand Down
2 changes: 1 addition & 1 deletion pkg/p2p/client.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -450,7 +450,7 @@ func (c *Client) tryConnect(ctx context.Context, peer peer.AddrInfo) {

func (c *Client) setupGossiping(ctx context.Context) error {
var err error
c.ps, err = pubsub.NewGossipSub(ctx, c.host)
c.ps, err = pubsub.NewFloodSub(ctx, c.host)
Copy link
Copy Markdown
Contributor

@coderabbitai coderabbitai bot Apr 17, 2026 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟠 Major

Confirm the scalability tradeoff of switching to FloodSub.

FloodSub forwards every message to every connected peer with no mesh or gossip optimization, so per-message bandwidth scales linearly with peer fan-out. GossipSub was specifically designed to bound this cost via mesh degree (D) + gossip. For a blockchain P2P network propagating headers/blocks/data, this change can materially increase egress bandwidth and duplicate deliveries as the network grows, and it removes score-based peer protection against spam/amplification that GossipSub provides.

Since the PR description doesn't include rationale or benchmarks, could you confirm:

  1. Expected peer count per node in production — is fan-out bounded enough that flooding is acceptable?
  2. Whether this is intended as a temporary debug aid (the source branch is p2p-debug) or a permanent change.
  3. Any measurements showing FloodSub behaves acceptably for the largest expected message sizes (e.g., blocks/data blobs).

Also note that downstream consumers such as pkg/sync/sync_service.go pass pubsub.DefaultMsgIdFn to goheaderp2p.NewSubscriber; that should still work (it's a generic PubSub helper, not GossipSub-specific), but message deduplication characteristics will differ in practice under FloodSub's flood semantics.

🤖 Prompt for AI Agents 
Verify each finding against the current code and only fix it if needed.

In `@pkg/p2p/client.go` at line 453, Switching c.ps from pubsub.NewGossipSub to
pubsub.NewFloodSub (the change at pubsub.NewFloodSub(ctx, c.host)) increases
per-message egress and removes GossipSub's mesh/gossip and scoring protections;
update the PR by either reverting to GossipSub or adding explicit justification
and measurements: state intended lifetime (temporary debug branch p2p-debug vs
permanent), expected production peer fan-out per node, and benchmark results
showing FloodSub bandwidth/duplicate delivery behavior for worst-case message
sizes (blocks/data blobs). Also mention impacts on downstream consumers such as
pkg/sync/sync_service.go which use goheaderp2p.NewSubscriber with
pubsub.DefaultMsgIdFn and explain how deduplication/delivery semantics will
change under FloodSub. Ensure this rationale is added to the PR description
and/or a comment near the pubsub.NewFloodSub call (c.ps) so reviewers can verify
the tradeoff.

Copy link
Copy Markdown
Contributor

@Wondertan Wondertan Apr 17, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is expected and something I suggested. It does indeed increase bandwidth requirements; however, it simplifies the system substantially as well when it comes to propagation issue debugging. The gossipsub is an overkill for the scale evolve networks are deployed at and if the network ever grows to hundreds of peers we can always migrated back to gossipsub in a backwards compatible manner

Copy link
Copy Markdown
Contributor

@coderabbitai coderabbitai bot Apr 17, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@Wondertan, thanks for the clear rationale — that makes complete sense. FloodSub is a reasonable fit for small, bounded networks where simplicity and debuggability outweigh bandwidth efficiency, and the backwards-compatible migration path to GossipSub if the network scales is a pragmatic safety net. I'll note this for future reviews.

ʕ •ᴥ•ʔ

✏️ Learnings added 
Learnt from: Wondertan
Repo: evstack/ev-node PR: 3263
File: pkg/p2p/client.go:453-453
Timestamp: 2026-04-17T10:00:53.638Z
Learning: In evstack/ev-node, the p2p layer intentionally uses FloodSub (pubsub.NewFloodSub) instead of GossipSub. This is a deliberate design decision suggested by Wondertan: FloodSub simplifies propagation debugging, and GossipSub is considered overkill for the small peer counts (well below hundreds) at which evolve networks are deployed. Migration back to GossipSub is considered feasible in a backwards-compatible manner if the network ever grows significantly. Do not flag this as a scalability concern.

Note: Learnings are effective only in the context of similar code segments. To apply general coding guidelines, please configure review instructions. You can manage existing learnings in the app.

🧠 Learnings used 
Learnt from: alpe
Repo: evstack/ev-node PR: 3146
File: block/internal/syncing/da_follower.go:128-156
Timestamp: 2026-03-12T14:41:30.304Z
Learning: In evstack/ev-node, DA priority heights (queued via `QueuePriorityHeight` in `block/internal/syncing/da_follower.go`) are untrusted, best-effort optimizations sourced from P2P hints. Dropping a hint on a transient fetch failure is intentional — the sequential catchup loop in `da.Subscriber` will cover every height eventually. Do not flag missing retry/re-queue logic for priority hints as a bug.

Learnt from: alpe
Repo: evstack/ev-node PR: 3131
File: block/internal/syncing/syncer_backoff_test.go:241-245
Timestamp: 2026-03-06T09:40:36.029Z
Learning: In evstack/ev-node, the module declares go 1.25.6. Since Go 1.22, loop variables are per-iteration by default, so loop variable capture is not a concern. Do not flag or fix loop variable capture in this codebase for any Go files; this rule applies to all Go files in the repository, not just the specific test file.

if err != nil {
return err
}
Expand Down
Loading