Related Alerts

high-p2p-out

Alert Severity

SEV3 as Safe Head progresses

Notice if this heightened activity continues, it could lead to a safe head stall which is SEV2-SEV1 depending on the outage timeout.

Alert Summary

The alert means the sequencer has high p2p activity (of > 100kb/s).

As an example, here you can see a spike from a nominal < 20kb/s activity to well above 100kb/s.

If enough data is spammed, a safe head halt will begin as the batcher can’t catch up to the pending blocks. This will produce a ‣ as seen on the on-call during ‣.

Investigation

1. Via the P2P dashboard, confirm if the p2p bandwidth remains elevated, over 100kb/s (or whatever the alert is for). During times of network congestion or general spam, peer gossipping can be heightened but will be managed by the node’s pubsub thresholds.

Inlining comment from @Proto:

the spike looks weird, but in absolute terms not too high. I think it may start spiking when the sequencer is publishing a lot of new blocks in quick succession when catching back up with the head? Nodes have a limited message window, do de-duplicate messages, and limit blocks per block-height, so I don’t think a spike can persist for very long.

Mitigation

1. For invalid gossip, proper peer scoring will ban these malicious peers.
2. For valid gossip where someone is just spamming the network, the peer may still be banned

References

• op-node's p2p gossiping thresholds in gossip.go