Related Alerts

Alert Severity

Severity depends on user impact. See ‣ for the appropriate severity level.

Alert Summary

This alert means that an L1 replica has diverged from the reference. The reference is usually an L1 infrastructure provider like Alchemy or Infura. This alert will fire when the L1 replica’s canonical chain does not match the reference for more than 5 minutes.

Investigation

  1. View the Replica Healthcheck dashboard to check if the replica is currently diverged. The red band on the graph below indicates a divergence.

    Screen Shot 2023-03-13 at 8.54.43 AM.png

  2. View the Geth dashboard to check if a reorg occurred. Spikes in the following graphs indicate a reorg:

    Screen Shot 2023-03-13 at 8.54.32 AM.png

Mitigation

  1. First, just wait. Geth usually processes reorgs within 10 minutes, and the divergence should solve itself.
  2. If the divergence persists, check the following:
    1. Check the reference’s status page (e.g., https://status.alchemy.com/) to make sure the reference is healthy.
    2. Make sure that the replica is running the latest version of the replica software. Failure to upgrade L1 replicas prior to a hardfork can lead to divergence.
    3. View the replica healthcheck logs for the affected replica to see where the diverged block is. Compare this diverged block with Etherscan, etc. to figure out why the divergence occurred.
  3. If the divergence is causing issues with L2 networks, switch to an infra provider while the issue is diagnosed.