Time Sync Check

Check if your node clocks are properly synchronized

Issue: Node clocks are not synchronized

Symptoms

As with all advanced checks, there is no obvious symptoms until you have missed a block.

Check

To check if you are having this issue, simultaneously check the time of your nodes. You can also optionally check the activity time stamps of your producer and relay logs but this is not necessary.

Root Cause and Avoidance

If you are not using a time synchronization service, the time kept by your servers will slowly drift away from the actual network time. Please do not take this lightly as in extreme cases, this may lead to you failing to mint blocks and I have personally met an SPO who experienced this issue.

Note that in some cases, especially with servers running from Cloud providers, this issue may not appear as there may be underlying architecture that already synchronizes server times.

Nevertheless, it is always good practice to run a time synchronization tool like chrony to avoid time issues from happening.

Remediation

Install chrony on all of your nodes. You can follow this tutorial from Coincashew to do this:

Note that whether you have already missed blocks or not, it is always recommended to run chrony on all of your nodes.

Last updated