Incident

Missed customer endorsements

2 customer endorsements were missed.

Error in the endorser shows:

β”‚ tezos-endorser-with-remote-signer-pb002 Sep 18 04:49:29 - 006-PsCARTHA.baking.endorsement: Error while injecting endorsement for delegate tz<snip> :                             
β”‚ tezos-endorser-with-remote-signer-pb002 Sep 18 04:49:29 - 006-PsCARTHA.baking.endorsement:   Error:                                                                                                          
β”‚ tezos-endorser-with-remote-signer-pb002 Sep 18 04:49:29 - 006-PsCARTHA.baking.endorsement:     Found no ledger corresponding to ledger://<snip>/ed25519/0h/1h.                           
β”‚ tezos-endorser-with-remote-signer-pb002 Sep 18 04:49:29 - 006-PsCARTHA.baking.endorsement:                                                                                                                   
β”‚ tezos-endorser-with-remote-signer-pb002 Sep 18 04:49:29 - 006-PsCARTHA.baking.endorsement:                                                          

Observation:

The ledger url in the error message is different from the Ledger url configured.

Diagnosis of the remote signer:

The right ledger url appears in /home/tezos/.tezos-client/secret_keys

However the wrong ledger url appears in /home/tezos/.tezos-signer/secret_keys

Corrective actions taken

  • fix the configuration error manually
  • add a check in the signer wrapper to mark the signer as offline when such configuration error occurs

Commit of check:

https://github.com/midl-dev/tezos-remote-signer-os/commit/daf8869d6cdda259e0023370a597014dd5d33860