Triage a failed night

Last updatedView as Markdown

A night fails at one step, and the steps after it never run. This page finds that step, says what its message means, and how to run the night again. Every message below is the text the step saves, with the parts that change in angle brackets.

Guided diagnosis

Where did last night go?

Start from Maintenance › Overview, Maintenance › Runs, or wp sisyphus list.
Question 1
Question 1

Is there a run that started at or after the site's run time last night?

Find the step that failed

Maintenance › Overview puts the sites whose last run failed first, with the step that failed, its message, and its GitHub run when the step started one. Maintenance › Runs lists the 20 most recent runs; Steps opens every step's status and message, and links each step to what it led to. From WP-CLI, wp sisyphus list finds the run and wp sisyphus show prints its steps:

A night that failed at sync bash
wp sisyphus show 15
Run 15 for builditbelieveit.com: failed, started 2026-10-27 06:15:33 UTC.
step       status   started                  finished                 message
preflight  passed   2026-10-27 06:15:33 UTC  2026-10-27 06:15:33 UTC  Ready to run builditbelieveit.com: dev is linchpindev.linchpin.build (Pressable 1833215), with Safety Net.
sync       failed   2026-10-27 06:15:35 UTC  2026-10-27 06:21:34 UTC  The sync failed: All 3 database backup slots on production are still in use
cleanup    pending
update     pending
resolve    pending
deploy     pending
test       pending
snapshot   pending

The failed row is the step to read. The rows after it stay pending, because a failed step fails the run. Times are UTC.

Run the night again

A failed run is finished, and nothing resumes it. Once the cause is fixed, Run now on the site's Overview card or page, or wp sisyphus run <site>, starts a new run from Preflight. That run overwrites dev again, so the sync runs again too: about 25 minutes for builditbelieveit.com. Or leave it for the site's next nightly run.

Restart is for a run that is still going, not a failed one. It fails the run at the step it reached, asks GitHub to cancel the workflow that step started, and starts a new run. The stopped step's message says who restarted it and what was cancelled.

Messages any step can give

MessageWhat happenedWhat to do
Timed out after <time>.The step was still waiting at its deadline. Each step's deadline is in ArchitectureRead the message the step saved before it timed out, on the Runs screen, and look at what it was waiting on
<class>: <message>The step threw, and the engine turned it into a failureUsually a bug in Sisyphus; the class says where
No step named "<name>" is registered any more.The run was created with a step the sisyphus_steps filter no longer returnsStart a new run; it takes today's steps
Abandoned: no tick was queued to finish it.The run was unfinished with nothing queued to tick it, so the next run started on the site failed itNothing; the new run is the one to follow
Restarted by <name>. or Restarted by <name> from WP-CLI.A person restarted the run. What the step stopped follows, such as Asked GitHub to cancel Deploy to Dev (run <id>).Follow the new run

Messages by step

The steps run in this order. A step that waits on something says so while it waits, and an outage, a 429 or a 5xx from api.linchpin.com or GitHub waits and asks again rather than failing.

preflight

MessageWhat to do
Sisyphus is not connected to api.linchpin.com: define SISYPHUS_API_CLIENT_ID and SISYPHUS_API_CLIENT_SECRET in wp-config.php.Define them; see Constants
<site> is not enrolled in maintenance.Register the site on Maintenance › Sites, or with wp sisyphus enroll
<site> is enrolled but not enabled.Turn on Run nightly, or wp sisyphus enroll <site> --enable
<site> has no dev target.Choose a dev on the site's map or settings, or --dev=<pressable-id>
<site> names its production site (<id>) as dev, which the sync would overwrite.Choose a Pressable staging site as dev
<site> has no repository.Set the repository, or map one in packagist-plugin-manager
Could not check <site> in the registry: <error>api.linchpin.com refused the request; the error carries its status and reason
Safety Net is not on <dev>, so nothing would scrub the production data the sync copies there. Run Deploy to Dev, which installs it.Run the site repository's Deploy to Dev with safety_net: true, then run the night again
<dev> reports its environment as <type>, not development, so Safety Net would not run there.Set dev's WP_ENVIRONMENT_TYPE to development

Preflight waits up to 15 minutes for a registry that does not answer before it times out.

sync

MessageWhat to do
The sync failed: <reason>api.linchpin.com's sync stopped; the reason is its own. Run 15 above is one: Pressable keeps three on-demand backups of each kind per site, and the slot the sync freed was still in use when it gave up. The api now waits up to half an hour for a slot
api.linchpin.com would not start the sync: <error>The api refused, as it does for a dev that is no longer a Pressable staging site. Fix the enrollment
Could not read sync <uuid>: <error>The api refused to say how the sync went; look the sync up in the api
Timed out after 3 hours.The transfer outlasted the plan's clone timeout; check the transfer in Pressable

cleanup

MessageWhat to do
Safety Net is not on <dev>, so production's data there is not scrubbed. Run Deploy to Dev, which installs it.Run Deploy to Dev, then run the night again
<dev> reports its environment as <type>, not development.Set dev's WP_ENVIRONMENT_TYPE to development
SAFETY_NET_DELETE_DATA keeps the users and orders on <dev>; the nightly run needs them deleted.Remove that setting on dev
Send emails is switched on on <dev>, so its emails reach real people.Switch Safety Net's send emails off on dev
Timed out after 30 minutes., after Waiting for Safety Net on <dev>: <steps undone>.Safety Net did not finish scrubbing; load dev and check its status at /wp-json/safety-net/v1/status

The cleanup says what Safety Net does and how the step reads it.

Any workflow step

update, resolve, deploy and snapshot each run a GitHub Actions workflow in the site's repository: Rolling Maintenance (maintenance.yml), Deploy to Dev (deploy-dev.yml) and Snapshots (snapshots.yml).

MessageWhat to do
<workflow> ended <conclusion>: <url>The workflow ran and did not succeed. Open the run at the URL; the failure is in its log
GitHub would not start <workflow>: <error>GitHub refused the dispatch. Check the workflow file is on the branch and takes a correlation_id input, and that the branch exists
Sisyphus cannot reach GitHub: it uses packagist-plugin-manager's GitHub App, which is not active or not configured on this site.Activate packagist-plugin-manager and configure its GitHub App
Preflight handed on no repository, so there is nothing to run on GitHub.Set the site's repository
The enrollment names no branch to run <workflow> on.Set the site's branches
Could not read <workflow> runs: <error>GitHub refused to list the workflow's runs; check the App can read Actions on the repository
<workflow> passed, but its result could not be checked: <error>The workflow passed, but a follow-up read of the branches or PRs was refused

update

MessageWhat to do
<maintenance branch> is missing <n> commits of <production branch>: GitHub would not merge them and Rolling Maintenance could not settle the conflict, so it left a maintenance-conflict issue for a person.Resolve the conflict the issue describes, so maintenance has every commit of the production branch, then run again
The enrollment names no maintenance branch.Set the maintenance branch

When Rolling Maintenance published the conflicts it could not settle, update passes and resolve answers them.

resolve

MessageWhat to do
Rolling Maintenance left conflicts in <branch>, but Sisyphus cannot reach ai.linchpin.com: Mantle is not active here or has no gateway key.Activate Mantle on this site with its gateway key
Rolling Maintenance left conflicts in <branch>, but Sisyphus cannot reach GitHub to answer them.Configure packagist-plugin-manager's GitHub App
Left the conflict for a person: <why>Nothing could answer it: Sonnet would not combine a hunk, a file has more than 25 hunks, or its conflict markers could not be read. Resolve it by hand through the maintenance-conflict issue
<branch> has no usable .sisyphus/conflicts.json.Rolling Maintenance's handover is missing or malformed; check its run
Rolling Maintenance did not take the answers: <branch> is still missing <n> commits of <base>. Its run says why, and its issue is open for a person.An answer failed resolve-merge's checks; its run names the file
GitHub would not create <branch>: <error>GitHub refused to create the resolutions branch; check the App can write contents
Could not answer <file>: <error>GitHub or ai.linchpin.com refused while answering that file. The error says which

Resolve describes the whole handover.

deploy

MessageWhat to do
Deploy to Dev ended <conclusion>: <url>The deploy failed: a build, the transfer to dev, or Safety Net's check on every site. Its run says which

A deployment log entry that is missing, or a log that cannot be read, is noted in the message and never fails the step.

test

MessageWhat to do
<n> of <total> tests in <suite> failed against dev: <tests>.Open each suite's Ghost Inspector result from the Runs screen. A real regression waits for a fix; a flaky test waits for its suite to be fixed
<suite> ended without a result, as a cancelled run does.Someone or something cancelled the suite in Ghost Inspector; run the night again
<suite> could not run: <why>Ghost Inspector would not start the suite
api.linchpin.com would not start the tests: <error>The api has no Ghost Inspector key (503 tests_unavailable), or the client lacks tests:run
wp-config.php names no api.linchpin.com credentials, so no tests could be asked for.Define the credentials
Timed out after 1 hour.The suites were still running; check them in Ghost Inspector

A site with no nightly suite passes the step, saying nothing ran against dev. That is not a failure.

snapshot

MessageWhat to do
Snapshots ended <conclusion>: <url>tools/snapshot.sh freeze failed; its run says why
The enrollment names no branch to promote snapshots into.Set where the site promotes to
The deploy step saved no commit, so there is nothing to freeze.A run from before deploy saved its commit; run the night again

<target> already has every update in <sha>, so there is nothing to promote. is a pass: the night had nothing new.

A run that seems stuck

A running step that has stopped changing is waiting on something, and its deadline fails it in the end. To see whether anything will tick it, look under Tools › Scheduled Actions for a pending sisyphus_run_tick action in the sisyphus group with the run's id, or run wp action-scheduler run --group=sisyphus to run the ticks that are due now.

With no tick queued, nothing will finish the run, and the next run started on the site fails it as abandoned. With a tick queued but a step going nowhere, Restart stops it, cancelling the workflow it started.

A night that never started

wp sisyphus schedule lists every enabled site, whether it is due and why:

DetailMeaning
Run <id> started after its last run time. Next run at <time>.That night's run started; find it with wp sisyphus list
Next run at <time>.The run time passed more than six hours ago with no run, so that night is skipped rather than run in the working day
Due since <time>.The next check, within 15 minutes, starts it. wp sisyphus schedule --start starts it now
Its run time, <time> <zone>, is not a time Sisyphus can read.Fix the run time (HH:MM) or time zone

A paused site is not listed: set it to Run nightly. A missed night with no explanation means the check did not run, because packagist.linchpin.com was down or Action Scheduler was not running; the sisyphus_schedule_check action's log under Tools, Scheduled Actions has each check's outcome, including an api.linchpin.com that did not answer.

Have a question about this page?

Linchpin assistant

Was this helpful?