Automation Center Reference Public

FAQ & Troubleshooting

Diagnose a stuck Automation Center: nothing ever becomes due, a new type will not appear in the builder, everything moved to Needs review, a run says success but changed nothing — plus the glossary and what to capture before escalating.

Guide version: r1 Module version: 0.3.3 Updated: 2026-09-04 Estimated time: 21 min 2 views

Reference

FAQ & Troubleshooting

You have a specific problem right now. Find it below, and read the why as well as the fix — most of these are the module behaving exactly as designed, and knowing which is which saves you from "fixing" a safety mechanism.

ℹ️

Almost every failure here is silent by design. The Automation Center is a scheduler that acts on payroll inputs, so wherever it is unsure it stops rather than guesses — and a thing that stopped produces no error page, because nobody was watching. The diagnosis is therefore nearly always reading state, not reading an error. Three screens hold that state: Overview (engine health), Runs (what happened), and Settings (what is registered).

Sixty-second triage

Before opening any question below, walk these five. In practice they resolve most reports, in this order, because each one makes the next meaningful.

Is the module on?
Settings → Manage Modules → Automation Center. The setting module_automations is seeded '0' — off. Off means the sidebar entry is gone, every route redirects, and the dispatch job returns immediately.
Is the automation active, with a next run?
Open the list. Only active automations that carry a next_run_at_utc in the past are ever picked up. draft, paused, needs_review and archived are all invisible to the dispatcher.
Is the dispatcher ticking?
php erpat cron:list must show automation_dispatch, and the Cron Manager must show recent runs for this tenant. The job is per-tenant (runsForGlobalScope() is false), so "cron works" on another tenant proves nothing here.
Is the type available?
Overview or Settings → the engine card. It lists the registry checksum, the available types, and a notices block naming everything the registry rejected and why. An empty list with no notices means no module contributed a type; a notice means one tried and failed.
Read the last run, not the automation.
The automation shows its current configuration. The run shows what actually executed — its own run_mode, its frozen config, its error code and its per-record items. When the two disagree, the run is the truth.

Nothing runs

My automation never becomes due ›

The dispatcher's claim query is narrow on purpose. From Automations_model::get_details(), an automation is due only when all three hold:

$where .= " AND $automations.status='active'";
$where .= " AND $automations.next_run_at_utc IS NOT NULL";
$where .= " AND $automations.next_run_at_utc<=" . $this->db->escape($due_before);

So work through them in that order:

Check	What you should see	If not
`module_automations`	Enabled in Manage Modules	The job self-gates on it and returns before doing anything. Nothing is logged as an error, because being switched off is not one.
Status	active	A saved automation starts as draft. It must be published to become active — saving is not publishing.
`next_run_at_utc`	A UTC timestamp	NULL means the pointer was cleared. That happens when the cron expression could not be parsed — see the next question.
The tick itself	`automation_dispatch` running every minute for this tenant	Advanced Cron is not driving this tenant. The module ships no scheduler of its own and cannot compensate.

One more, easily missed: an automation whose previous run is still open (queued or running) is skipped for this occurrence unless its overlap policy is queue_latest. The pointer still advances, and the dispatcher logs skipped: previous run still open. That is overlap protection working, not a stall — but if a run is stuck open, every later occurrence is quietly dropped.

Its next run went NULL and its status went to Needs review ›

The schedule could not produce another occurrence. From AutomationDispatchJob::advancePointer():

if ($next === null) {
    $CI->Automations_model->set_next_run((int) $automation->id, null);
    $CI->Automations_model->set_status((int) $automation->id, 'needs_review');
    return;
}

This is the deliberate alternative to a silent stall: an unparseable or never-matching cron expression surfaces as a status an operator can see, instead of an automation that simply never fires again. The classic cause is a date that cannot occur — 0 3 30 2 * (the 30th of February). Fix the expression, save, and publish again to restore active and a fresh pointer.

The dispatcher appears to have fired twice ›

Two cron drivers are enabled at once, and their locks do not intersect. This is a documented platform-level compromise, recorded as §20.5 of the implementation plan:

⚠

FlockLockProvider (the CLI driver) and DbLockProvider (the HTTP driver) use separate namespaces, so running both can double-fire the dispatcher. The operational rule is one driver only, and it belongs in your deploy runbook. Owner: the cron runtime, not this module.

Why you probably saw no damage. The blast radius is contained by the run idempotency key — a SHA-256 of automation_uuid | scheduled_for_utc | trigger_kind, held UNIQUE. The second driver's claim for the same occurrence resolves to the same key and creates no second run. Per-record writes are protected a second time by INSERT IGNORE on a UNIQUE (run_id, subject).

Note the key deliberately excludes the configuration revision, so editing an automation mid-tick cannot manufacture a second run of the same moment.

Fix: disable one driver. Pick the CLI worker or the HTTP endpoint, not both.

Types and the builder

My new automation type does not appear in the builder ›

Look at the engine card first. Overview and Settings both render $registry->skipped() as a notices block — the registry records the exact reason it rejected each definition, and reading it beats guessing. If the notices block is empty and your type is still absent, the sidecar was never seen at all, which narrows it to the first two causes below.

Cause	Notice you would see	Fix
The owner module is off	None — the type is valid, just unavailable	Every definition declares `module`, the owner's slug. While `module_<owner>` is off, `is_available()` returns false, so the type is excluded from `available()` and the builder. It does still appear in `all()` — that difference is your tell.
The sidecar threw	Usually none — the collector swallows it	`erpat_collect_module_config_arrays()` catches and logs a throwing sidecar rather than fatalling, so a side effect at file scope disappears quietly. Keep `config/automations.php` a pure array literal: no CI calls, no model loads, no `get_uri()`.
`handler.file` escapes the module	`'<key>' rejected — 'handler.file' escapes the declaring module directory`	Path containment is enforced: a handler must live inside its declaring module, and any `..` is refused before the filesystem is touched. Make the path module-relative.
A required key is missing	`missing the owner 'module' slug` · `missing the 'handler' block` · `missing 'handler.class'`	Add it. Validation happens at registration, so the type never reaches the builder in a half-declared state.
Missing interface	`<block> file could not be loaded — …`	PHP resolves `implements X` at compile time, so a handler implementing an interface it cannot see is a fatal at `require`. The registry loads the contracts first and traps the throw — but your class must actually implement the declared contract.
Key already taken	`'<key>' ignored, already declared by …`	First declaration wins — core, then modules alphabetically. Rename your type key; a later module cannot hijack a registered one.

????

The registry is fail-SAFE, not fail-closed. A broken sidecar is logged and skipped, never fatal — matching the collector that runs on every request. That is the right polarity here because this registry only decides what a tenant may build; it is not an authorization surface. The cost of that choice is exactly this failure mode, which is why the notices block exists.

The Action or Safety panel shows attendance fields for my type ›

Known gap, named so nobody treats it as a bug in their own sidecar: the builder's Action and Safety panels are hardcoded to the attendance type's fields. Today that is invisible, because attendance.auto_decision is the only registered type. A second type needs those two panels made registry-driven first — it is not something a well-formed sidecar can work around.

Two smaller gaps in the same area, all recorded in Research & Known Gaps: the defaults block is never applied, and of the five capability keys only capabilities.live is read.

Everything moved to Needs review

needs_review is a stop, not a crash. The executor sets it in exactly four situations, each of which fails the run with a distinct error code — so the run tells you which one, and you should read it before changing anything.

Run error code	What happened	What to do
`DEFINITION_UNAVAILABLE`	The type's owner module was switched off, or its handler could not be loaded. Availability is revalidated at execution, not only at claim time, because a module can be toggled off between the two.	Re-enable the owner module, or fix the handler and check the engine card's notices. Then publish again.
`CONFIG_NEEDS_REVIEW`	The stored configuration no longer validates against the current handler. A schema change under a stored config stops the run rather than being reinterpreted.	Open the automation in the builder, correct the flagged fields, save, preview, and publish. Expect the config hash to change — you will need a fresh Observe run before Live.
`PUBLISHER_UNAUTHORIZED`	The person who promoted this automation to Live no longer has the authority. Live runs only.	See the next section.
(none — pointer cleared)	The cron expression can never match, so no next occurrence exists.	Fix the expression and publish again.

ℹ️

A whole batch flipping at once almost always means one module went off. Every automation of that type revalidates against the same toggle on the same tick, so they all move together. Check what changed in Manage Modules before investigating them individually.

"The publisher no longer holds permission"

An automation acts with the authority of whoever promoted it, and a schedule outlives people. publisher_denial() re-checks that authority on every Live tick and fails closed — an unresolvable publisher denies, because "cannot tell, so proceed" means a deleted user's automation keeps changing records.

no publisherNever promoted through the Live path, so no authority backs it.

user goneThe publisher row no longer exists, or is soft-deleted.

inactiveThe publisher's account status is no longer active.

permission lapsedautomations_go_live was withdrawn from that user.

Fix: have someone who currently holds automations_go_live re-promote the automation. That records them as the new publisher, which is the point — authority is a standing delegation from a named, currently-authorised person, not a one-off grant.

⚠

Observe runs are deliberately not gated this way. They change nothing, and stopping a report because its author left the company would remove the very visibility someone needs to notice that. If you "fix" this by extending the check to Observe, you remove the safest thing in the module.

????

Developers: this check uses user_can($id, 'automations_go_live'). Never use the similarly named user_has_permit() — it ignores its $userid argument and evaluates the logged-in user, which in a cron process is nobody.

Runs and results

It said completed but nothing changed ›

The run was in Observe mode, and Observe doing nothing is Observe working. A completed Observe run means "I evaluated these records and this is what I would have done" — the outcomes are would_approve / would_reject, and the count lands in would_count, never approved_count.

????

Check the RUN's run_mode, not the automation's current setting. They can legitimately differ. The run row is stamped when the occurrence is claimed, and the executor then corrects it to the automation's mode at execution time so history reports what the run actually did. An automation showing "Live" today can own a run that executed as Observe yesterday.

That correction only ever moves toward safety in the case that matters: demoting a Live automation to Observe stops records changing now, including for runs already queued. A demotion that took effect one tick later would be useless in the situation it exists for.

If the run genuinely was Live and still changed nothing, read the reason counts — PAYROLL_LOCKED, NOT_YET_SETTLED and OUTSIDE_SCOPE all produce a clean completed run with zero approvals, and all three are the rule doing its job.

The run completed but the Run detail modal is empty ›

The modal renders automation_run_items. An empty list means no item rows were written, and there are three reasons for that — two of them silent.

The handler returned no items[]
Aggregate counts (processed, succeeded, skipped) and the per-record items[] array are separate keys on the chunk result. A handler can report healthy counts and return no items, and nothing complains. This is the usual cause for a newly-written handler.
The outcome was outside the vocabulary — rejected silently
record_item() validates outcome against automations_item_outcomes() and simply returns false:
```
if (!$run_id || !$subject_id || !in_array($outcome, automations_item_outcomes(), true)) {
    return false;
}
```
No exception, no log, no failed run. A typo like approve instead of approved discards every item in the chunk. The seven legal values are approved, rejected, would_approve, would_reject, skipped, failed, already_satisfied. A missing subject_id is discarded the same way.
You are looking past the first 50
run_detail() loads items with "limit" => 50. The aggregate counts above the list cover the whole run; the list itself does not. That is a display cap, not missing data.

ℹ️

reason_code is not validated the way outcome is — it is written verbatim. So a wrong reason code shows up in the UI rather than vanishing, which is the opposite failure mode and much easier to spot. See the reason-code gap below.

A run is stuck on running ›

First, confirm it is actually stuck. Unfinished is not failed: a run that exhausts the tick budget deliberately stays running with its cursor and resumes on the next tick. A large backlog legitimately spans many minutes.

Time budget	45 s per tick, leaving headroom inside the one-minute window
Chunks per tick	20 max, so one automation cannot monopolise a tick
Claims per tick	100 max, so a bad clock cannot flood the queue
Lease	600 s, taken by `start_run()`

If the cursor has not advanced across several ticks, the dispatcher is not reaching this run — go back to Nothing runs. Remember that while a run is open, later occurrences of the same automation are skipped, so one stuck run quietly suppresses the schedule.

A reason code I emitted is not in automations_reason_codes() ›

Expected, and harmless at runtime. That list is enforced nowhere. It has exactly one consumer — a test asserting the vocabulary reflects features that actually exist — because record_item() validates outcome but writes reason_code verbatim.

The list has drifted from all three real emitters. The shipped attendance handler alone emits five codes that are not in it: APPROVED, NOT_YET_SETTLED, OVERTIME_PRESENT, RULE_NOT_MET, TRANSITION_ERROR. Meanwhile the list declares several nobody emits. This is recorded as §20.8 with two non-equivalent fixes (descriptive vs normative); it is not a bug in your handler.

Practical advice: emit whatever code is truthful, and do not rely on that function for UI labels — it will be wrong.

I cannot promote to Live

_live_refusal() enforces four preconditions, in this order. It returns the first one that fails, so satisfy them top-down.

You hold automations_go_live
A separate permission from automations_publish, deliberately: someone who may schedule a rule that only reports is not thereby authorised to let it change payroll inputs unattended. Grant it in Roles. Failing this returns a plain no permission response, not one of the messages below.
The type declares it can act
"Live mode is not available yet. This automation type can currently run in Observe mode only." The type's capabilities.live is falsy. This is the type author's decision, not yours — no permission overrides it, and there is nothing to configure. Only the owning module can change it.
A completed run exists
"Run this automation in Observe mode first…" The automation has no last_run_at. An automation that has never completed a run has never been shown to be correct. Publish it in Observe, wait for a run to complete, then return.
That run observed this configuration
"The configuration has changed since the last completed run…" The last completed run's config_hash no longer matches the automation's. This is the subtle one: preview, then widen the scope, then go live, and the run that "proved" the rule was observing a different rule against a different set of records. Run once more in Observe, then promote.

⚠

Known gap — publish_permissions is declared but read by nothing. An operator holding automations_publish + automations_go_live but without attendance_approval can currently publish a Live attendance automation. The four checks above are what actually run. Treat domain permissions as an organisational control until this is wired.

Live refuses with a missing System actor

The attendance gateway resolves a System user at construction and refuses to act without one:

// modules/HumanResource/automation/Attendance_transition_gateway.php
$this->actor_id = function_exists('erpat_system_actor_id') ? (int) erpat_system_actor_id() : 0;

/**
 * An id of 0 is not an actor: `checked_by = 0` joins to nothing, so the approver
 * renders blank and one listing path attributes the approval to the record's own
 * creator. A run must refuse rather than write that.
 */
public function can_attribute()
{
    return $this->actor_id > 0;
}

erpat_system_actor_id() looks up [email protected] on the current database and returns 0 when it is absent — and 0 must be treated as a refusal, never as a fallback. The seed lives in the core migration application/migrations/20260903232432_seed_system_actor_user.php.

????

This is per-tenant. The actor id differs per database and is cached keyed by database name, because one cron tick walks several tenants in a single process. A tenant that has not had core migrations applied has no System actor, and its Live runs will refuse while another tenant's succeed. Fix: run php erpat migrate:latest against that tenant.

Overtime — did I just destroy data?

No. This is the question worth answering carefully, because the design is specifically built so the answer is always no.

☑

Hours are always stored

The measured overtime is written in every case, under every policy. No policy deletes, zeroes or refuses to record a number.

⚖

The policy records a decision

It sets attendance_metrics.ot_status — an attributable decision about the hours, kept separate from the hours themselves.

↺

Everything is reversible

Unapproved hours are withheld at read time, not erased at write time. Change the decision from the Overtime tab and the hours reappear.

Policy	Stored decision	What it means
skip (default)	`ot_status = ''`	Decide nothing. The attendance is approved; the overtime is queued for a person to rule on. Nothing is approved and nothing is rejected.
exclude	`ot_status = 'false'`	Approve the base hours, refuse the overtime — a real, attributable, reversible rejection, replacing the silent zeroing this superseded.
include	`ot_status = 'true'`	Authorise both the attendance and its overtime.

????

skip must not send 'false', and the distinction is load-bearing. Its help text promises that any record carrying overtime is left for a person — sending 'false' would record the System actor rejecting overtime it was explicitly configured not to rule on. Three policies, three genuinely different promises.

Glossary

Automation: A saved, scheduled rule: a type, a configuration, a cron expression, a scope, and a mode. Lifecycle statuses: draft, active, paused, needs_review, archived.
Type: The kind of automation, contributed by the module that owns the data — a handler plus a condition provider, declared in that module's config/automations.php. Registration is declarative only: no API, no event bus. Today the only registered type is attendance.auto_decision, owned by HumanResource.
Run: One execution of one occurrence. Carries its own frozen configuration, its own mode, a cursor, aggregate counts and a status: queued, running, completed, completed_with_exceptions, failed, skipped_overlap, cancelled, timed_out.
Run item: One record's outcome within a run, with its reason code. This is the per-record audit trail; the system_logs entry is one row per run, never one per record.
Observe (dry_run): The mode that evaluates and reports but changes nothing. Positive outcomes are would_approve / would_reject and are counted as would_count — never approved_count, which must only ever mean a record was actually changed.
Live: The mode that applies decisions through the owning module's own service. Gated by four preconditions, and the first live run is capped at the automation’s First live run limit (50 by default; the engine falls back to 25 only when no positive value is stored).
Publisher: The user recorded in published_by when the automation was promoted. A Live run executes on their standing authority, revalidated every tick.
Config hash: A fingerprint of the automation's configuration. Promotion to Live requires the last completed run's hash to match the current one — the mechanism that stops an Observe run from vouching for a rule it never saw.
Idempotency key: sha256(automation_uuid | scheduled_for_utc | trigger_kind), held UNIQUE. One due occurrence is exactly one run, however many times the dispatcher is invoked for that minute. The configuration revision is deliberately excluded.
Correlation id: A corr_… token minted per run that ties the dispatcher output, the run row, its items and the downstream business audit entries to one execution. Quote this when escalating.
Cursor: The handler-defined resume token saved after every chunk, so the next tick continues exactly where this one stopped rather than restarting.
Lease: A 600-second claim taken on a run by start_run(), so a crashed executor's run can be recovered rather than remaining locked forever.
Reason code: The machine-readable "why" on each run item. Written verbatim and validated nowhere — see the gap noted above.
Needs review: A deliberate stop. The automation is not deleted and not running; something it depended on changed and a person must look. Always paired with a failed run carrying an error code.
System actor: The [email protected] user that Live changes are attributed to, seeded per tenant by a core migration. Resolved by erpat_system_actor_id(), which returns 0 when unseeded — and 0 is a refusal, not a fallback.

How to escalate

Capture these before asking for help. Together they let someone else reconstruct the state without access to your screen, and each one answers a question that is otherwise the first thing they will have to ask.

Capture	Where	Why it matters
Run UUID and its status	Runs tab → the run row	Identifies the exact execution. Include the run's `run_mode` — not the automation's — and its error code if it failed.
Correlation id	Run detail modal	Ties the run to the dispatcher output and to the business audit entries the owning module wrote. This is the single most useful token.
Registry checksum	Settings → engine card	A stable fingerprint of the loaded catalogue. It changes when the registry changes, which is how you prove a deployment altered what is registered.
Registry notices (skipped list)	Settings or Overview → notices block	Names every definition the registry rejected, with the reason. Copy it whole, including "empty" — that is itself information.
`php erpat cron:list`	CLI	Confirms `automation_dispatch` is registered and shows its schedule and enabled state. Add the Cron Manager's recent runs for this tenant.
Automation status + `next_run_at_utc`	Automations list	Distinguishes "not scheduled" from "scheduled and not picked up" — two entirely different investigations.
Module toggles	Manage Modules	`module_automations` and the owner module of the type involved. A batch move to Needs review is nearly always one of these.
Tenant identity	—	Everything above is per-tenant: the tables, the System actor, the toggles and the cron fan-out. A report without a tenant is not reproducible.

ℹ️

Before reporting a missing feature, check it is shipped. Milestones 1–3 (foundation, builder + Observe, Live + clone) are in. M4 — the operations UI, analytics, notifications and both API surfaces — and M5 — hardening, README, OG image and behavioural tests — are pending. Notably, there are no automation notifications yet, so a failed run will not email anyone; you find it on the Runs tab. See Research & Known Gaps for the full list.

If the answer is not here, the two pages that most often contain it are Permissions, Authority & Scheduling for anything about who may do what and when things run, and Run Lifecycle & Execution for anything about how a run claims, chunks, resumes and closes.

Was this guide helpful?

FAQ & Troubleshooting

FAQ & Troubleshooting

Sixty-second triage

Nothing runs

Types and the builder

Everything moved to Needs review

"The publisher no longer holds permission"

Runs and results

I cannot promote to Live

Live refuses with a missing System actor

Overtime — did I just destroy data?

Glossary

How to escalate

Report a content problem

Couldn't find the instruction you need?

FAQ & Troubleshooting

Sixty-second triage

Nothing runs

Types and the builder

Everything moved to Needs review

"The publisher no longer holds permission"

Runs and results

I cannot promote to Live

Live refuses with a missing System actor

Overtime — did I just destroy data?

Glossary

How to escalate

Related guides

Couldn't find the instruction you need?