Skip to main content
Most plugin failures fall into a handful of categories: a signature the node does not trust, an invalid manifest, an install the agent refuses, a capability that was not granted, the cgroup killing the process, the IPC socket not coming up, or the GCS iframe trying to reach the network. This page maps each one to what you see and the commands that tell you which one you hit. Every ados plugin command exits with a code from the exit-code table. Pass --json in CI and branch on .code, not on the human text.

Signature and trust failures (exit 3)

ados plugin install checks the signature before anything is written to disk. The detail text names the cause:
  • The archive is unsigned. A default agent refuses it with archive is unsigned. Sign it with ados plugin sign or scripts/sign.sh.
  • The signer is not enrolled. The node has no /etc/ados/plugin-keys/<signer-id>.pem for the signer id in the archive. Install the publisher’s public PEM there (the file name must equal the signer id), or re-sign with a key the node already trusts.
  • The signer, the archive or the release is revoked. It is listed in /etc/ados/plugin-revocations.json (a release as <plugin_id>@<version>). A revoked signer, archive or release is refused in every mode. Re-sign with a fresh key, or install a release that is not revoked.
  • The signature does not verify under the enrolled key. The archive changed after signing, the wrong key is enrolled for that signer id, or the archive was signed with a different id than you think. Re-pack and re-sign:
Signature enforcement can only be relaxed on the agent itself, by running its services with ADOS_PLUGIN_REQUIRE_SIGNED=0 on a bench node. The agent then logs a warning, installs unsigned or unverifiable archives with no signer, and never grants them first-party status. The procedure is in Bench install and signing.

Manifest validation errors (exit 2)

ados plugin lint, sign and test validate manifest.yaml against the strict schema and exit 2 on failure; the agent rejects a malformed manifest at install the same way. Common causes:
  • Neither an agent: nor a gcs: block. At least one half is required.
  • A resource bound out of range. agent.resources.max_ram_mb must be between 8 and 4096 for the standard class (default 96).
  • A missing required field: id, version, name, compatibility.ados_version, agent.entrypoint, or gcs.entrypoint when the GCS half renders panels, tabs, agent_pages or node_surfaces.
  • A camelCase contributes key (missionTemplates, mapOverlays). Use mission_templates and map_overlays.
An unknown permission id is a warning, not an error: the manifest loads, but the permission grants nothing. Check the spelling against Permissions. Run the linter before you install:
lint exits 0 when the report passes and 1 when it carries an error or critical finding.

Install refusals (exit 4 and exit 8)

These manifests parse, but the agent refuses to install them:
  • agent.isolation: inprocess is refused for every plugin (exit 8). Use subprocess.
  • gcs.isolation: inline and agent.resources.class: heavy are refused unless the archive verifies against a first-party key (exit 4, refused: ..._first_party_only).
  • A board, tier, target profile or architecture the manifest does not cover (exit 8, incompatible: ...). supported_boards takes board ids or SoCs from Supported hardware, or "*".
  • ados plugin install --yes on a plugin whose risk is high or critical (exit 4, --yes refuses <risk>-risk plugins). Re-run without --yes and approve after review.

Capability denied at runtime

The agent re-resolves the required capability from the method on every request and rejects an ungranted call with capability_denied: <cap>; the handler never runs. On the GCS half the bridge answers permission_denied. This happens when the plugin calls a method whose capability was not granted (an optional permission left off, or one revoked while the plugin ran). Check and fix the grant:
A revoke takes effect at once, so a running plugin can start seeing capability_denied mid-session after an operator revokes a permission.

Token expiry

Tokens rotate without plugin action: the agent re-mints a token ahead of expiry and on expiry within a live session, and Mission Control pushes a fresh token into the iframe, which the SDK caches. A persistent capability_denied:token_expired on the GCS half means the host stopped pushing tokens; reload the panel.

The agent half crashes or restarts

Each agent half runs as ados-plugin-<id>-<hash8>.service inside the shared ados-plugins.slice, with the manifest’s bounds:
Find the exact unit name first; the <hash8> suffix comes from the id:
If the plugin’s memory crosses MemoryMax, the kernel OOM-kills it and systemd restarts it after 2 s. There is no start limit (StartLimitIntervalSec=0), so a crash-looping plugin keeps restarting and stays running in the plugin record. CPU is throttled, not killed: crossing CPUQuota slows the process, so a CPU-starved plugin looks like latency, not a crash. Read the cgroup accounting to tell an OOM from a throttle:
A rising oom_kill means raise agent.resources.max_ram_mb or shrink the working set. A rising nr_throttled means raise max_cpu_percent or do less work per tick.

IPC connect and timeout failures

The plugin host (ados-plugin-host) binds one Unix socket per plugin at /run/ados/plugins/<id>/host.sock. The unit passes the path in ADOS_PLUGIN_SOCKET and delivers the token as a systemd credential at $CREDENTIALS_DIRECTORY/ados-plugin-token, never in the environment. The runner connects, sends a hello handshake with the token, and only then issues requests. The unit has Requires=ados-plugin-host.service and PartOf=ados-supervisor.service.
  • Socket missing or connection refused. The plugin host is not serving that plugin yet. The runner keeps retrying, and systemd restarts it with no start limit. Check the host and the socket dir:
  • Handshake rejected. The server closes the connection with capability token invalid: <reason> when the token does not verify or names another plugin. The host rewrites the credential on every start, so a restart of the unit picks up a fresh one.
  • Host method not ready. A request for a subsystem that has not come up yet (for example the flight controller link) returns a not-ready error rather than data. Retry once the subsystem reports ready.

GCS iframe blocked from the network

The GCS half runs in sandbox="allow-scripts" with no allow-same-origin, on a document whose CSP has connect-src 'none'. WebRTC and nested frames are removed. So:
  • A fetch(), WebSocket or peer connection from the panel fails with a CSP violation or a missing constructor. Route the work through a bridge method (command.send, mission.write, cloud.read, records.*) or through your plugin’s agent half.
  • The iframe cannot read window.parent, localStorage or the host document. Anything you need from Mission Control comes over the bridge under a granted capability.
When a GCS panel renders blank, open the devtools console for that iframe and rule out, in order: a CSP block, a capability_denied or permission_denied from the bridge, and a throw in mount (the SDK does not report it to the host, so render your own error state).

Diagnostic commands

1

Read the plugin's own log

The agent half writes stdout and stderr to /var/log/ados/plugins/<id with dots as hyphens>-<hash8>.log. ados plugin logs asks the agent for that path and reads it:
An exit 5 with no log file at ... means the plugin never started, or its log was rotated out. Move to the unit status next.
2

Inspect install state and grants

A DENIED permission explains a runtime capability_denied. A disabled plugin with a recorded reason was disabled by a revocation.
3

Read the systemd unit

status shows the last exit reason and the restart count. cat shows the generated unit with its bounds and sandbox.
4

Pull the journal

For crashes before the plugin can log, or the plugin host’s view of the start:
A Killed process ... (oom) line confirms an OOM.
5

Query the Black Box store

When the Black Box store (ados-logd) is enabled, query across the whole agent to correlate a plugin failure with the rest of the node:
The store is off by default; without it, use journalctl -o json. See Black Box.

Next steps

Plugin CLI reference

Every command and the exit-code table.

Performance and budgets

Sizing max_ram_mb and max_cpu_percent.

Signing keys

Generate, enrol and rotate publisher keys.

Troubleshooting by area

Every troubleshooting page across ADOS.
Last modified on October 11, 2026