Troubleshooting
Find the message you see. agent-secrets prints a broker refusal as <message> (<CODE>) and a
helper refusal as <CODE>: <message>; the error reference,
generated from the code, lists every code. Every message about a secret also names where
agent-secrets looked - the broker AGENT_SECRETS_URL names (<store kind>), or the broker AGENT_SECRETS_URL names alone where the broker did not say what its store is - so a message of
this CLI’s is never read as another secrets CLI’s about the same name. It names no host, account or
project; a message about a broker that was never reached names the address it tried when that
address is a plain http(s) URL, and says (not printed: the value carries a credential or is not a plain http(s) URL) instead when it carries a user name, password, query, fragment or % escape,
with the failure named by its class alone (connection refused, name resolution failed, timed out, and so on). The messages below leave that part out.
The session has no broker identity
Section titled “The session has no broker identity”| Message | Cause | Fix |
|---|---|---|
AGENT_SECRETS_URL is required: it names the secrets broker this command reads every secret from, and this command reads no local secrets file (exit 2) |
The process does not know the broker’s address, and this CLI answers from no other store. | Set AGENT_SECRETS_URL to the broker’s public URL. A command run by agent-secrets NAME -- … keeps it. |
no session identity: the key dir … has no key.pem … and no helper socket is set or at … |
The process is neither in a container with a key nor on a machine with a helper. A Kubernetes pod that enrolled itself with agent-secrets enroll --pod --out <dir> reads this when a later command does not name <dir>: --out writes the key there, and every other form finds it only through AGENT_SECRETS_KEY_DIR. |
On a machine, run the agent under agent-secrets register --exec; in a container, its launcher must enroll it. In a pod that enrolled itself, run the command enroll --pod printed, which names the key directory (AGENT_SECRETS_KEY_DIR=<dir> agent-secrets renew --pod-token-file <file>), or export AGENT_SECRETS_KEY_DIR=<dir> for the pod. |
NOT_A_SESSION: pid … is not inside a registered machine session; a session root is started with agent-secrets register --exec -- <agent argv> |
The helper is running, but this process does not descend from a registered session. | Start the agent with agent-secrets register --wait 10 --exec -- <agent>. |
NOT_ENROLLED: this session is not enrolled with the broker yet |
The session registered, but the helper has not enrolled it yet: it was started without --wait, or the broker was unreachable. |
Start sessions with --wait 10. If it persists, check the helper’s log and that AGENT_SECRETS_URL reaches the broker. |
this machine is not logged in to the secrets broker; not an agent session (run: agent-secrets machine login) |
The helper holds no machine credential: the machine was never logged in, its login lapsed or was revoked, or its machine file (AGENT_SECRETS_MACHINE_FILE) is gone. |
Log the machine in. |
The helper’s log: the helper could not keep the machine login's refresh on disk; removed the machine file … (ERROR) |
A refresh rotated the login’s token, but the machine file could not be written: a full disk or an exceeded quota. The helper removes the machine file at once, since it names a token the broker rotated away and a restart presenting that token past its five-minute grace would revoke the machine. It holds the new login in memory, keeps working, and retries only the save every 15 s, with no further token request. A save that lands writes the file back. | Free space; the next save writes the file back. A restart before the next good save holds no login: log the machine in again. |
The helper’s log: … nor remove the machine file: a restart before the disk is writable again presents a token the broker has rotated away … (ERROR) |
The filesystem refuses even the removal (read-only). The file still names the rotated-away token. | Do not restart the helper until the filesystem is writable again: a restart before then presents that token, and the broker revokes the machine as a reuse, ending every session it enrolled. Once it is writable, the next save keeps the login. |
agent-secrets: helper unreachable at … |
The helper is not running, or the socket path is wrong. | Start agent-secrets-helper, or point AGENT_SECRETS_HELPER_SOCK at its socket. |
agent-secrets register: AGENT_SECRETS_HELPER_SOCK is unset and no helper socket is at … (machine sessions only; an agent box has a key dir instead) (exit 2) |
No helper listens on the default socket, $XDG_RUNTIME_DIR/agent-secrets/helper.sock, and AGENT_SECRETS_HELPER_SOCK names none. |
Start agent-secrets-helper, or set AGENT_SECRETS_HELPER_SOCK to the socket it listens on. |
the launcher credential reached its expiry; run: agent-secrets machine login, the broker refused the launcher credential (expired or revoked, …); run: agent-secrets machine login, or the broker refused the machine login's refresh: it was revoked, lapsed, or its refresh token was reused; run: agent-secrets machine login (from machine login-status) |
The credential expired or was refused: no refresh reached the broker for BROKER_LAUNCHER_CREDENTIAL_SECONDS, revoked by the person who approved it, a refresh token spent twice, a clock far off the broker’s, or an AGENT_SECRETS_URL that is not exactly the broker’s public URL. A token is spent twice when a copy of the machine file refreshed it elsewhere, or when the broker’s answer to a refresh was lost and the broker then stayed unreachable past its five-minute grace: the helper retries every 15 s for as long as the broker does not answer, and a retry after the grace counts as reuse, which revokes the machine and ends its sessions. |
Fix the clock or the URL if either is wrong, then log in again. |
ENROLLMENT_NOT_LIVE from agent-secrets renew (exit 76) |
The session ended: it was revoked (its launcher unenrolled it, or its machine login was revoked), its lease ran out while a broker was serving, or, for a machine’s own session, the machine login its Google identity traded for has ended (the row below). A lease that ran out while no broker was serving is not ended, so a broker outage alone does not cause this (when no broker is serving); a broker that runs but that the session cannot reach does. The renewer names the enrollment, HTTP status and code; the pod shim stops restarting the renewer, and the pod has none until it is relaunched. | Start a new session (a pod: relaunch it; a box: a new key and enrollment, run an agent in a container). |
Every session on one fleet machine stops renewing at once, the broker logging broker: machine session renewal refused: its machine login has ended enrollment=… (WARN) and answering ENROLLMENT_NOT_LIVE |
A machine’s own session renews only while the machine login its Google identity token traded for is live. That login has ended: it was revoked, it passed the hour’s expiry the identity token gave it and the machine has not traded again, or its account is no longer listed for its service in BROKER_VM_SERVICES. Every session of that box lapses within one BROKER_LEASE_SECONDS, with its grants. A person’s sign-in sessions on the box, and every pod, are unaffected. |
If the box should still be trading, check that it can reach the broker and still mint an identity token (AGENT_SECRETS_MACHINE_IDENTITY and its own variables), and that the account is in BROKER_VM_SERVICES and within the BROKER_VM_PROJECTS and BROKER_VM_INSTANCES pins, redeploying the broker after a settings change (Operate). The sessions enrol again on their own once a trade succeeds. If the fleet was dropped on purpose, this is that change taking effect. |
PROOF_INVALID |
The broker refused the proof itself: its iat is outside BROKER_PROOF_SKEW_SECONDS, its key does not match the enrollment, or AGENT_SECRETS_URL differs from the broker’s public URL. A renewer keeps retrying this code because a clock or URL correction can make the next proof valid. |
Check the clock and URL; for a key mismatch, start a new session. |
the broker's host … does not resolve or … name resolution failed, each naming GET/POST and AGENT_SECRETS_URL |
The call never reached a broker: the address’s host either resolves nowhere, or its name resolver timed out or failed. Usually a stale or mistyped AGENT_SECRETS_URL was inherited. It is not a refusal the broker gave. |
Set AGENT_SECRETS_URL to the broker’s current public URL in the profile your shells and agents start from, and start the session again. |
nothing answered at …: the connection was refused, nothing answered at …: no connection could be made, or … did not answer in time, each naming the call and AGENT_SECRETS_URL |
No connection to the broker was made or it did not answer: nothing listens at the address, the network or host is unreachable (a route, a firewall, a VPN or tunnel that ended), or the broker is too slow. | Check the address, the route to it, and that the broker is running. |
the broker at … answered but the exchange failed, naming the call and AGENT_SECRETS_URL |
A peer took the connection and then broke the exchange: a TLS certificate it would not trust or an HTTP server answering an https URL, or the stream ended or was reset after the request. It is not a broker refusal. |
Check the address’s scheme and port, TLS configuration and any proxy between the session and the broker. |
the request to the broker at … failed, naming the call and AGENT_SECRETS_URL |
The request failed in a way none of the rows above names; the error after the colon is the transport’s own. | Read that error; it says which layer failed. |
the broker's address is not a valid URL, naming the call and AGENT_SECRETS_URL (exit 1) |
AGENT_SECRETS_URL is not a URL the client can build a request from, such as a bad % escape or a port that is not a number. Nothing was sent. |
Set AGENT_SECRETS_URL to the broker’s public URL, escaping any %, @, /, ? or # in a user name or password. |
A pod’s renewer cannot read its token
Section titled “A pod’s renewer cannot read its token”| Message | Cause | Fix |
|---|---|---|
agent-secrets renew: ERROR: enrollment … cannot read its pod token (…); nothing was sent, so its lease lapses unless the file becomes readable |
The pod’s renewer (agent-secrets renew --pod-token-file) could not read its projected service-account token, or read it blank, so it sent the broker nothing. The broker never hears of it, so this line in the pod’s container log is the only record. The renewer does not exit 76 for it: at start it exits 2 and the worker shim starts it again 5 s later, and after a renewal it waits and reads the file again. If the file stays unreadable the enrollment’s lease lapses, and its grants end with it. |
Check the projected-token volume is mounted read-only at /var/run/legion/agent-secrets-token in the pod’s container (roles in an issue pod), with the file token present and non-empty, and that the container’s user can read it: the volume’s mode is 0440, readable through the pod’s fsGroup. Once the file is readable, the next renewal succeeds. |
The request was refused
Section titled “The request was refused”| Message | Cause | Fix |
|---|---|---|
request … to the broker AGENT_SECRETS_URL names … was denied (exit 77) |
The approver denied it, or one of the names is a service’s secret, which only that service’s sessions get. Nothing ran. | Ask the approver why, or ask for the names you may have. |
request … to the broker AGENT_SECRETS_URL names … was denied: <NAME>: a reason is necessary; ask again with --reason (exit 77) |
The request named a secret a person must approve — a human-tier secret, one your operator withheld from this session, or another person’s agent-tier secret — and carried no reason, or only spaces. The broker asks nobody to approve work the record does not describe, so it denied the whole request. Nothing ran. | Ask again with --reason saying what the credential is for, in a sentence the approver can decide on. |
request … to the broker AGENT_SECRETS_URL names … is still waiting for approval; nothing was run. Check it with: agent-secrets status … (exit 75) |
Nobody decided within --wait (30 minutes by default). |
Ask the approver the waiting for … line named (agent-secrets status prints it again): the person the broker recorded when the request was made, which can differ from whoever the secret’s description or your memory names. Rerun the command once it is granted, or with a longer --wait, a duration such as 1h. If that person’s approval on the broker’s page leaves the request still pending; it was not decided, something changed after the request was made (Approvals says what each change does). Usually the secret’s owner changed: run agent-secrets cancel <request id> and ask again, and the new request goes to the current owner. If instead the session’s operator withheld one of the names, the request waits on anyone, and the broker still serves that name to the session, the operator can approve it, so ask them. Otherwise no one can approve it: cancel it and request the names separately, one request per approver, leaving out any name the broker no longer serves (approving a request says what each line that page prints means). |
invalid value "…" for flag -wait: parse error (exit 2) |
--wait takes a duration with its unit (90s, 5m, 1h), not a bare number. register --wait is the exception: it takes whole seconds. |
Add the unit. |
request … to the broker AGENT_SECRETS_URL names … was expired or request … to the broker AGENT_SECRETS_URL names … was cancelled |
Nobody decided it before it expired, or the session cancelled it (agent-secrets cancel). A session that ended reads none of its requests: its calls are refused PROOF_INVALID. |
Ask again. |
no agent secret has this name (UNKNOWN_SECRET) |
No secret the broker serves has the name: none exists under the namespace in that name (DEEL_API_KEY is <namespace>deel-api-key), or the broker leaves it out because a tag is missing or malformed, it is on the wrong key, or it has no value yet, which the broker logs by name (agent secret policy refused, Operating the broker). |
Check the spelling (agent-secrets secret list lists every agent secret); ask the secret’s owner, or whoever runs the broker, to fix its tags or key or to put its value (manage a secret). Ask again once it is fixed: the broker reads a name it does not serve again before refusing it (up to ten such rereads per session in each broker process, refilled one every ten seconds), so the next request is served. |
the requested secrets need different approvers; request them separately (MIXED_APPROVERS) |
One command asked for secrets that different people approve. | Request them in separate commands. |
… allows at most <duration>; ask with agent-secrets --ttl <duration> (GRANT_REQUIRES_LIFETIME) |
The secret has a max-grant tag, and the command requested a longer lifetime (no --ttl asks for 12 hours, or until the session ends under default-grant=session). |
Run the command again with --ttl at or below the duration in the message. |
a request for one or more of these secrets is already pending: request … is still pending (REQUEST_PENDING) |
This session already has a request waiting for approval that names one of these secrets, with another set of names or another --ttl. A session has one pending request per secret; this holds for requests without --ttl too, so asking for A B while a request for A waits is refused. Only the identical request (the same names and the same --ttl, or none) joins the pending one. |
Wait for the named request to be decided (agent-secrets status <request id>), or end it with agent-secrets cancel <request id> and ask again with the names and --ttl you want. |
the grant from the broker AGENT_SECRETS_URL names … released no value for …; nothing was run |
The broker granted a name but returned no value for it. | Tell whoever runs the broker, naming the kind of store the message gives and the AGENT_SECRETS_URL the session has. |
read <NAME>: the broker could not mint a token from this secret: … (MINT_FAILED) |
The secret is a github-app or google-login kind, and GitHub or Google refused the mint or could not be asked: the message says why, such as Google login needs a new sign-in: invalid_grant (invalid_rapt), a GitHub App with no unsuspended installation on an account its github-owners tag lists, GitHub answered HTTP 401, or could not be reached. It names the HTTP status code and the class of a failed call, never the upstream’s own text. |
For a Google login, its owner signs in again and stores the new login with agent-secrets secret set. For a GitHub App, check that it is installed and not suspended on an account its github-owners tag lists (ask whoever runs the broker to add an account), and that the stored key is one of the App’s current keys. For a call that failed, check the broker’s egress to GitHub or Google. |
--github-repo and --github-read-only narrow one account's GitHub App token: name the account with --github-owner (exit 2) |
The command narrowed a GitHub App token without naming the account. Nothing reached the broker. | Add --github-owner <account>. |
--github-token hands over one account's GitHub App token: name the account with --github-owner (exit 2) |
The command asked for the token alone and named no account. A release holds one token for each installation, so the flag needs the account whose token the command gets. Nothing reached the broker. | Add --github-owner <account>. |
--github-token hands over one installation's token, and the release of <NAME> holds 2 installations (Ada, ada): ask again with a narrowing that leaves one installation, or without --github-token, and read the token out of the JSON object the release hands over (exit 1) |
The grant released a github-app value holding other than one installation, so there is no one token to hand over. A release of none reads holds no installation in place of the count. --github-owner alone does not always make it one: the mint compares the account without regard to case, so --github-owner ada mints for ada and for Ada alike. Nothing ran. |
Ask again with a narrowing that leaves one installation, such as --github-owner <the other account>. Where no narrowing does, drop --github-token and read .installations["<account>"].token out of the JSON object with jq. |
--github-token hands over a GitHub App installation token, and the grant released none: it read <the names> (exit 1) |
No name of the grant released a JSON object of installations: the grant holds no github-app secret, or every value it released is of another shape. The message names what it read. Nothing ran. |
Ask for the GitHub App secret in this command, or drop --github-token. |
GitHub answers HTTP 401 to a call whose Authorization header carried a github-app secret’s released value |
The released value is the JSON object of installations, not a token, so GitHub reads it as no credential. One narrowed release measured 496 characters of JSON, and the ghs_ token inside it answered 200. |
Ask again with --github-token, which hands the command that one installation’s token alone. Without the flag, read .installations["<account>"].token out of the object with jq. |
--github-owner must name one GitHub login … or --github-repo names … (exit 2) |
--github-owner is empty or not one GitHub login, or a --github-repo is not a repository name alone (it holds the account, such as acme/widgets, or a character GitHub does not allow), or there are more than 100. agent-secrets checks the flags as the broker does, before it asks for the secret. Nothing reached the broker. |
Give the account in --github-owner and each repository by its name alone, at most 100. |
github_app.owner must name one GitHub login …, github_app.repositories names … or github_app.repositories is empty … (INVALID_NARROWING) |
An API client sent a github_app body whose owner is not one GitHub login, whose repositories holds a name that is not a repository name alone or more than 100 names, or whose repositories is present but empty. Nothing was released. |
Send the account in owner and each repository by its name alone, at most 100; leave repositories out for every repository of the installation. |
this grant holds no github-app secret to narrow (NARROWING_NOT_APPLICABLE) |
The command used --github-owner with secrets none of which is a github-app kind. Nothing was released. |
Remove the --github-* flags, or ask for the GitHub App secret. |
the narrowing names an account a github-app secret of this grant does not serve: <NAME>'s github-owners tag does not list <account> (NARROWING_OWNER_NOT_LISTED) |
The account in --github-owner is not in the secret’s github-owners tag. A narrowing cannot add an account. Nothing was released, and GitHub was not asked. |
Name an account the secret’s github-owners tag lists, or ask whoever runs the broker to add it to the tag. |
read <NAME>: … has no installation on <account> (MINT_FAILED) after --github-owner |
The secret’s github-owners tag lists the account, but the App is not installed there, or its installation there is suspended. Nothing was released. |
Ask whoever runs the broker to install the App on the account, or name another account. |
read <NAME>: the narrowing cannot be served: … GitHub answered HTTP 422 to the narrowed token for <account>: likely a repository it names (…) that the installation does not reach … (NARROWING_REFUSED) |
GitHub refused the narrowing: a --github-repo it does not know or the installation cannot reach (misspelled, or not among the repositories the installation selected), or, with --github-read-only, read access the App does not hold of contents, metadata or pull requests. A missing Actions grant alone is not this: the broker asks again without actions: read and releases the other three. Nothing was released. |
Check the --github-repo names. If they are right, check the App installation’s repository selection and permissions on GitHub. |
read <NAME>: … GitHub's token holds more than read access …, … a permission past contents, metadata, pull requests and actions …, … has a repository selection other than "selected" … or … reaches a repository the narrowing does not name (MINT_FAILED) |
GitHub answered a narrowed token that holds more than the narrowing asked, which GitHub does not do: something between the broker and GitHub dropped or changed the narrowing. The broker released nothing and revoked the token at GitHub. | Tell whoever runs the broker to check its route to GitHub. |
the broker did not narrow the GitHub App token to …: it answered no narrowing, …; nothing was run |
The broker is from before narrowing: it ignored --github-owner and released the token for every repository, which agent-secrets did not give to the command. |
Tell whoever runs the broker to deploy a release that narrows GitHub App tokens. |
grant is expired, revoked, or its session ended (GRANT_NOT_LIVE) |
The grant ended between the decision and the read, one of its secrets has no value yet, or the secret was deleted or its tags changed since it was granted: the secret is now left out or denied, one granted automatically now needs approval, it now belongs to someone who did not approve the grant, or its max-grant is now shorter than the grant’s lifetime. A grant the model judge decided also ends when the airlock rules stop letting the judge decide that secret, or lower its seconds below the grant’s own lifetime. |
For a secret with no value, its owner puts the value: agent-secrets secret set has the broker serve it at once, and a value put any other way resumes the grant within about ten minutes, as the broker rereads the namespace every five and Secrets Manager’s listing can lag a change by up to five more. Otherwise, run the command again to ask anew, with --ttl within any max-grant. |
secret is not in the secrets store (SECRET_NOT_IN_STORE) |
The secret was deleted from the secret store by something that did not ask the broker to reread it (the AWS console or the AWS CLI), after the broker last read the namespace: outright, or with a recovery window (their default, which keeps the secret until the window passes and refuses every read of its value meanwhile). A delete through agent-secrets secret delete is reread at once, so its grants end GRANT_NOT_LIVE instead. |
Ask again within about ten minutes: the broker rereads the namespace every five, and Secrets Manager’s listing can lag a change by up to five more. The request is then refused UNKNOWN_SECRET unless the secret is back (a secret within its recovery window is restorable: agent-secrets secret restore). |
request object invalid (REQUEST_INVALID) |
The broker could not verify the signed request: usually a clock far off, or an AGENT_SECRETS_URL that is not the broker’s public URL. |
Check the clock and the URL. |
A request the judge could have decided is still pending
Section titled “A request the judge could have decided is still pending”The model judge approves a request or leaves it for the person who approves it today, and it
never denies, so a request it did not take waits as any other does
(Concepts). What it decided, and why, is one
request.judged row in the broker’s audit table and one line in that broker’s log,
broker judge: wrote a verdict, naming the record, the verdict, the model’s reason and, where a
check stopped the run, failed_check and failed_detail.
| What you find | Cause | Fix |
|---|---|---|
| No row and no line for the record | This broker runs no judge (its BROKER_JUDGE_* settings are unset), or the request joined one already pending, which the judge does not run for. Nothing is written either when the run passed its 30 seconds or the broker stopped serving under it, which logs broker judge: the run ended before it wrote anything. |
Ask the approver, as before. |
failed_check=rules-judge |
The request names more than one secret, or the airlock rules let the judge decide nothing for that secret. | Ask for one secret. Which secrets the rules give the judge is the deployment’s own choice, changed only in its rules file, which a person reviews. |
failed_check=lifetime |
The grant would last more than 15 minutes, more seconds than the rules let the judge decide for that secret, or until its session ends. With no --ttl the judge reads the broker’s default instead: 15 minutes for a human-tier secret, and the whole session for one tagged default-grant=session. |
Ask again with --ttl 15m or less, within the seconds that secret’s entry gives the judge. |
failed_check=policy |
The tag policy, read again at that moment, no longer answers approval for the name: its owner or tier changed, or it left the broker’s policy. | The approver the request names decides it, or the session cancels it and asks again under the tags now in force. |
failed_check=admin |
The secret is tagged as an administrator’s (admin: "true"), and a person approves every release of one. The judge refuses it whatever seconds the rules give that name, so a rules entry cannot move it off its approver (Concepts). |
That approver decides it. |
failed_check=withheld |
The session’s own person revoked a grant this session got without asking, so that secret is withheld from it and only its owner may approve it (Concepts). | That owner decides it. |
failed_check=issue |
The signed reason names no Dispatch issue key, Dispatch holds no issue with that key, the issue is done, or Dispatch could not be read; failed_detail says which. |
Ask again with an open issue’s key in --reason. The reason is signed, so an existing request cannot gain one. |
verdict=refer |
The model read the request and left it for a person. Its own words are the row’s and the line’s reason. |
The approver decides it. |
asked=true with model_err |
The model could not be reached, or did not answer within the judge’s 20 seconds. | The approver decides it. Tell whoever runs the broker if it keeps happening. |
applied=false |
The judge approved, and the approval wrote nothing: a person or an expiry decided the record first, which is no fault of the judge’s. | Read the record’s own state; the request is decided already. |
Managing a secret
Section titled “Managing a secret”These are the messages of agent-secrets secret
(manage a secret). A usage error exits 2 and writes
nothing. SIGHUP, Ctrl-C, Ctrl-\ or SIGTERM at the value prompt ends the CLI by that signal
(a shell reports 129, 130, 131 or 143) and writes nothing; every other failure exits 1.
| Message | Cause | Fix |
|---|---|---|
read the settings of the broker AGENT_SECRETS_URL names: … |
AGENT_SECRETS_URL does not reach the broker. |
Check the address; the CLI reads the namespace, key, account and region from it, and names that store in everything it prints. |
load your AWS sign-in: failed to get shared config profile, … |
--profile or AWS_PROFILE names a profile your AWS config does not have. |
Name a profile from ~/.aws/config. |
read your AWS sign-in: … get credentials: failed to refresh cached credentials, … |
The AWS SDK found no usable credentials. For an IAM Identity Center profile the rest of the message says which: failed to read cached SSO token file when you have not signed in with that profile; when the sign-in has expired, the SSO session has expired or is invalid for a profile that names its own sso_start_url, or refresh cached SSO token failed for one that names an sso_session. |
Run aws sso login --profile <profile>, and name that profile with --profile or AWS_PROFILE. |
your AWS sign-in … is in account …, but the agent secrets are in account … |
The sign-in is in another account than the agent secrets: every form, a read included, refuses it. | Sign in to the account the message names. |
--owner me: …: name the owner with --owner <email> (exit 2) |
The credential the command runs under names no person: an AWS sign-in that is no IAM Identity Center sign-in (a machine’s role, an assumed service role, an IAM user), or on GCP a token Google’s userinfo endpoint does not name, as for a token minted without an email scope, or names as a service account. That credential reads and writes all the same; only me needs a person. |
Name the owner outright with --owner <email>. |
…; the store refused … |
The store’s IAM refused the identity the message names after the store refused: the AWS sign-in’s ARN, or on GCP the lent CLOUDSDK_AUTH_ACCESS_TOKEN, the gcloud login, or what the Application Default Credentials are. The CLI refuses no credential for its kind; IAM decides. |
Run the command under a credential IAM lets do it, or ask whoever runs the deployment for the role. |
… your gcloud login, but gcloud holds no login: sign in once with `gcloud auth login` |
GCP: CLOUDSDK_AUTH_ACCESS_TOKEN is unset and there are no Application Default Credentials, so the command needed your gcloud login, and gcloud auth list names none. |
Run gcloud auth login once, then the command again, or set CLOUDSDK_AUTH_ACCESS_TOKEN. |
… your gcloud login, but no gcloud is on PATH: install the Google Cloud CLI, then sign in once with `gcloud auth login` |
GCP: as in the row above, on a machine with no gcloud on PATH. |
Install the Google Cloud CLI, then run gcloud auth login once. |
… your gcloud login, but CLOUDSDK_CORE_ACCOUNT names …, and gcloud holds no login for it (it holds …) … |
GCP: CLOUDSDK_CORE_ACCOUNT names an account gcloud holds no login for. The CLI runs as no other login in its place, even where gcloud holds only one. |
Sign in as that account with gcloud auth login, or set CLOUDSDK_CORE_ACCOUNT to one of the logins the message lists. |
… your gcloud login, but gcloud holds several logins (…) and none is active … |
GCP: gcloud auth list lists more than one login and marks none active, and CLOUDSDK_CORE_ACCOUNT is unset. |
Run the command again as CLOUDSDK_CORE_ACCOUNT=<account> agent-secrets secret …, naming one of the logins the message lists. |
… your gcloud login, but `gcloud auth print-access-token` failed for …: … or … but `gcloud auth list` failed: … |
GCP: gcloud could not lend your login, most often because it needs you to sign in again; the rest of the message is gcloud’s own. | Run gcloud auth login once more, as the account the message names. |
load the Google Cloud credentials file GOOGLE_APPLICATION_CREDENTIALS names: … |
GCP: GOOGLE_APPLICATION_CREDENTIALS is set, and the file it names cannot be read or is not a credentials file. While it is set, no other credentials are read, so no sign-in changes this. |
unset GOOGLE_APPLICATION_CREDENTIALS: without it, the command uses your Application Default Credentials or your gcloud login. |
"…": not an agent secret name: … (exit 2) |
The name is not uppercase letters, digits and single underscores starting with a letter. | Write the name as an agent asks for it: DEMO_DEPLOY_TOKEN, not demo-deploy-token. |
the value is read from standard input, which was empty or no value was entered at the prompt (exit 2) |
create or set got an empty value: empty standard input, or Enter or Ctrl-D alone at the prompt. Nothing was written. |
Pipe the value in, or type it at the prompt. |
the value must be valid UTF-8 (exit 2) |
The prompt or standard input contained bytes that Secrets Manager’s text value cannot preserve. Nothing was written. | Supply a UTF-8 text value. |
--owner me: the sign-in's email … is not an email the broker accepts as an owner (exit 2), or --owner is me, shared or a person's email, not … (exit 2) |
--owner would write an owner tag the broker refuses: the AWS Identity Center session name me reads, or the email --owner names, is not an email the broker accepts. Nothing was written. |
Name the owner outright with --owner <email>, an email the broker accepts. |
the control byte 0x… cannot be typed at the prompt (exit 2) |
An unhandled control byte was typed or pasted. The prompt discards input through the line ending and any bracketed paste’s end before reporting the error. | Finish with Enter or Ctrl-D, then type the value again or pipe it in: agent-secrets secret set NAME < value.txt. |
nothing was stored: the terminal stopped while reading the value; run … again and type the whole value (exit 2) |
Ctrl-Z (or another stop) came while the value was being typed. The prompt discards what the terminal held unread when it stopped, so nothing that arrived with or before Ctrl-Z reaches your shell, and refuses the whole entry, even if some of it had already been read. Keys typed after the stop took effect went to your shell. | After fg, press Enter to finish discarding the entry, then run the command in the message and enter the whole value again. |
no shell can bring the value prompt to the foreground of this terminal; pipe the value in: … < FILE (exit 2) |
The command was started in the background (&) from a shell that has since exited, such as a subshell or the terminal’s own shell, so no shell can give it the terminal with fg. It showed no prompt and wrote nothing. |
Run the command in the message with the value piped in, or run the command again in the foreground. |
no value prompt is available on this platform; pipe the value in: … < FILE (exit 2) |
This build of agent-secrets has no hidden prompt: the prompt runs on Linux and macOS for amd64 and arm64. Nothing was written. |
Run the command in the message with the value piped in. |
read the value at the terminal: the terminal hung up before the value ended |
The terminal closed, or its ssh session dropped, while a value was still being typed outside a paste. Nothing was written. | Run the command again and enter the whole value, or pipe it in: agent-secrets secret set NAME < value.txt. |
a value of more than one line must be piped in: … (exit 2) |
The prompt found more than one line and wrote nothing. A bracketed paste is discarded through its closing mark. Without brackets, draining ends after 200 ms of quiet, with no total limit, and Ctrl-C ends it; a paste-start mark split before the line ends holds the drain for one second for the rest of the mark: later bytes can reach the shell, and a second line delayed past the quiet window can leave the first already stored without a refusal. | Pipe multi-line values in: agent-secrets secret set NAME < key.pem. This also replaces a first line a split paste stored. |
read the value at the terminal: the paste was cut short before its closing mark |
The terminal closed, or its ssh session dropped, while a paste was still arriving at the prompt, or a paste fell quiet for 10 seconds before its closing mark came. The bound counts the quiet since the last input, so a paste still arriving is read on, and input that arrived after the paste was given up reached your shell. Nothing was written. | Run the command again, and paste the whole value, or pipe it in: agent-secrets secret set NAME < key.pem. |
… CreateSecret, … ResourceExistsException: …, or InvalidRequestException from a create |
A secret of that name exists, perhaps scheduled for deletion. | secret set gives an existing secret a new value; secret restore brings back a deleted one. |
InvalidRequestException from set or retag |
The secret is scheduled for deletion, which Secrets Manager refuses to change. | secret restore it first. |
… AccessDeniedException: User: … is not authorized to perform: secretsmanager:… |
IAM refuses that call to your sign-in. Two causes: the secret is another person’s, and your access does not cover it; or your access does not include the permission the message names, which can happen on your own secret too. delete or restore refused on your own secret means your access is not granted secretsmanager:DeleteSecret or secretsmanager:RestoreSecret. Operating the broker lists each form’s permissions. |
For another person’s secret, ask its owner. For your own, ask whoever administers IAM in the broker’s account to grant the permission the message names. |
rpc error: code = PermissionDenied desc = … |
GCP: IAM refuses that call to your ADC identity. Same two causes as AWS’s AccessDeniedException: another person’s secret, or your own access missing the permission the call needs. Operating the broker lists each form’s GCP permissions. |
For another person’s secret, ask its owner. For your own, ask whoever administers IAM on the project to grant the permission the call needs. |
a shared secret's owner and tier are an administrator's to change, after AccessDeniedException (AWS) or PermissionDenied (GCP) from a retag |
The secret is shared, whose owner and tier only an administrator may change; or you asked to make your own secret shared (--owner shared) and your access does not include that retag. |
For a shared secret, ask an administrator. For your own, ask whoever administers IAM (AWS) or GCP IAM (GCP) to grant the retag. |
the broker at … refuses … after the write to … (reason=…), after broker: refusing … from … (reason=…) |
The write stands in the secret store the message names, but the broker leaves the secret out for that reason: absent, or a refusal Operating the broker lists, such as no-current-value. |
Fix what the reason names (for no-current-value, secret set gives the secret a value) rather than repeating the write. |
the broker at … still serves … after its delete in … |
The delete stands in the secret store, but the broker’s reread found the secret served. | secret show it; once it shows deleted:, ask for the reread again (curl -X POST "$AGENT_SECRETS_URL/v1/secrets/<NAME>/reread"). |
the write to … stands, but the broker at … could not be asked to reread … |
The broker was unreachable or rate-limited the reread (RATE_LIMITED). |
Do not repeat the write: what happens next depends on the write, and asking for the reread later serves it at once, across every broker process, since the one that answers tells the others. A refusal names its wait in Retry-After: a second for the broker-wide limit, four seconds for that limit on Google Secret Manager, and for your own address’s twice whichever of those the broker runs on — two seconds against the default store, eight under Secret Manager. The broker-wide limit takes a burst of ten, so writing more than ten secrets at once meets it. |
… has no owner tag or no tier tag, and a retag sends both: name the missing one with --owner or --tier (exit 2) |
The secret is missing a tag, and retag sends both. |
Name the missing one too. |
none removes a tag in a retag of its own: run the removal and the other changes as two retags (exit 2) |
The retag named --kind none or --max-grant none beside another change. A removal is a write of its own, so nothing was written. |
Run the removal alone, then the other change as a second retag. |
a github-app secret needs --github-owners naming at least one GitHub account, or the broker refuses it as github-owners-missing (exit 2) |
The create or retag would leave a github-app secret without GitHub owners: --kind github-app with no --github-owners, or --github-owners none on a github-app secret. Nothing was written. |
Name the accounts with --github-owners, or remove the kind first. |
--github-owners: github-owners separates GitHub logins with spaces, not commas ("<login> <login>") (exit 2) |
The list was separated by commas. Nothing was written. | Separate the logins with spaces and quote the value: --github-owners "acme-org ada". |
another write changed the secret after this retag read it, so the store refused this write |
GCP: another person’s write changed the secret’s annotations between this retag’s read and its write. Nothing was written. | secret show it, then run the retag again on the tags it holds now. |
another write changed the secret after this retag read it, leaving kind … with github-owners …, which the broker refuses; a retag on AWS is safe only with one writer at a time: the tags are restored to what this retag read |
AWS: another writer retagged the secret between this retag’s read and its write, and the two writes together left a github-app secret without owners. This retag put back the tags it read, which undoes the other writer’s change too, and asked the broker to reread them. |
Agree with the other writer who retags it, then run the retag again. |
the retag in … stands, but the broker at … serves … as …, not …: it still sees the old tags. Run this retag again to ask it to reread them, and store no value in … until a retag exits 0 |
The retag was written, but the broker’s reread read the secret’s tags stale (Secret Manager’s metadata can lag a write by hours), so it still serves the secret as it was. A secret it serves as stored releases its stored value raw. | Run the same retag again until it exits 0. Store no App key or login in the secret before that. |
Listing and revoking machine logins and grants
Section titled “Listing and revoking machine logins and grants”These are the messages of agent-secrets machine list|revoke and agent-secrets grant list|revoke
(revoke a session or a grant),
which act under this machine’s login. A usage error exits 2; every other failure exits 1.
| Message | Cause | Fix |
|---|---|---|
IN_SESSION: pid … is inside a registered machine session, which acts on itself alone; run machine and grant commands from your own shell |
The command ran inside an agent session’s process tree. The helper signs these commands with the machine’s login only for a process outside every session’s tree. The check covers that tree only: a process a session sends out of it (( cmd & ), setsid -f, a tmux server it started) passes. Through these commands it can list and revoke your own machine logins, never a service’s, and the operator’s grants. |
Run it from your own shell. A session ends its own grants with agent-secrets revoke. |
IN_SESSION: pid … may be inside a registered machine session: …; run machine and grant commands from your own shell |
The helper could not follow the command’s ancestry to init: it could not read a parent’s /proc/<pid>/stat, the walk looped, or it ran past 64 processes. The message names which. It refuses a caller it cannot show is outside every session. |
Run it from your own shell, started outside every session. |
this machine is not logged in to the secrets broker; run: agent-secrets machine login |
The helper holds no machine credential, so there is no login to act under. | Log the machine in. |
the launcher credential is not valid (LAUNCHER_INVALID); this machine's login to the broker AGENT_SECRETS_URL names is expired or revoked (run: agent-secrets machine login) |
The broker refused the machine’s credential: it expired, someone revoked it, or the clock or AGENT_SECRETS_URL is off. |
Log the machine in again. |
no agent-secrets-helper at … (…): machine and grant commands act under this machine's login, which its helper holds (exit 2) |
No helper listens on the socket. These commands run on a machine whose helper holds a login. | Start agent-secrets-helper and log it in, or point AGENT_SECRETS_HELPER_SOCK at its socket. |
this machine's agent-secrets-helper is older than this client and cannot sign for machine and grant commands; restart it on this release |
The helper running predates these commands. | Upgrade the helper to the same release as agent-secrets, then run agent-secrets-helper install --url … with the new binary: the unit names the binary’s resolved path, so a plain systemctl --user restart runs the old binary when the upgrade landed in a new directory. Install restarts a helper its unit runs only when the unit changed and no session is registered. After an upgrade in place (the binary replaced at the same path, so the unit is unchanged) install restarts nothing: run systemctl --user restart agent-secrets-helper once no session is registered. A running helper that predates drain is not restarted either: install says so, and you re-run it with --restart-with-sessions at a quiet moment. Stop a helper you started by hand first, since install refuses while it answers. Then log the machine in: a helper from before machine files kept no login to resume. |
only the person who approved a person's machine login may revoke it or only the grant's approver or its enrollment's operator may revoke it (NOT_APPROVER) |
The login or grant is someone else’s. | Ask the person who approved it, or the session’s operator. |
no such machine login (NOT_FOUND) or no such grant (NOT_FOUND) |
No login or grant has that id, or the login is a service’s, which machine revoke neither lists nor ends. |
Copy the id from machine list or grant list. Revoke a service’s login on Dispatch’s machine-login page. |
launcher credential ids are UUIDs (CREDENTIAL_ID_INPUT) or grant ids are UUIDs (GRANT_ID_INPUT) |
The id is not a UUID. | Copy the id from machine list or grant list. |
Approving a request and logging a machine in
Section titled “Approving a request and logging a machine in”| Message | Cause | Fix |
|---|---|---|
only the record's approver may decide it (NOT_APPROVER), deciding a machine login in Dispatch |
A person’s machine login is that person’s alone to decide. | That person decides it on Dispatch’s machine-login page. |
not yours to decide; only <approver> may, on the broker’s approval page |
The request names someone else as its approver. A secret request is decided on that page alone, and the page says who may decide it rather than refusing you a code. | The named approver decides it there (approving a request). |
still pending; it was not decided, on the broker’s approval page |
The broker refused your approval though the request names you, usually because the secret’s owner changed while it waited (what an owner change does). | After an owner change, the new owner approves it if the request waited on anyone, and the approver it names can still deny it. |
only the person who approved a person's machine login may revoke it (NOT_APPROVER) |
You did not approve that person’s machine login. Machine logins lists your own machines’ and every service’s login, which anyone signed in may revoke. | Ask whoever approved it to revoke it. |
a service logs in with its own token … (SERVICE_LOGS_IN_WITH_TOKEN, when starting it or approving it) |
A service’s login opened through a code, which the broker no longer accepts or approves: the Legion daemon logs in with its own Kubernetes token. | Deny it or let it expire. Upgrade a daemon that still opens one. |
request is already decided or this machine login has already been decided (RECORD_TERMINAL) |
It was decided already, or the session that asked has ended. | Nothing to do. |
… expired before its approver acted on it (RECORD_TERMINAL) |
It waited past its expiry: a request’s or a machine login’s. | The agent or the machine asks again. |
no pending machine login has this code (NO_SUCH_CODE) |
The code is mistyped, already decided, or expired. | Check the code on the machine’s terminal; start a new login if it expired. |
confirmation code does not match (CODE_MISMATCH) |
The decision carried a different code from the login it names. | Look the code up again and decide from that page. |
this machine login's key already holds a live launcher credential (KEY_HOLDS_LIVE_CREDENTIAL) |
The machine signed two logins with one key, and one is already approved. | Deny this one; the machine is already logged in. |
| No Credential requests section and no Live grants | Dispatch has no broker connected (DISPATCH_AGENT_SECRETS_URL is unset). |
See Operating the broker. |
A secret request is decided on the broker’s own page, signed in to the broker; these are what that page says when it refuses.
| Message | Cause | Fix |
|---|---|---|
Sign-in unavailable: The secrets broker could not reach its sign-in provider, … (503) |
The broker could not read the provider’s discovery document. Machines are served as before. | Try again shortly. An operator reads GET /signin/status, whose client_check says issuer_unreachable (What the sign-in is doing). |
Sign-in refused: Only an account of the … Workspace may sign in to the secrets broker. or Your account's email address is not verified, … |
The account signed in is outside BROKER_SIGNIN_DOMAIN, or its email is unverified. |
Sign in with your Workspace account. |
Page expired: This page is more than an hour old, so nothing was decided. Open the request again. |
A decision is accepted for an hour from the moment the page was drawn. | Open the request again and decide it there. |
Refused: This form was not made for your sign-in, … or This form names a request its page did not print, … |
The form came from another sign-in, another browser, or a page that never printed that request. | Open the request again and decide it there. |
Not signed in: Your sign-in to the secrets broker has ended, so nothing was decided. |
The session lapsed (a day at most), was signed out, or an operator ran broker end-sessions for the login. |
Sign in again from the request’s page. |
DECIDE_ON_BROKER_PAGE (403), from a Dispatch tab’s Approve or Deny on a secret request |
Only the broker’s own page decides a secret request; the refusal names that page. | Open the page the refusal names. A tab older than the Dispatch release whose Inbox links it can be reloaded. |