Recover Administrator Access

Available only for:

Self-hosted Bytebase 3.22.0 and later. The steps on this page need 3.24.0 or later. For 3.22.0 to 3.23.x, see Earlier versions.
Shell access to the host, or kubectl access to the cluster, that runs Bytebase

If nobody can sign in to Bytebase, run bytebase recovery on the server to turn password sign-in back on or to set a new password. Recovery needs no downtime. Wait one minute before you sign in. Bytebase keeps running while you recover, and it picks up the change within one minute, without a restart. If a Workspace Admin can still sign in, reset passwords, roles, and 2FA from the console instead.

Choose what to run

Situation	Run
An administrator knows their password, but Disallow signin with email & password is on	`enable-password-signin`
A user forgot their password, or is locked out after too many failed attempts	`reset-password`
Disallow signin with email & password is on, and no administrator knows a password, for example because they all use SSO	`reset-password` for an administrator, then `enable-password-signin`
A user was removed from Members and has no workspace role	The interactive menu, to reset the password and add a role

Recovery can’t:

Create or reactivate a user. The user must already exist and be active. Service accounts and workload identities don’t work.
Change a role that a user already has. The menu adds a role only to a user with no workspace role at all: not directly, not through a group, and not through allUsers. When such a user tries to sign in, Bytebase gives them the Workspace Member role, even if the sign-in fails, so add the role first. If no Workspace Admin is left and every user already has a role, recovery can’t restore administrator access.
Reset two-factor authentication. Only another Workspace Admin can reset a user’s 2FA.
Bypass the workspace domain. When Members restriction is on, the email must belong to an allowed workspace domain.

If none of these fits your situation, contact us.

Run recovery

No downtime, then wait one minute. Leave Bytebase running: don’t stop it before recovery, and don’t restart it afterward. After the command succeeds, wait one minute for Bytebase to pick up the change before you sign in. Run the command inside the container that runs Bytebase. That container already knows how to reach the metadata database, where Bytebase stores its users and settings, so you don’t pass any connection details: no PG_URL, no volume, and no network flags. Back up the metadata database first. A snapshot from a managed PostgreSQL service needs no downtime. The procedures in Back up and restore stop Bytebase while they copy.

Docker
Kubernetes
Binary

The commands use the container name bytebase. Replace it with yours.

Check the version:
docker exec bytebase bytebase version
If it’s older than 3.24.0, follow Earlier versions instead.

Run the command for your situation:

docker exec bytebase bytebase recovery enable-password-signin

docker exec bytebase bytebase recovery reset-password --email <email> --password '<password>'

If you start Bytebase with your own --data or --port, add the same flags at the end of the command. docker exec doesn’t pass them on.

The commands work for both the Helm chart and the manifest from Deploy with Kubernetes. Both create a Pod named bytebase-0 with a container named bytebase. With several replicas, use any one of them.

Check the version:

kubectl -n <namespace> exec bytebase-0 -c bytebase -- bytebase version

If it’s older than 3.24.0, follow Earlier versions instead.

Run the command for your situation:

kubectl -n <namespace> exec bytebase-0 -c bytebase -- bytebase recovery enable-password-signin

kubectl -n <namespace> exec bytebase-0 -c bytebase -- bytebase recovery reset-password --email <email> --password '<password>'

The chart's escapePassword option is on

With the experimental escapePassword option, the server’s start script builds PG_URL, so a command started with kubectl exec doesn’t have it. Build it the same way before you run recovery:

kubectl -n <namespace> exec bytebase-0 -c bytebase -- /bin/sh -c 'export PG_URL=postgres://<user>:$(cat /mnt/bytebase-shared-volume/pg_escaped_password)@<host>:<port>/<dbname>; exec bytebase recovery enable-password-signin'

If you can’t use kubectl exec, for example because you change the cluster only through manifests or a pipeline, run recovery as a Job.

For a bytebase binary that runs directly on a host. Unlike a container, the command doesn’t share the server’s settings, so you pass them yourself.

Check the version:
bytebase version
If it’s older than 3.24.0, follow Earlier versions instead.
Run the command with the same binary, as the same OS user that runs the server. For embedded PostgreSQL, pass the server’s --data as an absolute path, and its --port if it sets one:
bytebase recovery enable-password-signin --data <data_dir>
For external PostgreSQL, pass the server’s PG_URL:
PG_URL='<pg_url>' bytebase recovery reset-password --email <email> --password '<password>'
If the server isn’t running, recovery starts the embedded PostgreSQL itself. Don’t start Bytebase until the command exits.

Each command does one thing and exits:

enable-password-signin switches off Disallow signin with email & password. It works only when an active Workspace Admin has a password, so that someone can sign in afterward.
reset-password sets a new password that meets the workspace password restriction, and clears the lockout from failed attempts. It doesn’t change the user’s role, 2FA, or SSO sign-in.

When it works, the command prints one line that ends with Live within one minute., for example:

Password updated for user admin@example.com. Live within one minute.

If reset-password also prints User <email> has no workspace role, add a role with the interactive menu before the user signs in.
If the command prints Error:, read the reason in Troubleshooting. When the success line appears above the error, your change was saved, and only a follow-up step failed, such as writing the audit log.
The command exits with status 0 only when everything succeeded. Running it again is safe: the second run changes nothing.

The password you pass with --password is saved in your shell history, and other users on the host can see it while the command runs. Change it after you sign in. To keep it hidden, use the interactive menu instead.

After recovery

Wait one minute, then sign in with the email and password. Bytebase applies the change within one minute. If you sign in earlier, the new password can be rejected, and five rejected attempts within 10 minutes lock the account for 10 minutes. If that happens, run the same reset-password command again to clear the lock, then wait one minute.
If you passed the password with --password, change it.
Repair and test the identity provider.
Switch Disallow signin with email & password back on in Settings > General > Account if it was on before.
Review the recovery events in the audit log. Each action records a warning-level event whose method starts with /bytebase.cli.Recovery/.

Run recovery as a Kubernetes Job

Use a Job when you can’t run kubectl exec. The Job’s Pod runs the command once, beside the server. It must use the same image as the server and connect to the metadata database the same way. A Job only works with external PostgreSQL, which is what the chart and the manifest use.

Get the exact image the server runs:
kubectl -n <namespace> get pod bytebase-0 -o jsonpath='{.status.containerStatuses[?(@.name=="bytebase")].imageID}'
Use the printed value as <image>, without the docker-pullable:// prefix if there is one. Don’t copy the tag from the StatefulSet: a tag such as latest can pull a newer version than the server runs.

For reset-password, store the new password in a Secret, so it isn’t written in the manifest:

kubectl -n <namespace> create secret generic bytebase-recovery --from-literal=password='<password>'

Create the Job. This example is for a server that reads PG_URL from a Secret, which is what the chart’s existingPgURLSecret option sets up:

apiVersion: batch/v1
kind: Job
metadata:
  name: bytebase-recovery
  namespace: <namespace>
spec:
  backoffLimit: 0
  template:
    spec:
      restartPolicy: Never
      serviceAccountName: <service_account>
      containers:
        - name: recovery
          image: <image>
          args: ["recovery", "reset-password", "--email", "<email>", "--password", "$(NEW_PASSWORD)"]
          env:
            - name: PG_URL
              valueFrom:
                secretKeyRef:
                  name: <pg_url_secret>
                  key: <pg_url_secret_key>
            - name: NEW_PASSWORD
              valueFrom:
                secretKeyRef:
                  name: bytebase-recovery
                  key: password

<service_account> is the service account of the server Pod.
Keep each argument as its own item, as shown. If you write the whole command as one item, it fails before recovery runs, and the error message shows the password.
Kubernetes fills in $(NEW_PASSWORD) from the Secret. If the variable isn’t defined, the command receives $(NEW_PASSWORD) as is and refuses it.
For enable-password-signin, use args: ["recovery", "enable-password-signin"] and remove NEW_PASSWORD.
backoffLimit: 0 stops Kubernetes from retrying a failed run. Never run recovery as a Deployment or StatefulSet, which would repeat it every time it exits.

Read the result. The Job succeeds when the command exits with status 0:
kubectl -n <namespace> logs job/bytebase-recovery

After you can sign in, delete the Job and the Secret:

kubectl -n <namespace> delete job/bytebase-recovery secret/bytebase-recovery --ignore-not-found

Then follow After recovery.

The server doesn't read PG_URL from a Secret

Copy what the Job needs from the server Pod’s spec, which kubectl -n <namespace> get pod bytebase-0 -o yaml prints:

How the server connects	What to copy into the Job
`PG_URL` written as a value, as in the manifest from Deploy with Kubernetes, or the chart’s `url` or `pgHost`, `pgUsername`, and `pgPassword` options	The server container’s `PG_URL` entry, as it is
The chart’s `existingPgPasswordSecret` option	The server container’s `PG_PASSWORD` and `PG_URL` entries, in that order, because `PG_URL` refers to `$(PG_PASSWORD)`
The chart’s `awsRdsIam` or `gcpCloudSqlIam` option	The server container’s `PG_URL` entry, and the server Pod’s service account, which carries the cloud identity
A `PG_URL` that is a file path, such as a mounted Secret	The volume and the mount that hold the file, at the same path
Mounted TLS certificates or other files	The same volumes and mounts
The chart’s experimental `escapePassword` option	The server Pod’s init container, its `bytebase-shared-volume` volume, and a script that builds `PG_URL`, as below

With escapePassword, the init container writes the escaped password to the shared volume. Keep the init container and the volume, and give the recovery container the mount and this script:

command: ["/bin/sh", "-c"]
args:
  - |
    export PG_URL=postgres://<user>:$(cat /mnt/bytebase-shared-volume/pg_escaped_password)@<host>:<port>/<dbname>
    exec bytebase recovery reset-password --email '<email>' --password "$NEW_PASSWORD"
volumeMounts:
  - name: bytebase-shared-volume
    mountPath: /mnt/bytebase-shared-volume

Inside the script, write "$NEW_PASSWORD" without parentheses, so the shell doesn’t treat any character of the password as part of the script.

Use the menu to add a role to a user, or to type the password where nobody can see it. Run bytebase recovery with no action, in the same place as the commands above, and add -it to docker exec or kubectl exec to get a terminal. Over SSH, connect with ssh -t.

docker exec -it bytebase bytebase recovery

The menu stays open until you enter q, so you can run several actions in a row. This example resets a password:

Select a recovery action:
1. Enable password sign-in
2. Reset user password
q. Exit
> 2
User email (blank to cancel): admin@example.com
Password requirements:
- Minimum length: 8 characters
New password:
Confirm new password:
Resetting user password...
Password updated for user admin@example.com. Live within one minute.

Enable password sign-in asks you to confirm with y, then does what enable-password-signin does.
Reset user password does what reset-password does. If the user has no workspace role, it then offers to add one: confirm, pick a role from the list, and confirm again. If you decline, the new password stays.

Each action is saved as soon as it finishes, so you can exit at any time. Then follow After recovery.

Troubleshooting

Terminal required

Error: recovery requires terminal input and output; without a terminal, run "bytebase recovery enable-password-signin" or "bytebase recovery reset-password --email <email> --password <password>"

The interactive menu needs a terminal, so that the password stays hidden. Add -it to docker exec or kubectl exec, or run one of the two commands, which need no terminal.

Embedded metadata database may be in use

Error: failed to open recovery metadata store: cannot connect to the embedded metadata database (...), and its data directory "/var/opt/bytebase/pgdata" may be in use: run recovery inside the Bytebase server's container, with the server's --data and --port; if no Bytebase server is running, start it once, then run recovery that way

Recovery couldn’t reach the server’s embedded PostgreSQL. Usually it runs in a second container that mounts the data volume, or with a different --port than the server. Run it in the server’s container, with the server’s --data and --port. If no server is running, the lock file is left over from a crash: start Bytebase once, then run recovery in its container.

Data directory not accessible

Error: failed to open recovery metadata store: unable to access --data directory /srv/bytebase: stat /srv/bytebase: no such file or directory

The --data path doesn’t exist where recovery runs. Pass the same --data as the server, as an absolute path.

Embedded metadata database not initialized

Error: failed to open recovery metadata store: embedded metadata database is not initialized at "/var/opt/bytebase/pgdata"

This error appears only when PG_URL isn’t set. The --data directory isn’t the one the server uses. Pass the server’s --data. If the server uses external PostgreSQL, pass its PG_URL instead.

Embedded metadata database not readable

Error: failed to open recovery metadata store: failed to inspect embedded metadata database: stat /srv/bytebase/pgdata/PG_VERSION: permission denied

Recovery runs as a different OS user than the server. Run it as the user that owns the data directory.

External metadata database not connected

Error: failed to open recovery metadata store: failed to connect to external metadata database: ...

This error appears only when PG_URL is set. The text after the last colon names the cause. Recovery needs everything the server uses to reach PostgreSQL: the network, the file when PG_URL is a file path (failed to read database URL from file), TLS files, or IAM credentials. Running in the server’s container provides all of them. For a Job, compare it with the server Pod’s spec.

Unknown command

Error: unknown command "recovery" for "bytebase"

The version is older than 3.22.0. Upgrade to 3.24.0 or later, then run recovery.

Unknown subcommand

Error: unknown command "enable-password-signin" for "bytebase recovery"

The version is 3.22.0 to 3.23.x, which has only the interactive menu. reset-password fails the same way, with Error: unknown flag: --email. Follow Earlier versions.

No usable workspace administrator

Error: no usable workspace administrator has an active end-user identity with a password credential

enable-password-signin needs an active Workspace Admin with a password. Run reset-password for an administrator, then try again. If no administrator is left, use the interactive menu to reset the password of a user with no workspace role and add them as Workspace Admin. If every user already has a role, recovery can’t restore administrator access.

Change not applied after one minute

Recovery saved the change to a different metadata database than the servers use. Make sure recovery uses the same PG_URL as the servers, including the database name.

Earlier versions

On 3.22.0 to 3.23.x, bytebase recovery has only the interactive menu, and the servers don’t see its change until they restart. So recovery needs downtime. Alternatively, upgrade to 3.24.0 or later first, then follow Run recovery.

Recover on 3.22.0 to 3.23.x

Because the server is stopped, recovery runs in a new, one-off container. Unlike docker exec, that container starts with nothing, so you pass it the same image and the same metadata database settings as the server.

Write down the exact version and image the server runs. Recovery must use the same version, and the restart must not upgrade Bytebase.
Stop every Bytebase server that uses the metadata database, and keep them stopped until recovery exits. On Kubernetes, scale the StatefulSet to 0, and pause helm upgrade and any GitOps sync, which would restore the replicas.
Back up the metadata: the data directory for embedded PostgreSQL, or the metadata database for external PostgreSQL.
Run bytebase recovery in a one-off container, with a terminal. For embedded PostgreSQL with Docker, mount the same data volume as the server:
docker run --rm -it --volume <host_data_dir>:/var/opt/bytebase <image> recovery
For external PostgreSQL with Docker, pass -e 'PG_URL=<pg_url>' instead of the volume. On Kubernetes, pass the same PG_URL, and press Enter if the menu doesn’t appear:
kubectl -n <namespace> run bytebase-recovery --rm -it --restart=Never --image=<image> --env='PG_URL=<pg_url>' --command -- bytebase recovery
Also pass anything else the server uses to reach PostgreSQL: the network, a mounted connection-string file or TLS certificate, and IAM credentials. On Kubernetes, when PG_URL comes from a Secret, a file, or IAM authentication, create a Pod with the same env, volumes, volumeMounts, and serviceAccountName as the StatefulSet’s Pod template, then kubectl exec -it into it. With a binary on a host, run the same binary as the same OS user, with the same --data or PG_URL.
Use the menu as described in Use the interactive menu. On these versions, nothing takes effect until you restart Bytebase.
Start Bytebase again on the image you wrote down, without a floating tag such as latest and without --pull always. On Kubernetes, pin the tag first, then restore the replica count.

AI

Security

General

Recover Administrator Access

Choose what to run

Run recovery

After recovery

Run recovery as a Kubernetes Job

Use the interactive menu

Troubleshooting

Terminal required

Embedded metadata database may be in use

Data directory not accessible

Embedded metadata database not initialized

Embedded metadata database not readable

External metadata database not connected

Unknown command

Unknown subcommand

No usable workspace administrator

Change not applied after one minute

Earlier versions

​Choose what to run

​Run recovery

​After recovery

​Run recovery as a Kubernetes Job

​Use the interactive menu

​Troubleshooting

​Terminal required

​Embedded metadata database may be in use

​Data directory not accessible

​Embedded metadata database not initialized

​Embedded metadata database not readable

​External metadata database not connected

​Unknown command

​Unknown subcommand

​No usable workspace administrator

​Change not applied after one minute

​Earlier versions

Choose what to run

Run recovery

After recovery

Run recovery as a Kubernetes Job

Use the interactive menu

Troubleshooting

Terminal required

Embedded metadata database may be in use

Data directory not accessible

Embedded metadata database not initialized

Embedded metadata database not readable

External metadata database not connected

Unknown command

Unknown subcommand

No usable workspace administrator

Change not applied after one minute

Earlier versions