# Two-factor Authentication Source: https://docs.bytebase.com/administration/2fa Two-factor authentication (2FA) provides an extra layer of security for member accounts. When signing in, you will be required to enter the security code generated by your Authenticator App. Bytebase implements rate limiting during MFA verification to protect against brute force attacks. Users are allowed a maximum of **5 failed MFA attempts within a 5-minute window**. The MFA temporary token expires after 5 minutes. See [Sign-in Restriction](/administration/sign-in-restriction#rate-limiting-for-login-attempts) for more details. ## Configuring 2FA using a TOTP mobile app ### Step 1: Download a TOTP app A time-based one-time password (TOTP) application automatically generates an authentication code that changes after a certain period of time. We recommend using cloud-based TOTP apps such as: * [Google Authenticator](https://googleauthenticator.net/) * [Authy](https://authy.com/) * [Microsoft Authenticator](https://www.microsoft.com/en-us/security/mobile-authenticator-app/) * [1Password](https://support.1password.com/one-time-passwords/) ### Step 2: Enable "Two-factor authentication" In the user's profile dashboard page, turn on the "Two-factor authentication" switch to enable 2FA for your account. enable-2fa ### Step 3: Scan the QR code with your TOTP app In the "Two-factor authentication" page, you will be prompted to scan the QR code with your TOTP app. Then enter the authentication code generated by your TOTP app to verify your identity. 2fa-setup-step1 If you can't scan the QR code, click **enter the text** to see the secret. You can then manually enter it in your TOTP application to get the authentication code. enter-your-secret ### Step 4: Download the recovery codes After verifying your TOTP app, you will be prompted to download your recovery codes. Under "Download your recovery codes", click **Download** to download your recovery codes to your device. Save them to a secure location because your recovery codes can help you get back into your account if you lose access. 2fa-setup-step2 ### Step 5: Click **I have saved my recovery codes** to finish After saving your two-factor recovery codes, click **I have saved my recovery codes** to enable two-factor authentication for your account. ## Enable Require 2FA for all users The workspace admin can require all users to enable two-factor authentication (2FA) in the Security setting page. This will ensure that all users have an extra layer of security for their accounts. enable-require-2fa ## Reset a user's two-factor authentication When a user loses both their authenticator app and their recovery codes, a workspace admin can reset their 2FA: 1. Go to **Settings > IAM & Admin > Members** and click the user's name to open their profile. 2. In the **Two-factor authentication** section, click **Disable**. The user can then sign in with their password and set up 2FA again. Available to workspace admins on self-hosted Bytebase. # Announcement Banner Source: https://docs.bytebase.com/administration/announcement Display a workspace-wide banner — a maintenance notice, a policy reminder — to everyone in the workspace. ## Prerequisites * **Workspace Admin** or **Workspace DBA** role ## Set the announcement 1. In the workspace, go to **Settings** > **General** and scroll to the **Announcement** section. 2. In **Title**, enter the banner text. To make the banner a link, fill in **Link** — clicking the banner then opens the link. Leaving **Title** empty hides the banner. 3. Pick a **Theme**: * **Normal**, **Warning**, or **Critical** — built-in color presets. * **Custom** — set your own **Background** and **Text** colors. 4. Click **Update**. The banner appears at the top for everyone in the workspace. # Archive Source: https://docs.bytebase.com/administration/archive Archiving is a soft delete. An archived resource keeps all of its data and history, disappears from the normal lists, and can be restored at any time. Instances and projects can be archived. Accounts — users, service accounts, and workload identities — use the same soft delete under a different name: they are **deactivated**. There is no separate archive dashboard. In both cases you find the hidden resources on the page you archived or deactivated them from, by changing that page's filter. ## Instances To archive an instance, go to **Instances**, open the row action menu, and click **Archive**. If databases on the instance are still assigned to projects, select **Force to archive** to unassign them. An archived instance and its databases stop appearing in instance lists, database lists, and the SQL Editor. To find archived instances, go to **Instances**, click the search box, choose the **State** scope, and select **Archived**. Select **All** to list active and archived instances together. The list defaults to **Active**. To restore, open the row action menu and click **Restore**. The **Archived** and **All** options appear only if you hold the `bb.instances.undelete` permission, which is granted to **Workspace Admin** and **Workspace DBA**. Archive and restore are also available on the instance detail page. ## Projects To archive a project, go to **Projects**, open the row action menu, and click **Archive**. You can also archive from the project's **Settings** > **General** > **Danger Zone**. To find archived projects, use the **State** scope on **Projects** the same way as for instances, then click **Restore**. Restoring is also available from the archived project's **Settings** > **General** > **Danger Zone**. The **Archived** and **All** options require the `bb.projects.undelete` permission, granted to **Workspace Admin** and **Workspace DBA**. ## Users, service accounts, and workload identities Each account type has its own page: **IAM & Admin** > **Users**, **Service Accounts**, and **Workload Identities**. To deactivate an account, open its row action menu and click **Deactivate**. To find deactivated accounts, turn on **Show inactive accounts** at the bottom of the page, then click **Restore** on the account you want back. ## Permanent deletion Archiving is reversible; permanent deletion is not. **Workspace Admin** and **Workspace DBA** can permanently delete an instance or a project from the same **Delete** action, whether or not it has been archived first. Deleting a still-active resource archives it first, then removes it. * Deleting an instance asks you to type the instance ID to confirm, and drops its databases along with it. * Deleting a project moves its databases to the default project. The default project itself cannot be archived or deleted. Environments have no archive state — deleting an environment removes it immediately. # Customize Logo Source: https://docs.bytebase.com/administration/customize-logo This document shows you how to add a logo to your workspace. ## Prerequisites * **Workspace Admin** or **Workspace DBA** role ## Procedure 1. Click **Settings** on the top bar. 2. Click **General** under **Workspace**, and scoll down till the **Branding** section is displayed. 3. Upload or drag your logo file to the **Logo** area according to the requirement on the screen. 4. Click **Update**, your logo will be uploaded and displayed on the top left. customize-the-logo # Database Group Source: https://docs.bytebase.com/administration/database-group Database groups are a powerful feature in Bytebase that allow you to manage collections of databases as a single unit. This simplifies operations when you need to apply consistent changes or run queries across multiple databases with similar structures. ## Overview A database group is a logical collection of databases that share common characteristics. Instead of selecting databases individually for each operation, you can define a group once and reuse it for batch operations. Database groups are particularly useful in the following scenarios: * **Multi-Tenant SaaS Applications**: Software as a Service providers often maintain separate database instances for each customer or tenant. Database groups enable you to apply updates across all tenant databases simultaneously while maintaining isolation. * **Geographic Distribution**: Organizations operating across multiple regions (US, EU, AP) often deploy separate database instances for compliance, latency, or data sovereignty requirements. Groups help manage these regional deployments efficiently. * **Database Sharding**: For applications using horizontal partitioning, database groups simplify managing sharded databases by treating all shards as a cohesive unit during schema migrations. ## Creating a Database Group ### Prerequisites * You must have appropriate project permissions * Target databases should already be added to your Bytebase project ### Steps 1. **Navigate to Database Groups** Within your project, click **Database > Groups** in the left sidebar. bb-project-groups-create 2. **Create New Group** Click **New database group** to open the creation dialog. 3. **Configure Group Settings** * **Name**: Provide a descriptive name for your database group * **Description** (optional): Add details about the group's purpose * **Database Selection Rules**: Define criteria to automatically include databases bb-new-db-group-with-raw ### Database Selection Methods #### Basic Filtering Select databases based on: * Environment (Development, Staging, Production) * Instance location * Database name patterns #### Advanced Filtering with Labels For more complex scenarios, use database labels with raw expressions: ``` Where resource.labels.YOUR_LABEL_KEY == "YOUR_LABEL_VALUE" and has(resource.labels.YOUR_LABEL_KEY) ``` **Example expressions:** * Filter by tenant: `Where resource.labels.tenant == "enterprise" and has(resource.labels.tenant)` * Filter by region: `Where resource.labels.region == "us-west" and has(resource.labels.region)` * Combine multiple labels: `Where resource.labels.tier == "premium" and resource.labels.region == "eu" and has(resource.labels.tier) and has(resource.labels.region)` 4. **Save the Configuration** Click **Save** to create the group. The new group will appear in your database groups list and be available for batch operations. ## Managing Database Groups ### Editing Groups To modify an existing group: 1. Navigate to **Database > Groups** 2. Click on the group name 3. Update the configuration as needed 4. Save your changes ### Deleting Groups To remove a database group: 1. Navigate to **Database > Groups** 2. Click the delete icon next to the group 3. Confirm the deletion **Note**: Deleting a group does not affect the underlying databases; it only removes the grouping configuration. # Manage License Source: https://docs.bytebase.com/administration/license Bytebase offers [3 pricing plans](https://www.bytebase.com/pricing): `Free`, `Pro`, `Enterprise`. | Plan | Deployment | Users | Pricing | | -------------- | ------------------ | --------- | --------------- | | **Free** | Self-host or Cloud | Up to 20 | Free | | **Pro** | Cloud only | Unlimited | \$20/user/month | | **Enterprise** | Self-host or Cloud | Custom | Custom | **Pro plan is cloud-only.** Self-hosted deployments require an Enterprise license. Existing Pro self-host licenses are grandfathered. Self-hosted Free instances can start a [14-day Enterprise trial](https://www.bytebase.com/contact-us/). Running more than one self-hosted Bytebase replica requires HA to be enabled in the license. If multiple replicas are detected without HA enabled, Bytebase disables backend runners for safety. As a result, background operations such as rollout task execution, plan checks, schema sync, and other scheduled runners will stop progressing until you scale back to a single replica or enable HA in the license. ## How Users Are Counted Your license has a single user limit. Two user counts are checked against it, each on its own: * **Workspace users** — members added to the workspace, who sign in to the Bytebase console. * **Active VCS users** — non-bot pull request / merge request authors observed by `bytebase-release` workflows on GitHub, GitLab, and Bitbucket. An author stays active for 90 days after they were last seen, then drops off automatically. Bytebase relies on the `bytebase-release` workflow to exclude bot accounts before reporting authors. Bytebase cannot match a VCS author to a workspace user, so the two counts stay independent — the same person may appear in both. Your license only needs to cover the larger count: a team with 10 workspace users and 20 active VCS users needs a 20-user license, not 30. When requesting a license, provide both numbers. Once the active VCS user count reaches the limit, a release check reporting a new author fails with a `resource_exhausted` error; authors already counted keep working. To admit new authors, increase the license user limit or wait for inactive authors to age out of the 90-day window. To review usage, go to **Settings > Subscription**, where the active VCS user count is shown. Download the full list as a CSV file with columns `vcs_type`, `user_id`, `user_name`, `display_name`, and `last_seen_at`. ## Configure Workspace License Go to **Settings > Subscription**, paste your license and click **Upload License**. Subscription page with plan stats, active VCS user count, and license upload box ## Configure Instance License Some features (e.g. [Dynamic Data Masking](/security/data-masking/overview)) require assigning a license to the instance. Enterprise-wide features like [SSO](/administration/sso/overview/) work without per-instance assignment. The instance quota on your license covers all instances in the workspace, so every instance can be assigned. You can assign instance licenses from: 1. **The instance page** — toggle **Assign License** directly on each instance. 2. **Settings > Subscription** — click the instance license stat or its edit icon to open the **Manage Instance License** drawer, tick the instances, and click **Confirm**. Manage Instance License drawer with instances selected 3. **Gated feature prompts** — when you encounter a feature requiring a license, click **Assign License** in the popup. Require instance license popup for data masking with Assign License button # Database Change Mode Source: https://docs.bytebase.com/administration/mode Bytebase supports two database change modes. In the UI, this setting appears as the **Default landing page**: * **Workspace**. Use issues to request, review, roll out, and version database changes. Suitable for teams with access control and a CI/CD workflow. * **SQL Editor**. Use SQL Editor to execute database changes directly. Suitable for single-person use or environments that do not require supervision. You may also choose this mode if you want to [embed SQL Editor into your own app](/tutorials/embed-sql-editor/). ## Configuration The first registered user is prompted to choose the mode during the setup wizard. Afterwards, the **Workspace Admin** can change it under **Settings** > **General**. Default landing page setting ## Workspace The default landing page is the workspace, where the default view is **My Issues**. ## SQL Editor The default landing page is the **SQL Editor**. # Password Restriction Source: https://docs.bytebase.com/administration/password ## Default restriction * Minimum length for password should no less than 8 characters * The password must contain at least one letter, regardless of upper case or lower case The restrictions above can be seen when your mouse hovers over the `?` icon. pwd-restrictions ## Configure restriction Go to **Settings** -> **General** from the left side bar in Workspace. Scroll down to the **Account** section, where you can alter the minimum length or components of your password. You can also decide whether users should reset their password after the first login or after a specified number of days. pwd-configuration # Risk Center Source: https://docs.bytebase.com/administration/risks The **Risk Center** (**CI/CD > Risk Center**) shows how Bytebase evaluates the risk level of database changes. Bytebase automatically assigns one of three risk levels — **High**, **Moderate**, or **Low** — to each database change. Currently, the risk level is determined purely based on the SQL statement types, with more sophisticated analysis planned. | Risk Level | Statement Types | | ------------ | -------------------------------------------------------- | | **High** | `DROP_DATABASE`, `DROP_TABLE`, `DROP_SCHEMA`, `TRUNCATE` | | **Moderate** | `DELETE`, `UPDATE`, `ALTER_TABLE`, `DROP_INDEX` | | **Low** | Everything else | If a change contains multiple statements, Bytebase takes the highest risk level found across all statements. ## Usage ### Custom Approval Custom Approval is mostly used by [UI workflow](/change-database/change-workflow/#ui-workflow). If you use [GitOps workflow](/change-database/change-workflow/#gitops-workflow), we recommend you to configure approval in the PR/MR process. Risk levels are available as the `risk_level` variable when defining [custom approval flows](/change-database/approval). You can use conditions like: ```text theme={null} risk_level == "HIGH" risk_level == "MODERATE" && resource.environment_id == "prod" ``` ### SQL Review [SQL review](/sql-review/review-policy/) returns the associated risk level for the checked SQL statement. Bytebase GitHub action can also surface the risk level under a [PR](https://github.com/bytebase/example-gitops-github-flow/pull/6#issuecomment-2731413296). sql-review # Roles and Permissions Source: https://docs.bytebase.com/administration/roles ## Overview Bytebase uses RBAC (Role-Based Access Control) to manage permissions. A **role** is a collection of permissions that can be granted to users and user groups. ### Permission Scope Permissions in Bytebase apply to two levels of resources: * **Workspace-level resources**: Settings, instances, environments, and overall workspace management * **Project-level resources**: Project settings, databases, issues, and project-specific operations ### Role Types Bytebase provides two types of roles: #### Built-in Roles **Workspace roles:** * `Workspace Admin` - Full administrative control * `Workspace DBA` - Database administration across all projects * `Workspace Member` - Basic access (automatically assigned to all users) **Project roles:** * `Project Owner` - Full control over project resources * `Project Developer` - Create and manage database changes; create Export issues for one-time exports * `Project Releaser` - Approve and release changes * `SQL Editor User` (formerly `Project Querier`) - Query in SQL Editor; run DDL/DML in the environments selected in the grant * `SQL Editor Read User` - Read-only query in SQL Editor * `Project Viewer` - Read-only access * `GitOps Service Agent` - Automated CI/CD workflows to create and execute database changes via GitOps #### Custom Roles Organizations can create custom roles with specific permission sets tailored to their needs. Custom roles are defined at the workspace level and can be granted at both workspace and project levels. **Granular Permissions:** Bytebase provides granular permissions for sensitive operations like masking policies. For example, instead of generic `bb.policies.create/update/delete` permissions, you can grant specific permissions like `bb.policies.createMaskingExemptionPolicy` or `bb.policies.updateMaskingRulePolicy` to control access to masking exemptions and global masking rules separately. ## Creating Custom Roles 1. Navigate to **IAM & Admin > Custom Roles** 2. Click **Add role** 3. Configure permissions for your new role add-custom-role Use **Import from role** to start with an existing role's permissions and modify them as needed. **Example:** To create a role that can approve and comment on issues but not execute them, create a `Project Approver` role by importing from `Project Releaser` and removing execution permissions. ## Assign Roles Roles (both built-in and custom) are assigned to users and groups through IAM policies at two levels. org-role-mapping ### Workspace level As a **Workspace Admin** (the `bb.workspaces.setIamPolicy` permission), click **IAM & Admin > Members** on the left sidebar, then **Grant Access**. You can grant both workspace roles and project roles here; a project role granted at the workspace level applies to **all projects**. Workspace-level grants carry no database scope or expiration. Every user holds at least one workspace role — **Workspace Member** by default — and may hold zero project roles. bb-workspace-grant-access **Inheritance:** Project IAM policies automatically inherit roles granted at the workspace level. For example, if a user is granted `Project Developer` at the workspace level, they have that role in all projects. ### Project level As a **Workspace Admin**, **Workspace DBA**, or **Project Owner** of the project (the `bb.projects.setIamPolicy` permission), click **Manage > Members** in the project, then **Grant Access**. Project-level grants only affect that project. bb-project-grant-access In the grant: 1. **Role** — the project role to grant, built-in or [custom](#creating-custom-roles). 2. **Reason** — an optional note stored with the binding. 3. **Databases** — for roles carrying SQL query permissions: **all databases** in the project, **manually selected** databases, schemas, and tables, or a set matched by a **CEL expression**. 4. **Environments** — for roles carrying `bb.sql.ddl` / `bb.sql.dml`: where those statements may run directly in SQL Editor. The environment scope is part of the permission itself — elsewhere, or when no environment is selected, SQL Editor directs the user to create a database change plan instead. 5. **Expiration** — an optional end time after which Bytebase revokes the binding automatically, bounded by the workspace [maximum role expiration](/security/database-permission/request/#maximum-role-expiration). ### Review and revoke The Members page lists every member with their granted roles, scopes, and expirations. From there you can edit a binding or **Revoke Access** — with the same permission as granting. bb-project-revoke-access ## Appendix ### Workspace Role Permissions By default, the first registered user is granted the `Admin` role, all following registered users are granted `Member` role. `Admin` can update any user's role later. | Workspace Permission | Member | DBA | Admin | | ----------------------------------- | ------ | --- | ----- | | Change own name and password | ✔️ | ✔️ | ✔️ | | Add new user | | | ✔️ | | View all users | ✔️ | ✔️ | ✔️ | | Change any user's role | | | ✔️ | | De-activate/re-activate user | | | ✔️ | | Change any user's name and password | | | ✔️ | | Manage identity providers (SSO) | | | ✔️ | | Add environment | | ✔️ | ✔️ | | View all environments | ✔️ | ✔️ | ✔️ | | Edit environment | | ✔️ | ✔️ | | Reorder environment | | ✔️ | ✔️ | | Archive environment | | ✔️ | ✔️ | | View all instances | | ✔️ | ✔️ | | Add instance | | ✔️ | ✔️ | | Edit instance | | ✔️ | ✔️ | | Archive instance | | ✔️ | ✔️ | | Sync instance schema | | ✔️ | ✔️ | | Create database | | ✔️ | ✔️ | | View all databases | | ✔️ | ✔️ | | Create project | ✔️ | ✔️ | ✔️ | | View all projects | | ✔️ | ✔️ | | Create issue | | ✔️ | ✔️ | | View all issues | | ✔️ | ✔️ | | Become issue assignee | | ✔️ | ✔️ | | Re-assign issue | | ✔️ | ✔️ | | Add comment to all issues | | ✔️ | ✔️ | | Subscribe to all issues | | ✔️ | ✔️ | | Alter schema | | ✔️ | ✔️ | | Change data | | ✔️ | ✔️ | | Configure SQL Review Policy | | ✔️ | ✔️ | | Manage version control system (VCS) | | | ✔️ | | Manage sensitive data | | ✔️ | ✔️ | | Manage database access control | | ✔️ | ✔️ | | Manage IM integration | | | ✔️ | | Change logo | | | ✔️ | ### Project Role Permissions Any user can create project. By default, the project creator is granted the `Project Owner` role. `Workspace DBA` and `Workspace Admin` assume the `Project Owner` role for all projects. | Project Permission | SQL Editor User | Project Developer | Project Releaser | GitOps Service Agent | Project Owner | Workspace DBA | Workspace Admin | | ---------------------------- | --------------- | ----------------- | ---------------- | -------------------- | ------------- | ------------- | --------------- | | Change project role | | | | | ✔️ | ✔️ | ✔️ | | Edit project | | | | | ✔️ | ✔️ | ✔️ | | Archive project | | | | | ✔️ | ✔️ | ✔️ | | Configure UI/GitOps workflow | | | | | ✔️ | ✔️ | ✔️ | ### Database Permissions Bytebase does not define database specific roles. Whether a user can perform certain action to the database is based on the user's Workspace role and the role of the project owning the database. | Database Permission | SQL Editor User | Project Developer | Project Releaser | GitOps Service Agent | Project Owner | Workspace DBA | Workspace Admin | | ------------------- | --------------- | ----------------- | ---------------- | -------------------- | ------------- | ------------- | --------------- | | Query | ✔️ | | | | ✔️ | ✔️ | ✔️ | | Edit database label | | | | | ✔️ | ✔️ | ✔️ | | Transfer database | | | | | ✔️ | ✔️ | ✔️ | ### Sheet Permissions User can save sheets from [SQL Editor](/sql-editor/overview). A sheet always belongs to a project. Sheet has three visibility levels: * Private * Project * Public #### Private Sheet | Permission | Creator | SQL Editor User | Project Developer | Project Owner | Workspace DBA | Workspace Admin | | ---------- | ------- | --------------- | ----------------- | ------------- | ------------- | --------------- | | Star | ✔️ | | | | | | | Read | ✔️ | | | | | | | Write | ✔️ | | | | | | | Delete | ✔️ | | | | | | #### Project Sheet | Permission | Creator | SQL Editor User | Project Developer | Project Owner | Workspace DBA | Workspace Admin | | ---------- | ------- | --------------- | ----------------- | ------------- | ------------- | --------------- | | Star | ✔️ | ✔️ | ✔️ | ✔️ | ✔️ | ✔️ | | Read | ✔️ | ✔️ | ✔️ | ✔️ | ✔️ | ✔️ | | Write | ✔️ | | | ✔️ | ✔️ | ✔️ | | Delete | ✔️ | | | ✔️ | ✔️ | ✔️ | #### Public Sheet | Permission | Creator | SQL Editor User | Project Developer | Project Owner | Others | | ---------- | ------- | --------------- | ----------------- | ------------- | ------ | | Star | ✔️ | ✔️ | ✔️ | ✔️ | ✔️ | | Read | ✔️ | ✔️ | ✔️ | ✔️ | ✔️ | | Write | ✔️ | | | ✔️ | | | Delete | ✔️ | | | ✔️ | | ### Issue Permissions | Issue Permission | Assignee | Creator | SQL Editor User | Project Developer | Project Releaser | GitOps Service Agent | Project Owner | Workspace DBA | Workspace Admin | | ------------------------- | -------- | ------- | --------------- | ----------------- | ---------------- | -------------------- | ------------- | ------------- | --------------- | | Create issue | N/A | N/A | | ✔️ | | ✔️ | ✔️ | ✔️ | ✔️ | | Change issue status | ✔️ | | | | ✔️ | ✔️ | Depends\* | ✔️ | ✔️ | | Edit name and description | ✔️ | ✔️ | | | | | | ✔️ | ✔️ | | Edit SQL Statement | | ✔️ | | | | | | | | | Subscribe/Unsubscribe | ✔️ | ✔️ | ✔️ | ✔️ | ✔️ | | ✔️ | ✔️ | ✔️ | | Add comment | ✔️ | ✔️ | ✔️ | ✔️ | ✔️ | | ✔️ | ✔️ | ✔️ | \* `Project Owner` can change issue status when the current active [Environment Rollout Policy](/change-database/environment-policy/rollout-policy) is set to **Require manual rolling out**. # Entra ID Source: https://docs.bytebase.com/administration/scim/entra-id ## Create enterprise application Sign in to the Entra ID Admin Center Dashboard. Select **Enterprise applications** and click **New application**. create-application Select **Create your own application**. Give your application a descriptive name, and select **Integrate any other application you don't find in the gallery (Non-gallery)** option, then click **Create**. create-own-application ## Create provision Go to the application detail page. Select **Provision User Accounts**. provision-user-accounts Click **Get Started** button. provision-get-started Change **Provisioning Mode** to **Automatic**. provision-automatic Go to your Bytebase console, navigate to **Security & Policy** -> **Users & Groups** page. Click **Sync From Entra ID (Azure AD)**. bytebase-sync-from-entra Copy the **Endpoint** and **Secret Token**. Bytebase endpoint implements SCIM protocol, please make sure you have configured [External URL](/get-started/self-host/external-url) and it's network accessible from Entra. bytebase-setting Go back to Entra console, paste the `Endpoint` and `Secret Token` above to `Tenant URL` and `Secret Token` respectively. Click **Test Connection** and save upon success. provision-admin-credentials ## Edit attribute mapping Continue the provision, click **Mappings** and click **Provision Microsoft Entra ID Groups**. provision-group Bytebase uses the group's `externalId` to uniquely identify a group. By default, Entra ID maps `objectId` to `externalId`, which is stable and recommended. You can optionally add a custom `email` attribute to sync the group email to Bytebase. If you have an existing SCIM configuration that maps `externalId` to `mail`, it will continue to work. However, we recommend switching to the default `objectId` mapping for stability, since object IDs do not change when a group's email is updated. ### Step 1 - Create a new `email` attribute Click **Show advanced options**, then click **Edit attribute list for Bytebase**. mapping-create-email-attr Add a new attribute `email` with type `String`, then click **Save**. mapping-email-attr-config ### Step 2 - Edit the mapping Edit the attribute mapping: * Click **Edit** for the `displayName` row. Change **Match objects using this attribute** to `No`. * Click **Edit** for the `externalId` row. Change **Match objects using this attribute** to `Yes` and set **Matching precedence** to `1`. * Add a new mapping row: set **email** to map to **mail**. mapping-edit-mapping The final mappings look like this. mapping-final ## Assign users and groups In order for your users and groups to be synced to Bytebase, you will need to assign them to your Entra SCIM application. Select **Users and groups** and click **Add user/group**. add-user-group Click **None selected** under the **Users and Groups**. Select the users and groups that you want to add to the SCIM application, and click **Select** and **Assign**. assign-user-group ## Turn on provisioning On the application overview page, click **Start provisioning**. To test syncing, we recommend starting with **Provision on demand** for a subset of users or groups. start-provision Afterwards, Entra will sync the users and groups to Bytebase periodically. # Okta Source: https://docs.bytebase.com/administration/scim/okta ## Get SCIM credentials from Bytebase 1. Log in to Bytebase as a **Workspace Admin**. 2. Navigate to **Settings** -> **General** -> **Network**. 3. Ensure **External URL** is configured (e.g., `https://bytebase.example.com`). 4. Navigate to **Settings** -> **Members**. 5. Click the **Directory Sync** button (or gear icon). 6. In the Directory Sync drawer, you'll find: * **SCIM Endpoint URL**: `https:///hook/scim/workspaces/` * **Secret Token**: Bearer token for authentication 7. Copy both values - you'll need them in the next steps. bytebase-setting ## Create SCIM application in Okta 1. Log in to the **Okta Admin Console** (`https://your-domain-admin.okta.com`). 2. In the left navigation, go to **Applications** -> **Applications**. 3. Click **Create App Integration**. 4. Select **SWA - Secure Web Authentication** and click **Next**. create-app-integration 5. Configure the application: * **App name**: `Bytebase SCIM` * **App login page URL**: Your Bytebase external URL * You can also check "Do not display application icon to users" and "This is an internal application that we created" since it's only used for data sync. create-swa-integration ## Configure API integration 1. Go to the **General** tab, in the **App Settings** section, click **Edit**. general-tab-edit 2. Find the **Provisioning** field and select **SCIM**, then save the setting. After saving, the **Provisioning** tab will appear in the application. provisioning-scim 3. Go to the **Provisioning** tab, edit the integration: * **SCIM connector base URL**: The SCIM Endpoint URL from Bytebase (e.g., `https://bytebase.example.com/hook/scim/workspaces/abc123`) * **Unique identifier field for users**: `userName` * **Authentication Mode**: HTTP Header * **Authorization**: The Secret Token from Bytebase scim-connection 4. Click **Test Connector Configuration** and save upon success. ## Enable provisioning features 1. Still on the **Provisioning** tab, click **To App** in the left sidebar. 2. Click **Edit** in the Provisioning to App section. 3. Enable the following features: * **Create Users** * **Update User Attributes** * **Deactivate Users** 4. Click **Save**. provisioning-to-app ## Configure user attribute mappings No changes are needed for the default attribute mapping configuration. ## Assign users to the application ### Assign individual users 1. Go to the **Assignments** tab. 2. Click **Assign** -> **Assign to People**. 3. Find and select the users you want to provision. 4. Click **Assign** next to each user. 5. Review the user's attribute values (you can customize per user if needed). 6. Click **Save and Go Back**. 7. Click **Done** when finished. ### Assign groups (preferred) 1. Go to the **Assignments** tab. 2. Click **Assign** -> **Assign to Groups**. 3. Select the groups whose members should be provisioned. assign-to-groups 4. Click **Assign**, review the attribute values, and click **Save and Go Back**. assign-groups-dialog assign-groups-attributes 5. Click **Done** when finished. Assigning a group provisions all members as users, but **does not create the group in Bytebase**. To sync groups, see [Configure group push](#configure-group-push) below. ## Configure group push To sync Okta groups to Bytebase (not just group members as users): 1. Go to the **Push Groups** tab. 2. Click **Push Groups**, search group by name or rule. 3. Click **Save**, then wait and check the status. push-groups After the status changes to **Active**, the groups are successfully synced to Bytebase. push-groups-active bytebase-groups-synced # Overview Source: https://docs.bytebase.com/administration/scim/overview SCIM (System for Cross-domain Identity Management) is a standard for provisioning and deprovisioning users and groups in an organization. Bytebase implements SCIM 2.0 and provides built-in support for the following identity providers: * [Entra ID (Azure AD)](/administration/scim/entra-id) * [Okta](/administration/scim/okta) | IdP | User | Group | Role | Interval | | ------------------- | ------------------- | ------------------------------------- | ---- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | Entra ID (Azure AD) | Sync name and email | Sync group name, email and membership | ❌ | [40 minutes](https://learn.microsoft.com/en-us/entra/identity/app-provisioning/application-provisioning-when-will-provisioning-finish-specific-user#how-long-will-it-take-to-provision-users) | | Okta | Sync name and email | Sync group name, email and membership | ❌ | [Configured in Okta](https://developer.okta.com/docs/reference/scim/scim-20/) | ## Prerequisites * You must be the **Workspace Admin** to configure SCIM. * Configure [External URL](/get-started/self-host/external-url). # Service Account Source: https://docs.bytebase.com/administration/service-account Service accounts are machine identities designed for automated processes and applications. Unlike human [users](/administration/user-groups/overview), service accounts authenticate via API keys and are intended for programmatic access. ## Workspace vs Project Level Service accounts can be created at two levels: * **Workspace level** — Has access governed by workspace IAM policies. Suitable for cross-project automation. * **Project level** — Scoped to a single project, following the principle of least privilege. Suitable for isolated automation within one project. ## Create a Service Account 1. For workspace-level: go to **IAM & Admin** > **Service Accounts** tab and click **Add Service Account**. 2. For project-level: go to **Project** > **Manage** > **Service Accounts** and click **Add Service Account**. Service accounts cannot be part of [user groups](/administration/user-groups/overview). Since service accounts are for automated processes with specific access needs, including them in groups could grant unintended permissions. This is considered an [anti-pattern](https://cloud.google.com/iam/docs/best-practices-service-accounts#groups). ## Service Account vs Workload Identity | | Service Account | [Workload Identity](/administration/workload-identity/overview) | | -------------- | -------------------------------------- | --------------------------------------------------------------- | | **Credential** | Long-lived API key | Short-lived OIDC token | | **Best for** | Scripts, Terraform, general API access | CI/CD pipelines (GitHub Actions, GitLab CI) | | **Security** | Key must be stored as a secret | No secrets to manage | If your automation runs in a CI/CD platform that supports OIDC, prefer [Workload Identity](/administration/workload-identity/overview) for better security. # Sign-in Restriction Source: https://docs.bytebase.com/administration/sign-in-restriction ## Rate Limiting for Login Attempts Bytebase implements rate limiting to protect against brute force attacks on user authentication. The system automatically tracks and limits failed login attempts: ### Password Authentication Phase * **Maximum attempts**: 10 failed attempts * **Time window**: 10 minutes * After exceeding the limit, the account will be temporarily locked ### Multi-Factor Authentication (MFA) Phase * **Maximum attempts**: 5 failed attempts * **Time window**: 5 minutes * After exceeding the limit, the MFA verification will be temporarily locked * MFA temporary token expires after 5 minutes These security measures help protect user accounts from unauthorized access attempts while ensuring legitimate users can still access their accounts. ## Sign-in Frequency **Sign-in Frequency** specifies the period that users are required to sign in again. In Bytebase Workspace, Go to **Settings** > **General** and scroll down to **Account** section. sign-in-frequency You need to restart Bytebase to make the change take effect. ## Disallow Sign-in with Email & Password Once [SSO](/administration/sso/overview) is configured, you can [enforce SSO sign-in](/administration/sso/overview/#enforce-sso-sign-in) for all users. ## Sign-in from Email Domains Go to **Settings** > **General**, scroll down to **Security** section. For **Workspace Domain**, you can configure allowed email domains for your workspace members. Click **Add domain** to add multiple domains as needed. After adding your domains, enable the **Members restriction** checkbox to enforce the restriction. Following domains are disallowed: * gmail.com * googlemail.com * outlook.com * hotmail.com * live.com * msn.com * yahoo.com * ymail.com * rocketmail.com * icloud.com * me.com * mac.com * aol.com * zoho.com * protonmail.com * gmx.com * gmx.net * mail.com * yandex.com * yandex.ru * fastmail.com * fastmail.fm * tutanota.com * 163.com * 126.com * sohu.com * qq.com * sina.com * sina.cn * aliyun.com * aliyun.cn * tom.com * 21cn.com * yeah.net set-domain Domain restriction applies to: * Sign-in page. (Note that the new restriction only applies to the accounts registered after the **Workspace Domain** was updated) sign-in-domain-restriction * **Add User** in **IAM & Admin** > **Users & Groups**. Users' email must be of the domain you set. add-user-domain-restriction # IdP-Initiated Flow Source: https://docs.bytebase.com/administration/sso/idp-initiated By default, SSO uses a **Service Provider-initiated (SP-initiated)** flow: users visit Bytebase, click "Sign in with X", get redirected to the identity provider, then back. **IdP-initiated** flow reverses this — users click the Bytebase tile directly from their identity provider's dashboard (e.g. Okta portal) and land in Bytebase already authenticated. This eliminates the extra step of visiting Bytebase first and works with both OAuth 2.0 and OIDC providers. ## Prerequisites 1. Configure [External URL](/get-started/self-host/external-url) for your Bytebase instance 2. Set up an [OAuth 2.0](/administration/sso/oauth2) or [OIDC](/administration/sso/oidc) SSO provider in Bytebase IdP-initiated flow in Bytebase works with OAuth 2.0 and OpenID Connect (OIDC) protocols. This feature uses a specialized endpoint to initiate the authentication flow from the identity provider's side. ## Configuration ### Step 1: Configure SSO Provider in Bytebase First, ensure you have already configured your SSO provider (OAuth 2.0 or OIDC) in Bytebase following the respective documentation: * [OAuth 2.0 Configuration](/administration/sso/oauth2) * [OIDC Configuration](/administration/sso/oidc) ### Step 2: Get Your IdP-Initiated URL Once your SSO provider is configured, the IdP-initiated flow URL follows this format: ``` {EXTERNAL_URL}/auth/idp-init?idp={IDENTITY_PROVIDER_ID} ``` Where: * `{EXTERNAL_URL}` is your Bytebase external URL (e.g., `https://bytebase.example.com`) * `{IDENTITY_PROVIDER_ID}` is the Identity Provider ID you specified when creating the SSO provider in Bytebase **Example**: If your external URL is `https://bytebase.example.com` and your Identity Provider ID is `okta-company`, the IdP-initiated URL would be: ``` https://bytebase.example.com/auth/idp-init?idp=okta-company ``` ### Step 3: Add Deep Linking (Optional) You can optionally specify where users should land after authentication by adding a `relay_state` parameter. If not provided, users will be directed to the Bytebase homepage after successful authentication. ``` {EXTERNAL_URL}/auth/idp-init?idp={IDENTITY_PROVIDER_ID}&relay_state={PATH} ``` **Example**: To redirect users to a specific project after login: ``` https://bytebase.example.com/auth/idp-init?idp=okta-company&relay_state=/projects/123 ``` The `relay_state` parameter is optional. When used, it must be a relative path (starting with `/`). External URLs are not permitted for security reasons. ## Okta Configuration After setting up your [OIDC SSO provider](/administration/sso/oidc) in Bytebase, configure the Initiate Login URL in your Okta application to enable IdP-initiated flow. 1. In your Okta Admin Console, go to **Applications** > **Applications**. 2. Select your existing Bytebase OIDC application (the one you created when setting up OIDC SSO). 3. Go to the **General** tab and click **Edit** in the **General Settings** section. 4. Set the **Initiate login URI** to your Bytebase IdP-initiated URL: ``` https://bytebase.example.com/auth/idp-init?idp=okta-company ``` Where `okta-company` is the Identity Provider ID you configured in Bytebase. okta-initiate-login-uri 5. Click **Save** to apply the changes. 6. Now when users click the Bytebase tile in their Okta End-User Dashboard, they will be automatically authenticated and redirected to Bytebase without needing to click any additional login buttons. To direct users to a specific page after login, add the `relay_state` parameter to your Initiate Login URI: ``` https://bytebase.example.com/auth/idp-init?idp=okta-company&relay_state=/projects/123 ``` ## Other Identity Providers For other identity providers (GitLab, Keycloak, Casdoor, etc.), the configuration typically involves: 1. Creating a bookmark/shortcut application in your IdP 2. Setting the target URL to your Bytebase IdP-initiated URL 3. Assigning the bookmark to appropriate users or groups Consult your identity provider's documentation for creating bookmark or web link applications. ## Security Considerations IdP-initiated flow in Bytebase includes several security measures: ### CSRF Protection The IdP-initiated flow uses the same state parameter mechanism as SP-initiated SSO to prevent Cross-Site Request Forgery (CSRF) attacks. Each authentication request generates a unique, single-use state token. ### Token Expiration State tokens expire after 10 minutes, limiting the window for potential replay attacks. ### Relay State Validation The `relay_state` parameter is validated to ensure it's a relative path, preventing open redirect vulnerabilities. External URLs are rejected. ### Existing Session Handling If a user already has an active Bytebase session when using IdP-initiated flow, the system will respect the existing session state. While IdP-initiated flow provides a better user experience, SP-initiated flows offer slightly stronger security guarantees because the authentication request originates from and is controlled by the service provider. For high-security environments, consider whether the convenience of IdP-initiated flow is appropriate for your security requirements. ## Troubleshooting ### Users redirected to homepage instead of intended destination If you're using the `relay_state` parameter but users are landing on the homepage: 1. Verify that the `relay_state` value is URL-encoded if it contains special characters 2. Ensure the path starts with `/` (e.g., `/projects/123`, not `projects/123`) 3. Check that the path exists and is accessible to the authenticated user ### IdP-initiated flow not working If clicking the application tile in your IdP dashboard doesn't authenticate users: 1. Verify your External URL is configured correctly in Bytebase 2. Check that the Identity Provider ID in the Initiate Login URI matches exactly what you configured in Bytebase (case-sensitive) 3. Ensure the OIDC SSO provider is properly configured and tested using SP-initiated flow first 4. Verify the Initiate Login URI is correctly set in your Okta application's General Settings ### Authentication succeeds but shows an error This may indicate an issue with token exchange or user provisioning: 1. Verify the OAuth/OIDC configuration in Bytebase has correct client ID and secret 2. Check the user information field mapping in your SSO configuration 3. Review Bytebase server logs for detailed error messages 4. Ensure the user's email in the IdP matches a valid email format ## Learn More * [OAuth 2.0 Configuration](/administration/sso/oauth2) * [OIDC Configuration](/administration/sso/oidc) * [External URL Configuration](/get-started/self-host/external-url) * [SSO Overview](/administration/sso/overview) # LDAP Source: https://docs.bytebase.com/administration/sso/ldap Lightweight Directory Access Protocol (LDAP) is a vendor-neutral software protocol used to lookup information or devices within a network. Bytebase supports using LDAP for configuring Single Sign-On (SSO). ## Configuration Basic information: * **Name**: the display name shown to your users (e.g. `JumpCloud`) * **Identity Provider ID**: a human-readable unique string, only lower-case alphabets and hyphens are allowed (e.g. `jumpcloud`) * **Domain**: the domain name to scope associated users (e.g. `jumpcloud.com`, optional) Identity provider information: * **Host**: the host of LDAP server (e.g. `ldap.jumpcloud.com`) * **Port**: the port number of the LDAP server, usually 389 for StartTLS and 636 for LDAPS * **Bind DN**: the Distinguished Name (DN) of the user to bind as a service account to perform search requests (e.g. `uid=system,ou=Users,dc=jumpcloud,dc=com`) * **Bind Password**: the password of the user to bind as a service account * **Base DN**: the base Distinguished Name (DN) to search for users (e.g. `ou=users,dc=jumpcloud,dc=com`) * **User Filter**: the LDAP search filter to match users during authentication. The `%s` placeholder will be replaced with the username entered during login (e.g. `(uid=%s)` searches for a user with matching `uid` attribute) * **Security protocol**: the security protocol to be used for establishing connections with the LDAP server User information field mapping: * **Email**: the attribute to be used as the Bytebase user email address (e.g. `mail`) * **Display name**: the attribute to be used as the Bytebase user display name (e.g. `displayName`, optional) * **Phone**: the attribute to be used as the Bytebase user phone number (e.g. `phone`, optional) ## User Filter Configuration The **User Filter** field is critical for LDAP authentication. It defines how Bytebase searches for users in your LDAP directory. ### Understanding the User Filter * The `%s` placeholder is **required** and will be replaced with the username entered during login * Filters must use valid LDAP filter syntax with proper parentheses * The filter should uniquely identify users to prevent authentication issues ### Common User Filter Patterns Different LDAP providers use different object classes and attributes: | Provider | Recommended User Filter | Description | | ---------------- | ------------------------------------------ | --------------------------------------------------------------- | | Okta | `(&(objectClass=inetOrgPerson)(uid=%s))` | Matches users with `inetOrgPerson` class and matching `uid` | | JumpCloud | `(&(objectClass=posixAccount)(uid=%s))` | Matches users with `posixAccount` class and matching `uid` | | Active Directory | `(&(objectClass=user)(sAMAccountName=%s))` | Matches users with matching `sAMAccountName` (Windows username) | | OpenLDAP | `(&(objectClass=inetOrgPerson)(uid=%s))` | Standard OpenLDAP user filter | ### Best Practices 1. **Use AND conditions**: Combine object class with the user identifier for more precise matching: * Good: `(&(objectClass=inetOrgPerson)(uid=%s))` * Avoid: `(uid=%s)` (too broad, may match non-user entries) 2. **Match your directory schema**: Verify the object class used in your LDAP directory: * Common classes: `inetOrgPerson`, `posixAccount`, `user`, `person` * Use LDAP browser tools to inspect your directory structure 3. **Test the filter**: Use the **Test Connection** button to verify your filter works before saving ### Common Issues | Issue Type | Description | Bad Example | Correction | | -------------------------- | ---------------------------------------------------------------------------------------------- | ------------------------------------ | ---------------------------------------- | | **Invalid filter syntax** | Parentheses must be balanced and properly nested following LDAP filter syntax | `&(objectClass=user)(uid=%s)` | `(&(objectClass=user)(uid=%s))` | | **Missing %s placeholder** | The `%s` placeholder must be present for username substitution | `(uid=username)` | `(uid=%s)` | | **Wrong attribute name** | Username attribute must match your LDAP schema (common: `uid`, `sAMAccountName`, `cn`, `mail`) | `(&(objectClass=user)(username=%s))` | `(&(objectClass=user)(uid=%s))` | | **Wrong object class** | Object class must match the user entries in your LDAP directory | `(&(objectClass=person)(uid=%s))` | `(&(objectClass=inetOrgPerson)(uid=%s))` | If you're unsure about your LDAP schema, use an LDAP browser tool (like Apache Directory Studio or ldapsearch) to inspect a user entry and identify the correct object class and username attribute. ## Examples ### Active Directory Users sign in with their **sAMAccountName** (the Windows logon name, e.g. `jdoe`), not their email address. The user information mapping fields take LDAP **attribute names** (e.g. `mail`), not actual values — entering a literal email address there leaves the user profile empty after login. This example covers on-premises Active Directory Domain Services. Microsoft Entra ID (formerly Azure AD) is not an LDAP server — configure it via [OIDC](/administration/sso/oidc-entra-id) instead. 1. Create (or reuse) a domain service account for Bytebase to bind as, e.g. `svc-bytebase`. It only needs read access to the users it should authenticate. 2. In Bytebase, go to **Settings > SSO** to create a new LDAP provider (all values are examples): * **Name**: `Active Directory` * **Identity Provider ID**: `active-directory` * **Domain**: `corp.example.com` * **Host**: `dc01.corp.example.com` (a domain controller; the AD domain name itself also works since it resolves to domain controllers) * **Port**: `389` * **Bind DN**: `CN=svc-bytebase,OU=Service Accounts,DC=corp,DC=example,DC=com` (AD also accepts the UPN form `svc-bytebase@corp.example.com`) * **Bind Password**: `YOUR_PASSWORD` * **Base DN**: `DC=corp,DC=example,DC=com` (scope it down to an OU like `OU=Employees,DC=corp,DC=example,DC=com` to limit who can sign in) * **User Filter**: `(&(objectClass=user)(sAMAccountName=%s))` * **Security protocol**: `StartTLS` * **Email**: `mail` (if `mail` is not populated in your directory, use `userPrincipalName`) * **Display name**: `displayName` ### JumpCloud The attribute `uid` is the username (e.g. `system`) not the email (e.g. `system@example.com`) in JumpCloud. 1. Follow the JumpCloud [Use Cloud LDAP](https://jumpcloud.com/support/use-cloud-ldap) to create an LDAP binding user and add Users to the LDAP directory. 2. In Bytebase, go to **Settings > SSO** to create a new LDAP provider (all values are examples): * **Name**: `JumpCloud` * **Identity Provider ID**: `jumpcloud` * **Domain**: `jumpcloud.com` * **Host**: `ldap.jumpcloud.com` * **Port**: `389` * **Bind DN**: `uid=YOUR_USERNAME,ou=Users,o=YOUR_ORG_DN,dc=jumpcloud,dc=com` * **Bind Password**: `YOUR_PASSWORD` * **Base DN**: `ou=Users,o=YOUR_ORG_NAME,dc=jumpcloud,dc=com` * **User Filter**: `(&(objectClass=posixAccount)(uid=%s))` * **Security protocol** `StartTLS` * **Email**: `mail` * **Display name**: `displayName` ### Okta The attribute `uid` is the username (e.g. `system`) not the email (e.g. `system@example.com`) in Okta. 1. Follow the Okta [Enable the LDAP interface](https://help.okta.com/en-us/Content/Topics/Directory/LDAP-interface-enable.htm) to enable LDAP interface for your directory. 2. In Bytebase, go to **Settings > SSO** to create a new LDAP provider (all values are examples): * **Name**: `Okta` * **Identity Provider ID**: `okta` * **Domain**: `okta.com` * **Host**: `YOUR_SUBDOMAIN.ldap.okta.com` * **Port**: `389` * **Bind DN**: `uid=YOUR_USERNAME,ou=users,dc=YOUR_SUBDOMAIN,dc=okta,dc=com` * **Bind Password**: `YOUR_PASSWORD` * **Base DN**: `ou=users,dc=YOUR_SUBDOMAIN,dc=okta,dc=com` * **User Filter**: `(&(objectClass=inetOrgPerson)(uid=%s))` * **Security protocol** `StartTLS` * **Email**: `mail` * **Display name**: `cn` # OAuth 2.0 Source: https://docs.bytebase.com/administration/sso/oauth2 OAuth is an open standard for token-based authentication and authorization which is used to provide Single Sign-On (SSO). Bytebase supports popular OAuth 2.0 providers (GitHub, Google, GitLab) and custom providers. ## Overview create-sso-dialog In the creating SSO dialog, you need to fill following fields: ### Basic information oauth2-basic-information * **Name**: the display name shown to your users (e.g. `Google` will be shown as `Sign in with Google`) * **Identity Provider ID**: a human-readable unique string, only lower-case alphabets and hyphens are allowed (e.g. `google-fsgs`) * **Domain**: the domain name to scope associated users (e.g. `google.com`, optional) ### Identity provider information The information is the base concept of [OAuth 2.0](https://oauth.net/2) and comes from your provider. oauth2-identity-provider-information * **Client ID** is a public identifier of the custom provider; * **Client Secret** is the OAuth2 client secret from identity provider; * **Auth URL** is the custom provider's OAuth2 login page address; * **Scopes** is the scope parameter carried when accessing the OAuth2 URL, which is filled in according to the custom provider; * **Token URL** is the API address for obtaining access token; * **User information URL** is the API address for obtaining user information by access token; * **Auth style** specifies how the endpoint wants the client ID & client secret sent. * **In params**: The `client_id` and `client_secret` will be sending in the POST body as application/x-www-form-urlencoded parameters. * **In header**: The `client_id` and `client_secret` will be sending in HTTP Basic Authorization. This is an optional style described in the OAuth2 RFC 6749 section 2.3.1. ### User information mapping For different providers, the structures returned by their user information API are usually not the same. That's why you need to fill an information mapping form for mapping user information into Bytebase user fields. Bytebase uses the mapping to import user profile fields when creating new accounts. The most important information is `Bytebase user's email`. It identifies the Bytebase account associated with the OAuth 2.0 login. oauth2-user-information-field-mapping * **Email** is the field name of primary email in 3rd-party user info; * **DisplayName** is the field name of display name in 3rd-party user info (optional); - **Phone**: is the field name of phone number in 3rd-party user info (optional). ## Built-in OAuth Provider OAuth 2.0 usually requires an authorization callback url in the configuration. Please make sure the [`--external-url`](/get-started/self-host/external-url) is set correctly. If you start Bytebase with `--external-url http://bytebase.example.com`, then the **authorization callback URL** will be `http://bytebase.example.com/oauth/callback`. Bytebase provides templates for configuring built-in OAuth providers. ### Google 1. Follow the [Using OAuth 2.0 to Access Google APIs](https://developers.google.com/identity/protocols/oauth2) to create OAuth 2.0 client credentials in [Google API Console](https://console.developers.google.com/). google-oauth-app-config 2. Open creating SSO dialog and select the **Google** template. 3. Update the **Client ID** and **Client secret** fields with the OAuth 2.0 client credential you just created. 4. After filling in all the required fields, try to click **Test connection**. 5. If everything is OK, click the **Create** button. ### GitHub 1. Follow [Creating an OAuth App in GitHub](https://docs.github.com/en/developers/apps/building-oauth-apps/creating-an-oauth-app) to create an OAuth app in GitHub, where you replace `YOUR_EXTERNAL_URL` in the image below with the external URL you [configured](https://www.bytebase.com/docs/get-started/self-host/external-url) in Bytebase Workspace. github-oauth-app-config 2. Open creating SSO dialog and select the **GitHub** template. 3. Update the **Client ID** and **Client secret** fields with the GitHub OAuth App you just created. 4. After filling in all the required fields, try to click **Test connection**. 5. If everything is OK, click the **Create** button. 6. Go to your GitHub [Public profile](https://github.com/settings/profile), make sure your **Public email** is set to the email you intend to use for SSO. Otherwise, the SSO will not work. gh-public-profile ### GitLab 1. Follow [Configure GitLab as an OAuth 2.0 authentication identity provider](https://docs.gitlab.com/ee/integration/oauth_provider.html) to create an OAuth 2 application in GitLab. gitlab-oauth-app-config 2. Open creating SSO dialog and select the **GitLab** template. 3. Update the **Client ID** and **Client secret** fields with the GitLab OAuth 2 application you just created. 4. After filling in all the required fields, try to click **Test connection**. 5. If everything is OK, click the **Create** button. ### Microsoft Entra (Azure AD) 1. Follow the [Register an application with the Microsoft identity platform](https://learn.microsoft.com/en-us/entra/identity-platform/quickstart-register-app) to create an Application in [Microsoft Entra admin center](https://entra.microsoft.com/). 2. Note the **Application ID** for later use. Click the **Endpoints** tab and note both the **OAuth 2.0 authorization endpoint** and **OAuth 2.0 token endpoint**. entra-endpoints 3. Create a client secret in the **Client credentials** tab and note the **Value** for later use. entra-client-secret 4. Click **IAM & Admin > SSO** in Bytebase workspace, then click **+ Create SSO**. Choose **OAuth 2.0** and select **Microsoft Entra** template. 5. Name it as you want, update the **Client ID**, **Client secret** and the **Endpoints** fields with the OAuth 2.0 client credential you just created. entra-sso-config 6. We will need to copy the **Authorization Redirect URL** and add it back to the **Redirect URIs** in the Microsoft Entra application. et-redirect-url 7. After filling in all the required fields, try to click **Test connection**. 8. If everything is OK, click the **Create** button. # OpenID Connect (OIDC) Source: https://docs.bytebase.com/administration/sso/oidc OpenID Connect (OIDC) is a simple identity layer on top of the OAuth 2.0 protocol. Bytebase supports using OIDC for configuring Single Sign-On (SSO). For **Microsoft Entra ID (Azure AD)**, see the dedicated [Entra ID OIDC guide](/administration/sso/oidc-entra-id) which covers app registration, API permissions, group claims, and troubleshooting. ## Configuration 1. Please make sure the [`--external-url`](/get-started/self-host/external-url) is configured correctly for the Bytebase workspace. If your start Bytebase with `--external-url http://bytebase.example.com`, then your application redirect URL should be `http://bytebase.example.com/oidc/callback`. 2. If you're unsure about the **Issuer** of your IdP, you can always use the [OpenID Connect Discovery](https://openid.net/specs/openid-connect-discovery-1_0.html) endpoint to find the correct value, e.g. `https://acme.okta.com/.well-known/openid-configuration`. Basic information: * **Name**: the display name shown to your users (e.g. `Google` will be shown as `Sign in with Google`) * **Identity Provider ID**: a human-readable unique string, only lower-case alphabets and hyphens are allowed (e.g. `google`) * **Domain**: the domain name to scope associated users (e.g. `google.com`, optional) Identity provider information: * **Issuer**: the issuer of the response (e.g. `https://accounts.google.com`) * **Client ID**: the client ID of your application * **Client secret**: the client secret of your application * **Scopes**: the scopes to request from the identity provider (e.g., `openid`, `profile`, `email`). Some providers also support a groups claim, which can be included by adding the `groups` scope. This is useful if you want to enable [**group syncing**](#group-syncing) as part of the authentication process. User information field mapping: * **Email**: the claims field to be used as the Bytebase user email address (e.g. `email`) * **Display name**: the claims field to be used as the Bytebase user display name (e.g. `name`, optional) * **Phone**: the claims field to be used as the Bytebase user phone number (e.g. `phone`, optional) * **Groups**: the claims field to be used as the Bytebase user groups (e.g. `groups`, optional). If this field is set, Bytebase will automatically perform [**group syncing**](#group-syncing) by default. ### Google 1. Follow the Google [OpenID Connect documentation](https://developers.google.com/identity/openid-connect/openid-connect) to create a new OAuth client ID with "Web application" as the **Application type**. 2. Configure the **Authorized redirect URIs** to be `{EXTERNAL_URL}/oidc/callback`. 3. In Bytebase, go to **Settings > SSO** to create a new OIDC provider (all values are examples): * **Name**: `Google` * **Identity Provider ID**: `google` * **Domain**: `google.com` * **Issuer**: `https://accounts.google.com` * **Client ID**: the client ID of your application * **Client secret**: the client secret of your application * **Email**: `email` * **Display name**: `name` ### GitLab 1. Follow the documentation of [configure GitLab as an OAuth 2.0 authentication identity provider](https://docs.gitlab.com/ee/integration/oauth_provider.html) to create a new OAuth application: 1. Configure the **Scopes** to include `openid`, `profile` and `email`. 2. Configure the **Redirect URI** to be `{EXTERNAL_URL}/oidc/callback`. 2. In Bytebase, go to **Settings > SSO** to create a new OIDC provider (all values are examples): * **Name**: `GitLab` * **Identity Provider ID**: `gitlab` * **Domain**: `gitlab.acme.com` * **Issuer**: `https://gitlab.acme.com` * **Client ID**: the application ID of your application * **Client secret**: the secret of your application * **Email**: `email` * **Display name**: `name` In some GitLab self-hosted setups, the **Issuer** is `http://gitlab.acme.com` (HTTP) instead of `https://gitlab.acme.com` (HTTPS) despite the latter being the URL used to access the instance. ### Okta 1. Follow the Okta [create OIDC app integrations documentation](https://help.okta.com/en-us/Content/Topics/Apps/Apps_App_Integration_Wizard_OIDC.htm) to create a new OIDC app integration with "Web Application" as the **Application type**. 1. Configure the **Sign-in redirect URIs** to be `{EXTERNAL_URL}/oidc/callback`. 2. Configure the **Assignments > Controlled access** to be **Allow everyone in your organization to access**. 2. In Bytebase, go to **Settings > SSO** to create a new OIDC provider (all values are examples): * **Name**: `Okta` * **Identity Provider ID**: `okta` * **Domain**: `acme.okta.com` * **Issuer**: `https://acme.okta.com` * **Client ID**: the client ID of your application * **Client secret**: the client secret of your application * **Email**: `email` * **Display name**: `name` ### Keycloak 1. Follow the Keycloak [create OIDC provider documentation](https://www.keycloak.org/docs/latest/server_admin/#assembly-managing-clients_server_administration_guide) to create a new "OpenID Connect" client. 1. Configure the **Client ID** to be `bytebase`. 2. Configure the **Valid redirect URIs** to be `{EXTERNAL_URL}/oidc/callback`. 3. Turn on the **Capability config > Client authentication** for the **Credentials** tab to be available (which will generate and display the client secret). 1. In some older versions, configure **Access Type** to "confidential" instead. 2. In Bytebase, go to **Settings > SSO** to create a new OIDC provider (all values are examples): * **Name**: `Keycloak` * **Identity Provider ID**: `keycloak` * **Domain**: `keycloak.acme.com` * **Issuer**: `https://keycloak.acme.com/auth/realms/master` * **Client ID**: `bytebase` * **Client secret**: the client secret of your application * **Email**: `email` * **Display name**: `name` ### Casdoor 1. Follow the Casdoor [Casdoor documentation](https://casdoor.org/docs/basic/core-concepts) to create a new application. 1. Configure the **Client ID** and **Client secret** to be `bytebase`. 2. Configure the **Valid redirect URIs** to be `{EXTERNAL_URL}/oidc/callback`. 2. In Bytebase, go to **Settings > SSO** to create a new OIDC provider (all values are examples): * **Name**: `Casdoor` * **Identity Provider ID**: `casdoor` * **Domain**: `` * **Issuer**: `` * **Client ID**: the client id of your application * **Client secret**: the client secret of your application * **Email**: `email` * **Display name**: `name` ### Authing 1. Follow the Authing [创建自建应用](https://docs.authing.cn/v2/guides/app-new/create-app/create-app.html) to create a new "标准 Web 应用" as "自建应用". 1. In **应用配置**, configure the **登录回调 URL** to be `{EXTERNAL_URL}/oidc/callback`. 2. In **协议配置**, configure the **id\_token 签名算法** to be `RS256`. 2. In Bytebase, go to **Settings > SSO** to create a new OIDC provider (all values are examples): * **Name**: `Authing` * **Identity Provider ID**: `authing` * **Domain**: `acme.authing.cn` * **Issuer**: `https://acme.authing.cn/oidc` * **Client ID**: the app ID of your application * **Client secret**: the app secret of your application * **Email**: `email` * **Display name**: `name` ## Group Syncing Bytebase supports syncing identity provider (IdP) groups with [Bytebase user groups](/administration/user-groups/) for providers that include a `groups` claim in their tokens. Group syncing is based on a one-to-one match using the **group title**. When a user logs in, Bytebase compares the group names from the IdP with existing Bytebase user group titles. If a match is found, the user is automatically added as a **Member** to the corresponding Bytebase user group. To ensure security and consistency, Bytebase also **automatically removes the user from any Bytebase groups that are not present in their IdP group claims**. Group syncing occurs during login, so if group membership changes in your IdP, users must log out and log back in for the changes to take effect. Some OIDC providers like **Okta** support the `groups` claim, but you’ll need to first **customize the tokens returned from Okta to include the `groups` claim**. For more information, refer to the [Okta documentation](https://developer.okta.com/docs/guides/customize-tokens-groups-claim/main/). ## Troubleshoot ### CORS If you click the login button and there is no response. It could be you are hitting the CORS error. You can verify this by inspecting the [browser network](https://developer.chrome.com/docs/devtools/network). Please ask your network admin to whitelist Bytebase host. Alternatively, you can configure [OAuth](/administration/sso/oauth2/). # OIDC with Microsoft Entra ID Source: https://docs.bytebase.com/administration/sso/oidc-entra-id This guide walks through configuring Microsoft Entra ID (formerly Azure Active Directory) as an [OIDC](/administration/sso/oidc) identity provider for Bytebase, including group synchronization. Entra ID can also be configured via [OAuth 2.0](/administration/sso/oauth2#microsoft-entra-azure-ad) if you don't need group syncing or OIDC-specific features. Please make sure the [`--external-url`](/get-started/self-host/external-url) is configured correctly for the Bytebase workspace. If you start Bytebase with `--external-url http://bytebase.example.com`, then your application redirect URL should be `http://bytebase.example.com/oidc/callback`. ## Step 1: Create App Registration 1. Navigate to [Azure Portal](https://portal.azure.com) 2. Go to **Microsoft Entra ID** (formerly **Azure Active Directory**) → **App registrations** 3. Click **New registration** 4. Configure the application: * **Name**: `Bytebase SSO` (or your preferred name) * **Supported account types**: Choose based on your requirements (single or multi-tenant) * **Redirect URI**: * Platform: `Web` * URI: `https://your-bytebase-domain.com/oidc/callback` 5. Click **Register** ## Step 2: Configure Authentication 1. In your app registration, go to **Authentication** 2. Verify your redirect URI under **Platform configurations** 3. Under **Implicit grant and hybrid flows**: * Check **ID tokens** (used for implicit and hybrid flows) 4. Click **Save** ## Step 3: Configure API Permissions 1. Go to **API permissions** 2. Click **Add a permission** → **Microsoft Graph** → **Delegated permissions** 3. Add the following permissions: * `openid` (Sign users in) * `profile` (View users' basic profile) * `email` (View users' email address) * `offline_access` (Maintain access to data) * `User.Read` (Sign in and read user profile) * `GroupMember.Read.All` (Read group memberships) - **Required for group syncing** 4. Click **Add permissions** 5. If required by your organization, click **Grant admin consent** ## Step 4: Configure Group Claims For group synchronization to work, configure group claims: 1. Go to **Token configuration** 2. Click **Add groups claim** 3. Configure the groups claim: * **Select group types**: Choose **Security groups** * **Customize token properties**: * For **ID**: Select `Group ID` (returns GUIDs) or `Cloud-only group display names` (returns readable names) * For better control, select **Groups assigned to the application** 4. Click **Add** ## Step 5: Create Client Secret 1. Go to **Certificates & secrets** 2. Click **New client secret** 3. Add description: `Bytebase OIDC` 4. Select expiration period (recommend 24 months) 5. Click **Add** 6. **Important**: Copy the secret value immediately (it won't be shown again) ## Step 6: Gather Configuration Information From the **Overview** page, collect: * **Application (client) ID**: Your client ID * **Directory (tenant) ID**: Your tenant ID Construct the issuer URL: ```text theme={null} https://login.microsoftonline.com/{tenant-id}/v2.0 ``` ## Step 7: Configure Bytebase OIDC 1. Log into Bytebase as an admin 2. Navigate to **Settings** → **SSO** → **Create** 3. Select **OIDC** as the provider type 4. Configure with the following settings: **Basic Information:** * **Name**: `Entra ID` * **Identity Provider ID**: `entra-id` * **Domain**: Your organization domain (optional) **Identity Provider Information:** * **Issuer**: `https://login.microsoftonline.com/{your-tenant-id}/v2.0` * **Client ID**: Your application client ID * **Client Secret**: Your client secret * **Scopes**: `openid profile email offline_access` **User Information Field Mapping:** * **Email**: `email` * **Display Name**: `name` * **Groups**: `groups` (Important for group syncing) 5. Click **Create** ## Step 8: Test the Configuration 1. Click **Test Connection** in Bytebase 2. You should be redirected to Microsoft login 3. After successful authentication, verify: * User information is correctly mapped * Groups are being returned (if configured) ## Group Synchronization If you configured group claims, Bytebase will automatically sync user groups: 1. Go to **Settings** → **Groups** in Bytebase 2. Create groups that match your Entra ID groups: * If using Group IDs, create groups with GUID names * If using display names, create groups with matching names 3. Assign appropriate roles to these groups Users will be automatically added to matching Bytebase groups upon login. ## Example Group Synchronization Flow 1. User logs in via Entra ID 2. ID token includes groups claim: `["c80adaad-11c0-4ad9-aa31-7ef37029aa2a", "6525b949-f790-4807-bf85-707fa6ec4cb4"]` 3. Bytebase receives and processes groups 4. User is automatically added to matching Bytebase groups 5. User receives permissions based on group membership ## Troubleshooting ### Groups Not Appearing 1. **Check Token Configuration**: Ensure groups claim is added for ID tokens 2. **Verify Permissions**: Ensure `GroupMember.Read.All` permission is granted 3. **Check Group Types**: Verify users are members of security groups 4. **Enable Debug Logging**: Run Bytebase with `--debug` flag to see claim details ### Common Issues 1. **Groups showing as GUIDs**: This is normal when using Group ID format 2. **Missing groups**: Ensure groups are assigned to the application if using that filter 3. **Token size limitations**: Consider using "Groups assigned to the application" for users with many groups ## Reference * [Microsoft identity platform and OpenID Connect protocol](https://docs.microsoft.com/en-us/azure/active-directory/develop/v2-protocols-oidc) * [Configure group claims for applications](https://docs.microsoft.com/en-us/azure/active-directory/develop/active-directory-optional-claims) # Overview Source: https://docs.bytebase.com/administration/sso/overview Single Sign-On (SSO) is an authentication method that enables users to securely authenticate with multiple applications and websites by using just one set of credentials. ## Supported Protocols Bytebase supports three SSO protocols: * **[OAuth 2.0](/administration/sso/oauth2)** — Authorization protocol. Bytebase gets a token and fetches your profile from the provider. Good for providers that don't support OIDC (e.g. GitHub). * **[OpenID Connect (OIDC)](/administration/sso/oidc)** — Identity layer built on top of OAuth 2.0. Adds a standardized ID token so Bytebase gets user info directly. Preferred when available. * **[LDAP](/administration/sso/ldap)** — Directory lookup protocol. Bytebase queries a user directory directly with username/password. No browser redirects involved. ## Authentication Flow There are two ways SSO authentication can start: * **SP-Initiated (default)** — User visits Bytebase, clicks "Sign in with X", gets redirected to the identity provider, then back to Bytebase after authentication. * **[IdP-Initiated](/administration/sso/idp-initiated)** — User clicks the Bytebase tile directly from their identity provider's dashboard (e.g. Okta portal) and lands in Bytebase already authenticated. Works with OAuth 2.0 and OIDC providers. ## User & Group Provisioning (SCIM) SSO handles **authentication** (who is this person), but not **provisioning** (creating/updating/deactivating accounts). For automatic user and group provisioning, Bytebase supports [SCIM 2.0](/administration/scim/overview) with [Entra ID](/administration/scim/entra-id) and [Okta](/administration/scim/okta). SCIM requires the IdP to communicate with Bytebase over HTTP. **LDAP does not support SCIM** — if you need SCIM with a provider like Microsoft Entra ID, you must configure SSO via [OIDC](/administration/sso/oidc-entra-id) or [OAuth 2.0](/administration/sso/oauth2#microsoft-entra-azure-ad), not LDAP. ## Which Protocol to Use | Identity Provider | Recommended Protocol | [SCIM](/administration/scim/overview) | Notes | | ---------------------------------------- | ---------------------------------------------- | ------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------- | | Google | [OIDC](/administration/sso/oidc#google) | — | Also available via [OAuth 2.0](/administration/sso/oauth2#google) | | GitHub | [OAuth 2.0](/administration/sso/oauth2#github) | — | GitHub does not support OIDC | | GitLab | [OIDC](/administration/sso/oidc#gitlab) | — | Also available via [OAuth 2.0](/administration/sso/oauth2#gitlab) | | Microsoft Entra ID (Azure AD) | [OIDC](/administration/sso/oidc-entra-id) | [Yes](/administration/scim/entra-id) | Also available via [OAuth 2.0](/administration/sso/oauth2#microsoft-entra-azure-ad) | | Okta | [OIDC](/administration/sso/oidc#okta) | [Yes](/administration/scim/okta) | Also available via [LDAP](/administration/sso/ldap#okta) (no SCIM) | | Keycloak | [OIDC](/administration/sso/oidc#keycloak) | — | | | Casdoor | [OIDC](/administration/sso/oidc#casdoor) | — | | | Authing | [OIDC](/administration/sso/oidc#authing) | — | | | JumpCloud | [LDAP](/administration/sso/ldap#jumpcloud) | — | | | Microsoft Active Directory (on-premises) | [LDAP](/administration/sso/ldap) | — | The on-premises domain controller. For Microsoft Entra ID (the cloud service, formerly Azure AD), use OIDC — see the Entra ID row above | | OpenLDAP | [LDAP](/administration/sso/ldap) | — | | **—** means Bytebase has no verified SCIM integration guide for that provider. Bytebase implements standard [SCIM 2.0](/administration/scim/overview), so other cloud IdPs that can push SCIM over HTTP may work but are untested. LDAP providers cannot use SCIM at all (see the warning above), and LDAP sign-in maps only the user's email and display name — group membership is not synced over LDAP. ## Prerequisites Configure [External URL](/get-started/self-host/external-url). ## Create SSO provider As a **Workspace Admin**, you can create a SSO provider following the steps below: 1. In Workspace, go to **IAM & Admin** -> **SSO** from left side bar. 2. Click **Create** to start creating SSO. 3. Fill in all the required fields. create-sso-dialog 4. **Test Connection** on bottom left. 5. If successfully connected, click **Update** on bottom right. ## Sign in with SSO Bytebase employs JIT (Just-In-Time) user provisioning. It will create the user the first time the user signs in. Once a valid SSO has been created, you can choose to sign in with the configured SSO provider. sign-in-with-github ## Enforce SSO Sign-in As `Workspace Admin`, you can enforce SSO sign-in for all users in Workspace. In **Workspace**, go to **Settings** > **General**, scroll down to **Account** section and switch on `Disallow signin with email & password`. Then users can only sign in with SSO. disallow-emailpass-only-sso Afterwards, when the user tries to sign in, the only option is to sign in with the configured SSO provider. only-sso In case of emergency, the admin can log in by navigating to `/auth/admin` and entering the email and password. # Users & Groups Source: https://docs.bytebase.com/administration/user-groups/overview ## User A `User` represents a human team member who can access and perform operations in Bytebase. For machine identities, see [Service Accounts](/administration/service-account) and [Workload Identity](/administration/workload-identity/overview). ## User Group A `User Group` organizes multiple users together for easier permission management. Workspace admins create groups and add users, then assign these groups to roles within projects. Bytebase does not support nested groups. A group can only contain users, it can't contain another group. Machine identities (service accounts and workload identities) cannot be part of user groups. Since they are for automated processes with specific access needs, including them in groups could grant unintended permissions. This is considered an [anti-pattern](https://cloud.google.com/iam/docs/best-practices-service-accounts#groups). ## Add group Within Workspace, go to **IAM & Admin** > **Users & Groups**, and **Add Group** from top-right. add-group Create an Email for this group, it'll serve as an account and *cannot be changed after creation*. Fill the group's name into the Title bar. You can **Add member** below, where they can be `Group member` as well as `Group owner`. add-group-detail Here we've created a `Contractor Group`, you can view or edit it under **Groups** page. view-under-groups You can see which group a user belongs to under **Users** page as well. view-under-users ## Grant roles to group Now that we've created this `Contractor Group`, we can assign corresponding permissions to these groups within any project. **Select Project** from top left. Enter `Basic Project`. enter-basic-project Go to **Manage** > **Members** where you can see the project's members and roles. Our `Contractor Group` is not among them before we **Grant Access** to the group from top right. Choose **Groups** and Select our group in **Grant Access** detail page. **Assign role** and **Confirm**. grant-access Now you can see the `Contractor Group` under **View by members** page as well as **View by roles** page within **Members** section of `Basic Project`. project-members-or-roles All members within this group now share permission to the project. # Workload Identity for GitHub Actions Source: https://docs.bytebase.com/administration/workload-identity/github-actions This guide explains how to configure Workload Identity for GitHub Actions to authenticate with Bytebase without storing long-lived credentials. ## Step 1: Create a Workload Identity in Bytebase 1. Go to **IAM & Admin** > **Users & Groups**. 2. Click **Add User** in the upper-right corner. 3. Select **Workload Identity** as the Type. 4. Fill in the configuration: | Field | Description | Example | | -------------- | ----------------------------------------------------------------------- | --------------------------------------------- | | **Name** | Display name for this identity | `GitHub Actions Deploy` | | **Email** | Unique email for this identity (must end with `@workload.bytebase.com`) | `github-actions-deploy@workload.bytebase.com` | | **Platform** | Select GitHub Actions | `GitHub Actions` | | **Owner** | GitHub organization or username | `my-org` | | **Repository** | Repository name | `my-repo` | | **Branch** | Branch name (use `*` for all branches) | `main` | 5. Click **Confirm** to create the Workload Identity. ## Step 2: Assign Roles After creating the Workload Identity, assign the `GitOps Service Agent` role to enable automated CI/CD workflows: 1. Go to your project's **Settings** > **Members**. 2. Click **Grant Access**. 3. Enter the Workload Identity email (e.g., `github-actions-deploy@workload.bytebase.com`). 4. Select the **GitOps Service Agent** role. 5. Click **Confirm**. The `GitOps Service Agent` role is designed for automated CI/CD workflows, allowing the identity to create and execute database changes. See [Roles and Permissions](/administration/roles) for details. ## Step 3: Configure GitHub Actions Workflow In your GitHub Actions workflow, add the following configuration: ### Request OIDC Token Add `id-token: write` permission and use the `actions/github-script` action to get the token: ```yaml theme={null} name: Deploy Database Changes on: push: branches: [main] permissions: id-token: write # Required for OIDC token contents: read env: BYTEBASE_URL: https://bytebase.example.com WORKLOAD_IDENTITY_EMAIL: github-actions-deploy@workload.bytebase.com jobs: deploy: runs-on: ubuntu-latest steps: - name: Checkout uses: actions/checkout@v4 - name: Get Bytebase Token id: bytebase-token uses: actions/github-script@v7 with: script: | const token = await core.getIDToken('https://github.com/${{ github.repository_owner }}'); core.setSecret(token); core.setOutput('token', token); - name: Exchange for Bytebase API Token id: exchange run: | RESPONSE=$(curl -s -X POST "${BYTEBASE_URL}/v1/auth:exchangeToken" \ -H "Content-Type: application/json" \ -d "{\"token\": \"${{ steps.bytebase-token.outputs.token }}\", \"email\": \"${WORKLOAD_IDENTITY_EMAIL}\"}") ACCESS_TOKEN=$(echo "$RESPONSE" | jq -r '.accessToken') echo "::add-mask::$ACCESS_TOKEN" echo "access_token=$ACCESS_TOKEN" >> $GITHUB_OUTPUT - name: Call Bytebase API run: | curl -s "${BYTEBASE_URL}/v1/projects" \ -H "Authorization: Bearer ${{ steps.exchange.outputs.access_token }}" ``` ## Complete Example Here's a complete workflow that creates a database change using Workload Identity: ```yaml theme={null} name: Database Schema Change on: push: branches: [main] paths: - 'migrations/**' permissions: id-token: write contents: read env: BYTEBASE_URL: https://bytebase.example.com WORKLOAD_IDENTITY_EMAIL: github-actions-deploy@workload.bytebase.com PROJECT: projects/my-project jobs: deploy: runs-on: ubuntu-latest steps: - name: Checkout uses: actions/checkout@v4 - name: Get OIDC Token id: oidc uses: actions/github-script@v7 with: script: | const token = await core.getIDToken('https://github.com/${{ github.repository_owner }}'); core.setSecret(token); core.setOutput('token', token); - name: Exchange Token id: auth run: | RESPONSE=$(curl -s -X POST "${BYTEBASE_URL}/v1/auth:exchangeToken" \ -H "Content-Type: application/json" \ -d "{\"token\": \"${{ steps.oidc.outputs.token }}\", \"email\": \"${WORKLOAD_IDENTITY_EMAIL}\"}") ACCESS_TOKEN=$(echo $RESPONSE | jq -r '.accessToken') if [ "$ACCESS_TOKEN" = "null" ] || [ -z "$ACCESS_TOKEN" ]; then echo "Failed to get access token" echo $RESPONSE exit 1 fi echo "::add-mask::$ACCESS_TOKEN" echo "access_token=$ACCESS_TOKEN" >> $GITHUB_OUTPUT - name: Create Plan id: plan run: | # Read migration SQL file SQL_CONTENT=$(cat migrations/latest.sql | jq -Rs .) RESPONSE=$(curl -s -X POST "${BYTEBASE_URL}/v1/${PROJECT}/plans" \ -H "Authorization: Bearer ${{ steps.auth.outputs.access_token }}" \ -H "Content-Type: application/json" \ -d "{ \"title\": \"Migration from GitHub Actions\", \"steps\": [{ \"specs\": [{ \"changeDatabaseConfig\": { \"target\": \"instances/prod/databases/mydb\", \"type\": \"MIGRATE\", \"sheet\": \"${SQL_CONTENT}\" } }] }] }") PLAN_NAME=$(echo $RESPONSE | jq -r '.name') echo "plan_name=$PLAN_NAME" >> $GITHUB_OUTPUT ``` ## Troubleshooting ### Token Exchange Fails If the token exchange returns an error: 1. **Verify the repository and branch**: Check that your workflow's repository, branch match the configured values in Bytebase. 2. **Check the audience**: Ensure the audience in your `getIDToken()` call matches `https://github.com/{owner}`. ### Permission Denied If API calls return permission errors: 1. Verify the Workload Identity has the `GitOps Service Agent` role assigned. 2. Check that the Workload Identity is a member of the target project. ### Debug Token Claims To inspect the OIDC token claims, decode the JWT: ```yaml theme={null} - name: Debug Token run: | echo "${{ steps.oidc.outputs.token }}" | cut -d. -f2 | base64 -d | jq . ``` This shows the token's claims including `sub`, `aud`, and `iss` that Bytebase validates. # Workload Identity for GitLab CI/CD Source: https://docs.bytebase.com/administration/workload-identity/gitlab-ci This guide explains how to configure Workload Identity for GitLab CI/CD to authenticate with Bytebase without storing long-lived credentials. ## Step 1: Create a Workload Identity in Bytebase 1. Go to **IAM & Admin** > **Users & Groups**. 2. Click **Add User** in the upper-right corner. 3. Select **Workload Identity** as the Type. 4. Fill in the configuration: | Field | Description | Example | | ------------------------- | -------------------------------------------------------------------------- | ----------------------- | | **Name** | Display name for this identity | `GitLab CI Deploy` | | **Email** | Unique email prefix (automatically appended with `@workload.bytebase.com`) | `gitlab-ci-deploy` | | **Platform** | Select GitLab CI | `GitLab CI` | | **Group / Username** | GitLab group or username (required) | `my-group` | | **Project** | Project name (leave empty to allow all projects) | `my-project` | | **Allowed Branches/Tags** | Select branch/tag restrictions | `All branches and tags` | | **Roles** | Assign workspace roles | `GitOps Service Agent` | 5. Click **Confirm** to create the Workload Identity. ## Step 2: Assign Roles After creating the Workload Identity, assign the `GitOps Service Agent` role to enable automated CI/CD workflows: 1. Go to your project's **Settings** > **Members**. 2. Click **Grant Access**. 3. Enter the Workload Identity email (e.g., `gitlab-ci-deploy@workload.bytebase.com`). 4. Select the **GitOps Service Agent** role. 5. Click **Confirm**. The `GitOps Service Agent` role is designed for automated CI/CD workflows, allowing the identity to create and execute database changes. See [Roles and Permissions](/administration/roles) for details. ## Step 3: Configure GitLab CI/CD Pipeline In your GitLab CI/CD pipeline, add the following configuration: ### Request OIDC Token Add `id_tokens` configuration to get the JWT token from GitLab: ```yaml theme={null} stages: - deploy deploy-database: stage: deploy image: alpine:latest id_tokens: GITLAB_OIDC_TOKEN: aud: https://gitlab.com variables: BYTEBASE_URL: https://bytebase.example.com WORKLOAD_IDENTITY_EMAIL: gitlab-ci-deploy@workload.bytebase.com before_script: - apk add --no-cache curl jq script: - | # Exchange GitLab OIDC token for Bytebase API token RESPONSE=$(curl -s -X POST "${BYTEBASE_URL}/v1/auth:exchangeToken" \ -H "Content-Type: application/json" \ -d "{\"token\": \"${GITLAB_OIDC_TOKEN}\", \"email\": \"${WORKLOAD_IDENTITY_EMAIL}\"}") ACCESS_TOKEN=$(echo "$RESPONSE" | jq -r '.accessToken') if [ "$ACCESS_TOKEN" = "null" ] || [ -z "$ACCESS_TOKEN" ]; then echo "Failed to get access token" echo "$RESPONSE" exit 1 fi # Verify the token by calling the user info API USER_INFO=$(curl -s "${BYTEBASE_URL}/v1/users/me" \ -H "Authorization: Bearer $ACCESS_TOKEN") echo "Authenticated as: $USER_INFO" rules: - if: $CI_COMMIT_BRANCH == "main" ``` ## Complete Example Here's a complete GitOps workflow that uses Workload Identity to deploy database migrations: ```yaml theme={null} # .gitlab-ci.yml stages: - review - deploy variables: BYTEBASE_URL: https://bytebase.example.com WORKLOAD_IDENTITY_EMAIL: gitlab-deploy@workload.bytebase.com # SQL Review on merge requests sql-review: stage: review image: bytebase/sql-review-action:latest id_tokens: GITLAB_OIDC_TOKEN: aud: https://gitlab.com script: - | # Exchange OIDC token for Bytebase token export BYTEBASE_TOKEN=$(curl -s -X POST "${BYTEBASE_URL}/v1/auth:exchangeToken" \ -H "Content-Type: application/json" \ -d "{\"token\": \"${GITLAB_OIDC_TOKEN}\", \"email\": \"${WORKLOAD_IDENTITY_EMAIL}\"}" \ | jq -r '.accessToken') # Run SQL review sql-review --url ${BYTEBASE_URL} --token ${BYTEBASE_TOKEN} \ --file-pattern "migrations/**/*.sql" rules: - if: $CI_PIPELINE_SOURCE == "merge_request_event" changes: - migrations/** # Deploy on merge to main rollout: stage: deploy image: bytebase/bytebase-action:latest id_tokens: GITLAB_OIDC_TOKEN: aud: https://gitlab.com script: - | # Exchange OIDC token for Bytebase token export BYTEBASE_TOKEN=$(curl -s -X POST "${BYTEBASE_URL}/v1/auth:exchangeToken" \ -H "Content-Type: application/json" \ -d "{\"token\": \"${GITLAB_OIDC_TOKEN}\", \"email\": \"${WORKLOAD_IDENTITY_EMAIL}\"}" \ | jq -r '.accessToken') # Create release and rollout bytebase-action rollout \ --url ${BYTEBASE_URL} \ --token ${BYTEBASE_TOKEN} \ --file-pattern "migrations/**/*.sql" \ --project projects/my-project \ --targets instances/prod/databases/mydb rules: - if: $CI_COMMIT_BRANCH == "main" changes: - migrations/** ``` For more details on GitOps workflows, see [GitOps Overview](/gitops/overview) and [Migration-Based Workflow](/gitops/migration-based-workflow/overview). ## Self-Hosted GitLab For self-hosted GitLab instances, update the audience (`aud`) to match your GitLab instance URL: ```yaml theme={null} id_tokens: GITLAB_OIDC_TOKEN: aud: https://gitlab.your-company.com ``` When creating the Workload Identity in Bytebase, ensure the configuration matches your self-hosted GitLab instance. ## Troubleshooting ### Token Exchange Fails If the token exchange returns an error: 1. **Verify the project path and branch**: Check that your pipeline's project path and branch match the configured values in Bytebase. 2. **Check the audience**: Ensure the `aud` in your `id_tokens` configuration matches your GitLab instance URL (e.g., `https://gitlab.com` for GitLab.com). 3. **Verify OIDC is enabled**: GitLab CI/CD OIDC tokens require GitLab 15.7 or later. ### Permission Denied If API calls return permission errors: 1. Verify the Workload Identity has the `GitOps Service Agent` role assigned. 2. Check that the Workload Identity is a member of the target project. ### Debug Token Claims To inspect the OIDC token claims, decode the JWT: ```yaml theme={null} script: - | echo "$GITLAB_OIDC_TOKEN" | cut -d. -f2 | base64 -d | jq . ``` This shows the token's claims including `sub`, `aud`, `namespace_path`, `project_path`, and `ref` that Bytebase validates. # Overview Source: https://docs.bytebase.com/administration/workload-identity/overview Workload Identity is a secure authentication method for CI/CD pipelines and external services using OpenID Connect (OIDC) tokens, eliminating the need for long-lived credentials. Unlike [Service Accounts](/administration/service-account) that require storing API keys as secrets, Workload Identity: * Uses short-lived tokens generated per job * Validates tokens against your CI/CD platform's identity provider * Restricts access to specific repositories, branches, and workflows ## Workspace vs Project Level Workload identities can be created at two levels: * **Workspace level** — Has access governed by workspace IAM policies. Suitable for cross-project CI/CD workflows. * **Project level** — Scoped to a single project, following the principle of least privilege. Suitable for project-specific pipelines. ## Supported Platforms Configure OIDC authentication for GitHub Actions workflows Configure OIDC authentication for GitLab CI/CD pipelines # AI Assistant Source: https://docs.bytebase.com/ai-assistant Bytebase only sends the database schema (table and column names) to the configured AI provider. It never sends the table data. You can check the [relevant code](https://github.com/bytebase/bytebase/blob/main/frontend/src/plugins/ai/logic/prompt.ts). AI Assistant uses the configured AI provider to augment the database development workflow. It powers [SQL Editor AI Assistant](/sql-editor/ai-assistant) and the [Page Agent](/page-agent). You can connect it to OpenAI, Azure OpenAI, Gemini, or Claude. Each provider works with its hosted API or with a custom endpoint — a private deployment, self-hosted model, or LLM gateway — that implements that provider's API. ## Enable AI Assistant Go to Bytebase console, click **Settings > General**. Scroll down to **AI Assistant** and check **Enable AI Assistant**. AI Assistant settings showing the provider, API key, endpoint, and model name fields Configure the following: * **AI Provider**. Choose **OpenAI**, **Azure OpenAI**, **Gemini**, or **Claude**. Selecting a provider prefills the default **API Endpoint** and **Model Name** below. * **API Key**. The API key for the selected provider. Use the **Find my key** link next to the field to reach the provider's key page. * **API Endpoint**. The provider's API endpoint. Prefilled with the default; override it to point at a private deployment, proxy, or gateway. The endpoint must implement the **selected provider's** API — OpenAI-compatible for OpenAI and Azure OpenAI, Gemini-compatible for Gemini, Anthropic-compatible for Claude (see [Connect to a custom endpoint](#connect-to-a-custom-endpoint)). When overriding, follow the same URL shape as the provider defaults below (e.g., Gemini uses the `v1beta` base URL; OpenAI/Azure/Claude use the full request URL). * **Model Name**. The model to use. Prefilled with the provider default; you can enter any model name the endpoint serves, including newer models if your endpoint supports them. Each provider prefills these defaults: | Provider | Default API Endpoint | Default Model Name | | ------------ | --------------------------------------------------------------------------------------------------------------------- | ------------------ | | OpenAI | `https://api.openai.com/v1/chat/completions` | `gpt-5.5` | | Azure OpenAI | `https://{resource name}.openai.azure.com/openai/deployments/{deployment id}/chat/completions?api-version=2024-06-01` | `gpt-5.5` | | Gemini | `https://generativelanguage.googleapis.com/v1beta` | `gemini-3.5-flash` | | Claude | `https://api.anthropic.com/v1/messages` | `claude-sonnet-5` | **Model Name** is a free-form field — Bytebase does not validate it against a fixed list. If you enter a model the endpoint doesn't serve, the error surfaces only when you use the assistant. In Bytebase Cloud (SaaS), Gemini is enabled automatically and the AI Assistant settings are hidden. ## Connect to a custom endpoint Every provider's **API Endpoint** can point at a custom URL instead of the vendor's hosted API — a private deployment, a self-hosted model, or an LLM gateway. Bytebase sends the selected provider's native request format to whatever URL you set, so the endpoint must implement that provider's API. To connect to one, select the matching **AI Provider**, then set **API Endpoint** to your service's URL and fill in the **API Key** and **Model Name** it expects. Pick the provider by the API your endpoint speaks. Most self-hosted models and gateways expose an OpenAI-compatible API, so **OpenAI** is the common choice; select **Gemini** or **Claude** if your endpoint implements Google's or Anthropic's API instead. ## Features * [SQL Editor AI Assistant](/sql-editor/ai-assistant) * [Page Agent](/page-agent) # Custom Approval Source: https://docs.bytebase.com/change-database/approval ## Overview Custom Approval enables organizations to implement approval workflows for database operations including schema and data changes, database creation, role requests, and just-in-time data access (which also covers data export). You define conditions and approval nodes directly on each approval flow to determine when it applies and who needs to approve. ### Key Features * **Condition-based routing**: Define conditions on each approval flow to match specific changes * **Multi-stage approvals**: Chain multiple approval nodes in sequence * **Role-based authorization**: Leverage built-in or custom roles for approvals * **First-match-wins ordering**: Drag-and-drop to reorder flows; the first matching flow applies * **No approval required**: Optionally skip approval for certain changes **Workflow Compatibility:** * **UI Workflow**: Custom Approval integrates seamlessly with the UI-based change workflow * **GitOps Workflow**: For GitOps, configure approvals in your VCS (GitHub/GitLab/Bitbucket) PR/MR process instead ## How Approval Works ### The Approval Process 1. **Condition Matching**: When a request is created, Bytebase evaluates the approval flows in order under the matching source type. The first flow whose condition matches is used 2. **Fallback**: If no flow matches under the source type, the **Fallback** source type flows are evaluated as a catch-all. You can also add a catch-all flow at the end of each source type (with a broad or empty condition) to serve as a per-source-type fallback 3. **Sequential Approval**: Approvers review and approve each node in order 4. **Execution**: After all approvals, the request proceeds to execution (for example, deployment, database creation, data export, or role change) ### Source Types Approval flows are organized by source type: | Source | Description | | ------------------------------- | ------------------------------------------------------------------------------------------------------------------------- | | **Change Database** | DDL and DML operations (schema changes and data modifications) | | **Create Database** | Database creation operations | | **Request Role** | Role/permission request operations | | **Request Just-In-Time Access** | [Access grant](/security/database-permission/grant/) requests — run a specific statement, optionally exporting the result | | **Fallback** | Catch-all flows when no source-specific flow matches | ### Conditions Each approval flow uses a condition expression built from variables like environment, project, database engine, SQL type, and affected rows. The available variables depend on the source type, and the condition builder dropdown in the UI shows all options. For **Change Database**, a `risk_level` variable (`"HIGH"`, `"MODERATE"`, `"LOW"`) is also available. It is automatically calculated by Bytebase based on the statement type. See [Risk Center](/administration/risks) for details. ## Configuration Navigate to **CI/CD > Custom Approval**. Approval flows are grouped by source type. For each source type, click **Add Rule** to create a new approval flow. ### Creating an Approval Flow 1. **Title** (required): Name the flow descriptively (e.g., "Production DDL Review") 2. **Description**: Optionally describe when and why this flow applies 3. **Condition** (required): Define when this flow triggers. You can build conditions using the visual condition builder, or click **Add raw expression** to write a CEL expression directly. Combine multiple conditions with **Add condition** or group them with **Add condition group** 4. **Approval nodes**: Add one or more approval stages. Each node specifies a required role (built-in or [custom role](/administration/roles)). Toggle **No approval required** to skip manual approval for matching changes Click **Create** to save the flow. ### Ordering and Priority Flows are evaluated in order (top to bottom) under each source type. **The first matching flow wins.** Drag-and-drop to reorder flows. Place more specific conditions above broader ones. If no source-specific flow matches, fallback flows are evaluated. ### Preset Templates When creating a flow, preset templates are available for common scenarios. Click a template to populate the title, condition, and approval nodes automatically. ### Examples | Source | Condition | Approval Nodes | | --------------------------- | ---------------------------------------------------------------- | -------------------- | | Change Database | `resource.environment_id == "prod"` | Project Owner -> DBA | | Change Database | `resource.environment_id == "dev"` | No approval required | | Create Database | `resource.environment_id == "prod"` | DBA | | Request Just-In-Time Access | `resource.table_name == "salary" && request.data_export == true` | Project Owner | | Request Role | `resource.environment_id == "prod"` | Project Owner -> DBA | ### Related Settings * [Self-approval](/change-database/settings#self-approval) — allow issue creators to approve their own changes (disabled by default) * [Rollout policy](/change-database/environment-policy/rollout-policy/) — configure automatic vs manual rollout and plan check enforcement # Batch Change Source: https://docs.bytebase.com/change-database/batch-change Bytebase allows you to change a collection of databases in a single workflow. These databases usually have a homogenous structure while belonging to different development environments, geographic locations, SaaS tenants, data centers or data partitions. ## Change databases from multiple environments A development pipeline usually consists of multiple environments. Each environment has its isolated database to store the respective data. A schema change will be propagated from each database following the development pipeline. You may have multiple databases in one stage as well. In Bytebase, you can select multiple databases from different environments to apply the database change. bb-db-edit-schema Then Bytebase will then create an issue to track the multi-database changes. You may deploy changes one database after another or batch deploy databases at the same stage. ## Database group Database groups allow you to manage and batch change multiple databases together. This is particularly useful for SaaS multi-tenant architectures, geographic deployments, or database partitioning scenarios. Learn how to create and manage database groups ### Batch change databases in a group 1. Within the project, click **Database > Groups** on the left-side bar. Click **Edit Schema**, choose **Database Group**, select the database group you'll operate and click **Next**. bb-db-group-edit 2. An issue with the selected databases will be previewed. Add the SQL in the text field and click **Create**. The issue is created and waiting for approval. bb-issue-batch-created # Changelog Source: https://docs.bytebase.com/change-database/change-history Every time Bytebase applies a change to a database, it records a **changelog** entry. Each entry captures a snapshot of the schema after the change, along with a link to the rollout task that produced it. The changelog was previously called change history. ## View the changelog Open a database and select the **Changelog** tab to see every change Bytebase has applied, most recent first. Each entry shows its status, when it was created, and the rollout it came from. The first migration on a database also creates a baseline entry that records the starting schema. Changelog tab listing every change applied to the database Open an entry to view the schema snapshot captured after the change. Bytebase diffs it against the previous entry's snapshot so you can see what the migration altered. The SQL that ran is available through the linked rollout task's run logs. Changelog entry detail with execution logs, the after-change schema snapshot diff, and the Rollback button ## Roll back a change From a changelog entry, select **Rollback** to revert the database to the schema snapshot from the previous changelog entry (the schema before the change). Bytebase seeds the [Synchronize Schema](/change-database/synchronize-schema) flow with that snapshot and generates the DDL to return the database to that state, which you then review and roll out like any other change. Rollback is available when: * The database does not belong to the `Default` project. * You have permission to alter the database's schema. * The entry's status is **Done** and the change produced an actual schema difference. * The database engine supports schema rollback: MySQL, TiDB, OceanBase, PostgreSQL, Oracle, and SQL Server. ## Revision Versioned migrations also record a **revision** on the database, tracked on the separate **Revision** tab. Revisions capture which migration versions have been applied so Bytebase knows what still needs to run. See [GitOps](/gitops/overview) for how revisions drive version-based rollouts. ### Import a revision If a version has already been applied outside Bytebase, import it as a revision so Bytebase marks it as done without re-running the SQL. On the **Revision** tab, click **Import**, then choose a source: * **From Release** — pick a project [release](/gitops/migration-based-workflow/release) and select one or more of its files. Files whose version already has a revision on the database are listed separately as already imported and can't be selected again. * **From Local Files** — upload one or more `.sql` files. Bytebase tries to parse a version from each filename; enter or edit it before confirming. Confirm to create the revisions. Deleting an imported revision removes the record only — it does not change the database schema. Using the state-based (SDL) workflow? Releases work the same way — see the [state-based release page](/gitops/state-based-workflow/release). Local files such as `schema/public.sql` or `tables/users.sql` usually have no parseable version, so enter one manually, and set **Revision Type** to `DECLARATIVE`. # Overview Source: https://docs.bytebase.com/change-database/change-workflow ## Database CI/CD Bytebase Database CI/CD provides a complete platform for managing database changes throughout the development lifecycle. By eliminating direct database access and enforcing structured processes, it ensures every change is safe, reviewed, and auditable. ## The Database CI/CD Lifecycle Every database change in Bytebase follows a structured lifecycle, regardless of which workflow you choose: ### 1. **Plan Change** Define what needs to change—whether it's a schema migration (DDL) or data modification (DML). Changes can target: * A single database * Multiple selected databases * A Database Group (databases sharing the same schema) ### 2. **Automatic SQL Review** Built-in SQL advisor validates every change before execution using configurable [Review Policies](/sql-review/review-policy): * **Syntax validation**: Catch SQL errors before deployment * **Schema rules**: Naming conventions, data types, and structure standards * **Performance checks**: Missing indexes, inefficient queries, table scans * **Security policies**: Prevent unsafe operations and data exposure * **Backward compatibility**: Ensure changes won't break existing applications * **Best practices**: Enforce organizational and industry standards You can configure SQL Review Policies at the environment or project level to enforce your team's specific standards and requirements. ### 3. **Approval Process** Flexible approval mechanisms based on your workflow choice: * **UI-Driven**: Built-in multi-level approval within Bytebase * **GitOps**: * Leverage pull request reviews in GitHub/GitLab/Bitbucket * Integrate with external systems like ServiceNow, Jira, or custom approval workflows * Condition-based routing: Define [custom approval rules](/change-database/approval) based on conditions like environment, risk level, SQL type, and more ### 4. **Multi-Environment Rollout Pipeline** Changes progress through your environments in a controlled pipeline: * **Stage-by-stage promotion**: Dev → Test → Staging → Production * **Configurable deployment paths**: Define custom environment chains * **Database Groups per stage**: Different database sets for each environment * **Gated progression**: Manual or automatic promotion between stages * **Environment-specific policies**: Different review rules per stage Within each stage, deployments execute with precision: * **Parallel execution**: Deploy across Database Groups simultaneously * **Sequential ordering**: Respect dependencies between databases * **Real-time monitoring**: Track progress and view execution logs * **Automatic retry**: Handle transient failures gracefully ### 5. **Rollback Capabilities** Instant recovery when things go wrong: * **DML Instant Rollback**: One-click recovery for UPDATE/DELETE operations * **Schema Rollback**: Generate reverse migration scripts for DDL changes * Automatic backup before risky DML operations ## Batch Changes with Database Groups Deploy changes at scale using Database Groups—logical collections of databases that share the same schema: * **Multi-tenant SaaS**: Update all tenant databases simultaneously * **Geographic distribution**: Roll out to regional databases in controlled waves * **Environment management**: Group databases by dev/staging/prod for systematic promotion Database Groups enable you to manage 10 or 10,000 databases with the same effort. ## Choose Your Workflow Bytebase offers two workflows to integrate with your existing processes: Visual, self-contained workflow managed entirely through Bytebase console Code-first approach integrated with your Git provider (GitHub, GitLab, Bitbucket) and CI/CD pipeline ## Which Workflow Should You Choose? ### Choose UI-Driven Workflow If: * **You prefer visual interfaces** — Point-and-click change management with immediate feedback * **You need centralized control** — All database changes managed in one dedicated platform * **Multiple teams involved** — DBAs, security, and compliance teams need visibility and approval rights * **No existing CI/CD for databases** — Get started quickly without setting up additional infrastructure * **Ad-hoc changes are common** — Hotfixes and emergency changes need quick turnaround **Best for:** Teams establishing database DevOps practices, organizations with dedicated DBAs, enterprises requiring multi-level approvals [Learn more about UI-Driven Workflow →](/change-database/plan) ### Choose GitOps Workflow If: * **Database schema = Application code** — Migration scripts live alongside your application * **Existing CI/CD pipelines** — Database changes follow your established deployment process * **Developer-centric culture** — Engineers own the full stack including database * **Git is your source of truth** — All changes tracked through version control * **Automation first** — Minimize manual intervention in deployments **Best for:** DevOps teams, microservices architectures, organizations practicing Infrastructure-as-Code [Learn more about GitOps Workflow →](/gitops/overview) ## Can I Use Both? **Yes!** Many teams use both workflows: * GitOps for routine application deployments * UI-Driven for complex migrations, hotfixes, or sensitive production changes * Different workflows for different teams or projects Both workflows share the same underlying CI/CD lifecycle, ensuring consistency regardless of how changes are initiated. # Environment Policy Source: https://docs.bytebase.com/change-database/environment-policy/overview Configure and manage your database environments including policies, permissions, and lifecycle operations. ## Environment tier ### Environment color You can configure any color for an environment either by inputting in **HEX** tab or choosing one in the palette. SQL Editor then displays the configured color tab. env-color-sql-editor ### Production indicator Once you mark an environment as a production environment, Bytebase will attach a shield icon 🛡️ besides the environment name. ## Rollout policy Control who can deploy changes to each environment and whether deployments happen automatically or require manual approval. Manage deployment permissions and automatic rollout settings ## SQL Review policy Configure automated SQL validation rules for each environment to ensure code quality and compliance. Different environments can have different strictness levels - for example, production might enforce stricter rules than development. Set up automated SQL validation rules and best practices enforcement ## Statement execution Whether a user can run DDL and DML statements directly in SQL Editor is governed by the `bb.sql.ddl` and `bb.sql.dml` [database permissions](/security/database-permission/overview/) carried by their granted roles — the built-in **SQL Editor User** role includes both. The environments where DDL/DML may run directly are part of the permission itself, selected when granting the role. Without the permission on a database's environment, SQL Editor directs the user to create a database change plan and follow the [change workflow](/change-database/change-workflow/) instead. There is no environment-level statement execution setting. ## Delete an environment To permanently delete an environment: 1. Navigate to the environment page 2. Click **Delete this environment** at the bottom of the page 3. Confirm the deletion Deleting an environment will also remove it from all related instances and databases. This action cannot be undone. # Rollout Permissions Source: https://docs.bytebase.com/change-database/environment-policy/rollout-policy Configure who can deploy database changes and what conditions must be met before rollout can proceed in Environment and Project. ## Configure rollout policy in Environment ### Role-based manual rollout **Default roles with permissions**: Users with the `bb.taskRuns.create` permission can perform rollouts. The following roles have this permission by default: `Workspace Admin`, `Workspace DBA`, `Project Owner`, `Project Releaser`. **Environment-specific roles**: If you want to use different roles to perform rollouts in different environments, you can specify environment-specific roles. This allows you to control who can deploy changes to specific environments, such as having different teams responsible for staging versus production deployments. ### Automatic rollout Select the `automatic` option to deploy changes without a manual rollout step. Changes then deploy automatically as soon as the [configured rollout requirements](#configurable-rollout-requirements-in-project) (approval and plan checks) are satisfied. Do not enable automatic rollout when using the GitOps workflow, as it can cause conflicts that result in failures. ## Configurable rollout requirements in Project Configure what conditions must be met before changes can be rolled out in each environment: ### Require issue approval before creating rollout The issue must be approved before creating a rollout. **Default**: Enabled ### Require plan checks to pass before creating rollout Plan checks must pass without error before creating a rollout. Warnings don't block — only errors do. **Default**: Enabled Plan checks include [SQL Review policy](/sql-review/review-policy) violations and other automated validations configured for your environment. # Online Schema Migration for MySQL Source: https://docs.bytebase.com/change-database/online-schema-migration-for-mysql [gh-ost](https://github.com/github/gh-ost) is a triggerless online schema migration tool for MySQL. Bytebase invokes gh-ost to migrate MySQL table schemas with minimal downtime. ## How it works Online schema migration tools operate through a careful orchestration of steps: 1. **Ghost table creation**: Creates a new table with the desired schema structure. 2. **Data migration**: Incrementally copies data from the original table to the ghost table while capturing ongoing changes (`INSERT`, `UPDATE`, `DELETE`). 3. **Table swap**: Atomically replaces the original table with the fully synchronized ghost table. The whole migration runs as a **single Bytebase task**. Once you start the rollout, ghost-table creation, data copy, replication catch-up, and cut-over all happen automatically in the background — there is no manual cut-over step. Bytebase uses a dedicated **`bbdataarchive`** database, separate from your application schema, to stage gh-ost's working tables. During the migration two temporary tables exist there: * **`__gho`** — the new table, being built with the desired schema and filled from the original. * **`
__ghc`** — the changelog table gh-ost uses to track DML against the original during the copy. At cut-over, gh-ost performs an atomic swap with two `RENAME TABLE`s: 1. The original table moves **out of your schema into `bbdataarchive`** as **`~
__del`** — preserved so you can verify the migration before dropping it (see Step 5). 2. The `_gho` table moves **out of `bbdataarchive` into your schema**, taking the original's name. It is now your new live table. The `_ghc` changelog is then dropped — it has no role after cut-over. The `_gho` is **not** dropped: it became your new table via the rename. So after a successful migration your application schema contains the new table, and `bbdataarchive` contains only the renamed `_del` original. ## Requirements and limitations ### Requirements * MySQL version 5.7 or higher * Row-based logging enabled * Primary key on the table * Primary database for migration (replica migration not currently supported) ### Limitations * Foreign key constraints are not supported * Triggers are not supported * Large tables may require extended migration time For a comprehensive list of requirements and limitations, refer to the [gh-ost documentation](https://github.com/github/gh-ost/blob/master/doc/requirements-and-limitations.md). ## Performing an online schema migration ### Step 1 - Enable online migration While creating a schema change plan, enable **Online migration** for the change. To customize gh-ost flags, click **Configure** next to the toggle. Enabling the toggle prepends a `-- gh-ost = { ... }` comment to your statement — this is the **gh-ost directive** that Bytebase parses at execution time to route the migration through gh-ost. With the toggle on and no customization, the statement becomes: ```sql theme={null} -- gh-ost = {} /* using default config */ ALTER TABLE employee ADD COLUMN nickname VARCHAR(255); ``` If you set flags via **Configure**, they're encoded as JSON, for example: ```sql theme={null} -- gh-ost = {"chunk-size":"1000","max-load":"Threads_running=25"} ALTER TABLE employee ADD COLUMN nickname VARCHAR(255); ``` Flags are read once when the issue is created and locked for the run. To change a flag, cancel the task and create a new issue. #### Writing the directive manually For GitOps workflows, or any case where you author SQL outside the Bytebase UI, add the directive yourself on its own line — Bytebase recognizes the same syntax. Use an empty object for default flags: ```sql theme={null} -- gh-ost = {} ALTER TABLE employee ADD COLUMN nickname VARCHAR(255); ``` Or pass any subset of these flags as JSON: `max-load`, `chunk-size`, `dml-batch-size`, `default-retries`, `cut-over-lock-timeout-seconds`, `exponential-backoff-max-interval`, `max-lag-millis`, `allow-on-master`, `switch-to-rbr`, `assume-rbr`, `heartbeat-interval-millis`, `nice-ratio`, `throttle-control-replicas`, `attempt-instant-ddl`, `assume-master-host`. See the [gh-ost command-line reference](https://github.com/github/gh-ost/blob/master/doc/command-line-flags.md) for what each flag does. ### Step 2 - Confirm gh-ost ran Roll out the resulting issue like any other DDL. Bytebase does not show a separate "gh-ost" badge on the rollout, so confirm the directive was picked up by checking the task log: * A **standard** MySQL DDL shows four sections — **Transaction**, **Command Execute**, **Transaction**, **Database Sync**. * A **gh-ost** migration shows a single **Database Sync** section. **Command Execute** is absent. If you see **Command Execute**, the directive was not parsed (most often because it is not on its own line). Fix the directive and re-create the issue. ### Step 3 - Watch progress in the backend logs The Bytebase UI shows only the start and end of a gh-ost migration. The full picture — cut-over events, the lock-and-rename duration, temporary-table cleanup, and any errors — lands in the **Bytebase server log**. Tail it and filter for `source=ghost/`. For example, a migration of `testdb.tmysql0` logs: | What's happening | Log line | | ----------------------------------------------------------- | -------------------------------------------------------------------------------------------------------- | | Migration starts on `testdb`.`tmysql0` | Migrating \`testdb\`.\`tmysql0\` | | Atomic cut-over — original and ghost tables are renamed | `Tables renamed` | | Writes against the original table were blocked for \~995 ms | Lock & rename duration: 994.696418ms. During this time, queries on \`tmysql0\` were blocked | | Bytebase drops the temporary changelog table | Dropping table \`bbdataarchive\`.\`\~tmysql0\_1778162448\_ghc\` | | Migration is done | Done migrating \`testdb\`.\`tmysql0\` | The **Lock & rename duration** is the only window in which writes against the original table are blocked — typically sub-second. In the raw log, gh-ost's printf-style messages pass through Bytebase's structured logger as-is, so you'll see `%s` placeholders alongside the values — for example, `msg="Lock & rename duration: %s..." !BADKEY=994.696418ms`. The values shown above are spliced in for readability. ### Step 4 - Cancel if needed A running gh-ost task can be canceled from the rollout page. Bytebase signals gh-ost to abort and drops the temporary `_gho` and `_ghc` tables from `bbdataarchive`. The original table in your application schema is left untouched. ### Step 5 - Post-migration cleanup gh-ost leaves the renamed `_del` original in `bbdataarchive` so you can verify the new table is correct before discarding it, and so a heavy `DROP TABLE` doesn't compete with traffic on the freshly cut-over table for I/O. Bytebase doesn't expose gh-ost's `--ok-to-drop-table` flag, so the drop is manual: 1. Verify your data integrity and application functionality against the new table. 2. Open the `bbdataarchive` database from the instance's database list — Bytebase syncs it like any other schema, so the `~yourtablename_{timestamp}_del` table is browseable and queryable. 3. Once verified, drop `bbdataarchive.~yourtablename_{timestamp}_del`. # Plan Source: https://docs.bytebase.com/change-database/plan A **Plan** is the core component of Bytebase’s CI/CD pipeline. It defines a series of schema or data changes to be applied to one or more databases. It can be converted into an **Issue** for review and deployment. ### 1. Create Plan In your project, navigate to **CI/CD** > **Plans** and click **+ New Plan**. ### 2. Select Databases Choose one or moretarget databases and click **Confirm**. ### 3. Add Changes Add SQL statements (multiple allowed by clicking **+ Add Change**) and click **Create**. The plan will be in `Draft` status with automatic SQL review. bb-plan-warning-detail ### 4. Submit for Review (Enterprise) Fix the warning, and click **Ready for Review**. The plan will be converted into an issue and await approval. Approval follows [configured rules](/change-database/approval). The plan detail page shows a **Review** section where you can follow the approval flow and the activity timeline while the change awaits approval. Review section on the plan detail page showing the pending approval flow and activity timeline ### 5. Deploy After approval, go to the **Rollout** section to deploy changes. You may configure the rollout policy to `Automatic` in the [environment](/change-database/environment-policy/rollout-policy). bb-issue-2-targets-to-rollout Deploy to `Test` first, then `Prod` after success. # Review Source: https://docs.bytebase.com/change-database/review SQL Review automatically validates your database changes against configured policies before they're executed. This ensures consistency and catches potential issues early in the change process. ## How It Works When you create a Plan, SQL Review automatically validates the statements and flags any violations: column-required ## Review Results SQL Review results appear directly in the Plan interface, showing: * **Errors**: Must be fixed before the change can proceed * **Warnings**: Should be reviewed but won't block execution * **Passed checks**: Confirmations that rules are satisfied ## Configuring Review Policies Review policies determine which rules are enforced for your database changes. You can configure these policies at the environment or project level. Learn how to set up and customize SQL Review policies for your environments and projects ## Review Rules Reference Bytebase provides comprehensive review rules for each database engine, covering: * Schema design best practices * Performance optimization * Security requirements * Naming conventions Browse the complete list of available review rules for each database engine ## Handling Review Feedback When SQL Review identifies issues: 1. **Review the findings**: Click on each issue to understand what needs to be fixed 2. **Update your SQL**: Modify the statements to comply with the rules 3. **Re-run the review**: The review automatically runs again when you save changes 4. **Proceed when cleared**: Once all errors are resolved, continue with the approval process ## Integration with Approval Flow SQL Review acts as the first gate in your change management process: * Review must pass before human approvers are notified * Failed reviews can be configured to block automatic rollout * Review results are included in approval notifications This automated validation reduces the burden on reviewers and ensures consistent quality standards across all database changes. # Data Rollback Source: https://docs.bytebase.com/change-database/rollback-data-changes Bytebase provides **Prior Backup** functionality that captures a snapshot of data before changes are applied. This safety mechanism stores the original state of affected rows, enabling you to revert data changes with **1-Click** when necessary. You can perform **multi-task rollbacks** to efficiently revert multiple related changes across databases in one operation. Backup data is stored within your own database instance in a dedicated `bbdataarchive` location. Bytebase does not automatically purge this backup data, so you can manually clean up the `bbdataarchive` database or schema as needed. ## Supported Databases | Database | Prerequisites | Prior Backup | 1-click Rollback | | ---------- | ------------------------------------------------------------------------------------- | ------------ | ---------------- | | MySQL | Create a `bbdataarchive` **database** on the instance where your database is located. | ✅ | ✅ | | MariaDB | Create a `bbdataarchive` **database** on the instance where your database is located. | ✅ | ✅ | | TiDB | Create a `bbdataarchive` **database** on the instance where your database is located. | ✅ | ✅ | | PostgreSQL | Create a `bbdataarchive` **schema** on the database. | ✅ | ✅ | | Oracle | Create a `bbdataarchive` **schema** on the database. | ✅ | ✅ | | SQL Server | Create a `bbdataarchive` **database** on the instance where your database is located. | ✅ | ✅ | ## Supported Operations Prior backup is feasible when meeting **all** of the following conditions: * `UPDATE` or `DELETE` * No mixed `UPDATE`/`DELETE` on the same table. * The SQL statement size is less than 2M. * No mixed DDL/DML. 1-click rollback is feasible when meeting **all** of the following conditions: * For `UPDATE`, the changed table has primary key and primary key columns are not updated. ## Enable Prior Backup You can enable the **Prior Backup** option either before or after creating a database change issue. bb-issue-backup-on If the Prior Backup switch appears disabled, navigate to the database page and click **Sync instance** to refresh the database metadata. ## Perform 1-Click and Multi-Task Rollback 1. After your database change has been successfully deployed, click the **Rollback available** link to initiate the rollback process. bb-issue-done-1 2. You may see multiple rollbackable changes available, depending on your deployment scenario. This could include a single change applied to multiple databases, multiple changes to one database, or a combination of both scenarios. bb-issue-done-2 3. Select which changes you want to rollback - you can choose individual changes or rollback all of them at once. This **multi-task rollback** capability allows you to efficiently revert multiple related changes in a single operation. bb-rollback-select 4. Bytebase automatically generates the appropriate rollback SQL statements based on the backup data. bb-rollback-issue-1 bb-rollback-issue-2 ## Inspect Backup Data You can examine the stored backup data using Bytebase's SQL Editor to verify what information has been preserved. bb-sql-editor-test bb-sql-editor-prod ## Appendix: Setting Up `bbdataarchive` Before using Prior Backup functionality, you must create the `bbdataarchive` storage location in your database instance. Follow the instructions for your specific database engine: ### MySQL, MariaDB, and TiDB These MySQL-compatible engines share the same setup. 1. **Create the backup database:** ```sql theme={null} CREATE DATABASE bbdataarchive; ``` 2. **Grant necessary privileges:** Replace `your_user` with the actual username that Bytebase uses to connect to your database. ```sql theme={null} GRANT ALL PRIVILEGES ON bbdataarchive.* TO 'your_user'@'%'; FLUSH PRIVILEGES; ``` ### PostgreSQL 1. **Create the backup schema:** ```sql theme={null} CREATE SCHEMA bbdataarchive; ``` 2. **Grant necessary privileges:** Replace `your_user` with the actual username that Bytebase uses to connect to your database. ```sql theme={null} GRANT ALL PRIVILEGES ON SCHEMA bbdataarchive TO your_user; GRANT ALL PRIVILEGES ON ALL TABLES IN SCHEMA bbdataarchive TO your_user; ``` 3. **If using `SET ROLE`, grant permissions to that role:** Replace `your_role` with the role used in `SET ROLE`. ```sql theme={null} GRANT USAGE, CREATE ON SCHEMA bbdataarchive TO your_role; ``` ### Oracle 1. **Create the backup user:** ```sql theme={null} CREATE USER bbdataarchive IDENTIFIED BY password; ``` 2. **Grant connection privileges:** ```sql theme={null} GRANT CREATE SESSION TO bbdataarchive; ``` 3. **Grant unlimited space quota:** Replace `tablespace_name` with your actual tablespace name. ```sql theme={null} GRANT QUOTA UNLIMITED ON tablespace_name TO bbdataarchive; ``` 4. **Grant privileges to the instance administrator:** Replace `admin_user` with the actual username that Bytebase uses as the instance administrator. ```sql theme={null} GRANT CREATE ANY TABLE TO admin_user; GRANT SELECT ANY TABLE TO admin_user; ``` ### SQL Server 1. **Create the backup database:** ```sql theme={null} CREATE DATABASE bbdataarchive; ``` 2. **Grant necessary privileges:** Replace `your_user` with the actual username that Bytebase uses to connect to your database. ```sql theme={null} USE bbdataarchive; GRANT CONTROL ON DATABASE::bbdataarchive TO your_user; ``` # Schema Rollback Source: https://docs.bytebase.com/change-database/rollback-schema-changes Bytebase records the history for each schema change. To roll back a schema change: * You can use [Schema Synchronization](/change-database/synchronize-schema) to synchronize the database to a history schema version. * You can open the database [changelog](/change-database/change-history/), select an entry that changed the schema, and roll back. Rollback is only offered on entries with a schema difference, so it does not apply to pure data changes. # Scheduled Rollout Source: https://docs.bytebase.com/change-database/scheduled-rollout When rolling out a change in Bytebase, you can choose to run it immediately or schedule it to a later time. ## Best Practices ### Role Responsibilities **Developers/Issue Creators:** * Create issues with migration scripts but do not handle rollouts directly * If timing requirements exist, specify them in one or more of these locations: * Issue title (e.g., "DB Migration - Deploy during maintenance window") * Issue description with detailed timing constraints * Labels (e.g., `rollout:night`, `rollout:maintenance-window`) * Comments explaining why specific timing is needed **Approvers/Releasers:** * Review rollout timing requirements provided by issue creators * Determine the actual rollout schedule based on: * Business impact assessment * System maintenance windows * Traffic patterns and downtime considerations * Dependencies on other deployments * Execute the scheduled rollout at the determined time This separation of concerns ensures that developers focus on creating quality migration scripts while experienced operators handle the critical timing decisions for production rollouts. ## Scheduling a Rollout When you roll out a task, choose **Schedule for later** and pick a date and time — the task starts automatically at that time. Choose **Run immediately** to start it right away. The time you pick is interpreted in your browser's local time zone. There is no time zone selector. After you schedule it, the pending task states the resolved time and its zone, so you can confirm which one applied — for example, *Waiting to execute after Aug 4, 2026, 2:22:15 PM GMT+2.* rollout-schedule Whether a rollout can proceed when plan checks fail is governed by the [rollout policy](/change-database/environment-policy/rollout-policy), not by scheduling. After you schedule the rollout, the task enters the **pending** state and waits for the scheduled time — you can't run it earlier. To stop a scheduled rollout, click **Cancel**. After clicking **Cancel**, you can **Restart** the rollout and run it immediately or reschedule it for a later time. The SQL statement is fixed once a rollout is created. Cancelling only stops the scheduled run — it doesn't unlock the statement for editing. To change the SQL, create a new change. ## Workflow Summary 1. **Issue Creation**: * Developer creates issue with complete migration script * Optionally specifies timing requirements in title, description, or labels * If timing is critical, provides justification for specific timing needs (e.g., maintenance window, low traffic period) 2. **Review & Approval**: * Approver reviews the migration script for correctness and safety * Validates timing constraints and business impact * Approves the issue once technical and timing requirements are satisfied 3. **Task Execution**: * By default, executes task immediately upon action * Releaser can choose to schedule the task for a specific time based on: * Issue creator's timing requirements * Operational constraints (maintenance windows, traffic patterns) * Business impact considerations This workflow ensures proper separation of responsibilities: developers focus on creating quality migration scripts, while approvers and releasers handle the rollout timing and execution decisions for production deployments. # Schema Editor Source: https://docs.bytebase.com/change-database/schema-editor Schema Editor provides a visual interface for modifying database structures without writing SQL. The Schema Editor is only available for **MySQL**, **TiDB**, and **PostgreSQL**. For other [supported databases](/introduction/supported-databases#schema-management), the **Schema Editor** button is not shown and schema changes are made with raw SQL. ## Getting Started The Schema Editor lives inside the change-plan editor. When you create or edit a schema change, the plan's SQL statement section shows a **Schema editor** button — click it to edit the schema visually instead of writing SQL. The editor opens as a side panel: an object tree on the left — with a search box, a **+** button, and right-click menus — and the editing pane on the right. As you make changes, the equivalent SQL is generated and shown live. Schema Editor panel showing the object tree, a table's column grid, and the live SQL preview ## Basic Operations ### Tables * **Create**: click **New table**, or right-click a schema's **Tables** node (or use the **+** button). * **Drop**: click the trash icon on a table row, or right-click the table > **Drop table**. Use **Restore** to undo before applying. * **Rename**: right-click a table that has pending changes > **Rename table**. ### Columns * **Add**: select a table and click **Add column**. * **Edit**: change column properties inline in the grid — name, type, default value, comment, **Not Null**, and **Primary** key. Primary keys can only be changed on newly created tables. ### Indexes and partitions For **MySQL** and **TiDB**, open a table and switch to the **Indexes** or **Partitions** tab, then click **Add index** or **Add partition**. ### Schemas For **PostgreSQL**, right-click the database node > **Create schema**, or a schema node > **Drop schema** / **Restore**. ## Apply Changes Click **Insert SQL**. The Schema Editor generates the DDL for your changes and inserts it into the plan's SQL statement. From there, review and run the plan as usual to roll out the change. # Settings Source: https://docs.bytebase.com/change-database/settings Customize how database changes work in your projects. Navigate to **Project** → **Settings** to access these options. ## Postgres Tenant Mode Perfect for multi-tenant PostgreSQL setups where each database has its own owner. When enabled, Bytebase automatically switches to the database's OWNER role before executing changes, ensuring proper permissions. **Location:** Project → Settings → Issue Related ## Self Approval By default, issue creators can't approve their own changes. To allow self-approval, enable **Allow self approval** in your project settings. **Location:** Project → Settings → Issue Related # Schema Synchronization Source: https://docs.bytebase.com/change-database/synchronize-schema Bytebase supports synchronizing a specified schema version of one source database to one or multiple target databases. You can use Schema Synchronization to: * Propagate the schema changes from the one database to others. * Revert the database schema to an old version. The overall workflow: workflow 1. Compare a source database schema with one or multiple target databases' schema. 2. Generate the suggested DDL statements (e.g. `ALTER TABLE`) to transit the source database schema state to each of the target database schema. 3. Create an issue. 4. Deploy the DDL statements to the targeted databases. ## Supported Objects | Object | MySQL | PostgreSQL | TiDB | Oracle | SQL Server | | ----------------------------- | ------------------------------------ | -------------------------------- | ---------------------------- | ----------------------------------------- | -------------------------- | | **Schema** | — | ✅ | — | ✅ | ✅ | | **Table** | ✅ | ✅ | ✅ | ✅ | ✅ | | **Column** | ✅ | ✅ | ✅ | ✅ | ✅ | | **Primary Key** | ✅ | ✅ | ✅ | ✅ | ✅ | | **Foreign Key** | ✅ | ✅ | ✅ | ✅ | ✅ | | **Index** | ✅ Regular, Unique, Fulltext, Spatial | ✅ Including Expression & Partial | ✅ | ✅ Regular, Unique, Bitmap, Function-based | ✅ Clustered, Non-clustered | | **Check Constraint** | ✅ (8.0+) | ✅ | ❌ (metadata sync limitation) | ✅ | ✅ | | **View** | ✅ | ✅ | ✅ | ✅ | ✅ | | **Materialized View** | — | ✅ | — | ✅ | ❌ | | **Generated/Computed Column** | ✅ STORED/VIRTUAL | ✅ STORED | ✅ | ✅ Virtual | ✅ Computed | | **Identity/Auto Column** | ✅ AUTO\_INCREMENT | ✅ SERIAL/IDENTITY | ✅ AUTO\_RANDOM | ✅ IDENTITY (12c+) | ✅ IDENTITY | | **Partitioning** | ✅ KEY, HASH, RANGE, LIST | ✅ | ✅ | ✅ RANGE, LIST, HASH, COMPOSITE | ❌ | | **Sequence** | — | ✅ | — | ✅ | ✅ | | **Trigger** | ✅ | ✅ | — | ✅ | ✅ | | **Function** | ✅ | ✅ | — | ✅ | ✅ | | **Procedure** | ✅ | ✅ | — | ✅ | ✅ | | **Package** | — | — | — | ✅ | — | | **Custom Type** | — | ✅ Enum only | — | ✅ | ✅ | | **Extension** | — | ✅ | — | — | — | | **Comments** | ✅ | ✅ | ✅ | ✅ | ✅ | ## How to use ### Step 1 - Go to the **Sync Schema** page Click the **Database > Sync Schema** button in the left navigation bar. ### Step 2 - Select source schema You need to select the project and source database schema version that you want to synchronize. select-source-schema ### Step 3 - Select target databases Select the target databases that will be synchronized with the chosen schema version in the previous step. select-target-databases After you have selected a target database, Bytebase will show a schema diff between the two databases and generate the DDL statements. And you can further edit the generated content. diff-preview ### Step 4 - Preview issue With the generated DDL statements, you can preview and create an **Edit Schema** issue. This kicks off the [change workflow](/change-database/change-workflow) for the applied database. # Transaction Mode Source: https://docs.bytebase.com/change-database/transaction-mode Supported databases: MySQL, PostgreSQL, Oracle, SQL Server, TiDB, Redshift Bytebase provides transaction management to ensure safe database changes. By default, all SQL statements are wrapped in a transaction, allowing automatic rollback if errors occur during execution. ## Enable/Disable Transactions When creating a plan, you can control whether SQL statements are executed within a transaction: * **Enabled (default)**: All statements run within a transaction boundary for safety * **Disabled**: Statements execute directly without transaction wrapping Transaction mode toggle ## Statements that cannot run in a transaction block Some PostgreSQL statements cannot run inside an **explicit transaction block**. When such a statement runs after `BEGIN` — or when a migration tool wraps it in a transaction — PostgreSQL rejects it with an error like: ``` ERROR: CREATE INDEX CONCURRENTLY cannot run inside a transaction block ``` For example, this fails because the statement is wrapped in an explicit transaction block: ```sql theme={null} BEGIN; CREATE INDEX CONCURRENTLY idx_users_email ON users(email); COMMIT; ``` While the same statement runs fine on its own under autocommit (no `BEGIN`): ```sql theme={null} CREATE INDEX CONCURRENTLY idx_users_email ON users(email); ``` You do not need to run these statements manually outside Bytebase. Bytebase automatically detects the most common ones and runs them outside the transaction — even in the default, transaction-enabled mode: * `CREATE INDEX CONCURRENTLY` * `DROP INDEX CONCURRENTLY` * `VACUUM` * `DROP DATABASE` Include them in your change like any other statement. Bytebase executes them after the surrounding transaction commits, so they never hit the error above — the plan UI notes this as "Non-transactional statements will be executed last." For other statements that cannot run in a transaction block but are **not** auto-detected — such as `REINDEX ... CONCURRENTLY`, `CREATE DATABASE`, `CREATE TABLESPACE`, `ALTER SYSTEM`, and some `ALTER TYPE ... ADD VALUE` cases — disable transaction mode so Bytebase runs the statements directly: ```sql theme={null} -- txn-mode = off REINDEX INDEX CONCURRENTLY idx_users_email; ``` `-- txn-mode = off` disables transaction wrapping for the **entire** script — if a later statement fails, earlier ones stay committed. Put a non-transactional statement in its own migration so the rest of your changes keep transactional, all-or-nothing safety. ## MySQL-Specific Settings For MySQL databases, Bytebase offers additional control over transaction isolation levels to manage concurrent access and data consistency: MySQL transaction isolation levels ### Available Isolation Levels * **READ UNCOMMITTED**: Lowest isolation, allows dirty reads * **READ COMMITTED**: Prevents dirty reads, allows non-repeatable reads * **REPEATABLE READ**: Default MySQL isolation, prevents dirty and non-repeatable reads * **SERIALIZABLE**: Highest isolation, prevents all phenomena but may impact performance ## GitOps In the GitOps workflow there is no UI toggle, so transaction mode and isolation level are controlled via comment directives at the top of the migration file: ```sql theme={null} -- txn-mode = off -- txn-isolation = READ COMMITTED ALTER TABLE users ADD COLUMN email VARCHAR(255); ``` Supported directives: * `-- txn-mode = on|off` — wrap the script in a transaction, or run statements directly without transaction wrapping * `-- txn-isolation = READ UNCOMMITTED | READ COMMITTED | REPEATABLE READ | SERIALIZABLE` — MySQL only Rules: * Directives must appear at the top of the file, before any non-comment SQL * Empty lines between directives are allowed; scanning stops at the first non-comment line * The order of the two directives does not matter * Whitespace around `=` is optional — `-- txn-mode = off` and `-- txn-mode=off` are both accepted; the spaced form shown above is the canonical style ## Best Practices * Keep transactions enabled for DDL and DML operations that modify data * Consider disabling transactions only for: * Large batch operations that manage their own transaction boundaries * Statements that cannot run within a transaction block — see [Statements that cannot run in a transaction block](#statements-that-cannot-run-in-a-transaction-block) * Choose appropriate isolation levels based on your concurrency requirements and performance needs # Webhook Source: https://docs.bytebase.com/change-database/webhook User can configure project-level webhooks to let Bytebase post messages to the configured webhook endpoint upon various events. Configure them under **Manage > Webhooks** in a project. Project webhooks listed under Manage > Webhooks ## Supported events * `Issue creation` - Post message when a new issue has been created. * `Issue approval needed` - Post message when an issue is pending approval. * `Issue sent back` - Post message when an issue has been sent back by an approver. * `Issue approved` - Post message when all approval steps are completed. * `Pipeline failed` - Post message when a pipeline has failed. * `Pipeline completed` - Post message when a pipeline has completed successfully. ### Direct message By default, a webhook posts to the channel at its configured URL. When **Enable direct messages** is turned on, Bytebase instead delivers the notification to the matched users through the workspace `IM` integration, and falls back to the channel URL if no user matches or delivery fails. The following events support sending direct messages/notifications to related users, make sure you have: 1. turned on the **Enable direct messages** option on the webhook. 2. configured the workspace-level `IM` integration under **Integration > IM** — set up once and shared by every project — so Bytebase can deliver the messages. * `Issue approval needed` - Notifies the pending approvers. * `Issue sent back` - Notifies the issue creator. * `Issue approved` - Notifies the issue creator. ## Supported webhook endpoints ### Slack [Official guide](https://api.slack.com/messaging/webhooks) #### Configure sending direct messages to related users 1. Go to [https://api.slack.com/apps](https://api.slack.com/apps). 2. Click **Create New App**. 3. Choose **From an app manifest**. 4. Pick your workspace to develop the app and click **Next**. 5. Replace the existing JSON with this manifest content and click **Next**. ```JSON theme={null} { "display_information": { "name": "Bytebase Bot" }, "features": { "bot_user": { "display_name": "Bytebase Bot", "always_online": false } }, "oauth_config": { "scopes": { "bot": [ "users:read", "users:read.email", "channels:manage", "groups:write", "im:write", "chat:write", "mpim:write" ] } }, "settings": { "org_deploy_enabled": false, "socket_mode_enabled": false, "token_rotation_enabled": false } } ``` 6. Click **Create**. 7. Click **Install to Workspace** and click **Allow**. 8. Go to **Features > OAuth & Permissions** and copy the **Bot User OAuth Token**. 9. Go back to Bytebase and paste the **Bot User OAuth Token** to the **Token** field under **Integration > IM**. 10. Go to **Manage > Webhooks** in a project, add a webhook, check all the events you want to send direct messages, and click **Create**. ### Discord [Official guide](https://support.discord.com/hc/en-us/articles/228383668-Intro-to-Webhooks) ### Google Chat [Official guide](https://developers.google.com/workspace/chat/quickstart/webhooks) ### Microsoft Teams [Official guide](https://support.microsoft.com/en-us/office/create-incoming-webhooks-with-workflows-for-microsoft-teams-8ae491c7-0394-4861-ba59-055e33f75498) Microsoft has [retired Office 365 Connectors](https://devblogs.microsoft.com/microsoft365dev/retirement-of-office-365-connectors-within-microsoft-teams/) in Teams. Use **Power Automate Workflows** to create incoming webhooks instead. 1. In Microsoft Teams, select **More options (⋯)** next to the channel you want to post to, then select **Workflows**. 2. Choose **Post to a channel when a webhook request is received**. 3. Authenticate with your account and select the Team and Channel, then click **Add workflow**. 4. Copy the generated webhook URL (it looks like `https://xxx.environment.api.powerplatform.com:443/powerautomate/automations/direct/workflows/...`). 5. Go to Bytebase **Manage > Webhooks** in a project, paste the URL and create the webhook. #### Configure sending direct messages to related users ##### Option 1: Chat Webhook via Power Automate You can use the **Post to a chat when a webhook request is received** workflow template to send alerts to a chat via the Flow bot. This template may not be available depending on your organization type, tenant configuration, or admin policies. Contact your IT admin if you don't see it. For a universally available approach, use Option 2. 1. In Microsoft Teams, select **More options (⋯)** next to a chat, then select **Workflows**. 2. Choose **Post to a chat when a webhook request is received**. If the template is not listed, you can build the flow manually in [Power Automate](https://make.powerautomate.com/) using the **When a Teams webhook request is received** trigger and the **Post card in a chat or channel** action. 3. Authenticate with your account and select the chat, then click **Add workflow**. 4. Copy the generated webhook URL. 5. Go to Bytebase **Manage > Webhooks** in a project, paste the URL and create the webhook. ##### Option 2: Azure AD Bot Framework (recommended) Register an Azure AD application with Bot Framework capabilities. This allows Bytebase to send 1:1 direct messages to any user matched by email without per-user workflow setup. 1. Go to [Azure Portal](https://portal.azure.com/) and navigate to **Microsoft Entra ID > App registrations**. 2. Click **New registration**, give it a name (e.g., "Bytebase Bot"), and register. 3. Note the **Application (client) ID** and **Directory (tenant) ID**. 4. Go to **Certificates & secrets**, click **New client secret**, and copy the **Secret value**. 5. Go to [Bot Framework Registration](https://dev.botframework.com/bots/new) or use the **Azure Bot** resource in Azure Portal to register a bot using the same Application ID. 6. In the bot's **Channels** settings, enable the **Microsoft Teams** channel. 7. Grant the following **Microsoft Graph API application permissions** to the app: * `User.Read.All` — to look up users by email * `TeamsAppInstallation.ReadWriteForUser.All` — to install the app for users * `Chat.Create` and `Chat.ReadWrite.All` — to get chat IDs for direct messaging 8. Click **Grant admin consent** for the permissions. 9. Go back to Bytebase and fill **Tenant ID**, **Client ID**, and **Client Secret** fields under **Integration > IM**. 10. Go to **Manage > Webhooks** in a project, add a webhook, check the events you want to send direct messages for, turn on **Enable direct messages**, and click **Create**. ### DingTalk [Official guide](https://developers.dingtalk.com/document/robots/custom-robot-access) #### Configure sending direct messages to related users 1. Go to [https://open-dev.dingtalk.com/](https://open-dev.dingtalk.com/). 2. Select **Application Development**, click **Create Application**. 3. Select **Credentials and Basic Information** on the left sidebar. Obtain the **Client ID** and **Client Secret**. 4. Select **Permission Management**, grant the following permissions: * `qyapi_get_member_by_mobile` * `qyapi_robot_sendmsg` 5. Select **Add Application Capability**, add the **Robot** capability. Configure the robot, and for **Message receiving mode** select `Stream mode`. In "Robot Configuration", click **Copy RobotCode** to obtain the **Robot Code**. Publish the robot. 6. Select **Version Management and Release**. Click **Create New Version**. Fill in the relevant information, then **Save** and **Directly Publish**. 7. Go back to Bytebase and fill **Client ID**, **Client Secret** and **Robot Code** fields under **Integration > IM**. 8. Go to **Manage > Webhooks** in a project, add a webhook, check all the events you want to send direct messages, and click **Create**. DingTalk allows to specify a list of keywords in the [security setting](https://developers.dingtalk.com/document/robots/customize-robot-security-settings) to protect webhook endpoint. You can add "Bytebase" to that keyword list. ### Feishu (Lark) [Official guide](https://www.feishu.cn/hc/zh-CN/articles/360024984973) Feishu (Lark) allows to specify a list of keywords in the [security setting](https://www.feishu.cn/hc/zh-CN/articles/360024984973#lineguid-RahdJr) to protect webhook endpoint. You can add "Bytebase" to that keyword list. #### Configure sending direct notifications to related users 1. Go to [https://open.feishu.cn/app](https://open.feishu.cn/app). 2. Click **Create Custom App**, fill the form and click **Create**. 3. Click **Add Features** on the left sidebar, add the **Bot** feature. 4. Click **Permissions & Scopes** on the left sidebar, find and add the following permissions: * `contact:user.id:readonly` * `im:message:send_as_bot` 5. Click **Create Version**. 6. Configure [availability](https://open.feishu.cn/document/home/introduction-to-scope-and-authorization/availability). 7. Go back to Bytebase and fill **App Id** and **App Secret** fields under **Integration > IM**. 8. Go to **Manage > Webhooks** in a project, add a webhook, check all the events you want to send direct messages, and click **Create**. ### WeCom WeCom does not provide its own official guide. Please follow this similar [setup](https://intl.cloud.tencent.com/zh/document/product/614/39581) from Tencent Cloud instead. #### Configure sending direct messages to related users 1. Go to [https://work.weixin.qq.com/wework\_admin/frame#apps](https://work.weixin.qq.com/wework_admin/frame#apps). 2. Click the tab **My Company**, and then you can find **Company ID** in the **Company Information**. 3. Click the tab **App Management**, and choose **Create an app** under **Self-built**. 4. Open the app, 1. Find **AgentId** and **Secret**. 2. Configure **Allowed users**. 3. Configure **Company's Trusted IP** to your Bytebase workspace IP. 5. Make sure the user's email in Bytebase is the same as the user's email (not External account) in WeCom. 6. Go back to Bytebase and fill **Corp Id**, **Agent Id** and **Secret** fields under **Integration > IM**. 7. Go to **Manage > Webhooks** in a project, add a webhook, check all the events you want to send direct messages, and click **Create**. # Bytebase 3.0.0 - Oct 24, 2024 Source: https://docs.bytebase.com/changelog/bytebase-3-0-0 AI Assistant for SQL Editor Check out our official 3.0 announcement - [Bytebase 3.0 - DevSecOps for Database](https://www.bytebase.com/blog/bytebase-3-0/) ## 🚀 New Features * [AI Assistant for SQL Editor](/sql-editor/ai-assistant/): Convert natural language into SQL, explain SQL queries and identifies potential problems. * Support 1-click rollback for SQL Server DML change. * Support gh-ost online schema change for MariaDB. * Add new SQL Review Rules: * Require `MAX_EXECUTION_TIME` for MySQL. * Require `MAX_STATEMENT_TIME` for MariaDB. * Require `ALGORITHM` and `LOCK` options for online DDL in MySQL/MariaDB. * Table comment checks for PostgreSQL/Oracle. * Column comment checks for PostgreSQL/Oracle. * Separate SQL review rules to disallow mixing statements in DDL or DML tasks. ## 🔔 Notable Changes ### Behavior We no longer allow the users to approve their own issues. ### API * CEL parsed expressions have been updated to remove unused sourceInfo, impacting the following APIs: `/v1/cel/batchDeparse`, `/v1/cel/batchParse`, `/v1/workspaces/*:getIamPolicy`, `/v1/workspaces/*:setIamPolicy`, `/v1/projects/{project}:getIamPolicy`, `/v1/projects/{project}:setIamPolicy`. * Force to set `group_email` while calling `POST /v1/groups` API to create a new group. For example, `POST /v1/groups?group_email=dba@bytebase.com` ## 🎄 Enhancements * Support table-level risk rule for DML in Risk center. * SQL Editor displays function signatures with parameters instead of just function names. * SELECT prompt prioritizes columns over tables in SQL Editor. * Ignore `bbdataarchive` schema for PostgreSQL during schema synchronization. * Add `allow_missing` parameter in APIs `PATCH /v1/groups/{group}` and `PATCH /v1/roles/{role}`. For example, you can use `PATCH /v1/groups/not_exist_group?allow_missing=true` to create a new group if the resource not found. * Use user-specified PostgreSQL role in SQL review. ## 🎠 Community * Thanks to [@blight19](https://github.com/blight19) for the PR: [#14114](https://github.com/bytebase/bytebase/pull/14114) fix: redis get database with proxy. ## ⚙️ Install and Upgrade * [Fresh install](/get-started/self-host/deploy-with-docker) * [Upgrade](/get-started/self-host/upgrade) *Before upgrading: 1) Back up the [metadata](/get-started/self-host/upgrade#back-up-and-restore) — in-place downgrade is not supported. 2) Do not run multiple containers on the same data directory. 3) Terraform users: upgrade Bytebase server first, then apply the new Terraform config.* # Bytebase 3.0.1 - Nov 07, 2024 Source: https://docs.bytebase.com/changelog/bytebase-3-0-1 Configurable DDL/DML Execution in SQL Editor ## 🚀 New Features * Add an environment policy to allow executing DDL/DML statements directly in SQL Editor. * Support data masking for BigQuery. * Support [rollback to a specific version in the change history](/change-database/change-history/) on the database page. * Add data access control and masking page under each project. ## 🔔 Notable Changes * Disallow issue creators from approving their own issues. * SQL review configuration API supports upsert via `PATCH`. For example, `PATCH /v1/reviewConfigs/not_exist_review_config?allow_missing=true` ## 🎄 Enhancements * Remove the export limit for audit logs. * Optimize CPU usage during synchronization on MySQL instances. ## 💰 Pricing Adjustments * The database group/batch change feature is now extended to the **Pro Plan** besides the **Enterprise Plan**. * The Pro plan is now billed per user instead of per database instance. [/pricing](https://www.bytebase.com/pricing) ## ⚙️ Install and Upgrade * [Fresh install](/get-started/self-host/deploy-with-docker) * [Upgrade](/get-started/self-host/upgrade) *Before upgrading: 1) Back up the [metadata](/get-started/self-host/upgrade#back-up-and-restore) — in-place downgrade is not supported. 2) Do not run multiple containers on the same data directory. 3) Terraform users: upgrade Bytebase server first, then apply the new Terraform config.* # Bytebase 3.1.0 - Nov 21, 2024 Source: https://docs.bytebase.com/changelog/bytebase-3-1-0 Google/GitHub SSO available in Pro Plan ## 🚀 New Features * Support PostgreSQL role selection in DDL/DML issues. * Add PostgreSQL database tenant mode option in project setting. * Display package metadata for ORACLE on database page and SQL Editor. * Support setting colors for environments for better visual distinction. * Allow workspace admin to disable data export functionality. * Add new SQL Review Rules for MySQL: * Prohibit DDL on specific tables for MySQL. * Prohibit DML on specific tables for MySQL. ## 🔔 Notable Changes * Redefine SQL Editor related permissions: * Permissions * `bb.databases.query` -> `bb.sql.select` * `bb.databases.queryDML` -> `bb.sql.dml` * `bb.databases.queryDDL` -> `bb.sql.ddl` * `bb.databases.queryExplain` -> `bb.sql.explain` * `bb.databases.queryInfo` -> `bb.sql.info` * `bb.databases.export` -> `bb.sql.export` * `bb.instances.adminExecute` -> `bb.sql.admin` * Roles * `roles/projectQuerier` -> `roles/sqlEditorUser` * API change: * Remove the `database` field of `sheet` in `SheetService`. * Disallow adding service account to a group via API (previously restricted in the UI) . * Deprecate `SQLService.Execute` method. * Default PostgreSQL role to run a DDL/DML issue: * Instance connection user (previously database owner) * If PostgreSQL database tenant mode is on for a project, the role remains database owner. * For existing projects: PostgreSQL tenant mode will be automatically enabled if multitenancy is enabled for any database group. ## 🎄 Enhancements * Expand audit log coverage to include more event types. * Upgrade 1-click rollback user experience for MySQL. * Upgrade parser to be compatible with PostgreSQL 15 syntax. * Add compatibility with Elasticsearch 7.x. * Support `DECLARE` and `SET` for SQL Server in SQL Editor. ## 💰 Pricing Adjustments * Move SSO OAuth2 Google and GitHub from **Enterprise** to **Pro Plan**. * Move self-service signup restriction from **Enterprise** to **Pro Plan**. ## ⚙️ Install and Upgrade * [Fresh install](/get-started/self-host/deploy-with-docker) * [Upgrade](/get-started/self-host/upgrade) *Before upgrading: 1) Back up the [metadata](/get-started/self-host/upgrade#back-up-and-restore) — in-place downgrade is not supported. 2) Do not run multiple containers on the same data directory. 3) Terraform users: upgrade Bytebase server first, then apply the new Terraform config.* # Bytebase 3.1.1 - Dec 05, 2024 Source: https://docs.bytebase.com/changelog/bytebase-3-1-1 Customizable homepage with shortcut list ## 🚀 New Features * Customizable homepage with shortcut list. * Query for all nodes in Redis Cluster. * Support for second-precision expiration time when granting project roles. ## 🔔 Notable Changes * Remove instance roles from Database messages. Call `GetInstance` or `ListInstanceRoles` to get instance role information. ## 🎄 Enhancements * Well-formatted JSON in SQL Editor. * Display PostgreSQL sequences in SQL Editor. * Display MySQL triggers in SQL Editor. * MySQL DUMP SCHEMA is 80% faster on large databases. * Support PostgreSQL roles during backup and restore. ## ⚙️ Install and Upgrade * [Fresh install](/get-started/self-host/deploy-with-docker) * [Upgrade](/get-started/self-host/upgrade) *Before upgrading: 1) Back up the [metadata](/get-started/self-host/upgrade#back-up-and-restore) — in-place downgrade is not supported. 2) Do not run multiple containers on the same data directory. 3) Terraform users: upgrade Bytebase server first, then apply the new Terraform config.* # Bytebase 3.1.2 - Dec 19, 2024 Source: https://docs.bytebase.com/changelog/bytebase-3-1-2 Tooltips for tables, columns, and PostgreSQL view comments in SQL Editor ## 🚀 New Features * Add tooltips for tables, columns, and PostgreSQL view comments in SQL Editor. * Support IM and webhook integration for Lark. * Display table and view definition for Redshift. ## 🔔 Notable Changes * Remove the masking policy API `v1/{instance}/{database}/policies/masking`. Use `v1/{instance}/{database}/metadata` instead and set the `columnConfigs` parameter. ## 🎄 Enhancements * Move the masking column page to the database detail page. * Support popup login modal on token expiration. ## 🐞 Bug fix * Fix the backup table name conflict in multi-tenant database group within the same instance. * Fix the cross-origin issue in SSO authentication (OIDC). * Fix bug causing QuotaExceededError in SQL Editor. ## ⚙️ Install and Upgrade * [Fresh install](/get-started/self-host/deploy-with-docker) * [Upgrade](/get-started/self-host/upgrade) *Before upgrading: 1) Back up the [metadata](/get-started/self-host/upgrade#back-up-and-restore) — in-place downgrade is not supported. 2) Do not run multiple containers on the same data directory. 3) Terraform users: upgrade Bytebase server first, then apply the new Terraform config.* # Bytebase 3.10.0 - Sep 11, 2025 Source: https://docs.bytebase.com/changelog/bytebase-3-10-0 Cassandra data masking ## 🚀 New Features * **Cassandra** - support data masking. * Add TLS certificate verification toggle while adding instance. ## 🔔 Notable Changes * Service accounts no longer consume user seats. * **API** – Deprecate `SQL_RESULT_SIZE_LIMIT` setting and `DATA_EXPORT` policy. Consolidated `SQLQueryRestrictionSetting` and `ExportDataPolicy` into the `DATA_QUERY` policy. * **Terraform** – Deprecate `sql_query_restriction` setting. Use the `query_data_policy` block instead, which supports `disable_export`, `maximum_result_rows`, `maximum_result_size`, and `timeout_in_seconds`. ## 🎄 Enhancements * **Bytebase Cloud** - Gemini AI is enabled automatically in the SaaS mode - Totally FREE! * **Bytebase GitHub Actions** `bytebase-action rollout` command now outputs a job summary on GitHub. [Example](https://github.com/bytebase/example-gitops-github-flow/actions/runs/17601287317) * Improve database creation page for clearer information. ## 🐞 Bug Fix * **SQL Server** - Fix cross-database reference issues in queries. ## ⚙️ Install and Upgrade * [Fresh install](/get-started/self-host/deploy-with-docker) * [Upgrade](/get-started/self-host/upgrade) *Before upgrading: 1) Back up the [metadata](/get-started/self-host/upgrade#back-up-and-restore) — in-place downgrade is not supported. 2) Do not run multiple containers on the same data directory. 3) Terraform users: upgrade Bytebase server first, then apply the new Terraform config.* # Bytebase 3.10.1 - Sep 25, 2025 Source: https://docs.bytebase.com/changelog/bytebase-3-10-1 Support key–value project labels ### 🚀 New Features * Add project labels, up to 64 key–value pairs. ### 🔔 Notable Changes * Environment Rollout Policy: * The **Issue Creator** and **Last Issue Approver** roles are being sunset. They no longer appear in new environments, while existing environments remain unaffected. Full removal is scheduled for the *next release.* * The **force rollout** mechanism will be replaced by configurable rollout conditions in the *next release*, providing more flexibility and stronger security. * Restrict **Disallow issue creation for SQL review with errors** option to project level only (was applied to both project and workspace). * Remove settings pages in **Editor mode**. * **API** - Change database label filter syntax to `labels.region == "asia"` (was `label == "region:asia"`). * **API** - Deprecate the standalone `DisableCopyDataPolicy` and migrates its functionality to `QueryDataPolicy`. ### 🎄 Enhancements * Improve Data Export issue page: * Support updating targets before issue creation. * Display tasks in a flat list instead of grouping by stages. * Add option to switch [transaction mode](/change-database/transaction-mode) in Plan. * **MySQL** - Add isolation level selection when transaction mode is on. * **Postgres** - Add role selection for executing SQL in Plan. * Support `instance_id` in risk condition. * Optimize advanced search with cached filters. ## ⚙️ Install and Upgrade * [Fresh install](/get-started/self-host/deploy-with-docker) * [Upgrade](/get-started/self-host/upgrade) *Before upgrading: 1) Back up the [metadata](/get-started/self-host/upgrade#back-up-and-restore) — in-place downgrade is not supported. 2) Do not run multiple containers on the same data directory. 3) Terraform users: upgrade Bytebase server first, then apply the new Terraform config.* # Bytebase 3.11.0 - Oct 10, 2025 Source: https://docs.bytebase.com/changelog/bytebase-3-11-0 Environment rollout policy update ### 🔔 Notable Changes * **Environment rollout policy update** * **Issue Creators** and **Last Issue Approvers** can no longer roll out issues. Manual rollouts now require specifying workspace/project roles or users with the `bb.taskRuns.create` permission. * The force rollout mechanism has been replaced by [configurable rollout requirements](/change-database/environment-policy/rollout-policy#configurable-rollout-requirements): * **Require Issue Approval** – ensures issues must be approved before rollout can proceed (default: enabled). * **Plan Check Enforcement** – controls rollout behavior based on plan check results (default: block on errors only). * Deprecate `bb.sql.export` permission and `roles/projectExporter` role. It’s merged into `SQL Editor User` role, which now can export directly in SQL Editor; Developer can still create Export issue as before. * Deprecate `request.row_limit` in the project IAM policy. Use `maximum_result_rows` in `QueryDataPolicy` instead. * **API** * Standardize CEL attribute naming with prefixes: `resource.*`, `statement.*`, `request.*` (e.g., environment\_id → resource.environment\_id) * Unify risk levels to single `RiskLevel` enum; changed from integers (`100`, `200`, `300`) to strings (`LOW`, `MODERATE`, `HIGH`); CEL expressions migrate from `format level == 300` to `level == "HIGH"` * Simplify approval template to singleton with flat role array; replace `approval_finding_done`/`approval_finding_error` with `approval_status` enum * Consolidate task types into `DATABASE_MIGRATE` (with `migrate_type`: DDL/DML/GHOST) and `DATABASE_SDL`. Split changelog Type into `Type` and `MigrationType` enums. Merged `DatabaseSchemaUpdate` and `DatabaseDataUpdate` into `DatabaseUpdate`. * **Terraform** (provider version: 3.11.1) * Use `RiskLevel` enum instead of numeric values for level in `bytebase_risk`. Example: [risk.tf](https://github.com/bytebase/terraform-provider-bytebase/blob/main/examples/setup/risk.tf) * Remove `disable_copy_data_policy`; moved into `query_data_policy` as the `disable_copy_data` field. Example: [environment.tf](https://github.com/bytebase/terraform-provider-bytebase/blob/main/examples/setup/environment.tf#L60) * Update structure of the `approval_flow` setting. Example: [approval\_flow.tf](https://github.com/bytebase/terraform-provider-bytebase/blob/main/examples/setup/approval_flow.tf) * Remove `row_limit` from `bytebase_iam_policy` setting. * Remove `Format on Save` feature from **SQL Editor**. ### 🎄 Enhancements * Support key-value instance labels. * Optimize **Schema Editor** performance. * Add pre-flight validation and circuit breaker to **gh-ost workflow**. * **AWS Elasticsearch** – Add cross-account support. ### 🐞 Bug Fixes * Add fallback behavior for **PostgreSQL** dump topological sorting to prevent blocking dump functionality. ## ⚙️ Install and Upgrade * [Fresh install](/get-started/self-host/deploy-with-docker) * [Upgrade](/get-started/self-host/upgrade) *Before upgrading: 1) Back up the [metadata](/get-started/self-host/upgrade#back-up-and-restore) — in-place downgrade is not supported. 2) Do not run multiple containers on the same data directory. 3) Terraform users: upgrade Bytebase server first, then apply the new Terraform config.* # Bytebase 3.11.1 - Oct 23, 2025 Source: https://docs.bytebase.com/changelog/bytebase-3-11-1 Apache Doris - Support system views and functions ## 🚀 Features * Apache Doris - Add support for system views and functions in SQL Editor. * Add support for configuring inactive session timeout. ## 🔔 Notable Changes * Remove Check() RPC and SQL Review functionality in SQL Editor. * Remove --check-plan option from bytebase-action rollout command. * Enforce proper create permissions for allow\_missing updates. ## 🎄 Enhancements * Support editing migration type in draft plans. * Support exporting results from multiple queries. * Add banner notifications for approved issues ready to roll out. * Show full signatures for functions and procedures instead of just names. * Add HTTP security headers in all responses. ## 🐞 Bug Fixes * Elasticsearch - Fix query parameter encoding in URLs. * SQL Server - Fix IDENTITY column handling during backup/restore. ## ⚙️ Install and Upgrade * [Fresh install](/get-started/self-host/deploy-with-docker) * [Upgrade](/get-started/self-host/upgrade) *Before upgrading: 1) Back up the [metadata](/get-started/self-host/upgrade#back-up-and-restore) — in-place downgrade is not supported. 2) Do not run multiple containers on the same data directory. 3) Terraform users: upgrade Bytebase server first, then apply the new Terraform config.* # Bytebase 3.12.0 - Nov 6, 2025 Source: https://docs.bytebase.com/changelog/bytebase-3-12-0 PostgreSQL - State-based GitOps workflow ## 🚀 Features * Add [State-based GitOps workflow](/gitops/state-based-workflow/overview) for PostgreSQL. * Add support for [IdP-initiated SSO flow](/administration/sso/idp-initiated). ## 🔔 Notable Changes * Remove identity provider permissions from the Workspace DBA role. * Remove support for URL-based secret retrieval via `URL` template format. * Deprecate changelist feature. * Add character length limits to prevent overly long entries and ensure data consistency. Length limits applied: * 200 characters: Titles and names (groups, issues, plans, projects, instances, users, etc.) * 1,000 characters: Short descriptions (group descriptions, operation reasons for cancel/skip/restart tasks) * 10,000 characters: Long descriptions (issue details, plan details) * 65,536 characters: Comments and long-form text (issue comments) * Update `APP_IM` configuration format in API and UI. Before: ```json theme={null} { "slack": { "enabled": true, "token": "xxx" }, "lark": { "enabled": true, ... }, "feishu": { "enabled": true, ... } } ``` After: ```json theme={null} { "settings": [ { "type": "slack", "slack": {"token": "xxx"} }, { "type": "lark", "lark": {} } ] } ``` * Allow direct data export in SQL Editor for users who can run the query — no approval issue is created. The approval flow still applies to export requests submitted through Data Export Center. You can also disable data export at the workspace level, so that SQL Editor queries cannot be exported directly by default. ## 🎄 Enhancements * Issue comment editor now supports searching and linking to other issues within the same project. * Support data export for multi-statement queries in SQL Editor. * Enhance SSO state management to align with OAuth 2.0 best practices. * Add 5-minute expiration for MFA temporary tokens. * Clarify label types with explicit naming: Project Labels, Issue Labels, Database Labels. ## 🐞 Bug Fixes * Fix the role selector reverting SQL edits during edit mode when the issue poller runs. * Fix the false positive `Primary Key missing` error during SQL Review when adding a PK in a separate statement for PostgreSQL. ## ⚙️ Install and Upgrade * [Fresh install](/get-started/self-host/deploy-with-docker) * [Upgrade](/get-started/self-host/upgrade) *Before upgrading: 1) Back up the [metadata](/get-started/self-host/upgrade#back-up-and-restore) — in-place downgrade is not supported. 2) Do not run multiple containers on the same data directory. 3) Terraform users: upgrade Bytebase server first, then apply the new Terraform config.* # Bytebase 3.12.1 - Nov 20, 2025 Source: https://docs.bytebase.com/changelog/bytebase-3-12-1 Audit logging to stdout ## 🚀 Features * Add [stdout audit logger](/security/audit-log/#emit-audit-logs-to-stdout) for integration with log aggregation systems. * Add folder-based organization to Worksheets, including: * Nested folders with drag-and-drop reordering * Inline actions (rename, star/unstar, sharing) * Filters for worksheet types * Add project-level data query policies (previously workspace-only). * Support [cross-account AWS IAM Authentication](/get-started/connect/aws#cross-account-iam-authentication). * **PostgreSQL** - Add [AI-driven custom SDL review](/gitops/state-based-workflow/sql-review-ci#ai-powered-validation-with-your-standards). ## 🔔 Notable changes * Add TLS configuration support for Vault connections. ## 🎄 Enhancements * Revamp issue search UI. * Add **GitOps Service Agent** role for automated CI/CD workflows. * **PostgreSQL** - Support SDL for materialized views, enum types, and extensions. * **BigQuery** - Add dry-run support in SQL Editor to validate queries before execution. ## ⚙️ Install and Upgrade * [Fresh install](/get-started/self-host/deploy-with-docker) * [Upgrade](/get-started/self-host/upgrade) *Before upgrading: 1) Back up the [metadata](/get-started/self-host/upgrade#back-up-and-restore) — in-place downgrade is not supported. 2) Do not run multiple containers on the same data directory. 3) Terraform users: upgrade Bytebase server first, then apply the new Terraform config.* # Bytebase 3.12.2 - Dec 4, 2025 Source: https://docs.bytebase.com/changelog/bytebase-3-12-2 Improvements and Bug Fixes ## 🎄 Enhancements * Enhance database and query permission checks. * Improve selector performance with lazy loading for large workspaces. * **SQL Server** - Add [visualized EXPLAIN support](/sql-editor/run-queries#explain-query) in SQL Editor. * **Doris** - Improve syntax compatibility. ## 🐞 Bug Fixes * Prevent DML statements from being incorrectly wrapped in CTEs in SQL Editor. * Fix popup handling for database change statements in SQL Editor. * Improve and isolate SQL Editor state management across browser tabs. * Format Cassandra, Spanner, and ClickHouse result data instead of showing raw protobuf. ## ⚙️ Install and Upgrade * [Fresh install](/get-started/self-host/deploy-with-docker) * [Upgrade](/get-started/self-host/upgrade) *Before upgrading: 1) Back up the [metadata](/get-started/self-host/upgrade#back-up-and-restore) — in-place downgrade is not supported. 2) Do not run multiple containers on the same data directory. 3) Terraform users: upgrade Bytebase server first, then apply the new Terraform config.* # Bytebase 3.13.0 - Dec 18, 2025 Source: https://docs.bytebase.com/changelog/bytebase-3-13-0 Support MCP integration and simplified approval flow configuration ## 🔔 Notable Changes * **Consolidate DDL and DML changes** so users no longer need to select them explicitly when initiating New Plan or configuring approval flow. * **Simplify approval workflows**: * Approval flows are defined directly using CEL expressions, without intermediate risk definitions. * Risk level is automatically evaluated using a standardized assessment framework. * `risk.level` is an optional CEL factor for CHANGE\_DATABASE approval flows. * For non-Terraform, existing approval flows are automatically migrated. * For Terraform, approval flows are defined via bytebase\_setting (settings/WORKSPACE\_APPROVAL) using approval\_flow\.rules.flow, and risk resources are removed.(e.g. [approval\_flow.tf](https://github.com/bytebase/terraform-provider-bytebase/blob/main/examples/setup/approval_flow.tf)) * Update SQL review configuration for Terraform: (e.g. [sql\_review.tf](https://github.com/bytebase/terraform-provider-bytebase/blob/main/examples/setup/sql_review.tf)) * SQL review rules use typed payload fields (`number_payload`, `string_payload`, `string_array_payload`, `naming_payload`, etc.). * Generic payload and comment fields are removed. * Rule level enums migrate from `SQLReviewRuleLevel_WARNING/ERROR` to `SQLReviewRule_WARNING/ERROR`. * Move rollout policy checkers from environment to project level: **Require issue approval** and **Plan check enforcement**. Project settings are checked if any checker is previously used via automatic migration. * Migrate user identifiers: User references change from `users/{id}` to `users/{email}` in APIs and storage. Audit logs store user emails directly. * Rename **Masking Exception** to **Masking Exemption**: * APIs rename `MaskingException` to `MaskingExemption`. * The action field is removed. * `member` is replaced with a `members` list. * For Terraform, `masking_exception_policy` is renamed to `masking_exemption_policy`. * **Disallow Copy Policy** now applies to Admin and DBA roles (previously exempted). * Refine database group permissions: * Roles with `bb.projects.get` gain `bb.databaseGroups.list/get`. * Roles with `bb.projects.update` gain `bb.databaseGroups.create/update/delete`. * For Terraform, please update the permissions of custom roles accordingly. * Update batch task APIs: The `reason` field is removed from `BatchRunTasks` and `BatchCancelTaskRuns`. `reason` is supported only for skip operations. * Remove **Schema Template** functionality. * Remove **Sync classification from comment**: * Column classification is no longer derived from column comments (`{classification id}-{comment}`). Classification is stored in Bytebase and managed via the [UI, API, or Terraform](/security/data-masking/data-classification). * The `requiredClassification` option is removed from the [table](/sql-review/review-rules#table.comment) and [column](/sql-review/review-rules#column.comment) comment convention SQL review rules, along with its error code `1303`. ## 🚀 Features * Support [MCP integration](/integrations/mcp). * Add **Workload Identity** as a dedicated account type for OIDC-based authentication, with [GitHub Actions support](/administration/workload-identity/github-actions). * Support direct messages for **Microsoft Teams**. ## 🎄 Enhancements * Improve the rollout page with a timeline view, auto-expanded task details, and collapsible task run logs with auto-refresh. * Allow filtering in SQL Editor results to quickly locate rows. * Allow selecting a database group to query from in the SQL Editor. * **Snowflake** - Support passphrase-protected private keys. * **PostgreSQL** - Add event trigger sync and SDL support. * **Trino** - Support TLS/SSL. * **Doris** - Improve SQL parser compatibility. ## ⚙️ Install and Upgrade * [Fresh install](/get-started/self-host/deploy-with-docker) * [Upgrade](/get-started/self-host/upgrade) *Before upgrading: 1) Back up the [metadata](/get-started/self-host/upgrade#back-up-and-restore) — in-place downgrade is not supported. 2) Do not run multiple containers on the same data directory. 3) Terraform users: upgrade Bytebase server first, then apply the new Terraform config.* # Bytebase 3.13.1 - Dec 30, 2025 Source: https://docs.bytebase.com/changelog/bytebase-3-13-1 Improvements and Bug Fixes ## 🎄 Enhancements * Update SQL review template configuration. ## 🐞 Bug Fixes * Fix issue label selector missing in new CI/CD layout. * Fix rollout date filter not working. * Fix SQL Editor horizontal scroll shortcut. * Fix SQL review showing incorrect line numbers for multi-statement SQL. ## ⚙️ Install and Upgrade * [Fresh install](/get-started/self-host/deploy-with-docker) * [Upgrade](/get-started/self-host/upgrade) *Before upgrading: 1) Back up the [metadata](/get-started/self-host/upgrade#back-up-and-restore) — in-place downgrade is not supported. 2) Do not run multiple containers on the same data directory. 3) Terraform users: upgrade Bytebase server first, then apply the new Terraform config.* # Bytebase 3.14.0 - Jan 15, 2026 Source: https://docs.bytebase.com/changelog/bytebase-3-14-0 Separate CI and CD phases and Redefine Webhook Events ## 🌟 Sequel to New CI/CD Experience * Database **CI (review)** and **CD (deployment)** are 2 separated phases, aligning with industry standard CI/CD practices (GitHub, GitLab): * **CI (review)** - issue page displays plan changes directly, no more context switching between tabs. * **CD (deployment)** - rollout is now a standalone page, clearly separating review from execution. * Rollout UI now supports release-based tasks — showing per-file execution status, command duration, and release info in task items. * Rollout creation: introduce creation dependency between issue and rollout, with 2 rollout creation modes - 1) automatic creation, if issue is approved AND SQL review check pass; 2) manual creation, in other cases, e.g. issue approved BUT review not passed. After rollout is created, issue status will be automatically set to `DONE`. * Rollout permission: as a result of introducing manual creation, `bb.rollouts.create` permission is added to **Project Releaser** role . All custom roles for rollout initiation are advised to be assigned `bb.rollouts.create` permission (it only controls who can create a rollout, but it does not bypass approval requirements or task execution). * Remove `auto_resolve_issue` and `allow_modify_statement` from **Project Settings**. * For Terraform, remove `allow_modify_statement` and `auto_resolve_issue` from the project resource, remove `auto_resolve_issue` from the workspace\_profile setting resource. ## 🔔 Other Notable Changes * Update [**Project Webhook events**](/change-database/webhook#supported-events). * For Terraform, the `notification_types` in the project webhook are changed to `ISSUE_CREATED`, `ISSUE_APPROVAL_REQUESTED`, `ISSUE_SENT_BACK`, `PIPELINE_FAILED` and `PIPELINE_COMPLETED`. * Improve permission guards and access control: * Stop hiding UI elements when users lack permissions and show a no permission alert instead. * Allow users with `bb.issues.create` permission to request Project roles directly from the UI. * Allow **Project Owners** to disable the self-service role request workflow in **Project Settings**. - Introduce more granular permission management: * Add new policy permissions: * `bb.policies.getMaskingRulePolicy` * `bb.policies.updateMaskingRulePolicy` * `bb.policies.createMaskingRulePolicy` * `bb.policies.deleteMaskingRulePolicy` * `bb.policies.getMaskingExemptionPolicy` * `bb.policies.updateMaskingExemptionPolicy` * `bb.policies.createMaskingExemptionPolicy` * `bb.policies.deleteMaskingExemptionPolicy` * Add new settings permissions: * `bb.settings.getEnvironment` * `bb.settings.setEnvironment` * `bb.settings.getWorkspaceProfile` * `bb.settings.setWorkspaceProfile` * Permission assignments are updated accordingly: * The **Workspace Admin** and **DBA** receive all these new permissions. * The **Project Owner** receives the new policy permissions. * The **Workspace Member** receives `bb.settings.getEnvironment` and `bb.settings.getWorkspaceProfile`. - Tighten **Workspace Members** permissions: * Revoke `bb.settings.get`, `bb.settings.list`, `bb.projects.list`. * If **Workspace Members** should not access the user list, create a custom role and revoke `bb.users.list` and `bb.users.get`. - For Terraform, support configure `allow_request_role`, `data_classification_config_id`, `enforce_sql_review`, `force_issue_labels`, `issue_labels`, `labels`, `require_issue_approval`, `require_plan_check_no_error` for the project resource. * Redesign the **Database Changelog** and **Revision** pages. * Remove **Schema Drift Detection** feature: * Remove `drifted` field from `Database message` in `v1/database_service.proto`. * Remove `drifted` filter option from `ListDatabases` API (previously supported `drifted == true` filter). * Remove `dump_version` field from `ChangelogPayload` in store proto. * Update core APIs `CreateSheet`, `CreatePlan`, `CreateRollout`, and `CreateRelease`, please refer to the latest API documentation for updated request/response definitions. * Sheet: Remove `engine`, `title`, `creator`, `create_time` fields; remove `UpdateSheet` RPC (sheets are now immutable). * Plan: Remove `rollout` field (use `has_rollout` boolean); remove `ChangeDatabaseConfig.type` field. * Rollout: Resource path changed from `projects/*/rollouts/*` to `projects/*/plans/*/rollout`; remove `rollout` field from `CreateRolloutRequest`. * Release: Move `File.type` to release level; remove `File.id` field; remove `SearchReleases` RPC. * Common: Remove `DatabaseChangeType` enum; consolidate `DATABASE_SDL` into `DATABASE_MIGRATE`. * Remove `maximum_connections` from **Instance Settings** and use `Parallel tasks per rollout` in **Project Settings** instead. For Terraform, remove `maximum_connections` from the instance resource. * Unify `TRUNCATE_TABLE` and `TRUNCATE` into a single `TRUNCATE` statement type for **Custom Approval** CEL expressions. * Releases are now identified by `release_id` (e.g., `my-release-RC00`) with automatic train/RC iteration numbering. The `title`, `digest` fields are removed from the Release API, and UpdateRelease returns `UNIMPLEMENTED`. For Terraform, update `bytebase_release` resource if using `title` or `digest` fields. ## 🚀 Features * Improve access and refresh token support. Add refresh tokens (previously only access tokens on the web were supported) and allow separate configuration of **access token duration** and **refresh token duration** (previously the sign-in frequency setting). * Allow setting `No approval required` in approval rules when configured conditions are met. * Support [**Workload Identity** with **GitLab**](/administration/workload-identity/gitlab-ci). ## 🎄 Enhancements * Add list sorting for projects, instances, and databases. * **Grant Exemption** supports CEL expressions and includes a database selector. * **PostgreSQL** - Support ENUM value additions via `ALTER TYPE ... ADD VALUE` in schema sync. * **MySQL** - Prettify generated DDL statements. ## 🐞 Bug Fixes * Fix AI setting cache modification issues. * **PostgreSQL** – Fix cyclic foreign key dependency handling in schema diff. * **TiDB** – Fix DML dry run handling for BATCH syntax. * **PostgreSQL** – Fix using identifiers as SELECT target field aliases. * **Oracle** – Fix SQL Review incorrectly flagging GRANT / REVOKE as reserved keywords. ## ⚙️ Install and Upgrade * [Fresh install](/get-started/self-host/deploy-with-docker) * [Upgrade](/get-started/self-host/upgrade) *Before upgrading: 1) Back up the [metadata](/get-started/self-host/upgrade#back-up-and-restore) — in-place downgrade is not supported. 2) Do not run multiple containers on the same data directory. 3) Terraform users: upgrade Bytebase server first, then apply the new Terraform config.* # Bytebase 3.14.1 - Jan 29, 2026 Source: https://docs.bytebase.com/changelog/bytebase-3-14-1 Improvements and Bug Fixes ## 🎄 Enhancements * Improve result masking performance for queries spanning multiple databases, and enhance masking coverage. * Show rollout link whenever plan has a rollout, regardless of issue status. * **Google Cloud SQL** — Support IAM authentication for connections. * **Azure SCIM** — Enhance SCIM PATCH API and fix SCIM sync issues. ## 🐞 Bug Fixes * Fix database group SQL checks bypassed issue. * Fix export data issues: add task actions (Run/Retry, Skip, Cancel) and fix skip/download functionality. * Fix page table showing errors from corrupted localStorage page size. * Fix minor issues for IM integration: direct messaging not working for new installation and settings page not showing configured data. * Fix missing users and approvers in some webhook notifications. * Fix nested transaction patterns that could cause deadlocks under high concurrency. ## ⚙️ Install and Upgrade * [Fresh install](/get-started/self-host/deploy-with-docker) * [Upgrade](/get-started/self-host/upgrade) *Before upgrading: 1) Back up the [metadata](/get-started/self-host/upgrade#back-up-and-restore) — in-place downgrade is not supported. 2) Do not run multiple containers on the same data directory. 3) Terraform users: upgrade Bytebase server first, then apply the new Terraform config.* # Bytebase 3.15.0 - Feb 12, 2026 Source: https://docs.bytebase.com/changelog/bytebase-3-15-0 Project-level Service Accounts & Workload Identities ## 🔔 Project-level Service Accounts & Workload Identities We introduce project-level Service Accounts and Workload Identities in addition to the existing workspace-level scope. This enables project-scoped machine identities to follow least privilege and reduce automation blast radius, while clearly separating machine identities from users and aligning them with the resource hierarchy. * **UI & scope changes** * Workspace Members page now has separate tabs for Users\&Groups, Service Accounts, and Workload Identities. * Service accounts and workload identities can now be created at both workspace and project levels, governed by their respective IAM policies. * Project-level identities are scoped to a single project to enable isolated automation. * The account selector for role assignment now supports users, groups, service accounts, and workload identities. Service accounts and workload identities require entering the full email address. * **Breaking changes (API / Terraform users)** * Machine identities are managed via dedicated APIs (`ServiceAccountService`, `WorkloadIdentityService`) instead of the User API. * IAM member prefixes updated: `user:{email}` → `serviceAccount:{email}` / `workloadIdentity:{email}` * Workspace-level Service Account and Workload Identity APIs now require explicit parent `workspaces/-` instead of an empty string.\ Affected APIs: `CreateServiceAccount`, `ListServiceAccounts`, `CreateWorkloadIdentity`, `ListWorkloadIdentities`.\ Endpoint change:\ `/v1/serviceAccounts` → `/v1/workspaces/-/serviceAccounts` * Terraform users must update IAM member prefixes and use the new service account/workload identity resources. [Latest provider documentation](https://registry.terraform.io/providers/bytebase/bytebase/latest/docs) ## 🔔 Other Notable Changes * **DDL and DML execution control update** * Environment condition is added to `bb.sql.ddl` and `bb.sql.dml` role grants as a replacement for the `disallow_ddl` and `disallow_dml` environment policy. The environment condition is only available when granting roles at the **project level** for a specific project. Roles granted at the **workspace level** (which apply to all projects) do not have the environment condition option. * Each project-level IAM Policy (i.e. role grant) has a single environment condition that applies to both `bb.sql.ddl` and `bb.sql.dml` permissions together. This means you cannot allow DDL but disallow DML (or vice versa) within the same role. If you need different environment restrictions for DDL and DML, the recommended practice is to create 2 separate roles — one with `bb.sql.ddl` and one with `bb.sql.dml` — each with its own environment condition. * `disallow_ddl` and `disallow_dml` are automatically migrated to be reflected in role grants in an OR manner — if for a specific environment, either `disallow_ddl` or `disallow_dml` is set to OFF, this environment condition will be configured for the role with `bb.sql.ddl` or `bb.sql.dml` permission. This could lead to **breaking changes** to DDL and DML control if your `disallow_ddl` and `disallow_dml` settings differ across environment policies. Please review after upgrading and follow the recommended practice to configure your role grants. * Add `bb.sql.ddl` and `bb.sql.dml` permissions to the **SQL Editor User** and **Project Owner** role. * For Terraform users, update the environment policy and your role grants accordingly. [Latest provider documentation](https://registry.terraform.io/providers/bytebase/bytebase/latest/docs) * **SQL Editor settings consolidation & policy updates** * Add a dedicated **SQL Editor** section under **Workspace Settings > General**, consolidating data export, data copying, admin data source access, max result size, max result rows, and max query time. * **Max result rows** can also be configured at the project level. * `DataSourceQueryPolicy` is merged into `QueryDataPolicy` and deprecated (auto-migrated). * For Terraform users, update `bytebase_policy` configuration. [Latest provider documentation](https://registry.terraform.io/providers/bytebase/bytebase/latest/docs) * **Role & permission adjustments** * Add `bb.taskRuns.create` permission to the **Project Owner** role. * Remove `bb.rollouts.create` permission from the **Project Developer** role (use **Project Releaser** or **Project Owner**). * Allow managing project IAM policy without the **Project Owner** role. * **Online migration configuration change** * Move gh-ost configuration from Plan spec to SQL directive in sheet content (`-- gh-ost = { ... }`). * Remove `enable_ghost` and `ghost_flags` from `ChangeDatabaseConfig` in the Plan API. * **Execution & validation improvements** * Skip DML dry-run checks when DDL statements are present to reduce false positives. Primarily applied to SQL Review rule `Validate the executability of DML statements`. * **Cleanup & removals** * Remove the **Archived** page (archived projects and instances now appear directly in the dashboard). * Remove `auto_enable_backup` and `skip_backup_errors` from project settings. * Deprecate the legacy issue page and route. ## 🚀 Features * **MongoDB** * Use native driver for queries by default, with fallback to `mongosh`. * SQL Editor now supports auto-complete, current statement highlighting, and syntax checking. * Support statement-type access control in SQL Editor, allowing administrators to control Read and Write permissions. * **Elasticsearch** * Support statement-type access control in SQL Editor, allowing administrators to control Read and Write permissions. ## 🎄 Enhancements * SQL Editor query results support multi-select via Cmd/Ctrl + Click for rows and columns. Copied data now includes column names. * Improve the SQL Editor database connection panel layout. * Normalize Unicode emails to prevent creating accounts with visually identical but technically different addresses. ## 🐞 Bug Fixes * Fix access token refresh on SQL Editor LSP websocket reconnection. * Fix incorrect Learn More link for online migration. * **Google Cloud SQL** - Fix IAM authentication while creating instances in Bytebase Cloud. * **PostgreSQL** - Support Common Table Expressions (CTE) in backup statement execution. ## ⚙️ Install and Upgrade * [Fresh install](/get-started/self-host/deploy-with-docker) * [Upgrade](/get-started/self-host/upgrade) *Before upgrading: 1) Back up the [metadata](/get-started/self-host/upgrade#back-up-and-restore) — in-place downgrade is not supported. 2) Do not run multiple containers on the same data directory. 3) Terraform users: upgrade Bytebase server first, then apply the new Terraform config.* # Bytebase 3.15.1 - Feb 26, 2026 Source: https://docs.bytebase.com/changelog/bytebase-3-15-1 Improvements and Bug Fixes ## 🎄 Enhancements * Add **SQL Editor Read User** role for read-only SQL Editor access (SELECT, EXPLAIN, INFO — no DDL/DML). * Enforce `bb.issues.update` permission requirement to edit issues, even for issue creators. ## 🐞 Bug Fixes * Fix DDL/DML error modal not showing. * Fix REST API `GetSchemaString` endpoint returning parse error. * Fix SQL Editor double-click-drag word selection in reverse direction. * **PostgreSQL** — Fix schema dump losing overloaded functions and missing quotes for CamelCase columns in index/constraint DDL. ## ⚙️ Install and Upgrade * [Fresh install](/get-started/self-host/deploy-with-docker) * [Upgrade](/get-started/self-host/upgrade) *Before upgrading: 1) Back up the [metadata](/get-started/self-host/upgrade#back-up-and-restore) — in-place downgrade is not supported. 2) Do not run multiple containers on the same data directory. 3) Terraform users: upgrade Bytebase server first, then apply the new Terraform config.* # Bytebase 3.16.0 - Mar 12, 2026 Source: https://docs.bytebase.com/changelog/bytebase-3-16-0 Just-In-Time (JIT) Data Access ## 🔔 Notable Changes * **Workspace API breaking changes** - Policy API: `/v1/policies` → `/v1/workspaces/{id}/policies`. All workspace-scoped APIs now require explicit workspace ID instead of `/v1/workspaces/-`. See [Change Details](#workspace-api-breaking-changes). * **User API breaking changes** - Decouple identity types and migrate Service Accounts and Workload Identities into separate data models. The unified User API no longer handles these identity types; `User.user_type` and `UserType` enum are removed. See [Change Details](#user-api-breaking-changes). * **Legacy service account email migration** - Legacy emails with `{name}@service.bytebase.com` and `{name}@{project}.service.bytebase.com` are auto-migrated. Use the dedicated Service Account and Workload Identity services introduced in 3.15.0. * **Resource ID migration** - Several API resource IDs migrate from sequential integers to opaque UUID strings (revision, changelog, issue comment, project webhook). Previously bookmarked integer IDs will no longer work. See [Change Details](#resource-id-migration). * Non-release database migrations now run in parallel; only release-based migrations remain sequential per database. * **Terraform provider 3.16.1 required** - Covers workspace policy API changes, UserType removal, resource ID migration, and JIT approval flow. See [Migration Guide](https://github.com/bytebase/terraform-provider-bytebase/blob/main/migration/3.16.1.md) ## 🚀 Features * **Just-In-Time (JIT) Data Access** - Users without database access can request approval to execute a specific read-only query. Enable JIT in project settings and configure approval rules with the new `REQUEST_ACCESS` source type. Once approved, the grant is scoped to that query and auto-expires after the configured duration. * Add **GitOps** landing page with guided setup for workload identity selection and CI/CD YAML generation. * **Elasticsearch & MongoDB** - Support dynamic data masking. Masking is configured per-collection through the Catalog using `objectSchema` (not the column-based configuration used by relational databases). Global masking rules and masking exemption are not supported for document databases at this time. ## 🎄 Enhancements * Redesign issue list with streamlined layout and improved information density. Support sorting by created/updated time, all approval status options (Checking, Pending, Approved, Rejected, Skipped) in advanced search filter, and more prominent approval status in issue detail. * Standardize timestamp display to relative time with absolute time tooltip. * Redesign **Create Instance** page as a full-page layout. * Support access-token authentication for Bytebase Action, enabling CI/CD pipelines to authenticate to Bytebase via workload identity federation. * Add pre-execution drift validation that detects schema changes before executing stale tasks. * Support copying the entire query result in SQL Editor. * Update default AI model placeholders to current-generation models (GPT-4o, Gemini 2.5 Flash, Claude Sonnet 4). * **MongoDB & Elasticsearch** - Preview query results in document view or table view. Live syntax checking and auto-complete in SQL Editor. * **BigQuery & Spanner** - Support Workload Identity Federation credentials for non-GCP hosted Bytebase. * **Oracle** - Add ROW STORE COMPRESS syntax support. * **PostgreSQL** - Support `search_path` resolution via current user in schema * **PostgreSQL & Oracle** - Improve schema sync accuracy. ## 🐞 Bug Fixes * Fix issues incorrectly moved to DONE by migration 3.14/0034. * Skip databases without environments during task creation. * Classify CALL/EXEC stored procedure statements as DML to allow execution in SQL Editor. * **MariaDB** - Fix SQL review plan check not blocking rollout on ERROR-level violations. * **MSSQL** - Fix error messages missing line number when rolling out multiple statements. * **Oracle** - Fix UTF-8 encoding issues in comment fields during schema sync. * **TiDB** - Fix DROP INDEX IF EXISTS walk-through, CHECK\_CONSTRAINTS query compatibility for TiDB \< 7.4.0, and SQL export resource extraction. ## ⚙️ Install and Upgrade * [Fresh install](/get-started/self-host/deploy-with-docker) * [Upgrade](/get-started/self-host/upgrade) *Before upgrading: 1) Back up the [metadata](/get-started/self-host/upgrade#back-up-and-restore) — in-place downgrade is not supported. 2) Do not run multiple containers on the same data directory. 3) Terraform users: upgrade Bytebase server first, then apply the new Terraform config.* ## 📃 Change Details ### Workspace API Breaking Changes **1. Policy API path changes (workspace-level policies only):** | Method | Before | After | | ------ | ------------------------------ | ------------------------------------------- | | Get | `/v1/{name=policies/*}` | `/v1/{name=workspaces/*/policies/*}` | | List | `/v1/policies` | `/v1/{parent=workspaces/*}/policies` | | Create | `/v1/policies` | `/v1/{parent=workspaces/*}/policies` | | Update | `/v1/{policy.name=policies/*}` | `/v1/{policy.name=workspaces/*/policies/*}` | | Delete | `/v1/{name=policies/*}` | `/v1/{name=workspaces/*/policies/*}` | Environment, instance, and database-level policy bindings are unchanged. **2. APIs that no longer accept** `workspaces/-` (must use `workspaces/{id}`): | Service | Affected Operations | | ----------------------- | -------------------------------------------------- | | ServiceAccountService | `CreateServiceAccount`, `ListServiceAccounts` | | WorkloadIdentityService | `CreateWorkloadIdentity`, `ListWorkloadIdentities` | | DatabaseService | `ListDatabases` | | WorkspaceService | `GetIamPolicy`, `SetIamPolicy` | ### User API Breaking Changes | Change | Details | | --------------------------------- | ------------------------------------------------------------------------------------------------- | | `User.user_type` field removed | Reserved field 5. Use dedicated Service Account / Workload Identity services. | | `UserType` enum removed | Deleted from `user_service.proto`. | | `WorkloadIdentityConfig` moved | From `User` message to `workload_identity_service.proto`. | | `ActuatorInfo.user_stats` removed | Replaced with `int32 activated_user_count`. | | `CreateUser` behavior | Only creates end users. Service accounts / workload identities must use their dedicated services. | | `ListUsers` behavior | Only returns end users. | ### Resource ID Migration Resource IDs in the following API resource names change from sequential integers to UUID strings: | Resource | Resource Name Pattern | ID Format Change | | --------------- | ----------------------------------------------- | ---------------- | | Revision | `instances/{id}/databases/{db}/revisions/{id}` | integer → UUID | | Changelog | `instances/{id}/databases/{db}/changelogs/{id}` | integer → UUID | | Issue Comment | `projects/{id}/issues/{uid}/issueComments/{id}` | integer → UUID | | Project Webhook | `projects/{id}/webhooks/{id}` | integer → UUID | Existing records receive randomly generated UUIDs during migration. Any previously bookmarked or cached integer IDs will stop working. # Bytebase 3.16.1 - Mar 26, 2026 Source: https://docs.bytebase.com/changelog/bytebase-3-16-1 Improvements and Bug Fixes ## 🎄 Enhancements * Change issue approval status label from "Done" to "Approved" for clarity. * Add fast-follow refresh mode so rollout status updates more quickly after user actions. * **CosmosDB** — Support more query syntax in SQL editor. ## 🐞 Bug Fixes * Fix internal error after login caused by InputOtp null check regression. * Fix popover dismissing when interacting with label selector during plan/issue creation. * Fix invalid issue type filter in Export Center. * Fix webhook detail page broken by UUID resource\_id migration. * Harden 3.16 upgrade migration scripts to be idempotent and handle edge cases. * **PostgreSQL** — Fix schema dump incorrectly including `pg_bitmapindex` system schema. ## ⚙️ Install and Upgrade * [Fresh install](/get-started/self-host/deploy-with-docker) * [Upgrade](/get-started/self-host/upgrade) *Before upgrading: 1) Back up the [metadata](/get-started/self-host/upgrade#back-up-and-restore) — in-place downgrade is not supported. 2) Do not run multiple containers on the same data directory. 3) Terraform users: upgrade Bytebase server first, then apply the new Terraform config.* # Bytebase 3.17.0 - Apr 9, 2026 Source: https://docs.bytebase.com/changelog/bytebase-3-17-0 Unified Plan Lifecycle View & High Availability ## 🔔 Notable Changes * **Unified Plan lifecycle view** — the Plan detail page shows the full lifecycle of a database change, giving developers one place to follow a change from draft to deployment. * The three phases of a database change — change, review, deploy — are shown in sections, allowing developers to edit change, view approval result, and execute deployment in this single page. * Issue page is now dedicated primarily for review and approval. Developers will need to go to the plan page to edit changes and create a rollout when needed. * **API breaking changes** — Worksheet API moves to project scope (`/v1/projects/{id}/worksheets`), `ListDatabases` now requires workspace ID (`/v1/workspaces/{id}/databases`), `branding_logo` moves from workspace profile setting to workspace message, `GRANT_REQUEST` issue type renamed to `ROLE_GRANT`, `UpdateSubscription` API deprecated in favor of `UploadLicense`, and classification description and level description fields removed with level type changed from string to number. See [Change Details](#api-breaking-changes). * **CEL filter syntax change** — Substring-search filters now use `contains()` instead of `matches()` across plan, database, instance, project, group, user, service account, workload identity, query history, access grant, and database metadata table filters. See [Change Details](#cel-filter-syntax-change). * **Read-only data source consolidation** — Each instance now allows at most one read-only data source. If more than one is configured, only the first is kept after migration; extra read-only data sources are removed. * **Remove Pro plan 20-user seat cap** — Pro plan no longer has a user limit. * **Terraform provider** — Updated for 3.17.0 API breaking changes. Supports get, create, update, and list identity providers. See [Migration Guide](https://github.com/bytebase/terraform-provider-bytebase/blob/main/migration/3.17.0.md). ## 🚀 Features * **High Availability (HA)** — Support licensed multi-replica deployments backed by a shared external PostgreSQL metadata database. Active replicas are tracked via heartbeats, and background runners coordinate safely across replicas. * **[Page Agent](/page-agent)** — Add an in-app AI assistant with tool-calling, threaded resumable conversations, DOM-aware autocomplete, and token usage tracking. * **Issue Approved webhook event** — Support a new webhook event type for issue approval, with redesigned Slack message format. * **MCP `query_database` tool** — Add a new MCP tool for executing SQL queries against managed databases with automatic database resolution. * **CosmosDB** — Support local emulator connection and dynamic data masking for all 13 query feature areas. ## 🎄 Enhancements * Support executing stored procedures (`CALL` / `EXEC`) in SQL Editor by classifying stored procedure execution as DML. * Support issue type filter in advanced search (Database Change, Role Grant, Database Export, Access Grant). * Replace data classification file upload with an inline JSON editor and enhance the data classification editor. * Support masking exemption by classification level, and redesign masking exemption list page. * Redesign data export creation UI. * Migrate the frontend from Vue to React across all major pages including settings, projects, databases, instances, plans, issues, and audit logs. * Upgrade SQL parsing engine for PostgreSQL, MySQL, MongoDB, and CosmosDB from ANTLR to omni parser, improving SQL syntax coverage, consistency across SQL review / schema diff / auto-completion, and error message quality. * Add configurable `--timeout` flag for `bytebase-action` for large SQL checks. * Expose `--enable-json-logging` flag in the Helm chart for structured log output. ## 🐞 Bug Fixes * Fix false schema diff noise caused by PostgreSQL trigger ordering and CRLF / whitespace normalization. * Fix SQL editor metadata sync retry flood. * Fix OAuth discovery to return usable URLs in self-hosted mode. * Fix stale debounced query reverts in advanced search. * Fix gh-ost directives in migration-based GitOps workflow. * **PostgreSQL** — Fix CRLF line endings causing query truncation in SQL splitter, schema load failure by excluding aggregates from function sync, array subscript handling in column reference normalization, whitespace between LIMIT and FOR UPDATE clause, and nested SELECTs in wrapper statements and CTEs for SQL review. * **MySQL** — Fix unqualified SET columns in prior backup for UPDATE JOIN, missing `multiStatements` param in IAM auth DSN, and role sync for anonymous users. * **TiDB** — Remove non-transaction statement handling. * **Oracle** — Strip trailing null bytes from schema definitions. ## ⚙️ Install and Upgrade * [Fresh install](/get-started/self-host/deploy-with-docker) * [Upgrade](/get-started/self-host/upgrade) *Before upgrading: 1) Back up the [metadata](/get-started/self-host/upgrade#back-up-and-restore) — in-place downgrade is not supported. 2) Do not run multiple containers on the same data directory. 3) Terraform users: upgrade Bytebase server first, then apply the new Terraform config.* ## 📃 Change Details ### API Breaking Changes **1. Worksheet API moves to project scope:** | Method | Before | After | | ------ | ----------------------------------- | ---------------------------------------------- | | List | `/v1/worksheets` | `/v1/{parent=projects/*}/worksheets` | | Get | `/v1/{name=worksheets/*}` | `/v1/{name=projects/*/worksheets/*}` | | Create | `/v1/worksheets` | `/v1/{parent=projects/*}/worksheets` | | Update | `/v1/{worksheet.name=worksheets/*}` | `/v1/{worksheet.name=projects/*/worksheets/*}` | | Delete | `/v1/{name=worksheets/*}` | `/v1/{name=projects/*/worksheets/*}` | **2. APIs that no longer accept** `workspaces/-` (must use `workspaces/{id}`): | Service | Affected Operations | | --------------- | ------------------- | | DatabaseService | `ListDatabases` | **3. Other breaking changes:** | Change | Affected Routes | Details | | -------------------- | ------------------------------------------------------------------------------------------------------------------------- | --------------------------------------------------------------------------------------------------------------------------- | | Branding logo | `GET/PATCH /v1/workspaces/{id}` | `branding_logo` removed from workspace profile setting; use `logo` field on workspace message | | Issue type enum | `POST /v1/{parent=projects/*}/issues`, `GET /v1/{parent=projects/*}/issues`, `POST /v1/{parent=projects/*}/issues:search` | `GRANT_REQUEST` renamed to `ROLE_GRANT` | | Subscription API | `PATCH /v1/subscription` | Deprecated; use `PATCH /v1/subscription/license` (`UploadLicense`) | | Data source ID | `POST /v1/{name=instances/*/databases/*}:query`, `POST /v1/{name=instances/*/databases/*}:export` | `data_source_id` auto-resolved server-side; at most one read-only data source per instance (extra removed during migration) | | Classification level | `GET/PATCH /v1/settings/{name}`, `POST /v1/queryHistories:search` | Level type changed from `string` to `int`; `description` field removed | ### CEL Filter Syntax Change Substring-search CEL filters now use `contains()` instead of `matches()`. Existing clients that send filters such as `title.matches("...")`, `name.matches("...")`, `resource_id.matches("...")`, `email.matches("...")`, `host.matches("...")`, `port.matches("...")`, `statement.matches("...")`, `query.matches("...")`, or `table.matches("...")` must switch to the corresponding `contains()` form. This applies to plan, database, instance, project, group, user, service account, workload identity, query history, access grant, and database metadata table filters. # Bytebase 3.17.1 - Apr 23, 2026 Source: https://docs.bytebase.com/changelog/bytebase-3-17-1 Improvements and Bug Fixes ## 🎄 Enhancements * **Instance TLS** — Support reading TLS certificates from a configured file path, and redesign TLS configuration with Disabled / TLS / Mutual TLS options. * **SQL Review** — Add `STATEMENT_DISALLOW_TRUNCATE` rule for Oracle, PostgreSQL, MySQL, and MSSQL. Improve `STATEMENT_WHERE_DISALLOW_FUNCTIONS_AND_CALCULATIONS` to only flag functions and calculations on indexed columns, and extend it to Oracle and PostgreSQL. ## 🐞 Bug Fixes * Fix access/role grant issue approval showing a spurious "project not found" error toast. * Fix Test Connection on instance create dropping passwords resolved from AWS Secrets Manager and other external secret backends. * **Oracle** — Fix PL/SQL parser to accept expressions (e.g. DATE literals) in `CREATE TABLE` partition range bounds. * **MSSQL** — Fix query span extraction for views whose WHERE clause contains a correlated `EXISTS` / `NOT EXISTS` subquery. * **Cassandra** — Fix request context propagation in query span extraction. ## ⚙️ Install and Upgrade * [Fresh install](/get-started/self-host/deploy-with-docker) * [Upgrade](/get-started/self-host/upgrade) *Before upgrading: 1) Back up the [metadata](/get-started/self-host/upgrade#back-up-and-restore) — in-place downgrade is not supported. 2) Do not run multiple containers on the same data directory. 3) Terraform users: upgrade Bytebase server first, then apply the new Terraform config.* # Bytebase 3.18.0 - May 14, 2026 Source: https://docs.bytebase.com/changelog/bytebase-3-18-0 Bytebase Cloud Upgrade ## ☁️ Bytebase Cloud Upgrade Bytebase Cloud is now a true multi-tenant SaaS. * One account, many workspaces — with a workspace switcher and self-serve leave/delete. * Passwordless sign-in by email code (no more passwords in Cloud). * Global SSO — sign in with Google, GitHub, or your org's IdP without picking a workspace first. * Self-serve subscription plan upgrade and cancel from inside the app. * Pro plan: unlimited user seats available, billed per user. ## 🔔 Other Notable Changes * **Plan edits surfaced in issue activity** — Plan edits now appear in the issue activity feed for approver visibility. * **API breaking changes** — `require_2fa` → `require_mfa`, `plan_spec_update` → `plan_update` (restructured payload), and `Issue.ApprovalStatus` moved to top-level. See [Change Details](#api-breaking-changes). * **Terraform provider 3.18.0 required** — Adds write-only sensitive fields and a provider-level `custom_header` block. Requires Terraform CLI 1.11+. See [Migration Guide](https://github.com/bytebase/terraform-provider-bytebase/blob/main/migration/3.18.0.md). ## 🚀 Features * **AWS RDS IAM auth for the Bytebase metadata database** — Authenticate the metadata PostgreSQL via AWS RDS IAM tokens. * **Google Chat webhook integration** — Add Google Chat as a project webhook destination. * **MariaDB** — Support DML rollback / prior-backup, matching MySQL behavior. * **CosmosDB** — Support cross-partition queries. ## 🎄 Enhancements * **Bytebase Action** — Add `--custom-header` flag for header-based access proxies in CI/CD pipelines. * Issues stuck in `CHECKING` can be retried via a new `RetryIssueApproval` API. * Improve SQL analysis reliability across PostgreSQL, MySQL, and MSSQL for advisor rules, query span, completion, and schema diff. * Improve PostgreSQL and CockroachDB metadata-backed schema diff reliability. * Show gh-ost start/end events in task run logs. * Show DDL/DML environment warnings across role grant, role request, and issue creation flows. * Show expired roles inline in the members table and member detail panel. * Show database group titles in plan selectors and human-readable instance/database names in member-role scopes. * Whitelist redirect URIs in OAuth dynamic client registration. * Restore audit logs for Login / Signup / ExchangeToken; add audit events for retry approval and email-code / password-reset paths. * Default the SQL Editor query role to the least-privileged SQL select role. * Improve the SQL statement matching for JIT access grants. * **TiDB** — Support extra DataSource connection parameters, including connection packet compression. * **TiDB** — SQL Editor `Ctrl+Enter` now runs the cursor statement, matching MySQL / PostgreSQL behavior. * **ClickHouse** — `AggregateFunction(...)` errors from `SELECT *` now suggest using `-Merge` + `GROUP BY` or `finalizeAggregation()`. ## 🐞 Bug Fixes * Fix multi-change plans targeting the same database group being rejected. * Fix `PIPELINE_COMPLETED` webhook event missing when failed tasks are skipped. * Fix per-sheet plan check summary preservation during approval checks. * **PostgreSQL** — Fix query span bugs affecting masking and query analysis. * **Databricks** — Apply user-selected row limit to SQL Editor queries. ## 📃 Change Details ### API Breaking Changes **1. `require_2fa` renamed to `require_mfa`** The field on `WorkspaceProfileSetting` reflects the broader MFA scope (no longer 2FA-only). | Field | Before | After | | ------------------------------------- | -------------------- | -------------------- | | `WorkspaceProfileSetting.require_2fa` | `require_2fa` (bool) | `require_mfa` (bool) | Affected routes: any caller updating workspace profile settings via `SettingService.UpdateSetting` writing `WorkspaceProfileSetting`. Update references in your config or IaC. **2. `IssueComment.plan_spec_update` renamed to `plan_update`** The per-spec event (single sheet replacement) is replaced by a before/after snapshot of `Plan.Spec[]`, so the comment carries the full diff of every plan spec change in one event. | Field | Before | After | | ------------------------------------------------------------------------------------------------------------ | ----------------------------------------------------------------------------------------- | ----- | | `IssueComment.plan_spec_update` — `PlanSpecUpdate { spec, from_sheet, to_sheet }` (single spec/sheet change) | `IssueComment.plan_update` — `PlanUpdate { from_specs, to_specs }` (repeated `Plan.Spec`) | | Affected routes: any consumer of `IssueService.ListIssueComments` or issue activity events that previously read `plan_spec_update`. Update to read `plan_update.from_specs` / `to_specs`. **3. `Issue.ApprovalStatus` moved to top-level `ApprovalStatus`** Nested enum `Issue.ApprovalStatus` is removed; the same enum now lives at the top level in `common.proto` and is referenced by both `Issue.approval_status` and `Plan.approval_status`. Enum values and ordinals are unchanged (`CHECKING=1`, `PENDING=2`, `APPROVED=3`, `REJECTED=4`, `SKIPPED=5`). | Type | Before | After | | ------------- | ---------------------------------- | ---------------------------- | | Enum location | `bytebase.v1.Issue.ApprovalStatus` | `bytebase.v1.ApprovalStatus` | Affected routes: clients that referenced the nested enum type by name (for example, generated Go or TS types). Update imports or type references; wire format is unchanged. ## ⚙️ Install and Upgrade * [Fresh install](/get-started/self-host/deploy-with-docker) * [Upgrade](/get-started/self-host/upgrade) *Before upgrading: 1) Back up the [metadata](/get-started/self-host/upgrade#back-up-and-restore) — in-place downgrade is not supported. 2) Do not run multiple containers on the same data directory. 3) Terraform users: upgrade Bytebase server first, then apply the new Terraform config.* # Bytebase 3.18.1 - May 28, 2026 Source: https://docs.bytebase.com/changelog/bytebase-3-18-1 Improvements and Bug Fixes ## 🎄 Enhancements * **Plan detail page improvements.** * **Refreshed layout.** Cleaner layout with at-a-glance plan-check status across all specs and consistent database-target display through every phase. * **Release-backed plan flow simplified.** Release-backed (GitOps) plans now show only two phases — CHANGES and DEPLOY — with minor UI refinements throughout. * **SaaS: MCP discovery works against Bytebase Cloud.** MCP clients (Claude Code, Cursor, etc.) can now complete the OAuth flow against `cloud.bytebase.com` without manual configuration. * **PostgreSQL** — Compatible with pgbouncer transaction pooling (statement cache disabled) and PostgreSQL 18 (built-ins like `uuidv7()` and 80+ others are recognized by SQL analysis and schema diff). * **TiDB** — DML rollback / prior-backup preview now works. Queries against columns added out-of-band trigger the standard metadata-resync-and-retry path (matching MySQL / PostgreSQL). * **gh-ost migrations honor the configured data-source SSH tunnel** for both the MySQL connection and the binlog reader. * **Query data source** — Non-read-only automatic SQL Editor queries now use the admin data source when the query data policy allows it. Read-only queries are unaffected. * **Demo mode removed.** The `--demo` server flag has been removed. This mode was for internal demos (it loads a baked-in SQL dump and a sample admin) and was never intended for production use — if you have `--demo` in your startup script, drop it and configure instances and users normally. ## 🐞 Bug Fixes * Workload identities are distinguished from service accounts in the members table, and the graphical CEL expression editor is restored for project member role grants. * Large target sets in the issue / plan target overflow view no longer freeze or mis-render. * The plan list creator filter actually returns users now, and the review badge ("Bypassed" / "Under Review") reads consistently with plan detail. * **PostgreSQL** — Multi-host failover honors Bytebase's configured TLS material on every host, not just the primary. * The Sync Database success toast renders the actual database name instead of the literal `{{name}}` placeholder. ## 🏗️ Terraform Update * **Terraform provider 3.18.1 required** — Marks the webhook URL as write-only and adds `APP_IM` support to `bytebase_setting`. See [Migration Guide](https://github.com/bytebase/terraform-provider-bytebase/blob/main/migration/3.18.1.md). ## ⚙️ Install and Upgrade * [Fresh install](/get-started/self-host/deploy-with-docker) * [Upgrade](/get-started/self-host/upgrade) *Before upgrading: 1) Back up the [metadata](/get-started/self-host/upgrade#back-up-and-restore) — in-place downgrade is not supported. 2) Do not run multiple containers on the same data directory. 3) Terraform users: upgrade Bytebase server first, then apply the new Terraform config.* # Bytebase 3.19.0 - Jun 04, 2026 Source: https://docs.bytebase.com/changelog/bytebase-3-19-0 Just-in-Time Data Export ## 🔔 Notable Changes * **Just-in-time data export** - Users can now request just-in-time data export access in the SQL editor and go through the approval workflow to export. Your existing data-export approval rules are migrated to the just-in-time approval flow. Please review and update the ordering of migrated rules according to the instruction on top of the Custom Approval page. See [Details](/security/database-permission/export#just-in-time-data-export). * **Terraform** - Just-in-time data export flows through `REQUEST_ACCESS` approval, please add data-export rules (`request.data_export == true`) under the `REQUEST_ACCESS` source in your Terraform configuration - otherwise the next `terraform apply` will drop the auto-backfilled rules and JIT exports skip approval. * **Active VCS user tracking for GitOps** — Non-bot PR/MR authors seen by `bytebase-release` workflows on GitHub, GitLab, and Bitbucket now count as active VCS users over a 90-day window and are enforced against the license user limit. Track usage and download the user list as CSV from the Subscription page. * **bytebase-action enforces the version compatibility window** — `bytebase-action check` and `bytebase-action rollout` now fail with an error (previously a warning) when the action version is too far from the server version. Pin your CI to a compatible action version, or use the `cloud` tag on Bytebase Cloud. See [Change Details](#bytebase-action-compatibility-window). ## 🎄 Enhancements * **MySQL** — Improve gh-ost prerequisite validation messages by distinguishing inaccessible binlog status, disabled binary logging, missing replication privileges, unsupported binlog format, and validation query failures. * **SQL Editor** — Saving an untitled worksheet now prompts for a title, and searching the result panel indicates when nothing matches. ## 🐞 Bug Fixes * **SQL Editor** — Fixed query execution and admin mode being blocked when opening the editor from a database page, along with several smaller UI fixes. Local editor state is now scoped per workspace on Bytebase Cloud. * Restore the custom expiration time option when granting project roles, honoring the workspace maximum role expiration cap. * Fix AI assistant compatibility with GPT-5-style models by omitting unsupported request parameters. * Fix hosted MCP clients (claude.ai web, ChatGPT, VS Code for the Web) being rejected during OAuth dynamic client registration on self-hosted instances. Callbacks are pinned to exact vendor hosts, so arbitrary `https://` redirects remain blocked. * **Self-hosted (Helm)** — Chart 1.1.3 restores `bytebase.version` as the single image knob; non-Azure deployments no longer silently resolve to the `latest` image. * **Spanner** — Fix SQL Editor queries failing with "disallowed query type" in normal mode. * **Doris/StarRocks** — Fix window functions inside a CTE body failing to parse in the SQL Editor. * **TiDB** — Fix prior-backup rollback for alias-target multi-table DELETE, cyclic view references crashing query analysis, and advisor line numbers for statements separated by blank lines. ## 📃 Change Details ### bytebase-action Compatibility Window `bytebase-action check` and `bytebase-action rollout` now **fail with an error** when the action/CLI version falls outside the supported compatibility window of the target Bytebase server (previously a warning). | Deployment | Compatibility requirement | | -------------- | ----------------------------------------------------------------------------------------------------------- | | Bytebase Cloud | Dated action versions (`cloud-YYYYMMDD`) within the last 7 days; use the `cloud` action tag to stay current | | Self-hosted | Same major version, within a 2-minor-version window of the server | | Version relationship | Behavior | | ------------------------- | ---------------------------------------- | | Exact match | Success log | | Within window, mismatched | Warn and continue (unchanged) | | Outside window | **Error — command exits non-zero (new)** | **Action required:** pin your CI to an action version within the window of your server version (self-hosted), or use the `cloud` tag (Cloud). ## ⚙️ Install and Upgrade * [Fresh install](/get-started/self-host/deploy-with-docker) * [Upgrade](/get-started/self-host/upgrade) *Before upgrading: 1) Back up the [metadata](/get-started/self-host/upgrade#back-up-and-restore) — in-place downgrade is not supported. 2) Do not run multiple containers on the same data directory. 3) Terraform users: upgrade Bytebase server first, then apply the new Terraform config.* # Bytebase 3.19.1 - Jun 18, 2026 Source: https://docs.bytebase.com/changelog/bytebase-3-19-1 Improvements and Bug Fixes ## 🎄 Enhancements * **Plan review on the plan detail page** — A unified Review section now lives directly on the plan detail page: approval flow, activity timeline, and rejection banner. * **SQL Editor query history** — Query history is reworked, and any history item now has a direct, shareable URL that reopens its exact statement and database connection. * **Faster large SQL scripts and reviews** — Splitting, DML transform, statement-range mapping, and SQL review now scale linearly on large multi-statement scripts, reducing the processing time of a 100k-line-SQL-statement by 98% (from 22s to 0.45s). * **StarRocks** — Dedicated StarRocks parser support: stops false syntax errors on StarRocks-specific DDL/DML (generated columns, async materialized views, `INSERT OVERWRITE`, CTE-prefixed `DELETE`/`UPDATE`), adds query lineage, and syncs StarRocks materialized views. * **Consistent date & time picker** — All date/time controls (run-task scheduler, role/permission expiration, time-range filters) now use one styled, app-themed picker; the whole field opens the picker, and custom expiration times are available again. ## 🐞 Bug Fixes * Fix table- and schema-scoped `bb.sql.ddl` / `bb.sql.dml` grants being silently denied for writes in the SQL Editor — such grants are now honored per write-target table. * **PostgreSQL** — Fix SQL Editor crash when a query returns a timestamp outside the supported range, and make metadata sync read columns and foreign tables from `pg_catalog` so privilege filtering can no longer hide them, producing valid, replayable schema dumps. * **MySQL** — Fix Sync Schema reporting false-positive diffs between identical databases (a regression from 3.17.0), and produce valid SQL for CHECK constraints and stored-routine headers in schema dumps. * **MSSQL** — Fix SQL Editor failing to resolve columns of an encrypted or otherwise unparseable view by falling back to synced column metadata. * Fix the instance Databases tab resetting to the first page after clicking Load more, and missing-license feedback when adding a read-only connection to an unlicensed instance. * Fix SQL review rules with required list values failing to save with an `InvalidArgument` error when no values were added. * Fix GitOps release checks reporting a `None` risk level — per-result and aggregate risk levels are now reported correctly in bytebase-action output. * **DingTalk** — Fix direct-message webhook notifications failing to resolve recipients when their phone numbers were not populated. ## ⚙️ Install and Upgrade * [Fresh install](/get-started/self-host/deploy-with-docker) * [Upgrade](/get-started/self-host/upgrade) *Before upgrading: 1) Back up the [metadata](/get-started/self-host/upgrade#back-up-and-restore) — in-place downgrade is not supported. 2) Do not run multiple containers on the same data directory. 3) Terraform users: upgrade Bytebase server first, then apply the new Terraform config.* # Bytebase 3.2.0 - Jan 02, 2025 Source: https://docs.bytebase.com/changelog/bytebase-3-2-0 Selectively sync databases from an instance ## 🚀 New Features * Selectively sync databases from an instance. * Show the definition of TABLE and VIEW for Redshift and ClickHouse. ## 🔔 Notable Changes * `Change History` module is renamed to `Changelog`. `ChangeHistory` API is removed, use `Changelog` API instead. * Changes initiated from the UI no longer carry the version, but changes initiated from GitOps still do. * For the same database, tasks with lower versions will block tasks with higher versions. ## 🎄 Enhancements * Support querying special date/time values(e.g., infinity) for PostgreSQL in SQL Editor. * Failed task run is now recorded to changelog, before it's not. ## ⚙️ Install and Upgrade * [Fresh install](/get-started/self-host/deploy-with-docker) * [Upgrade](/get-started/self-host/upgrade) *Before upgrading: 1) Back up the [metadata](/get-started/self-host/upgrade#back-up-and-restore) — in-place downgrade is not supported. 2) Do not run multiple containers on the same data directory. 3) Terraform users: upgrade Bytebase server first, then apply the new Terraform config.* # Bytebase 3.20.0 - Jul 02, 2026 Source: https://docs.bytebase.com/changelog/bytebase-3-20-0 Custom Theme ## 🔔 Notable Changes * **CI/CD change issues now open on the Plan detail page** — Opening a CI/CD database-change issue (from the issue list, a link, or a notification) now takes you to the Plan detail page — where the review happens, with approval flow, activity, and comments inline — instead of the Issue detail page. Access-request and data-export issues still open on the Issue detail page, and existing issue links keep working (they redirect). * **Cap the duration of access requests** — Admins can now enforce a maximum duration on access requests; a request can no longer exceed the workspace's configured cap. The workspace setting is renamed accordingly. See [Change Details](#access-request-expiration). * **Export Center and data-export approval flow deprecated** — The standalone Export Center and the separate data-export approval flow are deprecated; data export is handled through the just-in-time data-export approval flow in the SQL Editor. * **7 low-signal SQL review rules removed** — These rules only fired when the top-level statement was a `SELECT`, so they never applied to the DDL/DML change-review workflow. Existing review policies are cleaned automatically on upgrade. See [Change Details](#removed-sql-review-rules). * **Terraform provider 3.20.0 required** — Upgrade the Bytebase server first, then move to provider 3.20.0. It covers this release's breaking changes: `EXPORT_DATA` deprecated in the approval flow; announcement `level` deprecated in favor of `theme`; the workspace maximum request-expiration field renamed; and environment and issue-label colors moved to the color protocol message. It also adds Vault `token_type` support and fixes review-config state-diff and webhook-URL-in-state handling. Remove the seven deleted SQL review rule types from your review configs. See [Change Details](#terraform-impact). ## 🚀 Features * **Custom theme for SQL Editor and announcements** — Choose the SQL Editor color theme or define your own from Settings. Workspace announcements get the same treatment, with configurable background and text colors, replacing the deprecated announcement `level`. * **Self-hosted** — GCP Cloud SQL IAM authentication for the external PostgreSQL metadata database. On GKE with Workload Identity, the metadata database connects to Cloud SQL via IAM authentication with no static password — over public IP, private IP, or Private Service Connect (PSC), with [Helm chart support](/get-started/self-host/deploy-with-kubernetes#deploy-with-helm). See [Metadata database IAM authentication](/get-started/self-host/external-postgres#gcp-cloud-sql-iam-authentication). ## 🎄 Enhancements * **SQL parsing** — Improved Omni parser compatibility across multiple SQL engines, covering SQL Review, schema workflows, statement splitting, completion, and execution. For MariaDB specifically, the SQL Editor no longer flags valid MariaDB-only syntax as errors: sequences and `NEXT VALUE FOR`, `RETURNING` on INSERT/REPLACE/DELETE, system-versioned and application-time-period tables, parenthesized row constructors, and UUID/INET types. * **Plan detail** — Consolidated plan-lifecycle actions in the header, which now shows the plan's primary "what's next" action (or current status) at every stage, plus a consolidated review-activity timeline. * **gh-ost parameter configuration** — Restores the in-product editor for gh-ost online-migration flags (`max-load`, `chunk-size`, and more), so tuning no longer requires hand-writing the JSON directive. * **Vault** — The HashiCorp Vault external-secret integration adds a `token_type` field: the token can be provided as `PLAIN`, `ENVIRONMENT` (environment variable), or `FILE`. * Allow just-in-time requests for info-level permissions. ## 🐞 Bug Fixes * **GitOps** — Fix release checks not populating risk levels, which made high-risk statements such as `DROP TABLE` show `None` in the GitHub Action check output. * **Custom OAuth2 SSO** — Fix userinfo lookup failing against Oracle Cloud Infrastructure (OCI) IAM Identity Domains, which surfaced as a misleading JSON parse error. * Fix the "Manually select" database picker freezing the browser on large deployments, by loading one page on open with a "Load more" affordance instead of draining every page at once. * Fix the workspace title being uneditable on the setup page, and a missing project-scope feature check when listing database groups. * **MySQL/MariaDB** — Fix schema sync over-fetching foreign-key and trigger metadata across databases, and use the MariaDB-native `CHECK_CONSTRAINTS` query. * **gh-ost** — Trim a trailing semicolon so the attempt-instant-DDL path stays valid. ## 📃 Change Details ### Access Request Expiration Access requests are now bounded by the workspace maximum expiration, and the workspace setting field is renamed. | Change | Before | After | | ----------------------- | --------------------------------------------------------- | --------------------------------------------------- | | Workspace setting field | `maximum_role_expiration_in_seconds` | `maximum_request_expiration_in_seconds` | | Behavior | Requesters could set an expiration independent of the cap | Request duration is capped at the workspace maximum | The existing value is carried to the new field automatically on upgrade. ### Removed SQL Review Rules These 7 rules are removed — each only triggered on a top-level `SELECT` statement: | Removed rule | | ------------------------------------ | | `STATEMENT_SELECT_FULL_TABLE_SCAN` | | `STATEMENT_QUERY_MINIMUM_PLAN_LEVEL` | | `STATEMENT_DISALLOW_USING_FILESORT` | | `STATEMENT_DISALLOW_USING_TEMPORARY` | | `STATEMENT_MAXIMUM_JOIN_TABLE_COUNT` | | `STATEMENT_JOIN_STRICT_COLUMN_ATTRS` | | `STATEMENT_MAXIMUM_LIMIT_VALUE` | Stored review policies are cleaned automatically on upgrade, so existing configurations continue to load. Rules that also cover DML or VIEW statements are intentionally kept. ### Terraform Impact Move to `terraform-provider-bytebase` 3.20.0 after upgrading the server. See [Migration Guide](https://github.com/bytebase/terraform-provider-bytebase/blob/main/migration/3.20.0.md). | Change | Type | Action | | ------------------------------------------------------------------ | -------- | ------------------------------------------------------------------------------------------------------------- | | `EXPORT_DATA` approval source deprecated | Breaking | Remove `EXPORT_DATA` from approval-flow config; use the JIT data-export flow | | Announcement `level` → `theme` | Breaking | Replace `level` with `theme` (`background` / `text`) under `workspace_profile.announcement` | | Maximum request-expiration field renamed | Breaking | Rename `maximum_role_expiration_in_seconds` to `maximum_request_expiration_in_seconds` in `workspace_profile` | | Environment and issue-label colors moved to color protocol message | Breaking | Update color values to the new color message representation | | Vault `token_type` | Feature | Optionally set `PLAIN`, `ENVIRONMENT`, or `FILE` in `data_sources.external_secret` | | Review-config state diff / webhook URL in state | Fix | No action required | | Seven SQL review rules removed | Breaking | Remove the removed rule types (e.g. `STATEMENT_MAXIMUM_LIMIT_VALUE`) from your review config | ## ⚙️ Install and Upgrade * [Fresh install](/get-started/self-host/deploy-with-docker) * [Upgrade](/get-started/self-host/upgrade) *Before upgrading: 1) Back up the [metadata](/get-started/self-host/upgrade#back-up-and-restore) — in-place downgrade is not supported. 2) Do not run multiple containers on the same data directory. 3) Terraform users: upgrade Bytebase server first, then apply the new Terraform config.* # Bytebase 3.20.1 - Jul 16, 2026 Source: https://docs.bytebase.com/changelog/bytebase-3-20-1 Improvements and Bug Fixes ## 🎄 Enhancements * **Plan deployment** — Enhance the deploy section to show task execution details inline, without opening a separate detail panel. * Faster approval checks for multiple database changes, improved access-grants filtering, and `LIMIT`-aware row estimation. * **Maximum role expiration** — Restore the workspace setting that was renamed to **Maximum request expiration** in 3.20.0. The two are now separate settings: **Maximum role expiration** caps how long a requested role can last, while **Maximum request expiration** caps just-in-time access requests. For Terraform configuration and migration, see the [Migration Guide](https://github.com/bytebase/terraform-provider-bytebase/blob/main/migration/3.20.3.md). ## 🐞 Bug Fixes * Improve SQL Editor autocompletion and fix the high CPU usage it caused when entering or editing large multi-statement sheets. * Fix stale syntax-error markers lingering in the SQL Editor after a statement became valid. * Fix approval flow rule validation. * Show the actor on bypass-and-deploy activity entries instead of an anonymous entry. * Fix assorted UI issues: SQL Editor URL redirects and query-result display, schema editor preview for new tables, custom branding not refreshing, overly long statements slowing the page, and the initial setup flow. * **PostgreSQL** — Keep `public` in the session search path when a schema is selected in the SQL Editor, so extension functions installed in `public` resolve again. * **Oracle** — Fix schema sync failing when a view definition contains invalid UTF-8. * **MongoDB** — Render doubles in mongosh notation instead of scientific notation across query results, the detail panel, copied values, and exports. ## ⚙️ Install and Upgrade * [Fresh install](/get-started/self-host/deploy-with-docker) * [Upgrade](/get-started/self-host/upgrade) *Before upgrading: 1) Back up the [metadata](/get-started/self-host/upgrade#back-up-and-restore) — in-place downgrade is not supported. 2) Do not run multiple containers on the same data directory. 3) Terraform users: upgrade Bytebase server first, then apply the new Terraform config.* # Bytebase 3.21.0 - Jul 30, 2026 Source: https://docs.bytebase.com/changelog/bytebase-3-21-0 Label-driven Approvals and MySQL State-based GitOps ## 🔔 Notable Changes * **Label-driven approvals and draft-time issue labels** — Previously, issue labels could only be added at the Ready for Review step or after a plan was already in review — a real bottleneck for workspaces that require labels on issues. You can now add issue labels while drafting the plan, and Custom Approval conditions can reference them, opening up label-based routing of reviews (e.g., an `incident-mode` label selecting an incident approval path). * **MySQL state-based GitOps workflow (beta)** — Declarative (SDL) schema management now covers MySQL, in beta alongside PostgreSQL: declare the target schema, and Bytebase computes and applies the migration — covering routines, triggers, and events. See [docs](/gitops/state-based-workflow/overview). * **API breaking changes** — Several API methods moved to new services or were removed; REST paths are mostly unchanged and legacy request formats are converted transparently. gRPC/Connect callers and OpenAPI-generated SDKs should review before upgrading. See [change details](#api-breaking-changes). ## 🚀 Features * **SQL Editor** — Query histories and worksheets can now be listed across users and projects through new List APIs — for admins and integrations that need more than their own items — while the Search APIs stay scoped to the calling user (query histories: new `bb.queryHistories.list` permission, granted to workspace admins and project owners; worksheets: new `bb.worksheets.list` permission, not granted to any predefined role). * **Bytebase Cloud** — A new step-by-step setup guide walks new workspaces from creating a project through connecting an instance to running the first query. * **Self-hosted (Helm)** — Pin the Bytebase container image by digest. ## 🎄 Enhancements * Smoother first-run setup: when the workspace has no instance yet, you can create an instance directly from an empty project, and the databases found on that instance's first sync are assigned to that project. Connection failures now explain what went wrong. * **SQL review** — The maximum SQL file size is now a configurable built-in rule: set the level (warning/error) and the size limit (default 2MB). * **Plan detail** — Navigation is steadier: switching between changes, stages, and tasks preserves page context and works with browser history, and each change is labeled by its target databases rather than a generic label. * List pages keep your filters and scroll position across browser navigation, including paginated dashboards. * **SQL Editor** — Search within a result's detail view, and code assistance reconnects automatically after a disconnect. * **GitOps** — Active VCS user tracking now covers Azure DevOps pull-request authors. * **SQL parsing** — Broader SQL compatibility across PostgreSQL, Oracle, SQL Server, and Trino — notably, modern `pg_dump`/`pg_dumpall` scripts with psql meta-commands and inline `COPY` data now work in change workflows. * **PostgreSQL** — Schema sync, SDL dump, and Sync Schema now support composite types (`CREATE TYPE … AS`). * **Oracle** — Prior backup now deduplicates rows by `ROWID`, enabling multi-statement backups on tables with `CLOB`/`BLOB`/`NCLOB` columns; tables with unsupported `LONG`/`LONG RAW` columns are flagged before execution. ## 🐞 Bug Fixes * Skipped rollout tasks are now terminal — they no longer reappear as actionable or can be run after being skipped. * Fix sign-in and navigation traps: SSO logins no longer ask for a local password reset, the project selector no longer lands on a permission-denied page, and the SQL Editor no longer loops on a 403 after a session change. * Webhook fixes: Discord notifications are no longer delivered three times, and `PIPELINE_COMPLETED` now reports the completed stage's environment instead of always the first stage's. * Fix assorted UI issues: CosmosDB container selection loss after running a query, SQL Editor result-panel display, and disabled batch-action tooltips. * **Oracle** — Fix schema sync failing on malformed non-UTF-8 metadata in any schema object (extends the 3.20.1 view fix), and fix PL/SQL package bodies with block comments being split incorrectly, which sent trailing SQL\*Plus `/` delimiters to the database. ## 🏗️ Terraform * Upgrade to provider **3.21.0** after upgrading the server — [Migration Guide](https://github.com/bytebase/terraform-provider-bytebase/blob/main/migration/3.21.0.md). **Optional:** Spanner and BigQuery data sources take explicit `project_id`/`instance_id` (legacy `host` values keep working and convert automatically), and Vault external secrets support TLS configuration (`vault_ssl_ca`, `vault_ssl_cert`, `vault_ssl_key`, `skip_vault_tls_verification`). ## 📃 Change Details ### API Breaking Changes **1. Spanner and BigQuery data sources: `host` replaced by `project_id` / `instance_id`** The GCP resource identity moves out of the overloaded `host` field into dedicated `DataSource.project_id` / `instance_id` fields; `host` and `port` are now an optional Google API endpoint override (e.g. a Private Service Connect endpoint like `spanner-nonprod.p.googleapis.com`). | Engine | Before | After | | -------- | ------------------------------------------ | ----------------------------------------------- | | Spanner | `host: "projects//instances/"` | `project_id: ""`, `instance_id: ""` | | BigQuery | `host: ""` | `project_id: ""` | Affected routes: consumers of `InstanceService.GetInstance` / `ListInstances` that parse `host` — read `project_id` / `instance_id` instead. Writes in the legacy format are transparently converted, so existing API callers and Terraform configs keep working; `UpdateDataSource` accepts `project_id` / `instance_id` in `update_mask`. Existing data sources are migrated automatically on upgrade. **2. `SearchQueryHistories` moved to a project-scoped route, `parent` required** | | Before | After | | ------------------------------------ | -------------------------------- | ---------------------------------------------------- | | REST | `POST /v1/queryHistories:search` | `POST /v1/{parent=projects/*}/queryHistories:search` | | `SearchQueryHistoriesRequest.parent` | absent (searched all projects) | required; `projects/{project}` | Affected routes: callers of the old path get 404; Connect/gRPC callers omitting `parent` get `InvalidArgument`. The CEL `project ==` filter still works and composes with `parent`. **3. Changelog APIs moved to `ChangelogService`** `ListChangelogs` and `GetChangelog` moved from `DatabaseService` to the new `ChangelogService`. The old gRPC/Connect method names are removed; REST paths, permissions, and payloads are unchanged. | Method | Before | After | | --------------------- | ------------------------------------------------- | -------------------------------------------------- | | List / get changelogs | `DatabaseService.ListChangelogs` / `GetChangelog` | `ChangelogService.ListChangelogs` / `GetChangelog` | Affected routes: gRPC/Connect callers of the old method names, and OpenAPI-generated SDKs (operationIds are now `ChangelogService_*`). REST callers need no changes. **4. Query history APIs moved to `QueryHistoryService` (deprecation)** `SearchQueryHistories` and `GetQueryHistory` moved from `SQLService` to the new `QueryHistoryService`, which also hosts the new `ListQueryHistories` List API. The old `SQLService` method names keep working as deprecated aliases and will be removed in a future release. Permissions and payloads are unchanged; REST paths are unchanged except `SearchQueryHistories`, whose route moved — see item 2. | Method | Before (deprecated) | After | | ------------- | ----------------------------------------------------- | -------------------------------------------------------------- | | Query history | `SQLService.SearchQueryHistories` / `GetQueryHistory` | `QueryHistoryService.SearchQueryHistories` / `GetQueryHistory` | Affected routes: migrate gRPC/Connect callers before the aliases are removed; OpenAPI operationIds are now `QueryHistoryService_*`. REST callers need no changes. **5. `SQLService.DiffMetadata` replaced by `DatabaseService.DiffMetadata`** The anonymous `SQLService.DiffMetadata` and its REST path are removed. `DatabaseService.DiffMetadata` reads the source schema from the database resource — send only the database `name` and the complete `target_metadata` — and requires the new `bb.databases.diffMetadata` permission (granted to workspace admin/DBA and project owner/developer; add it to custom roles as needed). | | Before | After | | ------- | ---------------------------------------------- | -------------------------------------------------------------- | | Method | `SQLService.DiffMetadata` | `DatabaseService.DiffMetadata` | | REST | `POST /v1/schemaDesign:diffMetadata` | `POST /v1/{name=instances/*/databases/*}:diffMetadata` | | Request | `source_metadata`, `target_metadata`, `engine` | `name`, `target_metadata` (source and engine read server-side) | | Auth | None (anonymous) | IAM, `bb.databases.diffMetadata` | Affected routes: any caller of the old method or REST path; update the request shape and grant the permission. **6. `SQLService.AICompletion` removed — use `AIService.Chat`** | | Before | After | | ------ | ------------------------- | ---------------- | | Method | `SQLService.AICompletion` | `AIService.Chat` | Affected routes: any caller of `AICompletion`; the chat request is a superset of the old request. This also fixes AI dynamic suggestions failing on workspaces using the Claude provider. **7. `SQLITE` removed from the `Engine` enum (cleanup)** The console has not offered SQLite for years; this removes the leftover enum value and driver. Remove any remaining `SQLITE` references (API callers or Terraform `engine` values). **8. `Instance.sync_databases` reshaped** | Field | Before | After | | ------------------------- | ------------------------------------ | ---------------------------------------------------------------------------------------------------------- | | `Instance.sync_databases` | `repeated string` (empty = sync all) | `SyncDatabases` message (unset = sync all; set with an explicit list = sync exactly those, including none) | Affected routes: `InstanceService.CreateInstance` / `UpdateInstance` callers using this field. This also fixes selecting *zero* databases to sync — previously an empty selection synced everything. **9. `has_issue` Plan list filter removed** Affected routes: `PlanService.ListPlans` / `SearchPlans` callers still sending `has_issue` receive `InvalidArgument`. ## ⚙️ Install and Upgrade * [Fresh install](/get-started/self-host/deploy-with-docker) * [Upgrade](/get-started/self-host/upgrade) *Before upgrading: 1) Back up the [metadata](/get-started/self-host/upgrade#back-up-and-restore) — in-place downgrade is not supported. 2) Do not run multiple containers on the same data directory. 3) Terraform users: upgrade Bytebase server first, then apply the new Terraform config.* # Bytebase 3.3.0 - Jan 16, 2025 Source: https://docs.bytebase.com/changelog/bytebase-3-3-0 SOC 2 ## 🛡️ SOC 2 We have obtained SOC 2 Type 1 certification. This certification validates our commitment to maintaining the highest standards of security, availability, and confidentiality in our database DevSecOps platform. To request our SOC 2 Type 1 report, please [email us](mailto:sales@bytebase.com). ## 🔔 Notable Changes * Simplify data masking, both UI and API are affected, check [docs](/security/data-masking/overview/) and [API examples](https://github.com/bytebase/example-database-security/tree/main/masking) for details. Behavior changes: * Global masking takes precedence over column masking. Previously, column masking takes precedence over global masking. * Eliminate masking levels. For both global masking and column masking, you now configure the associated semantic type. Semantic type in turn maps to the corresponding masking algorithm. ## 🚀 New Features * Introduce **Release**, the new versioned database schema migration workflow. Create releases, which are versioned migration scripts bundled as a package to deploy, tell Bytebase which databases to deploy, and Bytebase will figure out which scripts to execute or skip if already executed. * Support CosmosDB. * Add project setting to allow users to self-approval their own issues. * Schema Synchronization supports PostgreSQL materialized view. * Preliminary support PostgreSQL 17. ## 🎄 Enhancements * Copy query results by selected columns or rows in SQL Editor. * Enhance Oracle data backup experience. * Improve the loading time for the database anomalies page. ## ⚙️ Install and Upgrade * [Fresh install](/get-started/self-host/deploy-with-docker) * [Upgrade](/get-started/self-host/upgrade) *Before upgrading: 1) Back up the [metadata](/get-started/self-host/upgrade#back-up-and-restore) — in-place downgrade is not supported. 2) Do not run multiple containers on the same data directory. 3) Terraform users: upgrade Bytebase server first, then apply the new Terraform config.* # Bytebase 3.3.1 - Jan 30, 2025 Source: https://docs.bytebase.com/changelog/bytebase-3-3-1 Support 1-click DML rollback for Oracle ## 🚀 New Features * Support 1-click DML rollback for Oracle. * Support AppRole method for Vault while connecting to an instance. ## 🎄 Enhancements * In SQL Editor, pressing `Cmd+Enter` will execute the query on the current line where the cursor is located. * Add compatibility with PostgreSQL extension CITUS. * Improve the performance for `GetChangelog` and `GetDatabaseSchema` APIs. ## 🔔 API Changes * Remove `concise` parameter from `GetChangelog` and `GetDatabaseSchema` APIs. ## ⚙️ Install and Upgrade * [Fresh install](/get-started/self-host/deploy-with-docker) * [Upgrade](/get-started/self-host/upgrade) *Before upgrading: 1) Back up the [metadata](/get-started/self-host/upgrade#back-up-and-restore) — in-place downgrade is not supported. 2) Do not run multiple containers on the same data directory. 3) Terraform users: upgrade Bytebase server first, then apply the new Terraform config.* # Bytebase 3.4.0 - Feb 13, 2025 Source: https://docs.bytebase.com/changelog/bytebase-3-4-0 More features moved to community plan ## 💰 Plan Adjustment * The Index Advisor feature is available in the **Community Plan** (free), enabling the suggestion of indexes for slow queries once OpenAI integration is set up. * The SQL Sharing feature is available in the **Community Plan** (free), allowing users to star and share worksheets with their team in the SQL Editor. ## 🔔 Notable Changes * The redundant `key` attribute of project has been removed. ## 🎄 Enhancements * Disallow using sensitive columns in WHERE clause for SQL Server. * Enhance SQL Server data backup experience. * Enhance PostgreSQL data backup experience. * The summary report of task check supports Redshift, including affected rows and statement reports. ## 🐞 Bug Fix * Show the correct timestamp scale in SQL Editor. * Fix SQL Server connection leak problem. ## ⚙️ Install and Upgrade * [Fresh install](/get-started/self-host/deploy-with-docker) * [Upgrade](/get-started/self-host/upgrade) *Before upgrading: 1) Back up the [metadata](/get-started/self-host/upgrade#back-up-and-restore) — in-place downgrade is not supported. 2) Do not run multiple containers on the same data directory. 3) Terraform users: upgrade Bytebase server first, then apply the new Terraform config.* # Bytebase 3.4.1 - Feb 27, 2025 Source: https://docs.bytebase.com/changelog/bytebase-3-4-1 Terraform provider manages more resources ## 🎄 Enhancements * Terraform provider manages more resources. * Support managing custom roles, semantic types, and database resources. * Support assigning instance licenses. * Support external secrets for instances. * Allow retrying online migration tasks. * Enable the following flags for online migration: `throttle-control-replicas` and `attempt-instant-ddl`. * Improve the automatic quotation feature in the SQL editor. * Consolidate the setting update UX for workspace, instance, environment, and project. ## 🔔 Notable Changes * External Approval Deprecated: Users should now use the Bytebase API for managing approvals and rollouts. * MySQL online migration sync and cutover tasks have been consolidated into a single task. As a prerequisite, you should first create a `bbdataarchive` database. * HTTP API update: Implemented strict request parsing, rejecting unknown fields. Response serialization now excludes unset values. ## ⚙️ Install and Upgrade * [Fresh install](/get-started/self-host/deploy-with-docker) * [Upgrade](/get-started/self-host/upgrade) *Before upgrading: 1) Back up the [metadata](/get-started/self-host/upgrade#back-up-and-restore) — in-place downgrade is not supported. 2) Do not run multiple containers on the same data directory. 3) Terraform users: upgrade Bytebase server first, then apply the new Terraform config.* # Bytebase 3.5.0 - Mar 13, 2025 Source: https://docs.bytebase.com/changelog/bytebase-3-5-0 Azure IAM Auth for Cosmos DB & Azure SQL ## 💰 Plan Adjustment * Refine the environment rollout policy by setting manual rollout as the default option, now available in the Community Plan (Free). ## 🔔 Notable Changes * Deprecated slow query feature. * To simplify the deployment process, the custom deployment configuration feature has been removed. Deployments will now follow the straightforward order of your defined environments. For canary deployments, create a separate `canary` environment. Then, point your database environment to this canary environment. * Deprecated multitenancy feature. * Deprecated built-in GitOps feature], now you can use [GitOps in a new way](https://www.bytebase.com/docs/tutorials/gitops-github-workflow/). * Removed custom project webhook endpoint. * API support pagination and filter. * ListProjects support `page_size` and `page_token`. [Doc](https://api.bytebase.com/#tag/projectservice/GET/v1/projects) * SearchProjects support filter by `name` and `resource_id`. [Doc](https://api.bytebase.com/#tag/projectservice/POST/v1/projects:search) * ListDatabases support filter by `environment/project/instance/name/engine/label/exclude_unassigned`. [Doc](https://api.bytebase.com/#tag/databaseservice/GET/v1/instances/\{instance}/databases) * ListUsers support filter by `name/email/user_type/state`. [Doc](https://api.bytebase.com/#tag/userservice/GET/v1/users) These API changes bring huge UI change and influence many features: * Will NOT list all projects anymore, users must fetch projects per page, or search projects by name/resource id. (For the project table & selector) * Will NOT list all databases anymore, users must fetch databases per page, or search databases by filter. (Almost all features are affected as long as it's using the database, for example, issue, release, rollout, database group, masking, SQL editor) * Will NOT list all users anymore, users must fetch users per page, or search users by name/email. (Many features are affected as long as it's using the user, for example, issue, release, group, member/IAM) ## 🚀 New Features * Support Azure IAM Auth for Cosmos DB & Azure SQL. * Sync schema for CockroachDB (PostgreSQL compatible syntax supported only). * Support configuring custom maximum query time in SQL Editor (Workspace Settings -> General -> Security -> Maximum query time). * Support webhook direct message for DingTalk. ## 🎄 Enhancements * View affected rows of SQL check result before issue creation. * Adjust SQL rule `Disallow setting volatile default value on columns` (PostgreSQL) to apply only to existing tables and the `ALTER TABLE ADD COLUMN` operation. * Extend API to allow updating the plan for database group deployment. ## ⚙️ Install and Upgrade * [Fresh install](/get-started/self-host/deploy-with-docker) * [Upgrade](/get-started/self-host/upgrade) *Before upgrading: 1) Back up the [metadata](/get-started/self-host/upgrade#back-up-and-restore) — in-place downgrade is not supported. 2) Do not run multiple containers on the same data directory. 3) Terraform users: upgrade Bytebase server first, then apply the new Terraform config.* # Bytebase 3.5.1 - Mar 27, 2025 Source: https://docs.bytebase.com/changelog/bytebase-3-5-1 Added Cosmos DB SQL filtering and projection support in SQL Editor ## 🚀 New Features * Support `WHERE` clauses and `SELECT` projections in Cosmos DB SQL syntax within SQL Editor. * Support connecting to OpenSearch using the Elasticsearch client library. ## 🎄 Enhancements * Expanded connection parameter support for PostgreSQL, MySQL, Microsoft SQL Server, and Oracle databases. * Optimized PostgreSQL table deletion by eliminating redundant DROP statements for constraints and indexes. * Support server-side identifier case sensitivity for OceanBase. * Enhanced binary data visualization. * Updated formatting options for boolean, binary, hexadecimal, and text representations. * Implemented customizable binary display preferences at both column and individual cell levels. ## ⚙️ Install and Upgrade * [Fresh install](/get-started/self-host/deploy-with-docker) * [Upgrade](/get-started/self-host/upgrade) *Before upgrading: 1) Back up the [metadata](/get-started/self-host/upgrade#back-up-and-restore) — in-place downgrade is not supported. 2) Do not run multiple containers on the same data directory. 3) Terraform users: upgrade Bytebase server first, then apply the new Terraform config.* # Bytebase 3.5.2 - Apr 10, 2025 Source: https://docs.bytebase.com/changelog/bytebase-3-5-2 Support Cassandra in SQL Editor ## 🚀 New Features * Support Cassandra in SQL Editor. ## 🔔 Notable Changes * List instance API returns at most 10 instances instead of all by default. The API requires using pagination parameter to fetch more. [Doc](https://api.bytebase.com/#tag/instanceservice/GET/v1/instances) * The statement field in Release API will be base64-encoded string. ## 🎄 Enhancements * Support displaying triggers for Oracle in SQL Editor. * Improve Elasticsearch parser accuracy. * Binary data in SQL editor now copies in its displayed format instead of the raw value. * Support "View Schema Text" for views in SQL Editor. * Service account emails now use workspace domain (if set) instead of default `@service.bytebase.com`. ## ⚙️ Install and Upgrade * [Fresh install](/get-started/self-host/deploy-with-docker) * [Upgrade](/get-started/self-host/upgrade) *Before upgrading: 1) Back up the [metadata](/get-started/self-host/upgrade#back-up-and-restore) — in-place downgrade is not supported. 2) Do not run multiple containers on the same data directory. 3) Terraform users: upgrade Bytebase server first, then apply the new Terraform config.* # Bytebase 3.6.0 - Apr 24, 2025 Source: https://docs.bytebase.com/changelog/bytebase-3-6-0 SOC 2 Type 2 Bytebase has earned [SOC 2 Type 2 certification](https://www.bytebase.com/blog/soc2-type2/) building on our earlier [Type 1 certification](https://www.bytebase.com/blog/soc2-type1) acquisition. ## 🚀 New Features * Add basic support for Trino database. * Support syncing [OIDC IdP groups](https://www.bytebase.com/docs/administration/sso/oidc/#group-syncing) to Bytebase groups via the groups claim. * Support Cassandra for executing statements in SQL editor and executing issues. * Able to automatically retry PostgreSQL lock timeouts. ## 🔔 Notable Changes * API change: * Remove `EnvironmentService`. The environment setting now resides in `SettingService`. `bb.environments.*` permissions are removed from custom roles. You can grant `bb.settings.update` to a role to allow it to update the environment setting. * Simplify `ApprovalNode`. Remove the predefined `groupValue` enum, only `role` is allowed. * Deprecate Anomaly Center. Schema drift is now visible in both the database list and on the database page. ## 🎄 Enhancements * Update Docker base image to Alpine to improve security posture. * Support filtering tasks by status in UI when the issue involves more than 10 databases. * Add support for `EXPLAIN` in MSSQL. * Support more read-only commands in SQL Editor for Redis. * Improve columns auto-completion in FROM clause for MSSQL. * Fix PostgreSQL infinity date display issue. * Hide dependent objects in SQL Editor. * Remove `FILTER_PREDICATES` from `EXPLAIN` query for Oracle. ## ⚙️ Install and Upgrade * [Fresh install](/get-started/self-host/deploy-with-docker) * [Upgrade](/get-started/self-host/upgrade) *Before upgrading: 1) Back up the [metadata](/get-started/self-host/upgrade#back-up-and-restore) — in-place downgrade is not supported. 2) Do not run multiple containers on the same data directory. 3) Terraform users: upgrade Bytebase server first, then apply the new Terraform config.* # Bytebase 3.6.1 - May 08, 2025 Source: https://docs.bytebase.com/changelog/bytebase-3-6-1 DML rollback supports unique indexes for major databases ## 🚀 New Features * DML rollback supports unique indexes for MySQL, PostgreSQL, Oracle and MSSQL. * Improve wizard for initial workspace creation. * Introduce **Parallel tasks per rollout** and **CI Data Sampling Size** options in project settings. ## 🎄 Enhancements * Allow removal of any roles — including `workspace member` — from any workspace user, including `allUser`. * Support setting custom expiration days when granting project roles. ## 🐞 Bug Fix * Fix PostgreSQL schema autocomplete for names with special characters. ## ⚙️ Install and Upgrade * [Fresh install](/get-started/self-host/deploy-with-docker) * [Upgrade](/get-started/self-host/upgrade) *Before upgrading: 1) Back up the [metadata](/get-started/self-host/upgrade#back-up-and-restore) — in-place downgrade is not supported. 2) Do not run multiple containers on the same data directory. 3) Terraform users: upgrade Bytebase server first, then apply the new Terraform config.* # Bytebase 3.6.2 - May 23, 2025 Source: https://docs.bytebase.com/changelog/bytebase-3-6-2 Enhance SQL Editor Experience ## 🚀 New Features * Enhance SQL Editor experience. * Support database group in the batch query. * Store the query result history. The history is only stored in the browser locally. * Lazy query. Only execute the query request on-demand when users click the tab. * Display SQL Server trigger body in SQL Editor. ## 🎄 Enhancements * Implement semantic versioning for GitOps. ## ⚙️ Install and Upgrade * [Fresh install](/get-started/self-host/deploy-with-docker) * [Upgrade](/get-started/self-host/upgrade) *Before upgrading: 1) Back up the [metadata](/get-started/self-host/upgrade#back-up-and-restore) — in-place downgrade is not supported. 2) Do not run multiple containers on the same data directory. 3) Terraform users: upgrade Bytebase server first, then apply the new Terraform config.* # Bytebase 3.7.0 - Jun 05, 2025 Source: https://docs.bytebase.com/changelog/bytebase-3-7-0 Support Batch Export ## 🚀 New Features * Support batch data export from multiple databases or a database group via issue. * Support batch data export from limited databases in the SQL Editor. * Support requesting any project roles. * Support displaying `CHECK` constraints in the SQL Editor. * **Trino** - Support data masking. * **MariaDB** - Support sql\_type in risk condition. * **SQL Server** - Support columnstore index. ## 🔔 Notable Changes ### 💰 Pricing * **Custom approval** is no longer restricted by instance limits. It now only requires an **Enterprise subscription**. Previously, it was limited to licensed instances — now it applies to any instance under an Enterprise plan. ### 🎨 GUI * Remove **Earliest allowed time**. Use **Scheduled time** during rollout instead. * The **gh-ost** and **backup** toggles are now set at the issue level, no longer at the task level. * The **SQL statement** is now defined at the issue level, no longer at the task level. Once a task has started or completed, the statement cannot be updated. ### 🧩 API * Remove `earliest_allowed_time` from `Plan.spec`. Use `run_at` in `BatchRunTasksRequest` instead. * Remove `steps` and `target` from `Plan`. Use `specs` and `targets` respectively. * Remove `PreviewPlan` RPC from `PlanService`. * Update `ChangeDatabaseConfig`: * Remove `BASELINE` from `Type`. * Remove `schema_version`. * Add `release`. * Update setting names, use the correct values as defined in [SettingName](https://github.com/bytebase/bytebase/blob/d950ad46fa88d1ad40299a780b34bd0fa28adc3a/proto/v1/v1/setting_service.proto#L111). ## 🎄 Enhancements * **PostgreSQL** - Issue tickets now honor any supplied `SET search_path`, so subsequent SQL statements (and their DML rollback logic) operate on the specified schema rather than defaulting to public. * **Trino** - Enhance auto-completion support. * **TiDB** - Improve syntax error detection. * **TiDB** - The "require primary key" rule now checks the source table when using `CREATE TABLE LIKE`. ## 🐛 Bug Fixes * **Doris** and **StarRocks** - Resolve compatibility issues. * **RedShift** - Fix connection database parameter issue. ## ⚙️ Install and Upgrade * [Fresh install](/get-started/self-host/deploy-with-docker) * [Upgrade](/get-started/self-host/upgrade) *Before upgrading: 1) Back up the [metadata](/get-started/self-host/upgrade#back-up-and-restore) — in-place downgrade is not supported. 2) Do not run multiple containers on the same data directory. 3) Terraform users: upgrade Bytebase server first, then apply the new Terraform config.* # Bytebase 3.7.1 - Jun 19, 2025 Source: https://docs.bytebase.com/changelog/bytebase-3-7-1 All database change features are now FREE in Community Plan ## 🔔 Notable Changes * All database change features are now **FREE** in **Community Plan**! See full details in [our updated pricing](https://www.bytebase.com/pricing). ## 🎄 Enhancements * Overhaul [docs site](/) with improved navigation, search, and Ask AI. * SQL Editor now highlights the statement under the cursor. * Renovate Review Center with a refreshed UI and better user experience. Review Center routes have changed from `/review-center` to `/plans`. * Streamline SSO creation flow for easier configuration. * **Terraform** - Add support for applying SSL certificates to database connections via the new `use_ssl` field. ## 🐛 Bug Fixes * Database Access Control now properly handles case-sensitive object names. * **MySQL** - Fix an issue when creating procedures via Schema Editor. * **OceanBase** - Fix 2 SQL Review rules: * Enforce maximum number of updated or deleted rows * Enforce maximum number of inserted rows ## ⚙️ Install and Upgrade * [Fresh install](/get-started/self-host/deploy-with-docker) * [Upgrade](/get-started/self-host/upgrade) *Before upgrading: 1) Back up the [metadata](/get-started/self-host/upgrade#back-up-and-restore) — in-place downgrade is not supported. 2) Do not run multiple containers on the same data directory. 3) Terraform users: upgrade Bytebase server first, then apply the new Terraform config.* # Bytebase 3.8.0 - Jul 03, 2025 Source: https://docs.bytebase.com/changelog/bytebase-3-8-0 Greatly improve schema sync/rollback for multiple DB engines ## 🔔 Notable Changes * Greatly improve schema sync/rollback for MySQL/PostgreSQL/TiDB/SQL Server/Oracle compatibility, [supporting most common database objects](https://docs.bytebase.com/change-database/synchronize-schema#supported-objects). * Sunset **Issue Subscription** feature. * Rename **Review Center** to **Plans**. ## 🎄 Enhancements * Add query result row limit support. * Add multi-domain configuration support. * Simplify default value input: No longer distinguish between expressions and values, but unify them into a text input field. * List all approvers on approval workflow, and display details on hover. ## 🐛 Bug Fixes * Fix JSON data type display issue for ClickHouse queries. * Add `use_ssl` field to Terraform SSL configuration. ## ⚙️ Install and Upgrade * [Fresh install](/get-started/self-host/deploy-with-docker) * [Upgrade](/get-started/self-host/upgrade) *Before upgrading: 1) Back up the [metadata](/get-started/self-host/upgrade#back-up-and-restore) — in-place downgrade is not supported. 2) Do not run multiple containers on the same data directory. 3) Terraform users: upgrade Bytebase server first, then apply the new Terraform config.* # Bytebase 3.8.1 - Jul 17, 2025 Source: https://docs.bytebase.com/changelog/bytebase-3-8-1 New Database CI/CD Experience Preview ## 🔔 Notable Changes * **New database CI/CD experience** is available in preview. Enable it from the profile dropdown. * `default_expression`, `default_string`, and `default_null` fields in `ColumnMetadata` are removed from the API. Use the unified `default` field instead. This affects APIs such as `GetDatabaseMetadata` and `GetChangelog`. ## 🚀 New Features * In **SQL Editor** and **audit log**, show masking reason for sensitive data. ## 🎄 Enhancements * Data export files are available for 24 hours with unlimited downloads. * **SQL Server** - Support Spatial Index in schema rollback and synchronization. * **Redshift** - Support turning off transaction wrapping using the `-- txn-mode=off` directive before the SQL statement. * **Terraform** - Support more policies: * rollout policy (for environments) * copy data policy (for environments and projects) * data source query policy (for environments and projects) * See [example configurations](https://github.com/bytebase/terraform-provider-bytebase/blob/main/examples/setup/environment.tf#L41). * Introduce a new 8-part series of [Terraform Provider Tutorials](/integrations/terraform/overview#tutorials). ## ⚙️ Install and Upgrade * [Fresh install](/get-started/self-host/deploy-with-docker) * [Upgrade](/get-started/self-host/upgrade) *Before upgrading: 1) Back up the [metadata](/get-started/self-host/upgrade#back-up-and-restore) — in-place downgrade is not supported. 2) Do not run multiple containers on the same data directory. 3) Terraform users: upgrade Bytebase server first, then apply the new Terraform config.* # Bytebase 3.9.0 - Jul 31, 2025 Source: https://docs.bytebase.com/changelog/bytebase-3-9-0 New database CI/CD experience ## 🌟 New Database CI/CD Experience New database CI/CD experience is now enabled by default. The new layout splits a monolithic page into dedicated tabs for clearer information display and efficient batch change management: * **Overview** tab displays general change details and approval status * **Changes** tab lists database changes and their targets * **Rollout** tab manages the deployment process * In addition to the layout update, the new design also introduces the following new features: * Multi-task rollback. * Plan draft to stage changes. Users can still switch back to the old layout via the profile dropdown in the top-right corner. ## 🔔 Other Notable Changes * Changelist is no longer available in the new CI/CD layout. Switch to the old layout to access it. You can draft a plan in the new CI/CD layout to stage multiple changes. * Remove `--rollout-title` flag from the `bytebase-action rollout` command (titles now inherit from associated plans and cannot be set separately). * Deprecate per-database secret variable. ## 🎄 Enhancements * Add transaction mode directives `--txn-mode=on/off` to toggle transaction wrapping, supporting MySQL/PostgreSQL/Oracle/SQL Server/TiDB/Redshift. * Improve Redshift compatibility. * Add Terraform plugin support for Project webhook settings and Instance external\_secret. ## 🐞 Bug Fix * Fix inconsistent user group display on page refresh. ## ⚙️ Install and Upgrade * [Fresh install](/get-started/self-host/deploy-with-docker) * [Upgrade](/get-started/self-host/upgrade) *Before upgrading: 1) Back up the [metadata](/get-started/self-host/upgrade#back-up-and-restore) — in-place downgrade is not supported. 2) Do not run multiple containers on the same data directory. 3) Terraform users: upgrade Bytebase server first, then apply the new Terraform config.* # Bytebase 3.9.1 - Aug 14, 2025 Source: https://docs.bytebase.com/changelog/bytebase-3-9-1 Support importing database revisions from releases or local files ## 🚀 New Features * Support importing database revisions from releases or local files. * Support reading `PG_URL` from a file with hot-reload. * Support IAM authentication for AWS OpenSearch. ## 🔔 Notable Changes * Service accounts will no longer be able to log in via the Bytebase GUI. ## 🪦 Sunset * Remove the deprecated `--pg` command-line flag. Use the `PG_URL` environment variable for external PostgreSQL connections instead. * Replace the `--disable-metric` flag with a Workspace setting. * Remove the `--disable-sample` flag. ## 🎄 Enhancements * Bytebase GitHub Actions will now try to reuse existing releases before creating new ones. If two releases have the same filenames and file contents, they are considered identical, reducing redundant releases when retrying. * Terraform — Add support to configure masking exceptions for all databases by skipping database, table, and column, applying the exception across the entire project. * Terraform — Add support for the `sync_databases` attribute in the instance resource, allowing you to specify which databases to sync (e.g., `["db1", "db2"]`). ## ⚙️ Install and Upgrade * [Fresh install](/get-started/self-host/deploy-with-docker) * [Upgrade](/get-started/self-host/upgrade) *Before upgrading: 1) Back up the [metadata](/get-started/self-host/upgrade#back-up-and-restore) — in-place downgrade is not supported. 2) Do not run multiple containers on the same data directory. 3) Terraform users: upgrade Bytebase server first, then apply the new Terraform config.* # Bytebase 3.9.2 - Aug 28, 2025 Source: https://docs.bytebase.com/changelog/bytebase-3-9-2 Support hard-delete for projects and instances ## 🚀 New Features * Add support for hard-delete of projects and instances (resources must be archived first). * Add support for adding/removing changes in draft plans. ## 🔔 Notable Changes * Streamline approval by enabling one-click ready for review. * Update `UpdateMask` values in `UpdatePolicy` to observe AIP. Use field names only; oneof type names are omitted. ([PR #17240](https://github.com/bytebase/bytebase/pull/17240)) * **Terraform** - Deprecate `column`, `member` and `action` field in the `masking_exception_policy`, use `columns`, `members` and `actions` instead. Example: [data\_masking.tf](https://github.com/bytebase/terraform-provider-bytebase/blob/main/examples/setup/data_masking.tf#L113) ## 🎄 Enhancements * Add support for empty environments in instance and database resources. * Enhance SQL Editor performance. * **Entra ID** - Normalize SCIM user emails to lowercase for authentication. * **PostgreSQL** - Add support for `CREATE RULE`. * **API** - Add support for pagination and filtering for group API. * **Terraform** - Add support for `raw_expression` in the `masking_exception_policy`. Once provided, it is used as the CEL expression, and all other ( `database`, `schema`, `table`, `columns`, `expire_timestamp`) fields are ignored. * **Terraform** - Add support for empty environments in instance and database resources. * **Terraform** - Add support for filters in group list data sources. ## ⚙️ Install and Upgrade * [Fresh install](/get-started/self-host/deploy-with-docker) * [Upgrade](/get-started/self-host/upgrade) *Before upgrading: 1) Back up the [metadata](/get-started/self-host/upgrade#back-up-and-restore) — in-place downgrade is not supported. 2) Do not run multiple containers on the same data directory. 3) Terraform users: upgrade Bytebase server first, then apply the new Terraform config.* # FAQ Source: https://docs.bytebase.com/faq ## How to reach us? * Sales & purchasing inquiries: [contact us](https://www.bytebase.com/contact-us) * Enterprise support (requires an active Enterprise contract): [help@bytebase.com](mailto:help@bytebase.com) * Community technical support: open a [GitHub Issue](https://github.com/bytebase/bytebase/issues) ## Supported languages English, 简体中文, Español, 日本語, Tiếng việt ## Certifications SOC 2 Type II, HIPAA. # Cloud Source: https://docs.bytebase.com/get-started/cloud [Bytebase Cloud](https://cloud.bytebase.com/) is a fully managed Database DevSecOps platform. Get started instantly without installing or managing any infrastructure. ## Key Features * **Instant Setup**: Ready in seconds * **Fully Managed**: We handle updates, backups, and monitoring * **Enterprise Ready**: SOC 2 compliant with built-in security * **Reliable Infrastructure**: Hosted on Google Cloud ## Prerequisites To connect your databases to Bytebase Cloud, add this IP address to your database firewall: **Required IP address:** `34.27.188.162` topology ## Create Your Workspace ### Step 1: Sign In Go to [cloud.bytebase.com](https://cloud.bytebase.com/) and sign in with Google, Microsoft, GitHub, Email. ### Step 2: Create Workspace Click **Create workspace**. You get one workspace per account. ### Step 3: Access Your Workspace Your workspace is usually ready in seconds. You'll be redirected automatically. If it takes longer, check your email for: * Workspace URL * Login credentials ## Add Your Team ### Add Members Only admins can add new members: 1. Go to **Settings** → **Members** 2. Click **Add Member** 3. Enter their email and choose a role ### Set Up Passwords After adding someone: 1. Click their name 2. Click **Set Password** 3. Create a temporary password 4. Send it to them securely ## Next Steps Add database instance Organize databases Configure access control Connect VCS (GitHub, GitLab) ## Need Help? * Check your workspace URL is correct - Clear browser cache - Try incognito mode Reset it at [cloud.bytebase.com](https://cloud.bytebase.com) Check that you've added the required IP address to your database firewall (see Prerequisites above) # AWS Database Connections Source: https://docs.bytebase.com/get-started/connect/aws Learn how to configure secure connections to AWS-managed databases using IAM authentication, Secrets Manager, and best practices for production deployments. ## Prerequisites: IAM Role Setup Use attached IAM roles for secure, key-free authentication on EC2 instances. This eliminates the need to manage access keys. References: [IAM roles for EC2](https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/iam-roles-for-amazon-ec2.html) | [IAM best practices](https://docs.aws.amazon.com/IAM/latest/UserGuide/best-practices.html) | [Using instance profiles](https://docs.aws.amazon.com/IAM/latest/UserGuide/id_roles_use_switch-role-ec2_instance-profiles.html) ### Create IAM Role 1. Go to [IAM Console → Roles](https://console.aws.amazon.com/iam/home#/roles) 2. Click **Create role** 3. Select trusted entity type: **AWS service** → **EC2** 4. Attach policies as needed: * For RDS IAM authentication - see [RDS/Aurora section](#rdsaurora-with-iam-authentication) * For Secrets Manager access - see [AWS Secrets Manager section](#aws-secrets-manager) 5. Name the role: `bytebase-instance-role` ### Attach IAM Role to EC2 **New EC2 Instance:** 1. Launch instance in [EC2 Console](https://console.aws.amazon.com/ec2/) 2. In **Advanced details** → **IAM instance profile**: Select `bytebase-instance-role` **Existing EC2 Instance:** 1. Select instance → **Actions** → **Security** → **Modify IAM role** 2. Select `bytebase-instance-role` → **Update IAM role** Deploy Bytebase on your EC2 instance - credentials are provided automatically through the instance metadata service. ### Alternative: IAM User with Access Keys Use only when running Bytebase outside AWS. See [why to use IAM roles instead of access keys](https://docs.aws.amazon.com/IAM/latest/UserGuide/id_credentials_access-keys.html#access-keys-alternatives). 1. Create an IAM user with required policies 2. Generate access keys 3. Set environment variables: ```bash theme={null} -e AWS_ACCESS_KEY_ID=xxx -e AWS_SECRET_ACCESS_KEY=yyy -e AWS_REGION=us-east-1 ``` Reference: [Managing access keys](https://docs.aws.amazon.com/IAM/latest/UserGuide/id_credentials_access-keys.html) ## RDS/Aurora with IAM Authentication Prerequisites: [IAM role](#prerequisites-iam-role-setup) with RDS connect permissions. References: [IAM database authentication](https://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/UsingWithRDS.IAMDBAuth.html) | [Connecting with IAM](https://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/UsingWithRDS.IAMDBAuth.Connecting.html) ### Step 1: Configure RDS/Aurora Instance 1. In [RDS Console](https://console.aws.amazon.com/rds/), modify your instance 2. Enable **IAM database authentication** under Database authentication options 3. Save changes (SSL is enabled by default) Reference: [Enabling IAM authentication](https://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/UsingWithRDS.IAMDBAuth.Enabling.html) ### Step 2: Grant Database Connect Permission Add this policy to your IAM role to allow RDS IAM authentication: ```json theme={null} { "Version": "2012-10-17", "Statement": [ { "Effect": "Allow", "Action": "rds-db:connect", "Resource": "arn:aws:rds-db:REGION:ACCOUNT_ID:dbuser:DB_RESOURCE_ID/bytebase" } ] } ``` Replace `REGION`, `ACCOUNT_ID`, and `DB_RESOURCE_ID` with your values. Find DB\_RESOURCE\_ID in RDS console → Configuration tab. For easier setup, you can use wildcards: `arn:aws:rds-db:*:*:dbuser:*/*` Reference: [IAM policy examples](https://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/UsingWithRDS.IAMDBAuth.IAMPolicy.html) ### Step 3: Create Database User **MySQL/Aurora MySQL:** ```sql theme={null} CREATE USER 'bytebase'@'%' IDENTIFIED WITH AWSAuthenticationPlugin AS 'RDS'; ALTER USER 'bytebase'@'%' REQUIRE SSL; GRANT ALL PRIVILEGES ON *.* TO 'bytebase'@'%'; ``` **PostgreSQL/Aurora PostgreSQL:** ```sql theme={null} CREATE USER bytebase; GRANT rds_iam TO bytebase; -- Grant appropriate permissions ``` References: [MySQL](https://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/UsingWithRDS.IAMDBAuth.DBAccounts.html#UsingWithRDS.IAMDBAuth.DBAccounts.MySQL) | [PostgreSQL](https://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/UsingWithRDS.IAMDBAuth.DBAccounts.html#UsingWithRDS.IAMDBAuth.DBAccounts.PostgreSQL) ### Step 4: Connect from Bytebase 1. Click **New Instance** in Bytebase 2. Configure basic connection: * **Host:** Your RDS endpoint * **Port:** 3306 (MySQL) or 5432 (PostgreSQL) * **Username:** `bytebase` * **Authentication:** Select `AWS RDS IAM` 3. Configure AWS credentials: * **Credential Source:** Select **Default** (recommended for same-account connections) * Automatically uses EC2 instance profile or environment variables * **Database Region:** Select your RDS region (e.g., `us-east-1`) * **Database:** Your database name 4. Test and save the connection For same-account connections on EC2, always use **Default** credential source. Bytebase automatically uses the instance profile to authenticate. Use **Specific Credentials** only for cross-account scenarios. ## Cross-Account IAM Authentication Available in Bytebase version 3.12.1 and later Connect to RDS databases in different AWS accounts using IAM role assumption. This allows Bytebase running in Account A to authenticate to databases in Accounts B, C, D, etc. ### Prerequisites * Bytebase running with an IAM role (EC2 instance profile or ECS task role) * Target RDS instances have IAM authentication enabled * Cross-account trust relationships configured ### Step 1: Create Target Account Role In each target AWS account (where databases reside): 1. Go to [IAM Console → Roles](https://console.aws.amazon.com/iam/home#/roles) 2. Click **Create role** 3. Select trusted entity: **Another AWS account** 4. Enter the source account ID (where Bytebase runs) 5. Optionally add an External ID for additional security 6. Name the role: `bytebase-target-db-role` 7. After creating the role, add an inline policy for RDS access: * Go to the **Permissions** tab * Click **Add permissions** → **Create inline policy** * Switch to **JSON** view and paste: ```json theme={null} { "Version": "2012-10-17", "Statement": [ { "Effect": "Allow", "Action": "rds-db:connect", "Resource": "arn:aws:rds-db:REGION:ACCOUNT_ID:dbuser:DB_RESOURCE_ID/bytebase" } ] } ``` For production, replace with specific values. Find your DB\_RESOURCE\_ID in RDS Console → your database → Configuration tab 8. Note the role ARN: `arn:aws:iam::TARGET_ACCOUNT:role/bytebase-target-db-role` ### Step 2: Configure Trust Relationship (Target Account) In the target account, configure the trust policy for `bytebase-target-db-role`: 1. Go to the role in IAM Console 2. Select **Trust relationships** tab 3. Click **Edit trust policy** 4. Update with this policy (replace with your source account details): ```json theme={null} { "Version": "2012-10-17", "Statement": [ { "Effect": "Allow", "Principal": { "AWS": "arn:aws:iam::SOURCE_ACCOUNT:role/bytebase-instance-role" }, "Action": "sts:AssumeRole", "Condition": { "StringEquals": { "sts:ExternalId": "your-external-id" // Optional but recommended } } } ] } ``` ### Step 3: Grant AssumeRole Permission (Source Account) In the source account (where Bytebase runs), add permission to assume the target role: 1. Go to your `bytebase-instance-role` in IAM Console 2. Click **Add permissions** → **Create inline policy** 3. Switch to **JSON** view and paste: ```json theme={null} { "Version": "2012-10-17", "Statement": [ { "Effect": "Allow", "Action": "sts:AssumeRole", "Resource": [ "arn:aws:iam::ACCOUNT_B:role/bytebase-target-db-role", "arn:aws:iam::ACCOUNT_C:role/bytebase-target-db-role" ] } ] } ``` 4. Name the policy: `AssumeTargetRoles` 5. Replace `ACCOUNT_B`, `ACCOUNT_C` with your target account IDs ### Step 4: Configure Database User Ensure the target RDS instance has: 1. **IAM authentication enabled** (RDS Console → Modify → Database authentication options) 2. **A database user configured for IAM auth** **For PostgreSQL:** ```sql theme={null} CREATE USER bytebase; GRANT rds_iam TO bytebase; -- Grant necessary permissions ``` **For MySQL:** ```sql theme={null} CREATE USER 'bytebase'@'%' IDENTIFIED WITH AWSAuthenticationPlugin AS 'RDS'; ALTER USER 'bytebase'@'%' REQUIRE SSL; -- Grant necessary permissions ``` ### Step 5: Configure Cross-Account Connection 1. Click **New Instance** in Bytebase 2. Configure basic connection: * **Host:** RDS endpoint in target account * **Port:** 3306 (MySQL) or 5432 (PostgreSQL) * **Username:** `bytebase` * **Authentication:** Select `AWS RDS IAM` 3. Configure AWS credentials: * **Credential Source:** Select **Specific Credentials** (required for cross-account) * **Access Key ID:** Leave empty (uses EC2 instance profile) * **Secret Access Key:** Leave empty (uses EC2 instance profile) * **Session Token:** Leave empty * **Role ARN:** `arn:aws:iam::TARGET_ACCOUNT:role/bytebase-target-db-role` * **External ID:** Optional security string (if configured in trust policy) * **Database Region:** Select target RDS region * **Database:** Your database name 4. Test and save the connection **Important:** For cross-account access, always use **Specific Credentials**. Leave Access Key ID, Secret Access Key, and Session Token empty when using EC2 instance profile - only provide the Role ARN. Use **Default** credential source only for same-account connections. How it works: 1. Bytebase uses the EC2 instance profile credentials 2. Assumes the specified role in the target account 3. Generates RDS IAM authentication tokens using the assumed role 4. Connects to the database using the token ### Example Setup **Scenario:** Bytebase in Account A (123456789012) connecting to RDS in Account B (987654321098) **Account B - Create role with trust relationship:** Configure the trust policy to allow the source account's role to assume this role: ```json theme={null} { "Version": "2012-10-17", "Statement": [ { "Effect": "Allow", "Principal": { "AWS": "arn:aws:iam::123456789012:role/bytebase-instance-role" }, "Action": "sts:AssumeRole", "Condition": { "StringEquals": { "sts:ExternalId": "your-secure-external-id" } } } ] } ``` **Account A - Allow Bytebase to assume the role:** ```json theme={null} { "Version": "2012-10-17", "Statement": [ { "Effect": "Allow", "Action": "sts:AssumeRole", "Resource": "arn:aws:iam::987654321098:role/bytebase-target-db-role" } ] } ``` Need to test cross-account authentication but only have one AWS account? See our guide on [testing cross-account authentication in a single account](/get-started/connect/aws-cross-account). ## DynamoDB DynamoDB connects via the AWS SDK, so region and credentials are read from the **Bytebase host environment**, not the instance form. The Host and Port fields do not apply and can be left blank. ### Step 1: Configure AWS Credentials and Region Bytebase reads from the AWS default credential chain, in order: 1. Environment variables (recommended for Docker/Kubernetes): ```bash theme={null} -e AWS_REGION=us-west-1 -e AWS_ACCESS_KEY_ID=xxx -e AWS_SECRET_ACCESS_KEY=yyy ``` 2. `~/.aws/credentials` and `~/.aws/config` 3. EC2 instance profile or EKS IRSA when Bytebase runs in AWS Reference: [AWS SDK credential resolution](https://docs.aws.amazon.com/sdkref/latest/guide/standardized-credentials.html) ### Step 2: Grant DynamoDB Permissions Attach this policy to the IAM role or user (scope `Resource` to specific table ARNs in production): ```json theme={null} { "Version": "2012-10-17", "Statement": [ { "Effect": "Allow", "Action": [ "dynamodb:ListTables", "dynamodb:DescribeTable", "dynamodb:ExecuteStatement", "dynamodb:ExecuteTransaction" ], "Resource": "*" } ] } ``` ### Step 3: Add the Instance in Bytebase 1. Click **New Instance** → select **DynamoDB**. 2. Leave **Host or Socket** and **Port** blank. 3. Click **Test Connection**, then save. After sync, the database appears as `{aws-account-id}-{region}` (e.g. `123456789012-us-west-1`). Each Bytebase DynamoDB instance covers one AWS region — configure separate instances for additional regions. ### Troubleshooting * **`Invalid Configuration: Missing Region`** — Set `AWS_REGION` (or `AWS_DEFAULT_REGION`) on the Bytebase host and restart. * **`NoCredentialProviders` / `failed to retrieve credentials`** — No credentials found. Set `AWS_ACCESS_KEY_ID` + `AWS_SECRET_ACCESS_KEY`, mount `~/.aws/credentials`, or attach an instance role. * **`AccessDenied` on `ListTables`** — IAM identity is missing the actions in Step 2. * **`failed to get caller identity`** — Credentials are missing or invalid; re-check the values from Step 1. ## AWS Secrets Manager Store database passwords securely in AWS Secrets Manager instead of Bytebase. Prerequisites: [IAM role](#prerequisites-iam-role-setup) with Secrets Manager permissions. ### Step 1: Grant Secrets Manager Access Add this policy to your IAM role to read secrets: ```json theme={null} { "Version": "2012-10-17", "Statement": [ { "Effect": "Allow", "Action": [ "secretsmanager:GetSecretValue", "secretsmanager:DescribeSecret" ], "Resource": "arn:aws:secretsmanager:REGION:ACCOUNT_ID:secret:SECRET_NAME-*" } ] } ``` Replace `REGION`, `ACCOUNT_ID`, and `SECRET_NAME` with your values. For easier setup, you can use wildcards: `arn:aws:secretsmanager:*:*:secret:*` Reference: [Secrets Manager IAM permissions](https://docs.aws.amazon.com/secretsmanager/latest/userguide/auth-and-access_identity-based-policies.html) ### Step 2: Create Secret 1. Go to [AWS Secrets Manager Console](https://console.aws.amazon.com/secretsmanager/) 2. Click **Store a new secret** 3. Select **Other type of secret** 4. Add key/value pair: Key = `DB_PASSWORD`, Value = your password 5. Name the secret (e.g., `bytebase-db-password`) 6. Complete creation and note the ARN Reference: [Creating secrets](https://docs.aws.amazon.com/secretsmanager/latest/userguide/create_secret.html) ### Step 3: Configure in Bytebase 1. In your database instance settings, find the password field 2. Click the key icon to use external secret 3. Select **AWS Secrets Manager** 4. Enter: * **Secret Name:** Your secret name from Step 2 * **Secret Key:** `DB_PASSWORD` 5. Test connection and save ## Database-Specific Configuration For specific database types running on AWS, see their configuration guides: See the PostgreSQL configuration above, with IAM authentication enabled as described in this guide. Follow the MySQL configuration with IAM authentication enabled as described in this guide. ## Best Practices 1. **Use IAM Roles over Access Keys**: Always prefer IAM roles when running on EC2 2. **Enable SSL/TLS**: All AWS database services support encrypted connections 3. **Use Secrets Manager**: Centralize password management with automatic rotation 4. **Follow Least Privilege**: Grant only necessary permissions to IAM roles 5. **Monitor Access**: Use CloudTrail to audit database access patterns ## Troubleshooting ### Connection Timeout * Verify security group rules allow traffic on database port * Check VPC routing and subnet configuration * Ensure database is publicly accessible or use VPN/bastion host ### IAM Authentication Failed * Verify IAM role has correct `rds-db:connect` permissions * Check database user was created with correct authentication method * Ensure SSL is enabled for the connection ### Secrets Manager Access Denied * Verify IAM role has `secretsmanager:GetSecretValue` permission * Check secret ARN matches the policy resource * Ensure secret exists in the correct region # Azure Database Connections Source: https://docs.bytebase.com/get-started/connect/azure Learn how to configure secure connections to Azure-managed databases using Managed Identity authentication and best practices for production deployments. ## Azure SQL with Managed Identity Authentication This guide demonstrates the most secure method for connecting to Azure SQL Database and Azure SQL Managed Instance using VM-attached managed identities, eliminating the need to manage credentials or connection strings. For alternative authentication methods and detailed configuration options, refer to: * [Azure SQL authentication methods overview](https://learn.microsoft.com/en-us/azure/azure-sql/database/authentication-aad-overview) * [Managed identities for Azure resources](https://learn.microsoft.com/en-us/entra/identity/managed-identities-azure-resources/overview) * [Configure Azure AD authentication for SQL Database](https://learn.microsoft.com/en-us/azure/azure-sql/database/authentication-aad-configure) * [Connect to Azure SQL with managed identity](https://learn.microsoft.com/en-us/azure/azure-sql/database/authentication-azure-ad-user-assigned-managed-identity) ### Step 1: Create Azure VM with System-Assigned Managed Identity 1. **Create VM with Managed Identity** * Go to [Azure Portal → Virtual Machines](https://portal.azure.com/#blade/HubsExtension/BrowseResource/resourceType/Microsoft.Compute%2FVirtualMachines) * Click **Create** → **Azure virtual machine** * Configure VM settings as needed * Under **Management** tab: * Enable **System assigned managed identity**: Set to **On** * Complete VM creation > **Security Best Practice:** System-assigned managed identities are automatically managed by Azure and tied to the VM lifecycle. This eliminates credential management and reduces security risks. Learn more: [Managed identity best practices](https://learn.microsoft.com/en-us/entra/identity/managed-identities-azure-resources/managed-identity-best-practice-recommendations) 2. **Deploy Bytebase on the VM** Deploy Bytebase on your Azure VM. The VM's managed identity is automatically available - no credential configuration needed. ### Step 2: Configure Azure SQL Database 1. **Enable Microsoft Entra Authentication** * Navigate to your Azure SQL Server in [Azure Portal](https://portal.azure.com) * Go to **Settings** → **Microsoft Entra ID** * Click **Set admin** and select an Entra admin account * Click **Save** to enable Entra authentication Reference: [Configure Entra authentication for Azure SQL](https://learn.microsoft.com/en-us/azure/azure-sql/database/authentication-aad-configure) 2. **Grant Database Access to Managed Identity** * Note your VM's managed identity name (same as VM name) * Connect to Azure SQL using the Entra admin account * Run the following for each database: ```sql theme={null} -- Create user for the VM's managed identity CREATE USER [your-vm-name] FROM EXTERNAL PROVIDER; -- Grant appropriate permissions (adjust as needed) ALTER ROLE db_datareader ADD MEMBER [your-vm-name]; ALTER ROLE db_datawriter ADD MEMBER [your-vm-name]; ALTER ROLE db_ddladmin ADD MEMBER [your-vm-name]; -- For full database management in Bytebase: ALTER ROLE db_owner ADD MEMBER [your-vm-name]; ``` > **Production Best Practice:** Follow the principle of least privilege. Grant only the minimum permissions required for your use case. See [Azure SQL Database permissions](https://learn.microsoft.com/en-us/azure/azure-sql/database/logins-create-manage). ### Step 3: Connect from Bytebase 1. Access Bytebase on your VM (typically `http://localhost:5678`) 2. Click **New Instance** 3. Configure the connection: * **Host:** Your Azure SQL server name (e.g., `yourserver.database.windows.net`) * **Port:** 1433 * **Database:** Target database name * **Authentication:** Select `Azure Default Credential` 4. Test and save the connection Bytebase automatically uses the VM's managed identity through Azure's Instance Metadata Service (IMDS) for authentication. **Advantages of this approach:** * No passwords or connection strings to manage * Automatic credential rotation handled by Azure * Enhanced security through Azure RBAC * Simplified compliance and auditing For troubleshooting, see [Troubleshoot managed identity authentication](https://learn.microsoft.com/en-us/azure/azure-sql/database/authentication-azure-ad-user-assigned-managed-identity#troubleshooting). ### Alternative: User-Assigned Managed Identity For more granular control or cross-resource scenarios: 1. **Create User-Assigned Managed Identity** * Go to [Managed Identities](https://portal.azure.com/#blade/HubsExtension/BrowseResource/resourceType/Microsoft.ManagedIdentity%2FuserAssignedIdentities) * Create a new identity with a descriptive name * Assign it to your VM under **Settings** → **Identity** → **User assigned** 2. **Configure Database Access** ```sql theme={null} CREATE USER [managed-identity-name] FROM EXTERNAL PROVIDER; ALTER ROLE db_owner ADD MEMBER [managed-identity-name]; ``` 3. **Set Environment Variable** (if using multiple identities) ```bash theme={null} export AZURE_CLIENT_ID= ``` Reference: [User-assigned managed identities](https://learn.microsoft.com/en-us/entra/identity/managed-identities-azure-resources/how-manage-user-assigned-managed-identities) ## Azure Database for PostgreSQL ### Entra ID Authentication 1. **Enable Entra Authentication** * In Azure Portal, navigate to your PostgreSQL server * Go to **Authentication** → **Active Directory admin** * Set an Entra admin and save 2. **Create Database User** Connect as the Entra admin and run: ```sql theme={null} -- For system-assigned managed identity CREATE ROLE "your-vm-name" LOGIN; GRANT ALL PRIVILEGES ON DATABASE yourdb TO "your-vm-name"; ``` 3. **Connect from Bytebase** * **Host:** Your PostgreSQL server name * **Port:** 5432 * **Username:** VM name (for system-assigned identity) * **Authentication:** Select `Azure Default Credential` Reference: [Azure Database for PostgreSQL Entra authentication](https://learn.microsoft.com/en-us/azure/postgresql/single-server/concepts-azure-ad-authentication) ## Azure Database for MySQL ### Entra ID Authentication 1. **Enable Entra Authentication** * Navigate to your MySQL server in Azure Portal * Go to **Active Directory admin** * Set an Entra admin 2. **Create Database User** ```sql theme={null} CREATE AADUSER 'your-vm-name'; GRANT ALL PRIVILEGES ON *.* TO 'your-vm-name'; ``` 3. **Connect from Bytebase** * **Host:** Your MySQL server name * **Port:** 3306 * **Username:** VM name * **Authentication:** Select `Azure Default Credential` Reference: [Azure Database for MySQL Entra authentication](https://learn.microsoft.com/en-us/azure/mysql/single-server/concepts-azure-ad-authentication) ## Private Endpoints and VNet Integration For enhanced security using private endpoints: ### Configure Private Endpoint 1. **Create Private Endpoint** * Navigate to your database resource * Go to **Networking** → **Private endpoint connections** * Click **+ Private endpoint** * Select your VNet and subnet * Complete the configuration 2. **Update DNS** * Use Azure Private DNS zones for automatic resolution * Or manually configure DNS for the private endpoint 3. **Connect from Bytebase** * Deploy Bytebase in the same VNet or peered VNet * Use the private endpoint DNS name for connection Reference: [Private endpoints for Azure SQL](https://learn.microsoft.com/en-us/azure/azure-sql/database/private-endpoint-overview) ## Best Practices 1. **Use Managed Identities**: Eliminate password management overhead 2. **Enable Private Endpoints**: Restrict database access to your VNet 3. **Follow Least Privilege**: Grant minimal required permissions 4. **Enable Auditing**: Use Azure SQL Auditing for compliance 5. **Use Azure Key Vault**: For scenarios requiring password storage ## Troubleshooting ### Authentication Failed * Verify managed identity is properly assigned to VM * Check database user was created with `FROM EXTERNAL PROVIDER` * Ensure Entra authentication is enabled on the database ### Connection Timeout * Verify network security group rules * Check VNet and subnet configuration * Ensure database firewall rules allow your IP ### Private Endpoint Issues * Verify DNS resolution for private endpoint * Check VNet peering if using multiple VNets * Ensure network security allows traffic ### Managed Identity Not Working * Restart VM after enabling managed identity * Check Azure RBAC assignments * Verify IMDS endpoint is accessible from VM # GCP Database Connections Source: https://docs.bytebase.com/get-started/connect/gcp Learn how to configure secure connections to GCP-managed databases using service account authentication, Secret Manager, and best practices for production deployments. ## Prerequisites: Service Account Setup Use attached service accounts for secure, key-free authentication on: * **GCE** - VMs with attached service accounts * **GKE** - Pods with Workload Identity References: [Service accounts](https://cloud.google.com/iam/docs/service-account-overview) | [Best practices](https://cloud.google.com/iam/docs/best-practices-for-managing-service-account-keys) | [ADC](https://cloud.google.com/docs/authentication/application-default-credentials) ### Create Service Account 1. Go to [IAM & Admin → Service Accounts](https://console.cloud.google.com/iam-admin/serviceaccounts) 2. Create service account named `bytebase` 3. Grant roles as needed: * `Cloud SQL Client` and `Cloud SQL Instance User` - for Cloud SQL * `Secret Manager Secret Accessor` - for Secret Manager 4. Note the email: `bytebase@PROJECT_ID.iam.gserviceaccount.com` ### Attach Service Account **Option A: GCE VM** 1. Create VM in [Compute Engine](https://console.cloud.google.com/compute/instances) 2. Set service account: `bytebase@PROJECT_ID.iam.gserviceaccount.com` 3. Set access scopes: "Allow full access to all Cloud APIs" **Option B: GKE with Workload Identity** ```bash theme={null} # Create Kubernetes service account kubectl create serviceaccount bytebase-ksa # Bind to Google service account kubectl annotate serviceaccount bytebase-ksa \ iam.gke.io/gcp-service-account=bytebase@PROJECT_ID.iam.gserviceaccount.com # Allow impersonation gcloud iam service-accounts add-iam-policy-binding bytebase@PROJECT_ID.iam.gserviceaccount.com \ --role roles/iam.workloadIdentityUser \ --member "serviceAccount:PROJECT_ID.svc.id.goog[NAMESPACE/bytebase-ksa]" ``` Reference: [Workload Identity](https://cloud.google.com/kubernetes-engine/docs/how-to/workload-identity) Deploy Bytebase on your resource - credentials are provided automatically. ### Alternative: Service Account Keys Use only when running Bytebase outside GCP. See [why to avoid service account keys](https://cloud.google.com/iam/docs/best-practices-for-managing-service-account-keys#avoid). 1. Create a service account with required roles 2. Download the JSON key file 3. Set environment variable: ```bash theme={null} -e GOOGLE_APPLICATION_CREDENTIALS=/path/to/key.json ``` Reference: [Service account keys authentication](https://cloud.google.com/docs/authentication/provide-credentials-adc#local-key) ## Cloud SQL with IAM Authentication Prerequisites: [Service account](#prerequisites-service-account-setup) with Cloud SQL roles. References: [IAM authentication](https://cloud.google.com/sql/docs/mysql/iam-authentication) | [Configure instances](https://cloud.google.com/sql/docs/mysql/create-edit-iam-instances) ### Step 1: Configure Cloud SQL Instance 1. In [Cloud SQL](https://console.cloud.google.com/sql/instances), edit your instance 2. Add flag: `cloudsql_iam_authentication` = `on` 3. Save (SSL is enabled by default) ### Step 2: Add Service Account User **Using gcloud:** ```bash theme={null} gcloud sql users create bytebase@PROJECT_ID.iam.gserviceaccount.com \ --instance=INSTANCE_NAME \ --type=cloud_iam_service_account ``` **Using Console:** Instance → Users → Add User Account → Cloud IAM → Enter service account email References: [MySQL](https://cloud.google.com/sql/docs/mysql/add-manage-iam-users) | [PostgreSQL](https://cloud.google.com/sql/docs/postgres/add-manage-iam-users) ### Step 3: Connect from Bytebase 1. Click **New Instance** in Bytebase 2. Configure connection details: * **Host:** Your Cloud SQL connection name (`PROJECT_ID:REGION:INSTANCE_ID`) * Find this in Cloud SQL console → Instance details * **Port:** 3306 (MySQL) or 5432 (PostgreSQL) * **Username:** * MySQL: `bytebase` (service account name only) * PostgreSQL: `bytebase@PROJECT_ID.iam` (with project ID) * **Authentication:** Select `Google Cloud SQL IAM` 3. Click **Test Connection** then **Create** ## GCP Secret Manager Store database passwords securely in Google Cloud Secret Manager instead of Bytebase. Prerequisites: [Service account](#prerequisites-service-account-setup) with `Secret Manager Secret Accessor` role. ### Step 1: Create Secret 1. Go to [Secret Manager Console](https://console.cloud.google.com/security/secret-manager) 2. Click **Create Secret** 3. Enter secret name (e.g., `db-password`) and your database password as value 4. Click **Create** and note the resource name: `projects/PROJECT_ID/secrets/SECRET_NAME` ### Step 2: Configure in Bytebase 1. In your database instance settings, find the password field 2. Click the key icon to use external secret 3. Select **GCP Secret Manager** 4. Enter the secret resource name from Step 1 5. Test connection and save ## Private IP and VPC Peering For Cloud SQL instances using private IP: ### Step 1: Configure VPC Peering 1. Enable Service Networking API 2. Reserve IP range for services: ```bash theme={null} gcloud compute addresses create google-managed-services-default \ --global \ --purpose=VPC_PEERING \ --prefix-length=16 \ --network=default ``` 3. Create private connection: ```bash theme={null} gcloud services vpc-peerings connect \ --service=servicenetworking.googleapis.com \ --ranges=google-managed-services-default \ --network=default ``` Reference: [Private IP configuration](https://cloud.google.com/sql/docs/mysql/configure-private-ip) ### Step 2: Connect from Bytebase 1. Deploy Bytebase in the same VPC or a peered VPC 2. Use the private IP address for the Cloud SQL instance 3. Configure connection as normal with private IP ## Database-Specific Configuration For specific database types running on GCP: Follow standard MySQL configuration with IAM authentication enabled as described above. ## Best Practices 1. **Use Service Accounts over Keys**: Always prefer attached service accounts 2. **Enable Private IP**: Use VPC peering for enhanced security 3. **Use Secret Manager**: Centralize password management 4. **Follow Least Privilege**: Grant only necessary IAM roles 5. **Enable Audit Logging**: Monitor database access with Cloud Audit Logs ## Troubleshooting ### Connection Refused * Verify Cloud SQL instance allows connections from your IP/network * Check authorized networks configuration * Ensure VPC peering is properly configured for private IP ### IAM Authentication Failed * Verify service account has `Cloud SQL Instance User` role * Check database user was created with correct type * Ensure `cloudsql_iam_authentication` flag is enabled ### Secret Manager Access Denied * Verify service account has `Secret Manager Secret Accessor` role * Check secret resource name is correct * Ensure secret exists in the correct project ### Private IP Connection Issues * Verify VPC peering is active * Check firewall rules allow traffic * Ensure Bytebase is deployed in the correct VPC # Overview Source: https://docs.bytebase.com/get-started/connect/overview Learn how to connect Bytebase to your database instances and configure basic connection settings. ## Quick Start RDS, Aurora with IAM auth Cloud SQL, Spanner with service accounts Azure SQL with managed identity ## Basic Connection Before configuring connection parameters, ensure network connectivity: **Network Requirements:** * Verify network routing between Bytebase and your database instance (e.g., VPN, private networks) * Configure firewall rules to allow Bytebase to connect to your database port * For cloud databases, add Bytebase to security groups or IP allowlists * Bytebase Cloud users: [Whitelist required IPs](/get-started/cloud#prerequisites) **Connection Parameters:** 1. **Host**: Database server address * Docker (standard): Use `host.docker.internal` for localhost databases * Docker (--network host): Use `127.0.0.1` for localhost databases 2. **Port**: Database port number (e.g., 5432 for PostgreSQL, 3306 for MySQL) 3. **Username & Password**: Database credentials with appropriate permissions Additional parameters vary by database type - see the database-specific guides below. ## Advanced Connection Options ### Read-Only Connections Configure separate read-only connections for enhanced security and performance. Read-only connections are used for: * SQL Editor queries with [data source restrictions](/sql-editor/settings/data-source-restriction) * [Data export](/security/database-permission/export) **Setup:** 1. Create a read-only database user or configure a read-replica 2. In Bytebase, click **+** next to **Connection Info** 3. Enter the read-only connection details 4. Save the configuration ### SSH Tunnel Use SSH tunneling to connect through a bastion host when your database is behind a firewall or in a private network. **Setup:** 1. Enter your database connection details 2. Enable **SSH Connection** and select **Tunnel + Private Key** 3. Configure SSH settings: * **SSH Host**: Bastion host address * **SSH Port**: SSH port (typically 22) * **SSH User**: Username for SSH authentication * **Private Key** or **Password**: SSH credentials 4. Test and save the connection ### Connection Parameters Customize connection behavior with database-specific parameters: | Parameter | Description | Example | Databases | | ----------------- | ---------------------- | --------- | ----------------- | | `sslmode` | SSL connection mode | `require` | PostgreSQL | | `connect_timeout` | Connection timeout | `10` | PostgreSQL, MySQL | | `readTimeout` | Read operation timeout | `30s` | MySQL, SQL Server | **Database Documentation:** * [PostgreSQL Parameters](https://www.postgresql.org/docs/current/libpq-connect.html) * [MySQL Parameters](https://github.com/go-sql-driver/mysql#parameters) * [SQL Server Parameters](https://pkg.go.dev/github.com/microsoft/go-mssqldb#section-readme) ### Secret Manager Integration Never store database passwords in Bytebase. Use your existing secret manager for automatic rotation, compliance, and centralized control. **Key benefits:** * Meet SOC2/HIPAA compliance requirements * Automatic password rotation without downtime * Complete audit trail of credential access * Centralized management across all systems **Supported integrations:** * **[AWS Secrets Manager](/get-started/connect/aws#aws-secrets-manager)** - For AWS deployments * **[GCP Secret Manager](/get-started/connect/gcp#gcp-secret-manager)** - For GCP deployments * **HashiCorp Vault** - Enterprise secret management #### HashiCorp Vault **Vault Setup:** 1. Create secret in Vault KV v2: * Engine: `secret` * Path: `bytebase` * Key: `DB_PASSWORD` Requires Vault KV v2 engine with `read` and `list` permissions on both `/data/` and `/metadata/` paths **Configuration in Bytebase:** 1. Enter Vault URL 2. Choose authentication: **Token** or **AppRole**. For **Token**, choose how Bytebase reads the token: * **Plain text** — the token value itself. * **Environment variable** — the name of an environment variable on the Bytebase server that holds the token. * **File path** — the path to a file on the Bytebase server that holds the token. 3. Specify secret location **Environment variable** and **File path** read from the Bytebase server, so they aren't available on Bytebase Cloud (SaaS) — only **Plain text** is offered there. ## Instance Management Configure instance-level settings to optimize how Bytebase interacts with your databases. ### Sync Interval Manually trigger schema metadata sync at any time: * **Database level**: Sync an individual database's schema metadata * **Instance level**: Sync all databases and detect newly added or removed databases Bytebase automatically syncs schema metadata when changes are made through Bytebase. For external changes, set an automatic sync interval to keep metadata current for SQL review and data masking. We recommend 24 hours for most use cases—shorter intervals can overload your database server and Bytebase service. ### Maximum Connections Control the number of concurrent connections Bytebase can establish to your database. Finding the right balance is important: * **Too high**: May overwhelm your database server * **Too low**: May cause task queuing and execution delays Consider these factors when setting the limit: * Team size and concurrent SQL Editor usage * Available database connection capacity * Parallel operations (migrations, backups, syncs) ### Deleting Instances #### Archive Instance To temporarily remove an instance from active use while preserving its configuration: 1. Navigate to the instance page 2. Click **Archive** to move the instance to archived status 3. Archived instances remain accessible from **Settings > Archived** Before archiving an instance, ensure all its databases are unassigned from projects. Databases actively assigned to projects cannot be archived. #### Hard Delete Instance To permanently remove an instance and all its associated metadata from Bytebase: 1. **Archive the instance first**: * Go to the instance page * Click **Archive this instance** (requires all databases to be unassigned from projects) on the bottom of the page 2. **Permanently delete the archived instance**: * Navigate to **Settings > Archived** * Select the archived instance * Click **Hard-Delete** to permanently remove the instance Hard deletion is irreversible. All instance configuration, connection settings, and metadata stored in Bytebase will be permanently removed. This does not affect the actual database itself, only Bytebase's records and configuration for that instance. # Self-Host vs Cloud Source: https://docs.bytebase.com/get-started/self-host-vs-cloud Use **Self-Host** for private databases (VPC, on-premises). Use **Cloud** for public cloud databases (Neon, Supabase, RDS with public IP). | | Self-Host | Cloud | | ------------------- | ---------------------------------- | --------------------------- | | **Database Access** | Private networks, VPC, on-premises | Public cloud databases only | | **Available Plans** | Free, Enterprise | Free, Pro, Enterprise | | **Setup** | \~5 minutes (Docker) | Instant | | **Maintenance** | You manage updates | Automatic | | **Location** | Your infrastructure | Google Cloud US | The **Pro plan** is cloud-only. Self-hosted deployments use the Free plan or require an Enterprise license. Self-hosted Free instances can start a [14-day Enterprise trial](https://www.bytebase.com/contact-us/). Deploy with Docker in 5 minutes Start instantly at cloud.bytebase.com # Deploy with Docker Source: https://docs.bytebase.com/get-started/self-host/deploy-with-docker ## Installation ```bash theme={null} docker run --rm --init \ --name bytebase \ --publish 8080:8080 --pull always \ --volume ~/.bytebase/data:/var/opt/bytebase \ bytebase/bytebase:latest ``` Bytebase is now accessible at [http://localhost:8080](http://localhost:8080). ## Troubleshooting ### Docker Compatibility Docker version must be at least 20.10.24. We recommend using the [latest Docker version](https://docs.docker.com/engine/install/). If you cannot upgrade Docker, pass `--security-opt seccomp=unconfined` to `docker run` as a best effort to avoid the `pthread_create failed` error. If you encounter a "manifest not found" error, this may be due to: * **Architecture mismatch**: The docker image only supports `linux/amd64` and `linux/arm64` architectures. If it doesn't match your OS architecture, you may supply `--platform linux/amd64` as a best effort. * **Outdated Docker version**: Your Docker version is too old and doesn't support manifest lists. ### Air-gapped Environment If you need to transfer images to a completely air-gapped environment: ```bash theme={null} # Save the image to a tar file docker save bytebase/bytebase:latest > bytebase-latest.tar # Transfer the tar file to your target environment # Then load it on the target system docker load < bytebase-latest.tar # Tag and push as described above docker tag bytebase/bytebase:latest your-registry.acme.com/library/bytebase:latest docker push your-registry.acme.com/library/bytebase:latest ``` ### Port Conflicts If you see `bind: address already in use` error, it means the port 8080 is already in use on your host. You need to either stop the existing process using the port or configure Bytebase to use a different port via `--publish :8080`. ### Connect to Database on Same Host * If you **run Bytebase inside Docker on Linux**, then you need to supply the additional `--network host` flags in `docker run` command. This allows Bytebase to connect to database instance running on the same host with `localhost`. * If you **run Bytebase inside Docker Desktop on Mac** , then you need to use `host.docker.internal` to connect to database instance running on the same host. ### Colima Support Due to the vm mechanism of [Colima](https://github.com/abiosoft/colima), try to use the `--mount` option when starting colima as shown below: ```bash theme={null} mkdir ~/volumes colima start --mount ~/volumes:w ``` # Deploy with Kubernetes Source: https://docs.bytebase.com/get-started/self-host/deploy-with-kubernetes ## Deployment Single-replica deployment still works as before. For multi-replica deployment, see [High Availability](/get-started/self-host/high-availability/). Here is a sample Kubernetes YAML file `bytebase.yaml` describing the minimal components and configuration required to run Bytebase in Kubernetes. ```yaml theme={null} apiVersion: apps/v1 kind: StatefulSet metadata: name: bytebase namespace: default spec: replicas: 1 selector: matchLabels: app: bytebase template: metadata: labels: app: bytebase spec: containers: - name: bytebase image: bytebase/bytebase:latest imagePullPolicy: Always # Configure external PostgreSQL following the guide: # https://www.bytebase.com/docs/get-started/self-host/external-postgres env: - name: PG_URL value: 'postgresql://:@:/' args: [ '--port', '8080', ] ports: - containerPort: 8080 livenessProbe: httpGet: path: /healthz port: 8080 initialDelaySeconds: 300 periodSeconds: 300 timeoutSeconds: 10 --- apiVersion: v1 kind: Service metadata: name: bytebase-entrypoint namespace: default spec: # Optional type: ClusterIP selector: app: bytebase ports: - protocol: TCP port: 80 targetPort: 8080 ``` 1. Start Bytebase with the following command: ```bash theme={null} kubectl apply -f bytebase.yaml ``` 2. Make sure everything worked by listing your deployments: ```bash theme={null} kubectl get statefulsets ``` Do the same check for your services: ```bash theme={null} kubectl get services ``` ## Deploy with Helm Bytebase provides an official Helm chart for simplified Kubernetes deployments. For comprehensive configuration options and advanced settings, visit the [Bytebase Helm Chart on Artifact Hub](https://artifacthub.io/packages/helm/bytebase/bytebase). The current official Helm chart deploys a single replica and does not yet expose HA-specific settings such as replica count and the `--ha` flag. If you want to run Bytebase in HA mode today, use a custom manifest based on this guide or customize the chart template before deploying. ### Installation Deploy Bytebase using Helm with your external PostgreSQL database: ```bash theme={null} helm -n \ --set "bytebase.option.externalPg.url=" \ --set "bytebase.version=" \ install bytebase-repo/bytebase ``` Quote the whole `key=value` pair. A PostgreSQL DSN contains `?` and `&`, which the shell would otherwise treat as a glob and a background operator. To let the chart construct `PG_URL` from external PostgreSQL fields, set the host, port, username, database, and authentication-specific values instead of `bytebase.option.externalPg.url`. For AWS RDS IAM authentication: ```bash theme={null} helm -n \ --set "bytebase.option.externalPg.pgHost=" \ --set "bytebase.option.externalPg.pgPort=5432" \ --set "bytebase.option.externalPg.pgUsername=" \ --set "bytebase.option.externalPg.pgDatabase=" \ --set "bytebase.option.externalPg.awsRdsIam.enabled=true" \ --set "bytebase.option.externalPg.awsRdsIam.region=" \ --set "bytebase.version=" \ install bytebase-repo/bytebase ``` For GCP Cloud SQL IAM authentication: ```bash theme={null} helm -n \ --set "bytebase.option.externalPg.pgUsername=" \ --set "bytebase.option.externalPg.pgDatabase=" \ --set "bytebase.option.externalPg.gcpCloudSqlIam.enabled=true" \ --set "bytebase.option.externalPg.gcpCloudSqlIam.instanceConnectionName=::" \ --set "bytebase.version=" \ install bytebase-repo/bytebase ``` See [Configure External PostgreSQL](/get-started/self-host/external-postgres/#iam-authentication-for-managed-postgresql) for the required AWS or GCP IAM setup. ### Uninstallation To remove the Bytebase deployment: ```bash theme={null} helm delete --namespace ``` ## HA Deployment To run Bytebase with multiple replicas on Kubernetes: 1. Configure an external PostgreSQL database with `PG_URL`. 2. Set `replicas` to more than `1`. 3. Add the `--ha` flag to every Bytebase replica. 4. Put the replicas behind one stable ingress, gateway, or service entrypoint. 5. Make sure the workspace license has HA enabled. Example: ```yaml theme={null} apiVersion: apps/v1 kind: StatefulSet metadata: name: bytebase namespace: default spec: replicas: 2 selector: matchLabels: app: bytebase serviceName: bytebase template: metadata: labels: app: bytebase spec: containers: - name: bytebase image: bytebase/bytebase:latest env: # Replace the <> placeholders. See: # https://www.bytebase.com/docs/get-started/self-host/external-postgres - name: PG_URL value: 'postgresql://:@:/' args: - '--port' - '8080' - '--ha' ports: - containerPort: 8080 ``` # External Access Configuration Source: https://docs.bytebase.com/get-started/self-host/external-access This guide covers how to configure external access for your Bytebase deployment across different deployment methods. Bytebase service itself does not provide native HTTPS support. We recommend using a reverse proxy (Nginx, Caddy) on the same VM for Docker deployments, or ingress/gateway for Kubernetes deployments. ## Docker Deployment with Reverse Proxy When deploying Bytebase with Docker on a VM, use a reverse proxy for external access and HTTPS termination. ### Nginx Configuration For Docker deployments using Nginx as a reverse proxy: ```nginx theme={null} http { map $http_upgrade $connection_upgrade { default upgrade; '' close; } server { listen 80; listen [::]:80; # Listen HTTPS listen 443 ssl; listen [::]:443 ssl; server_name bytebase.example.com; # SSL cert and key ssl_certificate /path/to/certificate/file; ssl_certificate_key /path/to/private/key/file; location ~ ^/(v1:adminExecute|lsp) { # Point to the actual Bytebase service, NOT the nginx domain proxy_pass http://127.0.0.1:8080; # If Bytebase runs on the same host proxy_http_version 1.1; # Enables WebSocket which is required for SQL Editor autocomplete proxy_set_header Upgrade $http_upgrade; proxy_set_header Connection $connection_upgrade; } location / { # Point to the actual Bytebase service, NOT the nginx domain proxy_pass http://127.0.0.1:8080; # If Bytebase runs on the same host } proxy_read_timeout 3600; proxy_send_timeout 3600; } } ``` #### Common Issue: 502 Bad Gateway If you're getting 502 errors, ensure `proxy_pass` points to the actual Bytebase service, not the nginx domain itself (which creates a loop): ```nginx theme={null} # ❌ WRONG - Creates proxy loop proxy_pass http://bytebase.example.com; # ✅ CORRECT - Points to Bytebase service proxy_pass http://127.0.0.1:8080; # Same host proxy_pass http://bytebase:8080; # Docker Compose service name proxy_pass http://10.0.0.5:8080; # Different host IP ``` ### Caddy Configuration For Docker deployments using Caddy (automatic HTTPS with Let's Encrypt): ```caddy theme={null} bytebase.example.com { # Automatic HTTPS with Let's Encrypt # Reverse proxy to Bytebase reverse_proxy localhost:8080 { # Timeouts for long-running operations transport http { read_timeout 3600s write_timeout 3600s } } } ``` To use this Caddy configuration: 1. Install Caddy on your VM 2. Save the configuration to `/etc/caddy/Caddyfile` 3. Run: `caddy reload` ## Kubernetes Deployment For Kubernetes deployments, use ingress controllers or gateways to configure external access with HTTPS support. ### Nginx Ingress Controller Deploy Bytebase with [Nginx Ingress Controller](https://kubernetes.github.io/ingress-nginx/deploy/): ```yaml theme={null} apiVersion: networking.k8s.io/v1 kind: Ingress metadata: name: bytebase-ingress namespace: default annotations: # Enable HTTPS redirect nginx.ingress.kubernetes.io/ssl-redirect: 'true' # WebSocket support for SQL Editor nginx.ingress.kubernetes.io/proxy-read-timeout: '3600' nginx.ingress.kubernetes.io/proxy-send-timeout: '3600' spec: ingressClassName: nginx tls: - hosts: - bytebase.example.com secretName: bytebase-tls-secret rules: - host: bytebase.example.com http: paths: - path: / pathType: Prefix backend: service: name: bytebase-service port: number: 8080 ``` ### Kubernetes Gateway API For modern Kubernetes deployments using Gateway API: ```yaml theme={null} apiVersion: gateway.networking.k8s.io/v1 kind: HTTPRoute metadata: name: bytebase-route namespace: default spec: parentRefs: - name: gateway namespace: default hostnames: - bytebase.example.com rules: - matches: - path: type: PathPrefix value: / backendRefs: - name: bytebase-service port: 8080 --- apiVersion: gateway.networking.k8s.io/v1 kind: Gateway metadata: name: gateway namespace: default spec: gatewayClassName: nginx listeners: - name: https protocol: HTTPS port: 443 tls: mode: Terminate certificateRefs: - name: bytebase-tls-secret - name: http protocol: HTTP port: 80 # Redirect HTTP to HTTPS allowedRoutes: namespaces: from: Same ``` ### Service Configuration Ensure your Bytebase service is configured correctly: ```yaml theme={null} apiVersion: v1 kind: Service metadata: name: bytebase-service namespace: default spec: selector: app: bytebase ports: - protocol: TCP port: 8080 targetPort: 8080 type: ClusterIP ``` ## Additional Configuration ### Configure External URL For production usage, configure the External URL to match your domain. See [Configure External URL](/get-started/self-host/external-url) for details. ### WebSocket Support SQL Editor autocomplete requires WebSocket support. All configurations above include the necessary WebSocket settings. Key endpoints that require WebSocket: * `/v1:adminExecute` - For SQL execution * `/lsp` - For Language Server Protocol (autocomplete) ### Troubleshooting * **WebSocket issues**: Verify proxy/ingress WebSocket configuration * **502 errors**: Check Bytebase service status * **Timeout errors**: Increase proxy timeout settings (see examples above) # Configure External PostgreSQL Source: https://docs.bytebase.com/get-started/self-host/external-postgres ## PostgreSQL Setup PostgreSQL 14 or above. External PostgreSQL is required for [High Availability](/get-started/self-host/high-availability/). HA mode is not supported with Bytebase's embedded PostgreSQL. ### Database Create a database named `bytebase` with UTF-8 encoding. UTF-8 encoding is required for proper system operation. ### Authentication Choose one authentication method for the metadata database. The authentication method determines how you create the database user and how you set `PG_URL`. The connection string must follow the standard PostgreSQL URI format. See the [official PostgreSQL documentation](https://www.postgresql.org/docs/current/libpq-connect.html#LIBPQ-CONNSTRING-URIS) for complete syntax details. #### Password Authentication (Default) Use password authentication for standard PostgreSQL deployments. Create a user `bytebase` with a password and one of the following privilege levels: ##### Option 1: Superuser This option is not available on managed database services like AWS RDS or Google Cloud SQL. Grant PostgreSQL superuser privileges: ```sql theme={null} ALTER ROLE bytebase SUPERUSER; ``` ##### Option 2: Database Owner Make the user own the database: ```sql theme={null} ALTER DATABASE bytebase OWNER TO bytebase; ``` ##### Option 3: Schema Create Privilege Grant CREATE privilege on the public schema: ```sql theme={null} GRANT CREATE ON SCHEMA public TO bytebase; ``` ##### Option 4: Database Owner Role Grant the db\_owner role to the user: ```sql theme={null} GRANT db_owner TO bytebase; ``` Set `PG_URL` to a PostgreSQL connection URI that includes the password: ```bash theme={null} PG_URL="postgresql://bytebase:@:5432/bytebase" ``` The user and database names are literal — they are the `bytebase` user and `bytebase` database created above. #### IAM Authentication for Managed PostgreSQL Use IAM authentication when the metadata database is AWS RDS/Aurora PostgreSQL or GCP Cloud SQL for PostgreSQL. This removes the static PostgreSQL password from `PG_URL`; Bytebase uses the cloud credentials available in its runtime environment to authenticate each metadata database connection. Only enable one metadata database IAM provider at a time. Keep using password authentication for other PostgreSQL providers. ##### AWS RDS IAM Authentication Use AWS RDS IAM authentication when Bytebase stores metadata in RDS for PostgreSQL or Aurora PostgreSQL. Required setup: * Enable IAM database authentication on the RDS or Aurora PostgreSQL instance. * Create the PostgreSQL user for Bytebase. * Grant the user the required Bytebase metadata database privileges, such as database owner, schema `CREATE`, or equivalent privileges. * Grant the PostgreSQL user the `rds_iam` role. * Grant the Bytebase runtime AWS principal `rds-db:connect` for that database user. * Use verified TLS, such as `sslmode=verify-full`, with a trust store that validates the RDS certificate. * Provide AWS credentials through the standard AWS credential chain, such as EKS IRSA, ECS task roles, EC2 instance profiles, environment variables, or shared AWS config. Set `PG_URL` without a password and add the AWS RDS IAM parameters: ```bash theme={null} PG_URL="postgresql://bytebase@:5432/bytebase?sslmode=verify-full&bytebase_aws_rds_iam=true&bytebase_aws_region=" ``` Bytebase-specific connection parameters: | Parameter | Description | | ------------------------------ | ------------------------------------------------------------- | | `bytebase_aws_rds_iam=true` | Enables AWS RDS IAM authentication for the metadata database. | | `bytebase_aws_region=` | AWS region used to sign RDS IAM authentication tokens. | ##### GCP Cloud SQL IAM Authentication Use GCP Cloud SQL IAM authentication when Bytebase stores metadata in Cloud SQL for PostgreSQL. Required setup: * Enable Cloud SQL IAM database authentication on the Cloud SQL PostgreSQL instance. * Add the Google service account as a Cloud SQL IAM database user. For PostgreSQL, use the IAM database username format without the `.gserviceaccount.com` suffix, for example `bytebase@.iam`. * Grant the IAM database user the required Bytebase metadata database privileges, such as database owner, schema `CREATE`, or equivalent privileges. * Grant the Bytebase runtime Google principal the Cloud SQL permissions it needs, such as `Cloud SQL Client` and `Cloud SQL Instance User`. * Provide Google credentials through Application Default Credentials, such as GKE Workload Identity, a GCE attached service account, or `GOOGLE_APPLICATION_CREDENTIALS`. Set `PG_URL` without a password and add the GCP Cloud SQL IAM parameters. The example percent-encodes `@` in the username as `%40` because `@` is a URI delimiter: ```bash theme={null} PG_URL="postgresql://bytebase%40.iam@localhost:5432/bytebase?bytebase_gcp_cloud_sql_iam=true&bytebase_gcp_cloud_sql_instance_connection_name=::" ``` Bytebase-specific connection parameters: | Parameter | Description | | --------------------------------------------------------------------------- | ------------------------------------------------------------------- | | `bytebase_gcp_cloud_sql_iam=true` | Enables GCP Cloud SQL IAM authentication for the metadata database. | | `bytebase_gcp_cloud_sql_instance_connection_name=` | Cloud SQL instance connection name used by the Cloud SQL connector. | #### File Path (Kubernetes & Secret Management) For Kubernetes deployments and secret manager integration, set `PG_URL` to a file path containing either a password-based or IAM-enabled connection string. Bytebase automatically picks up the updated connection string when the file content changes, enabling seamless secret rotation. ```bash theme={null} PG_URL="/var/secrets/pg-connection/url" ``` Any readable file path works. This example matches the volume mount shown under [Secret-Based Configuration](#secret-based-configuration). ## Running with Docker This bash script demonstrates how to add an external PostgreSQL database as the metadata store when running the bytebase container. When connecting to a PostgreSQL instance running on the same host machine, use [host.docker.internal](https://docs.docker.com/desktop/networking/#i-want-to-connect-from-a-container-to-a-service-on-the-host) as the hostname. ```bash theme={null} docker run --rm --init \ -e PG_URL=postgresql://:@:/ \ --name bytebase \ --publish 8080:8080 --pull always \ --volume ~/.bytebase/data:/var/opt/bytebase \ bytebase/bytebase:latest ``` `` is the database Bytebase stores its own metadata in, not one of the databases you manage with it. ## Running with Kubernetes ### Direct Configuration Configure the PostgreSQL connection directly in your deployment manifest: ```yaml theme={null} env: - name: PG_URL value: 'postgresql://:@:/' ``` `` is the database Bytebase stores its own metadata in, not one of the databases you manage with it. ### Secret-Based Configuration For enhanced security, store your PostgreSQL connection string in a Kubernetes Secret: #### Using Secret as Environment Variable Add the following environment variable configuration to your deployment's `spec.templates.spec.containers.env` section: ```yaml theme={null} env: - name: PG_URL valueFrom: secretKeyRef: name: key: ``` #### Using Secret as File Mount Mount the secret as a file and point `PG_URL` to the file path. This approach supports automatic secret rotation - when the Kubernetes Secret is updated, the mounted file content is automatically refreshed, and Bytebase will pick up the new connection string without requiring a restart: ```yaml theme={null} spec: containers: - name: bytebase env: - name: PG_URL value: "/var/secrets/pg-connection/url" volumeMounts: - name: pg-secret mountPath: "/var/secrets/pg-connection" readOnly: true volumes: - name: pg-secret secret: secretName: bytebase-pg-secret items: - key: connection-string path: url ``` When using file-based secrets, Kubernetes automatically updates the mounted file content when the Secret is updated (typically within a minute). Bytebase monitors the file for changes and automatically reloads the connection string, enabling seamless secret rotation without downtime. # Configure External URL Source: https://docs.bytebase.com/get-started/self-host/external-url The external URL determines how users access your Bytebase instance. This configuration is required for [SSO](/administration/sso/overview) and [SCIM](/administration/scim/overview). A Fully Qualified Domain Name (FQDN) is preferred over an IP address. ## Configuration Methods ### Option 1: Command Line Pass the `--external-url` flag when starting Bytebase: ```bash theme={null} ./bytebase --external-url "https://bytebase.example.com" ``` ### Option 2: Web UI 1. Navigate to **Settings** → **General** under **Workspace** 2. Enter your External URL (e.g., `https://bytebase.example.com`) 3. Click **Update** # High Availability Source: https://docs.bytebase.com/get-started/self-host/high-availability Bytebase supports high availability (HA) for self-hosted deployments. In HA mode, you run multiple Bytebase replicas behind the same external endpoint. The replicas coordinate through the shared metadata PostgreSQL database, so scheduled work such as rollout tasks, plan checks, schema sync, and cancellation handling remains safe across replicas. ## Requirements * Use an external PostgreSQL database for Bytebase metadata * Start every replica with the `--ha` flag * Put all replicas behind the same load balancer / ingress / service * Configure the same External URL for the deployment * Use a license with HA enabled HA mode does not work with Bytebase's embedded PostgreSQL. If `PG_URL` is not set, Bytebase will refuse to start in HA mode. ## How It Works * Each replica sends heartbeats to the metadata PostgreSQL database * Distributed coordination uses PostgreSQL advisory locks and row-level claiming * Long-running tasks are owned by a replica; stale work is detected if that replica stops heartbeating * Cancel signals are propagated across replicas through PostgreSQL `NOTIFY/LISTEN` ## Kubernetes Example The example below shows the minimum HA-specific changes compared with a single-replica deployment: ```yaml theme={null} apiVersion: apps/v1 kind: StatefulSet metadata: name: bytebase namespace: default spec: replicas: 2 selector: matchLabels: app: bytebase serviceName: bytebase template: metadata: labels: app: bytebase spec: containers: - name: bytebase image: bytebase/bytebase:latest env: # Replace the <> placeholders. See: # https://www.bytebase.com/docs/get-started/self-host/external-postgres - name: PG_URL value: "postgresql://:@:/" args: - "--port" - "8080" - "--ha" ports: - containerPort: 8080 ``` Use an ingress, gateway, or load balancer in front of the replicas so clients always access Bytebase through one stable URL. ## Operational Notes * HA is mainly for Kubernetes or other orchestrated environments. A single `docker run` deployment is still a single-node setup. * Replicas do not require embedded PostgreSQL or shared local metadata storage. * If multiple replicas are running without HA enabled in the license, Bytebase disables backend runners to avoid unsafe execution. ## Related Guides * [Production Setup](/get-started/self-host/production-setup/) * [Deploy with Kubernetes](/get-started/self-host/deploy-with-kubernetes/) * [Configure External PostgreSQL](/get-started/self-host/external-postgres/) * [Configure External URL](/get-started/self-host/external-url/) # Network Architecture and External Access Source: https://docs.bytebase.com/get-started/self-host/network-architecture This guide explains the network connectivity options for Bytebase features that require external access, including GitOps workflows, SSO authentication, and webhooks. ## Features Requiring External Access * **GitOps Workflows**: CI/CD platforms (GitHub Actions, GitLab CI, etc.) need to call Bytebase API * **SSO Authentication**: OAuth2/OIDC/SAML providers need callback access * **Webhooks**: External services sending events to Bytebase ## Three Network Patterns ### 1. Bytebase Cloud Use [Bytebase Cloud](https://cloud.bytebase.com/) for instant setup without infrastructure management. External services connect directly. **Best for:** Quick testing, evaluation, and small teams ### 2. Self-Hosted Bytebase (Production) Keep Bytebase inside your private network for security. **With cloud CI/CD** (GitHub.com, GitLab.com, Bitbucket, Azure DevOps): * Works without self-hosted runners * Cloud CI/CD can call Bytebase directly (or via the required exposed endpoint) **With self-hosted CI/CD** (GitHub Enterprise Server, self-hosted GitLab, Jenkins): * Install self-hosted runners/agents inside your network * Runners connect to your internal Bytebase without exposing it to the internet **Best for:** Production setups with strict network boundaries. ### 3. Self-Hosted Local Testing with Reverse Proxy Temporarily expose local Bytebase for testing using reverse proxy tools such as: * [ngrok](https://ngrok.com/) * [VS Code port forwarding](https://code.visualstudio.com/docs/editor/port-forwarding) * Other tunneling services Never use reverse proxy tools in production. They are for local testing only. **Important for Self-Hosted:** Remember to configure the External URL in Bytebase **Settings > General** with your public URL for SSO callbacks and other integrations to work properly. # Production Setup Source: https://docs.bytebase.com/get-started/self-host/production-setup This page provides a comprehensive checklist for setting up Bytebase in a production environment. ## System Requirements ### CPU and RAM | Users & Instances | Bytebase | [External PostgreSQL](/get-started/self-host/external-postgres/) | | ----------------------------- | ----------------- | ---------------------------------------------------------------- | | Up to 5 users / 5 instances | 1 CPU / 2 GB RAM | 1 CPU / 2 GB RAM | | Up to 20 users / 20 instances | 2 CPU / 4 GB RAM | 1 CPU / 2 GB RAM | | Up to 50 users / 50 instances | 4 CPU / 8 GB RAM | 2 CPU / 4 GB RAM | | 250+ users or 50+ instances | 8 CPU / 16 GB RAM | 2 CPU / 4 GB RAM | ### Storage Bytebase uses a PostgreSQL database to store SQL statements and other metadata. * If using cloud services (RDS, Cloud SQL) that support disk expansion, you can start with 20 GB and increase the size later as usage grows * Otherwise, start with 100 GB to avoid running out of space Teams that frequently work with large SQL files may need more. ## Production Setup Checklist * Review system requirements (see above) * [Configure External PostgreSQL](/get-started/self-host/external-postgres/) * Decide whether you need [High Availability](/get-started/self-host/high-availability/) * [Configure External Access](/get-started/self-host/external-access) * [Configure External URL](/get-started/self-host/external-url) * Pin Docker image to specific version instead of `latest` ([changelog](https://docs.bytebase.com/changelog)) While version tags like `3.7.0` are better than `latest`, Bytebase may still add patches to existing versions. For a truly fixed version, use the SHA256 digest from [Docker Hub](https://hub.docker.com/r/bytebase/bytebase/tags). ```bash theme={null} # Pin to exact image using digest (prevents any updates) docker pull bytebase/bytebase:3.7.0@sha256:00b436855b08739aad02fd52e41b5d74c03db9c70c49ce5de42376347cd1403c ``` ## High Availability Checklist If you plan to run Bytebase with multiple replicas: * Use external PostgreSQL. HA is not supported with embedded PostgreSQL * Start every replica with `--ha` * Put replicas behind one ingress, gateway, or load balancer * Set the same External URL for the whole deployment * Confirm your license has HA enabled before scaling above one replica ### Monitoring * Telemetry metrics available at `/metrics` endpoint ### Cloud Provider Recommendations If deploying on a cloud provider, use the recommended stack: #### AWS * **Compute**: EC2 or ECS/EKS * **Database**: RDS for PostgreSQL #### GCP * **Compute**: GCE or GKE * **Database**: Cloud SQL for PostgreSQL #### Azure * **Compute**: Virtual Machines or AKS * **Database**: Azure Database for PostgreSQL # Upgrade Source: https://docs.bytebase.com/get-started/self-host/upgrade ## Upgrade Process This guide walks through upgrading Bytebase with Docker. **Complete each step sequentially to ensure a smooth and safe upgrade:** ### Step 1: Stop and Remove the Existing Container First, gracefully stop and remove your current Bytebase container. This prevents data corruption by ensuring no concurrent access to the metadata. ### Step 2: Back Up Your Metadata 🚨 **Critical:** Before proceeding, create a complete backup of your metadata. This backup serves as your safety net for rolling back if needed. Refer to the "Back up and Restore" section below for comprehensive backup procedures. ### Step 3: Launch the Updated Version Start Bytebase with the new version using your existing Docker run command, updating only the image tag to the desired version. **Production Safety:** Validate your upgrade procedure in a staging environment before executing it in production. ## Back up and Restore The procedure depends on where Bytebase stores its metadata: * **Embedded PostgreSQL** — the default. You did not set `PG_URL`, and the metadata lives in the data directory mounted into the container. Back up that directory. * **[External PostgreSQL](/get-started/self-host/external-postgres/)** — you set `PG_URL`, and the metadata lives in the database named in that connection string. Clone that database. ### Embedded PostgreSQL The embedded PostgreSQL listens only on a Unix socket inside the container and shuts down together with Bytebase, so it cannot be cloned from another host. Back up the mounted data directory instead. The examples below assume the `docker run` command from [Deploy with Docker](/get-started/self-host/deploy-with-docker/), which mounts `~/.bytebase/data` at `/var/opt/bytebase`. Substitute your own host path. #### Back Up the Data Directory 1. Stop Bytebase so PostgreSQL shuts down cleanly: ```bash theme={null} docker stop bytebase ``` Add `docker rm bytebase` if you did not start the container with `--rm`. 2. Copy the host directory somewhere safe. An archive travels well: ```bash theme={null} sudo tar czf ~/bytebase-backup-$(date +%Y%m%d).tar.gz -C ~/.bytebase/data . ``` A plain directory copy is equally valid for a local rollback, as long as it preserves ownership and permissions: ```bash theme={null} sudo cp -a ~/.bytebase/data ~/bytebase-backup ``` Copy the directory only while Bytebase is stopped. A copy taken from a running instance captures PostgreSQL mid-write and may not restore. #### Restore the Data Directory 1. Stop your Bytebase instance completely. 2. Move the current data directory aside rather than deleting it, then put the backup in its place: ```bash theme={null} sudo mv ~/.bytebase/data ~/.bytebase/data.old sudo mkdir -p ~/.bytebase/data sudo tar xzf ~/bytebase-backup-.tar.gz -C ~/.bytebase/data ``` From a directory copy instead: ```bash theme={null} sudo cp -a ~/bytebase-backup/. ~/.bytebase/data ``` 3. Start Bytebase with the previous version (the version you were running before the failed upgrade). 4. After confirming successful restoration, remove `~/.bytebase/data.old`. Restore as root, and with a command that preserves ownership — `tar` as root and `cp -a` both do, a plain `cp -r` does not. Bytebase repairs the permissions of the data directory itself, but not the ownership of the files inside it, and PostgreSQL refuses to start when it does not own them. ### External PostgreSQL Replace `` in the statements below with the database named in your `PG_URL` — the same value you supplied when you [configured External PostgreSQL](/get-started/self-host/external-postgres/). To check which one you are using: ```bash theme={null} psql "$PG_URL" -c 'SELECT current_database();' ``` #### Back Up the Metadata Database Connect to another database on the same PostgreSQL server — `postgres` always exists — and clone the metadata database: ```sql theme={null} CREATE DATABASE _backup WITH TEMPLATE ; ``` `WITH TEMPLATE` requires that no other session is connected to the source database. Stop Bytebase first, or PostgreSQL fails with `source database ... is being accessed by other users`. #### Restore the Metadata Database 1. Stop your Bytebase instance completely. 2. Preserve the current database by renaming it: ```sql theme={null} ALTER DATABASE RENAME TO _old; ``` 3. Restore from your backup: ```sql theme={null} CREATE DATABASE WITH TEMPLATE _backup; ``` 4. Start Bytebase with the previous version (the version you were running before the failed upgrade). 5. After confirming successful restoration, remove temporary databases: ```sql theme={null} DROP DATABASE _old; DROP DATABASE _backup; ``` # Step 4: Add an Instance Source: https://docs.bytebase.com/get-started/step-by-step/add-an-instance **Database Instance** or simply **Instance** models after a single database server accessed via a host:port address or a primary/replica cluster. A typical database instance could be your on-premises MySQL instance, a PostgreSQL cluster, an AWS RDS instance etc. Each **Database Instance** belongs to an **Environment**. ## Prerequisites * **Workspace Admin** or **Workspace DBA** role ## Add instance If you choose to use sample data during onboarding, Bytebase will start two embedded sample PostgreSQL instances accessible from `/tmp:8083` and `/tmp:8084`. Go to **Instances** and click **Create**. Pick the database engine, then fill in the **Basic Info** — the instance name and the environment it belongs to. Choose engine and fill basic info Next, scroll down to the **Connection** section and enter how Bytebase reaches the database. Configure the connection For **Host or Socket**, if you run Bytebase inside Docker and connect to a database on the same host, use `host.docker.internal`. Check out [Instance Connection Configuration](/get-started/instance) for the detailed guide. # Step 6: Change Schema Source: https://docs.bytebase.com/get-started/step-by-step/change-schema Database changes follow four stages: **Plan**, **Review**, **Approve**, and **Rollout**. Plans become Issues when submitted for review. ## Create a Plan 1. Navigate to **CI/CD** > **Plans** and click **+ New Plan**. 2. Select target databases and click **Confirm**. 3. Add your SQL statement or use **Schema Editor** for visual design. ## Create Issue & Run SQL Review 1. Click **Create** and the plan will be in `Draft` status with automatic SQL review. SQL review checks showing a NOT NULL warning 2. Fix any warnings and click **Ready for Review** to convert the plan into an issue and await approval. ## Approve (Enterprise) 1. Issue awaits approval per [configured rules](/change-database/approval). Approvers click **Approve**. ## Deploy 1. Click **Run** to deploy manually, or configure automatic rollout in environment settings. Deploy stage rolling out to Test then Prod 2. Issue marked `Done` when complete. ## Summary Plan → Review → Approve → Rollout issue-lifecycle See [Database Change Workflow](/change-database/change-workflow/) for details. # Step 5: Create a Project Source: https://docs.bytebase.com/get-started/step-by-step/create-a-project **Project** is a logic unit to model a team effort. It's similar to the project concept in other dev tools such as Jira, GitLab. **Project** is the container to group logically related **Databases**, **Issues** and **Users** together. In Bytebase, A **Database** or an **Issue** always belongs to a single **Project**. Make sure you have configured environments, instances, and members properly at the workspace level. ## Create project Go to **Projects** and click **Create**. Give the project a **Project Name**; the **Project ID** is derived from the name and cannot be changed later. Create a project ## Transfer a database into the project To make changes to a database, you need to place it in a project first. Once a project is created, you need to fill it with some Databases. On the project's **Database** > **Databases** page, click **Create** to add a new database. Project databases To bring an existing database in, go to the workspace **Databases** page, select the database(s), and click **Transfer Project**, then pick the target project. Transfer a database into the project ## Manage the members in the project To manage fine-grained database access, check out [Database Permission](/security/database-permission/overview/). A project is only visible to its related **users**. The project creator assumes `Project Owner`role and can manage project members. Go to the project page. Click **Manage** > **Members** on the left sidebar. **Grant Access** to add new project members from workspace level accounts. You can assign project-level roles to the members. grant-access # Step 1: Deploy with Docker Source: https://docs.bytebase.com/get-started/step-by-step/deploy-with-docker It's recommended to [run Bytebase with Docker](/get-started/self-host/deploy-with-docker) which is the easiest way to get you started. Run the following command to start Bytebase on container port `8080` and map it to localhost port `8080`. ```bash theme={null} docker run --rm --init \ --name bytebase \ --publish 8080:8080 --pull always \ --volume ~/.bytebase/data:/var/opt/bytebase \ bytebase/bytebase:latest ``` Visit Bytebase at [localhost:8080](http://localhost:8080/). If you've run Bytebase before, you can reset all data by running this command: ```bash theme={null} rm -rf ~/.bytebase/data ``` # Step 7: Query Data Source: https://docs.bytebase.com/get-started/step-by-step/query-data Bytebase also provides a web-based SQL Editor for querying data. Unlike traditional client-side SQL clients, this server-side SQL Editor can enforce [fine-grained data access control](/security/database-permission/overview/), [data masking](/security/data-masking/overview/), [audit logging](/security/audit-log/), etc. ## Access SQL Editor SQL Editor can be accessed from the **SQL Editor** button in the top bar. SQL Editor button in the top bar It's also available from a database's detail page. SQL Editor link on the database detail page ## Query data Select a database to start. Selecting the hr_prod database in SQL Editor Run the query. Query result in SQL Editor Check out the detailed [SQL Editor](/sql-editor/overview) docs to explore other features. # Step 2: Register Account Source: https://docs.bytebase.com/get-started/step-by-step/register-accounts After deploying Bytebase successfully, you need to register accounts for your team members. ## Create the first admin account The first registered account is the admin account and will be granted a **Workspace Admin** role. All following registered accounts will be regular accounts. welcome-page After creating the admin account, Bytebase walks you through a short setup wizard — **Basic info**, **Setup**, and **Default landing page**. In the final step, choose your **Default landing page** ([Database Change Mode](/administration/mode)): pick **Workspace** or **SQL Editor**. Keep the default and you can always change it later. Choose the default landing page ## Manage users or members Bytebase also supports [SSO](/administration/sso/overview) and [SCIM (Directory Sync)](/administration/scim/overview). As a **Workspace Admin**, you can manage all users under **IAM & Admin** > **Users**. Click **Create** on the upper-right to add a user, or click a user row to update their information like email. Groups are managed under **IAM & Admin** > **Groups**. Manage users under IAM & Admin # Step 3: Configure Environments Source: https://docs.bytebase.com/get-started/step-by-step/set-up-environments **Environment** models after various environments in the development pipeline such as `test`, `uat`, `staging`, `prod`. ## Prerequisites * **Workspace Admin** or **Workspace DBA** role. ## View environments Go to **Environments** to view the two predefined environments - `Test` and `Prod`. Each environment carries settings such as an **Environment Tier** (mark as production), a **Rollout Policy**, and a **SQL Review** policy. Environments page Keep the default settings as-is. You can add more environments using **Create** or reorder them at any time. # File Organization Source: https://docs.bytebase.com/gitops/best-practices/file-organization Proper file organization is crucial for maintainable GitOps workflows. This guide covers repository structures and directory layouts that scale with your team. ## Repository Structure ### Monorepo Approach Database migrations alongside application code: ``` my-app/ ├── src/ # Application code ├── migrations/ # Database migrations │ ├── versioned/ │ │ ├── 001__init.sql │ │ └── 002__add_users.sql │ └── schema/ │ └── public.sql # SDL if using declarative ├── .github/workflows/ │ └── bytebase-gitops.yml # CI/CD integration └── README.md ``` **Benefits:** * Migrations versioned with application code * Atomic commits for schema + code changes * Single source of truth **Best for:** * Small to medium teams * Tight schema-code coupling * Monolithic or modular monolith architectures ### Separate Repository Dedicated database repository: ``` database-schemas/ ├── app-db/ │ ├── migrations/ │ └── schema/ ├── analytics-db/ │ └── migrations/ └── .gitlab-ci.yml ``` **Benefits:** * Separation of concerns * Independent deployment cycles * Multiple teams/databases **Best for:** * Larger organizations * Dedicated database teams * Microservices with shared databases Choose based on your team structure. Co-located migrations work well for small teams with tight schema-code coupling. Separate repos fit larger organizations with dedicated database teams. ## Directory Layout ### For Migration-Based Workflow **Option 1: Organized by Category** ``` migrations/ ├── baseline/ │ └── 000__initial_schema.sql # Initial baseline ├── features/ │ ├── 001__users.sql │ ├── 002__products.sql │ └── 003__orders.sql └── hotfixes/ └── 004__fix_index.sql ``` **Benefits:** * Clear organization by purpose * Easy to navigate * Separates routine changes from emergencies **Option 2: Flat Structure** ``` migrations/ ├── 001__initial_schema.sql ├── 002__add_users.sql ├── 003__add_products_dml.sql └── 004__add_indexes.sql ``` **Benefits:** * Simple and straightforward * Chronological ordering * Easy to understand ### For State-Based Workflow (SDL) The recommended layout is a `schema/` directory holding Bytebase's multi-file schema export - one object per file, grouped by type. The [SQL Review CI](/gitops/state-based-workflow/sql-review-ci) and [Release](/gitops/state-based-workflow/release) examples match this layout with `schema/**`: ``` schema/ └── schemas/ ├── public/ │ ├── tables/ │ │ ├── users.sql │ │ └── orders.sql │ ├── views/ │ │ └── active_users.sql │ ├── functions/ │ │ └── get_user_count.sql │ └── sequences.sql └── analytics/ └── tables/ └── events.sql ``` Alternative layouts also work - one file per schema (`schema/public.sql`, `schema/analytics.sql`) or split by object type (`01_tables.sql`, `02_indexes.sql`, ...). ``` schema/ ├── tables/ │ ├── users.sql │ └── orders.sql ├── views/ │ └── active_users.sql ├── procedures/ │ └── update_user_status.sql ├── triggers/ │ └── users_audit_trigger.sql └── events/ └── purge_expired_sessions.sql ``` MySQL has no schema namespace, so the export layout is flat. A single file holding the whole database schema also works. **Benefits:** * One object per file makes code reviews and Git diffs precise * Matches the Bytebase export, so adoption requires no manual reorganization * Predictable file structure ### Hybrid Approach Combine both workflows for different purposes: ``` database/ ├── schema/ # SDL for structure (DDL) │ ├── public.sql │ └── analytics.sql ├── migrations/ # Migrations for data (DML) │ ├── 001__seed_roles_dml.sql │ └── 002__migrate_users_dml.sql └── .github/workflows/ ├── schema-cicd.yml # SDL pipeline └── data-cicd.yml # Migration pipeline ``` **Best for:** * Teams wanting declarative schema management * Projects requiring data migrations * Gradual migration from versioned to SDL ## Documentation Structure Maintain supporting documentation alongside migrations: ``` database/ ├── migrations/ ├── schema/ ├── docs/ │ ├── CHANGELOG.md # Schema changelog │ ├── DEPENDENCIES.md # Schema-app dependencies │ └── ROLLBACK_PLAN.md # Rollback procedures └── test-data/ └── seed.sql # Test data for development ``` *** ## Next Steps Learn version numbering and file best practices Set up branching strategies and pipelines # Git and CI/CD Source: https://docs.bytebase.com/gitops/best-practices/git-and-cicd Choose the right Git branching strategy and CI/CD patterns for your team's workflow. ## Git Branching Strategies ### GitHub Flow (Simple) ``` main ─────────────○─────────────○─────→ ╲ ╱ ╱ feature-branch hotfix-branch ``` **Workflow:** 1. Create feature branch from `main` 2. Develop migration 3. Open PR, SQL review runs 4. Merge to `main` 5. CI creates release and deploys **Best for:** Continuous deployment, small teams ### GitLab Flow (Environment Branches) ``` main ─────────○─────────○─────────────→ ╲ ╲ staging production ``` **Workflow:** 1. Merge to `main` → deploys to dev 2. Merge `main` to `staging` → deploys to staging 3. Merge `staging` to `production` → deploys to prod **Best for:** Progressive environment promotion ### Git Flow (Release Branches) ``` main ──────────────────○────────○────→ (production) ╲ ╱ ╱ develop ─○─○─────○─── (dev) ╲ ╱ feature-branch ``` **Workflow:** 1. Develop in `feature` branches 2. Merge to `develop` → deploys to dev/staging 3. Create `release` branch → final testing 4. Merge to `main` → deploys to production **Best for:** Scheduled releases, large teams ## CI/CD Integration Patterns ### Pattern 1: Automated Review + Manual Deploy ```yaml theme={null} # .github/workflows/sql-review.yml on: pull_request jobs: sql-review: runs-on: ubuntu-latest steps: - uses: actions/checkout@v4 - name: SQL Review run: | # Call Bytebase API to check release curl -X POST "$BB_URL/v1/projects/my-project/releases:check" ``` ```yaml theme={null} # .github/workflows/deploy.yml on: workflow_dispatch: # Manual trigger jobs: deploy: runs-on: ubuntu-latest steps: - name: Create Release run: | # Create release via API ``` **Best for:** Production environments requiring manual control ### Pattern 2: Fully Automated Pipeline ```yaml theme={null} # .github/workflows/database-cicd.yml on: push: branches: [main] paths: ['migrations/**'] jobs: deploy: runs-on: ubuntu-latest steps: - uses: actions/checkout@v4 - name: Create Release id: release run: | RELEASE=$(curl -X POST "$BB_URL/v1/projects/my-project/releases" ...) echo "release=$RELEASE" >> $GITHUB_OUTPUT - name: Create Plan id: plan run: | PLAN=$(curl -X POST "$BB_URL/v1/projects/my-project/plans" \ -d '{"release": "${{ steps.release.outputs.release }}"}') echo "plan=$PLAN" >> $GITHUB_OUTPUT - name: Create Rollout run: | curl -X POST "$BB_URL/v1/projects/my-project/rollouts" \ -d '{"plan": "${{ steps.plan.outputs.plan }}"}' ``` **Best for:** Dev/staging environments, high-frequency deployments Complete CI/CD setup examples for GitHub, GitLab, Azure DevOps, and Bitbucket ## Testing Strategies ### Test in Non-Production First Always follow environment progression: ``` Local Dev → CI Testing → Dev Environment → Staging → Production ``` **Never skip environments for production changes.** ### Use Sampling for Large Fleets For database groups with 100+ databases: ```json theme={null} { "project": { "ci_sampling_size": 20 } } ``` During SQL review, validates on 20 random databases for faster feedback. ### Maintain Test Data Create sample datasets that mirror production: ```sql theme={null} -- test-data/seed.sql -- Realistic test data for development INSERT INTO users (username, email) VALUES ('alice', 'alice@example.com'), ('bob', 'bob@example.com'); INSERT INTO orders (user_id, total) VALUES (1, 99.99), (2, 149.99); ``` Test migrations against this data locally before committing. ### Schema Compatibility Testing Before deploying schema changes: 1. **Deploy new schema to staging** 2. **Test old application version against new schema** 3. **Verify backward compatibility** 4. **Deploy application update** 5. **Remove deprecated columns/tables later** This enables zero-downtime deployments. ### Local Testing Workflow **Before committing:** ```bash theme={null} # 1. Apply migration locally psql $LOCAL_DB -f migrations/new_migration.sql # 2. Test with sample data psql $LOCAL_DB -f test-data/seed.sql # 3. Run application tests npm test # 4. Verify rollback (if applicable) psql $LOCAL_DB -f migrations/rollback.sql ``` ### CI Testing Checklist Automated tests should verify: * ✅ SQL syntax is valid * ✅ Migration applies successfully * ✅ No SQL review rule violations * ✅ Schema changes don't break existing queries * ✅ Performance impact is acceptable * ✅ Rollback script works (if provided) *** ## Next Steps Configure review rules and security practices Step-by-step CI/CD setup guides # Migration Guidelines Source: https://docs.bytebase.com/gitops/best-practices/migration-guidelines Follow these guidelines to write maintainable, safe, and effective database migrations. ## Version Numbering Strategies ### Strategy 1: Timestamp-Based (Recommended) ``` 20250120143000__add_user_email.sql 20250121091500__create_orders_table.sql ``` **Format:** `YYYYMMDDHHmmss` **Advantages:** * ✅ No merge conflicts with parallel development * ✅ Chronological ordering * ✅ Supports distributed teams **Disadvantages:** * ⚠️ Less human-readable * ⚠️ Doesn't convey significance **Best for:** Teams with frequent parallel development ### Strategy 2: Semantic Versioning ``` v1.0.0__initial_release.sql v1.1.0__add_user_profiles.sql v1.1.1__fix_profile_constraint.sql v2.0.0__redesign_authentication.sql ``` **Format:** `v..` **Advantages:** * ✅ Conveys change significance * ✅ Aligns with application versioning * ✅ Clear breaking change indication **Disadvantages:** * ⚠️ Requires coordination * ⚠️ Merge conflicts possible **Best for:** Teams with coordinated releases ### Strategy 3: Sequential with Milestones ``` 001__initial_schema.sql 002__add_users.sql ... 100__v1_0_release.sql 101__add_analytics.sql ... 200__v2_0_release.sql ``` **Advantages:** * ✅ Simple and clear * ✅ Milestones mark releases **Disadvantages:** * ⚠️ Merge conflicts in parallel work * ⚠️ Gaps can be confusing **Best for:** Small teams, sequential development ## Migration File Best Practices ### Keep Migrations Small ```sql theme={null} -- 005__add_user_email.sql ALTER TABLE users ADD COLUMN email TEXT; ``` * Single, clear purpose * Fast execution * Easy to review * Simple to rollback ```sql theme={null} -- 005__big_refactor.sql ALTER TABLE users ADD COLUMN email TEXT; ALTER TABLE users ADD COLUMN phone TEXT; CREATE TABLE user_preferences (...); CREATE TABLE user_sessions (...); INSERT INTO user_preferences ...; -- 500 more lines ``` * Multiple unrelated changes * Long execution time * Hard to review * Difficult to troubleshoot **Guideline:** One logical change per file ### Include Rollback Planning Document rollback approach in comments: ```sql theme={null} -- 010__add_payment_methods_table.sql -- Rollback: DROP TABLE payment_methods; -- Impact: Requires app deployment v2.5.0+ CREATE TABLE payment_methods ( id SERIAL PRIMARY KEY, user_id INTEGER REFERENCES users(id), card_last_four TEXT NOT NULL, created_at TIMESTAMP DEFAULT NOW() ); ``` ### Transaction Handling **Bytebase Automatic Transactions**: Bytebase automatically wraps all SQL statements in a migration file within a single transaction. You typically don't need to add explicit `BEGIN`/`COMMIT` statements. **When Bytebase's automatic transactions are sufficient:** * Single-file migrations with multiple statements * Standard DDL operations (CREATE, ALTER, DROP) * Simple DML operations (INSERT, UPDATE, DELETE) **When you might need explicit transaction control:** * **Batched operations** requiring commits between chunks * **Long-running data migrations** that need progress checkpoints * **Statements that cannot run in a transaction block** — Bytebase auto-detects the common PostgreSQL ones (`CREATE INDEX CONCURRENTLY`, `DROP INDEX CONCURRENTLY`, `VACUUM`, `DROP DATABASE`) and runs them outside the transaction for you. For others, disable transaction mode. See [Statements that cannot run in a transaction block](/change-database/transaction-mode#statements-that-cannot-run-in-a-transaction-block) **Example of batched migration (when needed):** ```sql theme={null} -- 015__batch_archive_logs_dml.sql -- Note: This uses explicit batching for large data operation DO $$ DECLARE batch_size INT := 10000; deleted_count INT; BEGIN LOOP DELETE FROM logs WHERE id IN ( SELECT id FROM logs WHERE created_at < '2024-01-01' LIMIT batch_size ); GET DIAGNOSTICS deleted_count = ROW_COUNT; EXIT WHEN deleted_count < batch_size; COMMIT; -- Commit each batch PERFORM pg_sleep(0.1); -- Throttle END LOOP; END $$; ``` **Database-specific transaction support:** * ✅ PostgreSQL: DDL in transactions (except `CONCURRENTLY` operations) * ❌ MySQL: DDL commits immediately (implicit commit) * ✅ SQL Server: DDL in transactions ### Add Comments for Complex Logic ```sql theme={null} -- 020__migrate_legacy_permissions_dml.sql -- Migrates old role system to new permission model -- Old: roles.name -> New: permissions.scope + permissions.action -- Mapping: -- 'admin' -> 'database:*' -- 'editor' -> 'database:write' -- 'viewer' -> 'database:read' UPDATE permissions SET scope = CASE WHEN roles.name = 'admin' THEN 'database' WHEN roles.name = 'editor' THEN 'database' WHEN roles.name = 'viewer' THEN 'database' END, action = CASE WHEN roles.name = 'admin' THEN '*' WHEN roles.name = 'editor' THEN 'write' WHEN roles.name = 'viewer' THEN 'read' END FROM roles WHERE permissions.role_id = roles.id; ``` ## Documentation ### Maintain Migration Changelog Document significant schema changes: ```markdown theme={null} # Database Changelog ## v2.0.0 (2025-01-20) - Added `user_preferences` table for customization - Migrated legacy role system to new permissions model - **Breaking**: Removed deprecated `user_settings` table ## v1.5.0 (2025-01-15) - Added email column to users table - Created indexes on frequently queried columns ``` ### Document Schema Dependencies Track dependencies between application and schema: ```markdown theme={null} # Schema Dependencies ## users.email Column - Added: v1.5.0 (migration 010) - Required by: Auth Service v2.0+ - Can remove after: All instances upgraded to v2.1+ ``` This helps coordinate schema cleanup with application deployments. ## Common Anti-Patterns to Avoid ### ❌ Modifying Applied Migrations **Don't:** ```sql theme={null} -- 005__add_email.sql (already deployed) ALTER TABLE users ADD COLUMN email TEXT; -- Changed to VARCHAR ``` **Do:** ```sql theme={null} -- 006__fix_email_type.sql ALTER TABLE users ALTER COLUMN email TYPE VARCHAR(255); ``` ### ❌ Skipping Version Numbers **Don't:** ``` 001__init.sql 002__add_users.sql 010__add_products.sql ← Why skip 003-009? ``` **Do:** ``` 001__init.sql 002__add_users.sql 003__add_products.sql ``` ### ❌ Mixing DDL and DML Without Suffix **Don't:** ```sql theme={null} -- 015__add_orders.sql CREATE TABLE orders (...); INSERT INTO orders VALUES (...); -- DML without _dml suffix ``` **Do:** ```sql theme={null} -- 015__add_orders.sql (DDL only) CREATE TABLE orders (...); -- 016__seed_orders_dml.sql (DML separate) INSERT INTO orders VALUES (...); ``` ### ❌ Long-Running Migrations in Production **Don't:** ```sql theme={null} -- 020__huge_migration.sql UPDATE users SET legacy_field = NULL; -- Locks 10M rows for 10 minutes ``` **Do:** ```sql theme={null} -- 020__migrate_users_batch_dml.sql -- Batch processing with throttling (see Performance section) ``` *** ## Next Steps Learn branching strategies and testing patterns Optimize migration performance # Overview Source: https://docs.bytebase.com/gitops/best-practices/overview This guide provides battle-tested patterns and recommendations for production GitOps workflows with Bytebase. ## What You'll Learn This best practices guide covers: * **File Organization** - Repository structures and directory layouts for different team sizes * **Migration Guidelines** - Version numbering strategies, file best practices, and common pitfalls * **Git and CI/CD** - Branching strategies, CI/CD patterns, and testing approaches * **SQL Review and Security** - Review configuration and security best practices * **Performance** - Optimization techniques for migrations ## Who Should Read This This guide is designed for: * **DevOps Engineers** setting up database CI/CD pipelines * **Database Administrators** migrating to GitOps workflows * **Development Teams** implementing database version control * **Platform Engineers** building self-service database platforms ## Prerequisites Before diving into best practices, you should have: * Basic understanding of [GitOps Workflow](/gitops/overview) * Familiarity with either [Migration-Based](/gitops/migration-based-workflow/overview) or [State-Based](/gitops/state-based-workflow/overview) workflows * [GitOps integration installed](/gitops/installation) ## Quick Reference ### Critical Best Practices **File Management:** * Use timestamp-based versioning for parallel development * Keep migrations small and focused (one logical change per file) * Never modify applied migrations **Testing:** * Always test in non-production first: Local → Dev → Staging → Production * Maintain test data that mirrors production * Test schema compatibility before deployment **Security:** * Use service accounts for CI/CD * Grant least privilege database access * Store secrets in secret management systems **Performance:** * Batch large data migrations * Use online schema changes for large tables * Create indexes concurrently when possible *** ## Get Started Repository structures and directory layouts Version numbering and migration best practices Branching strategies and testing patterns Review configuration and security practices Migration optimization techniques # Performance Optimization Source: https://docs.bytebase.com/gitops/best-practices/performance Optimize migration performance in your GitOps workflow. ## Performance Optimization ### Batch Large Data Migrations For DML on large tables, process in batches: ```sql theme={null} -- 050__archive_old_logs_dml.sql -- Instead of single large DELETE: -- ❌ DELETE FROM logs WHERE created_at < '2024-01-01'; -- Use batched approach: DO $$ DECLARE batch_size INT := 10000; deleted_count INT; BEGIN LOOP DELETE FROM logs WHERE id IN ( SELECT id FROM logs WHERE created_at < '2024-01-01' LIMIT batch_size ); GET DIAGNOSTICS deleted_count = ROW_COUNT; EXIT WHEN deleted_count < batch_size; COMMIT; -- If supported PERFORM pg_sleep(0.1); -- Throttle END LOOP; END $$; ``` **Benefits:** * Reduces lock contention * Prevents transaction log bloat * Allows monitoring progress * Can be paused/resumed ### Use Online Schema Changes for Large Tables For MySQL tables > 1M rows: ```sql theme={null} -- Instead of: -- ❌ 010__add_index.sql CREATE INDEX idx_users_email ON users(email); -- Use gh-ost: -- ✅ 010__add_index_ghost.sql ALTER TABLE users ADD INDEX idx_users_email (email); ``` Configure gh-ost for zero-downtime MySQL migrations ### Optimize Index Creation Create indexes concurrently when possible: ```sql theme={null} -- PostgreSQL: Non-blocking index creation CREATE INDEX CONCURRENTLY idx_users_email ON users(email); -- MySQL: Use online DDL ALTER TABLE users ADD INDEX idx_email (email), ALGORITHM=INPLACE, LOCK=NONE; ``` **Note:** `CONCURRENTLY` requires: * Cannot be in transaction block * Takes longer than regular index creation * May fail if concurrent writes conflict ### Timing Considerations **Best times for migrations:** * ✅ Low-traffic hours (nights, weekends) * ✅ During maintenance windows * ✅ After application deployments **Avoid:** * ❌ Peak business hours * ❌ During critical operations (month-end, sales events) * ❌ Without communication to stakeholders ### Monitor Migration Performance Track execution time during development: ```sql theme={null} -- Add timing to migration \timing on -- Migration SQL here \timing off ``` Set expectations for production: ```sql theme={null} -- 015__large_migration.sql -- Expected duration: 5-10 minutes -- Locks: users table (write lock) -- Tested on: 2M row table ``` ## Performance Monitoring ### Track Migration Metrics Monitor these metrics: | Metric | Target | Alert Threshold | | --------------------- | ---------- | --------------- | | Migration duration | \< 5 min | > 10 min | | Lock wait time | \< 1 sec | > 5 sec | | Rows affected | Documented | > Expected | | Rollback success rate | 100% | \< 100% | ### Database Health Checks Before and after migrations: ```sql theme={null} -- Check table sizes SELECT schemaname, tablename, pg_size_pretty(pg_total_relation_size(schemaname||'.'||tablename)) AS size FROM pg_tables WHERE schemaname = 'public' ORDER BY pg_total_relation_size(schemaname||'.'||tablename) DESC; -- Check index usage SELECT schemaname, tablename, indexname, idx_scan, pg_size_pretty(pg_relation_size(indexrelid)) AS size FROM pg_stat_user_indexes WHERE idx_scan < 100 ORDER BY pg_relation_size(indexrelid) DESC; ``` *** ## Next Steps Solutions for common GitOps issues Configure gh-ost for zero-downtime changes Return to GitOps overview # SQL Review and Security Source: https://docs.bytebase.com/gitops/best-practices/sql-review-and-security Configure SQL review rules and implement security best practices for production GitOps workflows. ## SQL Review Configuration Configure SQL review rules to enforce standards across your team. Configure 200+ linting rules for automated validation ### Recommended Rules **Critical Rules (ERROR level):** * ❌ `DROP DATABASE` * ❌ `DROP TABLE` (without confirmation) * ❌ Missing `WHERE` clause in `UPDATE`/`DELETE` * ❌ `NOT NULL` on existing columns without default * ❌ Charset changes on existing columns **Warning Rules (WARN level):** * ⚠️ Missing indexes on foreign keys * ⚠️ Column without comments * ⚠️ Table without primary key * ⚠️ Large `IN` clause (> 1000 items) **Info Rules:** * 💡 Consider partitioning for large tables * 💡 Index naming convention suggestions ### Example Policy ```json theme={null} { "rule_list": [ { "type": "naming.table", "level": "ERROR", "payload": { "format": "^[a-z_]+$" } }, { "type": "statement.select.no-select-all", "level": "WARNING" }, { "type": "column.required", "level": "WARNING", "payload": { "column_list": ["created_at", "updated_at"] } } ] } ``` ### Review Severity Levels Configure how different rule violations are handled: | Level | Behavior | Use Case | | ----------- | -------------------------- | ---------------------------------------- | | **ERROR** | Blocks merge | Dangerous operations, critical standards | | **WARNING** | Allows merge with approval | Best practices, style guidelines | | **INFO** | Informational only | Suggestions, optimization tips | ## Security Best Practices ### Use Service Accounts Create dedicated service accounts for CI/CD: ```bash theme={null} # Don't use personal accounts ❌ export BB_TOKEN="user-alice-token" # Use service accounts ✅ export BB_TOKEN="service-account-cicd-token" ``` **Service account setup:** 1. Create service account in Bytebase 2. Grant the `GitOps Service Agent` role for automated CI/CD workflows 3. Store token in CI/CD secrets 4. Rotate tokens regularly The `GitOps Service Agent` role is specifically designed for CI/CD integrations with minimal permissions required for automated deployments. Learn about service account authentication ### Least Privilege Database Access Configure Bytebase with minimal database permissions: **For schema changes:** ```sql theme={null} GRANT CREATE, ALTER, DROP ON DATABASE app_db TO bytebase_user; ``` **For readonly access:** ```sql theme={null} GRANT SELECT ON ALL TABLES IN SCHEMA public TO bytebase_readonly; ``` Avoid using superuser accounts. ### Protect Sensitive Migrations For migrations containing sensitive data: ```sql theme={null} -- 099__seed_api_keys_dml.sql -- WARNING: Contains sensitive data -- Ensure this file is not committed to version control INSERT INTO api_credentials (service, key) VALUES ('payment_gateway', '${PAYMENT_API_KEY}'), ('email_service', '${EMAIL_API_KEY}'); ``` **Alternatives:** * Store secrets in secret management systems (AWS Secrets Manager, HashiCorp Vault) * Reference secrets via environment variables in CI/CD * Use Bytebase secret integration Configure database connections with secret managers ### Secrets Management **Option 1: CI/CD Secrets** ```yaml theme={null} # .github/workflows/deploy.yml env: BYTEBASE_TOKEN: ${{ secrets.BYTEBASE_TOKEN }} DB_PASSWORD: ${{ secrets.DB_PASSWORD }} ``` **Option 2: Secret Manager** ```yaml theme={null} # Use AWS Secrets Manager - name: Get secrets run: | aws secretsmanager get-secret-value \ --secret-id bytebase/cicd \ --query SecretString ``` **Option 3: External Secret Store** ```bash theme={null} # Use HashiCorp Vault export BYTEBASE_TOKEN=$(vault kv get -field=token secret/bytebase) ``` ### Audit and Compliance Enable comprehensive audit logging: ```json theme={null} { "project": { "audit_log_retention_days": 365 } } ``` **What gets logged:** * All schema changes * Who approved changes * When deployments occurred * Access to sensitive data * Policy violations Configure audit logging for compliance ### Network Security **Restrict Bytebase Access:** * Use VPN or private networking for production * Enable IP allowlisting * Use TLS for all connections * Implement firewall rules **Database Connection Security:** ```yaml theme={null} # Use SSL/TLS for database connections database: ssl: enabled: true ca_cert: /path/to/ca.pem verify_mode: require ``` ### Role-Based Access Control Configure appropriate roles: | Role | Permissions | Use Case | | ------------- | -------------------------- | ----------------------- | | **Owner** | Full access | Team leads, admins | | **DBA** | Schema changes, admin mode | Database administrators | | **Developer** | Create issues, query data | Application developers | | **Releaser** | Deploy to production | Release engineers | | **Querier** | Query data only | Analysts, support | Configure role-based access control ### Code Review Security Security checklist for PR/MR reviews: * ✅ No hardcoded secrets or passwords * ✅ No `SELECT *` exposing sensitive columns * ✅ Proper `WHERE` clauses to prevent mass updates * ✅ No `DROP` statements without explicit approval * ✅ Appropriate indexes to prevent performance issues * ✅ Data access follows compliance requirements *** ## Next Steps Optimize migration performance Complete reference of 200+ rules Configure audit logging Protect sensitive data # Installation Source: https://docs.bytebase.com/gitops/installation This guide covers the setup requirements for connecting Version Control Systems (VCS) with Bytebase. For understanding network patterns and external access options, see the [Network Architecture guide](/get-started/self-host/network-architecture). ## Bytebase Cloud **No installation required** for cloud VCS providers: * GitHub.com * GitLab.com * Bitbucket Cloud * Azure DevOps Services These services connect directly to Bytebase Cloud. ### Self-Hosted GitLab * Install a [GitLab Runner](https://docs.gitlab.com/runner/) on your GitLab server * Ensure your VPC firewall rules allow connections between Self-Hosted GitLab and Bytebase Cloud ## bytebase-action Version Compatibility Use a `bytebase-action` image that matches your Bytebase deployment: * For Bytebase Cloud, use `bytebase/bytebase-action:cloud`. * For self-hosted Bytebase, use `bytebase/bytebase-action:`, for example `bytebase/bytebase-action:3.14.0`. `bytebase-action check` and `bytebase-action rollout` verify compatibility before running. Cloud action builds must not be newer than the Bytebase Cloud server build and must be no more than 7 days behind it. Self-hosted action builds must use the same major version and be no more than 2 minor versions behind the Bytebase server; newer action builds are not compatible with older servers. ## Bytebase Self-Host Self-hosted Bytebase requires specific setup based on your VCS provider. ### Cloud VCS Providers When using cloud VCS with self-hosted Bytebase, you need a self-hosted runner: **GitHub.com** * Install a [GitHub self-hosted runner](https://docs.github.com/en/actions/hosting-your-own-runners/about-self-hosted-runners) * Ensure runner can access your Bytebase instance **GitLab.com** * Install a [GitLab Runner](https://docs.gitlab.com/runner/) * Register runner with your GitLab.com project **Bitbucket Cloud** * Install a [Bitbucket self-hosted runner](https://support.atlassian.com/bitbucket-cloud/docs/runners/) * Configure Bitbucket Pipelines to use the runner **Azure DevOps Services** * Install a [Self-hosted Azure Pipelines Agent](https://docs.microsoft.com/en-us/azure/devops/pipelines/agents/agents) * Register agent with your Azure DevOps organization ### Self-Hosted GitLab * Install a [GitLab Runner](https://docs.gitlab.com/runner/) on your GitLab server * Ensure VPC connectivity between GitLab and Bytebase servers * Network policies allow communication * If your GitLab runner lacks internet access, pre-load the bytebase-action Docker image ## Local Testing To expose local Bytebase for testing with cloud VCS services, use reverse proxy tools like ngrok. See [Network Architecture guide](/get-started/self-host/network-architecture#3-self-hosted-local-testing-with-reverse-proxy) for setup details. For development and testing only. Never use in production. # Develop Source: https://docs.bytebase.com/gitops/migration-based-workflow/develop Create SQL migration files following naming conventions that enable proper version tracking and execution order. ## File Naming Convention Migration filenames must follow this structure: ``` _.sql ``` **Components:** 1. **Version** (required) - Optional `v` or `V` prefix, followed by one or more numbers separated by dots * Pattern: `^[vV]?(\d+(\.\d+)*)` * Examples: `v1.2.3`, `1.0`, `V2` 2. **Underscore** (`_`) separator 3. **Description** - Human-readable description using underscores or hyphens 4. **`.sql`** file extension ## Version Formats Choose a versioning strategy that fits your team: **Timestamp-Based** - Recommended for teams with parallel development ``` 20250120143000_add_user_email.sql 20250121091500_create_orders_table.sql ``` Format: `YYYYMMDDHHmmss` ✅ No merge conflicts ✅ Chronological ordering ✅ Supports distributed teams ⚠️ Less human-readable **Semantic Versioning** - Meaningful version numbers ``` v1.0.0_initial_release.sql v1.1.0_add_user_profiles.sql v1.1.1_fix_profile_constraint.sql v2.0.0_redesign_authentication.sql ``` ✅ Conveys change significance ✅ Aligns with application versioning ⚠️ Requires coordination ⚠️ Merge conflicts possible **Simple Sequential** - Easy to understand ``` 001_initial_schema.sql 002_add_users.sql 003_add_products.sql ``` ✅ Simple and clear ✅ Milestones mark releases ⚠️ Merge conflicts in parallel work ## Migration Type (MySQL Only) For zero-downtime MySQL schema changes, add this comment at the top of your file: ```sql theme={null} -- gh-ost = {} ALTER TABLE users ADD COLUMN email VARCHAR(255); ``` This uses [gh-ost](https://github.com/github/gh-ost) to apply changes without blocking your database. Use `{}` to run with default flags, or pass gh-ost flags as a JSON object: ```sql theme={null} -- gh-ost = {"max-lag-millis":"1500","cut-over-lock-timeout-seconds":"10"} ALTER TABLE users ADD COLUMN email VARCHAR(255); ``` Learn more about gh-ost *** ## Next Steps Set up automated validation in your CI/CD pipeline Deploy your migrations to databases # Limitations Source: https://docs.bytebase.com/gitops/migration-based-workflow/limitations While migration-based workflow is powerful and flexible, be aware of these considerations: ## Version Conflicts **Issue:** Multiple developers creating migrations with the same version number. **Solution:** Use timestamp-based versioning (`YYYYMMDDHHmmss`) to avoid conflicts. ## Migration Order Dependencies **Issue:** Complex dependencies between migrations can be hard to manage. **Solution:** * Keep migrations small and focused * Use descriptive names * Document dependencies in comments * Consider state-based workflow for pure schema changes ## Rollback Complexity **Issue:** Reverting migrations requires writing reverse scripts. **Solution:** * Plan rollback strategy before deploying * Test rollback scripts in staging * Consider using Bytebase's data rollback feature for DML changes Automatic rollback for INSERT/UPDATE/DELETE operations ## Large-Scale Deployments **Issue:** Deploying to hundreds or thousands of databases takes time. **Solution:** * Use database groups for fleet management * Configure parallel execution limits * Enable sampling for validation (check N random databases) Production-ready workflow patterns and optimizations *** ## Next Steps Learn about the declarative alternative Production-ready workflow patterns Solutions for common issues Return to GitOps overview # Overview Source: https://docs.bytebase.com/gitops/migration-based-workflow/overview The migration-based workflow is the **imperative** approach to database schema management, where you write incremental SQL files that explicitly describe each change to apply. This is the traditional approach familiar to database administrators and developers using tools like Flyway, Liquibase, or Rails migrations. ## How It Works Each migration file contains explicit DDL or DML statements: ```sql theme={null} -- migrations/001__create_users.sql CREATE TABLE users ( id SERIAL PRIMARY KEY, username TEXT NOT NULL UNIQUE ); -- migrations/002__add_email.sql ALTER TABLE users ADD COLUMN email TEXT; -- migrations/003__seed_admin_dml.sql INSERT INTO users (username, email) VALUES ('admin', 'admin@example.com'); ``` Bytebase executes these files sequentially based on version numbers and tracks which versions have been applied to each database through the revision system, ensuring migrations are never re-executed. ## When to Use Migration-Based Workflow ✅ **Use migration-based workflow when:** * You need data migrations (INSERT, UPDATE, DELETE operations) * You require precise control over the execution order * Your team is familiar with traditional migration tools * You're working with any database type (MySQL, PostgreSQL, SQL Server, MongoDB, etc.) * You need to perform complex multi-step transformations * You want explicit versioning of each database change ## Complete Workflow The migration-based workflow consists of three stages: ``` 1. Develop → 2. SQL Review (PR/MR) → 3. Release (Bytebase) ``` ### Stage 1: Develop Write migration files following the naming convention and commit to your repository. ### Stage 2: SQL Review (PR/MR) Open a pull/merge request. Automated SQL review runs in CI/CD. ### Stage 3: Release (Bytebase) After merge, CI/CD triggers Bytebase to create a release and deploy to target databases. ## Tutorials *** ## Next Steps Learn about file naming conventions and versioning strategies Set up automated SQL validation in your CI/CD pipeline Deploy migrations to your databases Understand constraints and considerations # Release Source: https://docs.bytebase.com/gitops/migration-based-workflow/release After PR/MR approval and merge, your CI/CD pipeline triggers Bytebase to create a release and deploy the migrations. Non-bot PR/MR authors observed by `bytebase-release` count as active VCS users against your plan's user limit. See [How Users Are Counted](/administration/license#how-users-are-counted). ## How Releases Work A **Release** is an immutable package containing all your SQL migration files: * Linked to VCS commit for full traceability * Files validated and stored with SHA256 checksums * Can be deployed to multiple environments * Supports progressive rollout strategies **Inside Bytebase, the release triggers:** 1. **Plan Generation** - Defines target databases and rollout strategy 2. **Rollout Execution** - Creates tasks for each database 3. **Revision Tracking** - Records applied migrations to prevent duplicates ## CI/CD Integration Examples Use a compatible `bytebase-action` image. For Bytebase Cloud, use `bytebase/bytebase-action:cloud`. For self-hosted Bytebase, replace `:cloud` with your Bytebase server version, for example `:3.14.0`. ```yaml theme={null} # .github/workflows/release.yml name: Database Release on: push: branches: [main] paths: - 'migrations/**' jobs: release: runs-on: ubuntu-latest steps: - uses: actions/checkout@v4 - name: Create Bytebase Release uses: bytebase/sql-review-action@v1 with: command: rollout bytebase-url: ${{ secrets.BYTEBASE_URL }} bytebase-token: ${{ secrets.BYTEBASE_SERVICE_ACCOUNT_TOKEN }} file-pattern: 'migrations/**/*.sql' project: projects/my-project targets: instances/prod/databases/mydb ``` ```yaml theme={null} # .gitlab-ci.yml rollout: image: bytebase/bytebase-action:cloud stage: deploy only: - main script: - | bytebase-action rollout \ --url $BYTEBASE_URL \ --service-account $BYTEBASE_SERVICE_ACCOUNT \ --service-account-secret $BYTEBASE_SERVICE_ACCOUNT_SECRET \ --file-pattern "migrations/**/*.sql" \ --project projects/my-project \ --targets instances/prod/databases/mydb ``` ```yaml theme={null} # azure-pipelines.yml trigger: branches: include: - main paths: include: - migrations/** jobs: - job: Release pool: vmImage: 'ubuntu-latest' steps: - task: Docker@2 inputs: command: run arguments: > -e BYTEBASE_URL=$(BYTEBASE_URL) -e BYTEBASE_SERVICE_ACCOUNT=$(BYTEBASE_SERVICE_ACCOUNT) -e BYTEBASE_SERVICE_ACCOUNT_SECRET=$(BYTEBASE_SERVICE_ACCOUNT_SECRET) -v $(System.DefaultWorkingDirectory):/workspace bytebase/bytebase-action:cloud rollout --file-pattern "/workspace/migrations/**/*.sql" --project projects/my-project --targets instances/prod/databases/mydb ``` ```yaml theme={null} # bitbucket-pipelines.yml pipelines: branches: main: - step: name: Database Release image: bytebase/bytebase-action:cloud deployment: production script: - | bytebase-action rollout \ --url $BYTEBASE_URL \ --service-account $BYTEBASE_SERVICE_ACCOUNT \ --service-account-secret $BYTEBASE_SERVICE_ACCOUNT_SECRET \ --file-pattern "migrations/**/*.sql" \ --project projects/my-project \ --targets instances/prod/databases/mydb ``` ## Deployment Strategies **Progressive rollout across environments:** ```yaml theme={null} # Deploy to dev, then staging, then production targets: > instances/dev/databases/mydb, instances/staging/databases/mydb, instances/prod/databases/mydb ``` **Multi-tenant deployment using database groups:** ```yaml theme={null} # Deploy to entire production fleet targets: projects/my-project/databaseGroups/production-fleet ``` Manage database fleets with groups ## Idempotency Guarantee Bytebase tracks which migration versions have been applied to each database via the **revision system**. When creating a release: 1. **Check revisions** - Query which versions already exist 2. **Skip applied** - Migrations with matching versions are skipped 3. **Execute new** - Only unapplied versions are executed 4. **Record success** - Create revision only on successful completion This ensures: * ✅ Safe to run the same release multiple times * ✅ Migrations never double-execute * ✅ Environment parity (dev migrations auto-skip in prod) *** ## Next Steps Understand constraints and considerations Production-ready workflow patterns # SQL Review CI Source: https://docs.bytebase.com/gitops/migration-based-workflow/sql-review-ci Automated SQL review validates your migration files during pull/merge requests, catching issues before they reach production. ## What Gets Validated **Automated checks include:** * SQL syntax validation * Policy rule enforcement (200+ configurable rules) * Naming convention compliance * Risk assessment for dangerous operations * Schema compatibility checks Configure 200+ linting rules for automated validation ## Setup **Prerequisites:** 1. [Set up SQL Review Policy](/sql-review/review-policy) - Define your SQL standards 2. [Install GitOps Integration](/gitops/installation) - Connect Bytebase to your VCS Use a compatible `bytebase-action` image. For Bytebase Cloud, use `bytebase/bytebase-action:cloud`. For self-hosted Bytebase, replace `:cloud` with your Bytebase server version, for example `:3.14.0`. ## CI/CD Integration Examples ```yaml theme={null} # .github/workflows/sql-review.yml name: SQL Review on: pull_request: paths: - 'migrations/**' jobs: sql-review: runs-on: ubuntu-latest steps: - uses: actions/checkout@v4 - name: SQL Review uses: bytebase/sql-review-action@v1 with: bytebase-url: ${{ secrets.BYTEBASE_URL }} bytebase-token: ${{ secrets.BYTEBASE_SERVICE_ACCOUNT_TOKEN }} file-pattern: 'migrations/**/*.sql' ``` GitHub PR SQL Review ```yaml theme={null} # .gitlab-ci.yml sql-review: image: bytebase/bytebase-action:cloud stage: test only: - merge_requests script: - | bytebase-action check \ --url $BYTEBASE_URL \ --service-account $BYTEBASE_SERVICE_ACCOUNT \ --service-account-secret $BYTEBASE_SERVICE_ACCOUNT_SECRET \ --file-pattern "migrations/**/*.sql" ``` GitLab PR SQL Review ```yaml theme={null} # azure-pipelines.yml trigger: none pr: branches: include: - main paths: include: - migrations/** jobs: - job: SQLReview pool: vmImage: 'ubuntu-latest' steps: - task: Docker@2 inputs: command: run arguments: > -e BYTEBASE_URL=$(BYTEBASE_URL) -e BYTEBASE_SERVICE_ACCOUNT=$(BYTEBASE_SERVICE_ACCOUNT) -e BYTEBASE_SERVICE_ACCOUNT_SECRET=$(BYTEBASE_SERVICE_ACCOUNT_SECRET) -v $(System.DefaultWorkingDirectory):/workspace bytebase/bytebase-action:cloud check --file-pattern "/workspace/migrations/**/*.sql" ``` Azure DevOps PR SQL Review ```yaml theme={null} # bitbucket-pipelines.yml pipelines: pull-requests: '**': - step: name: SQL Review image: bytebase/bytebase-action:cloud script: - | bytebase-action check \ --url $BYTEBASE_URL \ --service-account $BYTEBASE_SERVICE_ACCOUNT \ --service-account-secret $BYTEBASE_SERVICE_ACCOUNT_SECRET \ --file-pattern "migrations/**/*.sql" ``` Bitbucket PR SQL Review ## Review Bot Feedback The SQL review bot posts detailed feedback as PR/MR comments: * ✅ **Passed checks** - Migration meets all policy requirements * ⚠️ **Warnings** - Best practice violations (non-blocking by default) * ❌ **Errors** - Policy violations (blocking by default) * 📊 **Risk assessment** - Evaluation of potential impact * 📝 **Detailed explanations** - Why rules failed with fix suggestions *** ## Next Steps Deploy your migrations after review approval Explore all available SQL review rules # Overview Source: https://docs.bytebase.com/gitops/overview ## Goal GitOps for database CI/CD brings the same rigor and automation used in application development to database schema management. Database schema changes are equivalent to API changes in microservices - they define contracts between services, and misalignment can break production systems. By treating databases as dependencies that require versioning, compatibility management, and progressive rollout strategies, schema changes gain the same safety and predictability as application code. ## Why Database Schema Changes Matter Database schema changes are fundamentally similar to API changes: * They define contracts between services * Breaking changes can cause application failures * They require versioning and compatibility considerations * They benefit from progressive rollout and feature flagging Bytebase enables database-as-code workflows, allowing you to manage database changes through your version control system (VCS) with the same process and confidence as application code. ## Core Capabilities ### Database-as-Code Workflow * **Unified CI/CD Pipeline** - Integrate database changes seamlessly into your existing CI/CD workflows * **Version Control Integration** - Manage schema changes alongside application code in your VCS * **Automated Change Detection** - Intelligently detect and apply only necessary changes, ensuring idempotency * **Progressive Deployment** - Roll out changes across environments (development → staging → production) with configurable automation and approval gates ### Enterprise-Grade Safety * **SQL Review and Validation** - Automatic policy enforcement during pull requests * **Rollback Capabilities** - Support for safe schema rollbacks when needed * **Multi-Region Support** - Deploy to isolated regional databases with separate or unified Bytebase deployments * **Batch Operations** - Apply changes across multiple databases consistently ## GitOps Workflow Bytebase GitOps follows a streamlined 3-stage pipeline: ``` Develop → Review (PR/MR) → Release (Bytebase) ``` ### 1. Develop Developers create SQL files in feature branches following your team's Git workflow (GitHub Flow, GitLab Flow, trunk-based, etc.). **What happens:** * Write SQL migration or schema files * Follow naming conventions for proper versioning * Test changes locally when possible * Commit to version control ### 2. Review (Pull Request / Merge Request) When developers open a pull/merge request, automated validation ensures changes meet your organization's standards. **What happens:** * **Automated SQL Review** - Validates syntax and enforces policies * **Risk Assessment** - Identifies high-risk operations * **Schema Compatibility** - Checks for breaking changes * **Team Review** - Human review of schema changes alongside code ### 3. Release (Bytebase) After PR/MR approval and merge, Bytebase orchestrates the deployment. This single stage encompasses **Plan → Rollout → Revision** tracking, all managed within Bytebase. **What happens in Bytebase:** 1. **Release Creation** - Immutable package of SQL files linked to VCS commit 2. **Plan Generation** - Defines deployment strategy and target databases 3. **Rollout Execution** - Creates and executes tasks across environments 4. **Revision Tracking** - Records applied migrations to prevent duplicates The Release stage in Bytebase involves several coordinated steps: * **Plan**: Defines which databases to target and rollout strategy * **Rollout**: Executes the plan, creating stages for progressive deployment * **Revision**: Tracks which migrations have been applied to each database For detailed information about these components, see: * Plan creation and targeting strategies * Rollout execution and approval workflows * Revision-based idempotency These topics are covered in detail within each workflow documentation. **Key Principle**: Bytebase automatically detects previously applied changes by checking revision history and skips them, ensuring safe re-deployment and idempotent operations across all environments. ## Two Workflow Approaches Bytebase supports two complementary approaches for managing database schema changes. Choose the one that fits your team's needs, or use both for different scenarios. ### Migration-Based Workflow The **imperative** approach where you write incremental change scripts that describe how to modify the schema. ```sql theme={null} -- migrations/001__create_users.sql CREATE TABLE users (id INT PRIMARY KEY, name TEXT); -- migrations/002__add_email.sql ALTER TABLE users ADD COLUMN email TEXT; ``` **Best for:** * Teams familiar with traditional database migration tools * Complex changes requiring specific execution order * Data migrations (INSERT, UPDATE, DELETE operations) * All supported databases (MySQL, PostgreSQL, SQL Server, etc.) * Situations requiring fine-grained control over each change **How it works:** Each migration file contains explicit DDL/DML statements. Files execute sequentially based on version numbers. Bytebase tracks which versions have been applied to prevent re-execution. Complete guide to migration-based approach with examples and best practices ### State-Based Workflow (SDL) The **declarative** approach where you define the desired end-state of your database schema, and Bytebase automatically generates the migration DDL. ```sql theme={null} -- schema/public.sql (complete desired state) CREATE TABLE public.users ( id INTEGER PRIMARY KEY, name TEXT NOT NULL, email TEXT ); ``` **Best for:** * Teams adopting infrastructure-as-code principles * Pure schema changes (DDL only, no data operations) * Simplified Git diffs showing schema evolution * PostgreSQL and MySQL databases (currently) * Reducing complexity around migration ordering **How it works:** You maintain the complete schema definition. Bytebase compares your desired state with the current database state and automatically generates the necessary ALTER/DROP statements to reach the desired state. Complete guide to state-based (SDL) approach with validation rules and examples ## Choosing Between Workflows | Aspect | Migration-Based | State-Based (SDL) | | -------------------- | ------------------------------- | ----------------------------------- | | **Approach** | Imperative (how to change) | Declarative (desired end state) | | **Files** | Multiple migration scripts | Single schema definition per module | | **Git History** | Shows individual changes | Shows current state + diffs | | **Operations** | DDL + DML | DDL only | | **Data Changes** | ✅ Supported | ❌ Not supported | | **Database Support** | All databases | PostgreSQL and MySQL (currently) | | **Learning Curve** | Familiar to DBAs | Familiar to DevOps/IaC users | | **Control** | Explicit control over each step | Automatic diff generation | | **Rollback** | Write reverse migration | Define previous state | **Can I use both?** Yes! You can use state-based workflow for schema structure and migration-based workflow for data operations in the same project. ## Next Steps Traditional approach with incremental changes Modern declarative schema management Set up VCS integration and runners Production-ready workflow patterns # Develop Source: https://docs.bytebase.com/gitops/state-based-workflow/develop Create SDL files that define your complete database schema in a declarative format. ## Getting Started To use state-based workflow, you must start by exporting your database schema. Bytebase provides a built-in export feature that generates SDL files in the multi-file format, ready to commit to your Git repository. ### Quick Start Workflow 1. **Export** - Click `Export Schema` → `Multi-File (ZIP)` in Bytebase database detail page 2. **Extract** - Unzip the downloaded file into a `schema/` directory in your Git repository; the [SQL Review CI](/gitops/state-based-workflow/sql-review-ci) and [Release](/gitops/state-based-workflow/release) examples match files under `schema/**` 3. **Verify** - Check the exported structure follows the [multi-file format](#file-organization) 4. **Commit** - Add files to Git and push to your repository 5. **Make Changes** - Edit the SDL files to modify your schema 6. **Deploy** - Commit changes to trigger deployment via GitOps The exported ZIP contains a complete, ready-to-use schema structure. You can immediately commit it to Git without any manual organization. ### How to Export Schema 1. **Navigate to Database Detail Page** - Go to your database in Bytebase 2. **Click "Export Schema" Button** - Find it in the action buttons at the top 3. **Choose Export Format**: * **Multi-File (ZIP)** - Recommended for GitOps workflow * Downloads organized schema as a ZIP file * Each object in its own file (e.g., `schemas/public/tables/users.sql` for PostgreSQL, `tables/users.sql` for MySQL) * Ready to extract and commit to Git * **Single File** - All objects in one SQL file * Useful for quick review or legacy workflows * Requires manual splitting if you want multi-file format ### What Gets Exported The multi-file export automatically includes: * **Tables** - With inline indexes, comments, and owned sequences * **Views** - Regular views with their definitions * **Materialized Views** - With their indexes and refresh settings * **Functions & Procedures** - User-defined functions and stored procedures * **Sequences** - Independent sequences consolidated per schema * **Comments** - All object and column comments * **Tables** - With inline indexes, foreign keys, check constraints, generated columns, partitions, and table options * **Views** - Regular views with their definitions * **Functions & Procedures** - User-defined functions and stored procedures * **Triggers** - With their timing, event, and body * **Events** - Scheduled events with their schedule and body ## File Organization Bytebase uses a **multi-file format** where each database object is stored in a separate file. PostgreSQL objects are organized by schema and object type; MySQL has no schema namespace (schema equals database), so it uses a flat layout organized by object type only. This structure provides clear organization and makes code reviews easier. ### Directory Structure ``` schemas/ ├── public/ │ ├── tables/ │ │ ├── users.sql │ │ ├── orders.sql │ │ └── products.sql │ ├── views/ │ │ ├── active_users.sql │ │ └── order_summary.sql │ ├── materialized_views/ │ │ └── sales_summary.sql │ ├── functions/ │ │ └── get_user_count.sql │ ├── procedures/ │ │ └── update_user_status.sql │ └── sequences.sql ├── analytics/ │ ├── tables/ │ │ └── events.sql │ └── views/ │ └── daily_stats.sql └── internal/ └── tables/ └── audit_logs.sql ``` ``` tables/ ├── users.sql ├── orders.sql └── products.sql views/ ├── active_users.sql └── order_summary.sql functions/ └── get_user_count.sql procedures/ └── update_user_status.sql triggers/ └── users_audit_trigger.sql events/ └── purge_expired_sessions.sql ``` ### Organization Rules * **One object per file** - Each table, view, function, etc. has its own file * **Schema-based grouping** - Objects are organized under their schema directory * **Type-based subdirectories** - Within each schema, objects are grouped by type (`tables/`, `views/`, etc.) * **Sequences consolidated** - Independent sequences are grouped in a single `sequences.sql` file per schema * **Related objects bundled** - Table files include their indexes, comments, and owned sequences * **One object per file** - Each table, view, function, procedure, trigger, and event has its own file * **Flat type-based layout** - Objects are grouped by type at the top level (`tables/`, `views/`, `triggers/`, ...); there is no schema level * **Related objects bundled** - Table files include their indexes, foreign keys, check constraints, partitions, and table options inline in the `CREATE TABLE` statement ### File Naming Files are named after the database object they define: * Tables: `schemas/{schema}/tables/{table_name}.sql` * Views: `schemas/{schema}/views/{view_name}.sql` * Materialized Views: `schemas/{schema}/materialized_views/{materialized_view_name}.sql` * Functions: `schemas/{schema}/functions/{function_name}.sql` * Procedures: `schemas/{schema}/procedures/{procedure_name}.sql` * Sequences: `schemas/{schema}/sequences.sql` (consolidated) * Tables: `tables/{table_name}.sql` * Views: `views/{view_name}.sql` * Functions: `functions/{function_name}.sql` * Procedures: `procedures/{procedure_name}.sql` * Triggers: `triggers/{trigger_name}.sql` * Events: `events/{event_name}.sql` ### Example: Table File Content A table file includes the table definition and all related objects: ```sql theme={null} -- schemas/public/tables/users.sql CREATE TABLE public.users ( id INTEGER NOT NULL DEFAULT nextval('public.users_id_seq'::regclass), username TEXT NOT NULL, email TEXT NOT NULL, created_at TIMESTAMP NOT NULL DEFAULT CURRENT_TIMESTAMP, CONSTRAINT users_pkey PRIMARY KEY (id), CONSTRAINT users_username_key UNIQUE (username), CONSTRAINT users_email_key UNIQUE (email) ); COMMENT ON TABLE public.users IS 'Application users'; COMMENT ON COLUMN public.users.username IS 'Unique username for login'; CREATE INDEX idx_users_email ON public.users(email); CREATE INDEX idx_users_created_at ON public.users(created_at); COMMENT ON INDEX idx_users_email IS 'Index for email lookups'; -- Owned sequence CREATE SEQUENCE public.users_id_seq; ALTER SEQUENCE public.users_id_seq OWNED BY public.users.id; ``` ```sql theme={null} -- tables/users.sql CREATE TABLE `users` ( `id` int NOT NULL AUTO_INCREMENT, `username` varchar(64) NOT NULL COMMENT 'Unique username for login', `email` varchar(255) NOT NULL, `created_at` timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP, PRIMARY KEY (`id`), UNIQUE KEY `uk_users_email` (`email`), UNIQUE KEY `uk_users_username` (`username`), KEY `idx_users_created_at` (`created_at`) ) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COMMENT='Application users'; ``` Everything belonging to the table - indexes, foreign keys, check constraints, partitions, comments, and table options - lives inline in the single `CREATE TABLE` statement, matching `SHOW CREATE TABLE` conventions. ### Example: Materialized View File Content (PostgreSQL) Materialized view files include the view definition and its indexes: ```sql theme={null} -- schemas/public/materialized_views/sales_summary.sql CREATE MATERIALIZED VIEW public.sales_summary AS SELECT DATE_TRUNC('day', created_at) AS sale_date, COUNT(*) AS total_orders, SUM(amount) AS total_amount FROM public.orders GROUP BY sale_date; COMMENT ON MATERIALIZED VIEW public.sales_summary IS 'Daily sales aggregation'; CREATE INDEX idx_sales_summary_date ON public.sales_summary(sale_date); COMMENT ON INDEX idx_sales_summary_date IS 'Index on sale date for fast lookups'; ``` ## SDL Syntax Requirements PostgreSQL SDL enforces strict conventions to ensure maintainability: ### 1. Schema Qualification Required All objects must include schema prefix: ```sql theme={null} ✅ Correct - Fully qualified names CREATE TABLE public.users (...); CREATE INDEX idx_name ON public.users(name); CREATE FUNCTION public.get_user(...) RETURNS ...; ``` ```sql theme={null} ❌ Incorrect - Missing schema CREATE TABLE users (...); CREATE INDEX idx_name ON users(name); ``` ### 2. Table-Level Constraints PRIMARY KEY, UNIQUE, FOREIGN KEY, and CHECK constraints must be defined at table level with explicit names: ```sql theme={null} ✅ Correct - Table-level with names CREATE TABLE public.users ( id INTEGER NOT NULL, -- NOT NULL allowed at column level email TEXT NOT NULL, -- NOT NULL allowed at column level created_at TIMESTAMP DEFAULT NOW(), -- DEFAULT allowed at column level CONSTRAINT users_pkey PRIMARY KEY (id), CONSTRAINT users_email_key UNIQUE (email), CONSTRAINT users_email_check CHECK (email LIKE '%@%') ); ``` ```sql theme={null} ❌ Incorrect - Column-level constraints CREATE TABLE public.users ( id INTEGER PRIMARY KEY, -- Must be at table level email TEXT UNIQUE, -- Must be at table level age INTEGER CHECK (age >= 0) -- Must be at table level ); ``` **Allowed at column level:** * `NOT NULL` * `DEFAULT` * `GENERATED ALWAYS AS` * `SERIAL` / `BIGSERIAL` ### 3. Named Constraints All table constraints require explicit names: ```sql theme={null} ✅ Correct - Named constraints CONSTRAINT users_pkey PRIMARY KEY (id) CONSTRAINT users_email_key UNIQUE (email) CONSTRAINT fk_orders_users FOREIGN KEY (user_id) REFERENCES public.users(id) CONSTRAINT check_positive_age CHECK (age > 0) ``` ```sql theme={null} ❌ Incorrect - Unnamed constraints PRIMARY KEY (id) UNIQUE (email) FOREIGN KEY (user_id) REFERENCES public.users(id) CHECK (age > 0) ``` **Naming conventions:** * Primary keys: `{table}_pkey` * Unique constraints: `{table}_{column}_key` * Foreign keys: `fk_{table}_{referenced_table}` * Check constraints: `check_{description}` or `{table}_{column}_check` ### 4. Foreign Key References Foreign keys must use fully qualified table names: ```sql theme={null} ✅ Correct - Fully qualified reference CREATE TABLE public.orders ( id INTEGER NOT NULL, user_id INTEGER NOT NULL, CONSTRAINT orders_pkey PRIMARY KEY (id), CONSTRAINT fk_orders_users FOREIGN KEY (user_id) REFERENCES public.users(id) ); ``` ```sql theme={null} ❌ Incorrect - Unqualified reference CONSTRAINT fk_orders_users FOREIGN KEY (user_id) REFERENCES users(id) ``` ### 5. Named Indexes All indexes must have explicit names: ```sql theme={null} ✅ Correct - Named indexes CREATE INDEX idx_users_email ON public.users(email); CREATE UNIQUE INDEX idx_users_username ON public.users(username); ``` ```sql theme={null} ❌ Incorrect - Unnamed index CREATE INDEX ON public.users(email); ``` MySQL SDL accepts standard `CREATE` syntax as produced by `SHOW CREATE` and the Bytebase schema export. The recommended workflow is to start from the export and edit it, keeping these conventions in mind: * **Unqualified object names** - There is no schema namespace; write `users`, not `mydb.users`. References to the file's own database are normalized to the unqualified form. * **Inline table definitions** - Indexes, foreign keys, and check constraints are written inline in the `CREATE TABLE` statement, matching `SHOW CREATE TABLE` output. Standalone `CREATE INDEX` statements are also accepted. * **No `DELIMITER` blocks** - Write function, procedure, trigger, and event bodies directly; SDL files are parsed as a whole, so no delimiter switching is needed. * **Omit `DEFINER` clauses** - Bytebase does not manage definers; the export omits them, and they don't participate in schema comparison. * **`AUTO_INCREMENT` counter values are ignored** - A table option like `AUTO_INCREMENT=1024` is runtime state, not schema; the export strips it and the comparison never generates a diff for it. ## Supported Statements SDL files support these PostgreSQL statements: * `CREATE TABLE` * `CREATE INDEX` / `CREATE UNIQUE INDEX` * `CREATE VIEW` * `CREATE SEQUENCE` * `CREATE FUNCTION` * `ALTER SEQUENCE ... OWNED BY` (for serial columns) **Not Allowed:** * `ALTER TABLE`, `DROP` (Bytebase generates these) * `INSERT`, `UPDATE`, `DELETE` (use migration-based workflow) * Transaction control (`BEGIN`, `COMMIT`) SDL files support these MySQL statements: * `CREATE TABLE` * `CREATE INDEX` / `CREATE UNIQUE INDEX` / `CREATE FULLTEXT INDEX` / `CREATE SPATIAL INDEX` * `CREATE VIEW` / `CREATE OR REPLACE VIEW` * `CREATE FUNCTION` * `CREATE PROCEDURE` * `CREATE TRIGGER` * `CREATE EVENT` **Not Allowed:** * `ALTER`, `DROP`, `RENAME`, `TRUNCATE` (Bytebase generates the migration DDL) * `CREATE TABLE ... AS SELECT` (imperative; cannot be represented as a schema snapshot) * `INSERT`, `UPDATE`, `DELETE`, `SELECT` (use migration-based workflow) SDL rejects top-level DML only. Function, procedure, trigger, and event **bodies** can still contain DML that runs when the object executes - review those bodies with the same care as data changes. ## Complete Multi-File SDL Example ### schemas/public/tables/users.sql ```sql theme={null} CREATE TABLE public.users ( id INTEGER NOT NULL DEFAULT nextval('public.users_id_seq'::regclass), username TEXT NOT NULL, email TEXT NOT NULL, created_at TIMESTAMP NOT NULL DEFAULT CURRENT_TIMESTAMP, status TEXT NOT NULL DEFAULT 'active', CONSTRAINT users_pkey PRIMARY KEY (id), CONSTRAINT users_username_key UNIQUE (username), CONSTRAINT users_email_key UNIQUE (email), CONSTRAINT check_email_format CHECK (email ~* '^[A-Z0-9._%+-]+@[A-Z0-9.-]+\.[A-Z]{2,}$'), CONSTRAINT check_status_values CHECK (status IN ('active', 'inactive', 'suspended')) ); COMMENT ON TABLE public.users IS 'Application users'; COMMENT ON COLUMN public.users.email IS 'User email address (unique)'; CREATE INDEX idx_users_email ON public.users(email); CREATE INDEX idx_users_created_at ON public.users(created_at); CREATE INDEX idx_users_status ON public.users(status) WHERE status != 'inactive'; -- Owned sequence for id column CREATE SEQUENCE public.users_id_seq; ALTER SEQUENCE public.users_id_seq OWNED BY public.users.id; ``` ### schemas/public/tables/user\_profiles.sql ```sql theme={null} CREATE TABLE public.user_profiles ( user_id INTEGER NOT NULL, bio TEXT, avatar_url TEXT, CONSTRAINT user_profiles_pkey PRIMARY KEY (user_id), CONSTRAINT fk_user_profiles_users FOREIGN KEY (user_id) REFERENCES public.users(id) ON DELETE CASCADE ); COMMENT ON TABLE public.user_profiles IS 'Extended user profile information'; ``` ### schemas/public/views/active\_users.sql ```sql theme={null} CREATE VIEW public.active_users AS SELECT id, username, email, created_at FROM public.users WHERE status = 'active'; COMMENT ON VIEW public.active_users IS 'View of all active users'; ``` ### schemas/public/functions/get\_user\_count.sql ```sql theme={null} CREATE FUNCTION public.get_user_count() RETURNS INTEGER LANGUAGE sql AS $$ SELECT COUNT(*) FROM public.users; $$; COMMENT ON FUNCTION public.get_user_count() IS 'Returns total count of users'; ``` ### schemas/public/sequences.sql ```sql theme={null} -- Independent sequences (not owned by any table column) CREATE SEQUENCE public.custom_id_seq START WITH 1000 INCREMENT BY 1 NO MINVALUE NO MAXVALUE CACHE 1; COMMENT ON SEQUENCE public.custom_id_seq IS 'Custom sequence for special IDs'; ``` ### tables/users.sql ```sql theme={null} CREATE TABLE `users` ( `id` int NOT NULL AUTO_INCREMENT, `username` varchar(64) NOT NULL, `email` varchar(255) NOT NULL, `status` varchar(16) NOT NULL DEFAULT 'active', `created_at` timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP, PRIMARY KEY (`id`), UNIQUE KEY `uk_users_email` (`email`), UNIQUE KEY `uk_users_username` (`username`), KEY `idx_users_created_at` (`created_at`), CONSTRAINT `check_status_values` CHECK (`status` IN ('active', 'inactive', 'suspended')) ) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COMMENT='Application users'; ``` ### tables/orders.sql ```sql theme={null} CREATE TABLE `orders` ( `id` int NOT NULL AUTO_INCREMENT, `user_id` int NOT NULL, `amount` decimal(10,2) NOT NULL, `created_at` timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP, PRIMARY KEY (`id`), KEY `idx_orders_user_id` (`user_id`), CONSTRAINT `fk_orders_users` FOREIGN KEY (`user_id`) REFERENCES `users` (`id`) ON DELETE CASCADE ) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4; ``` ### views/active\_users.sql ```sql theme={null} CREATE OR REPLACE VIEW `active_users` AS SELECT `id`, `username`, `email`, `created_at` FROM `users` WHERE `status` = 'active'; ``` ### events/purge\_expired\_sessions.sql ```sql theme={null} CREATE EVENT `purge_expired_sessions` ON SCHEDULE EVERY 1 DAY DO DELETE FROM `sessions` WHERE `expires_at` < NOW(); ``` *** ## Next Steps Set up SDL validation in your CI/CD pipeline Deploy your SDL changes to databases # Limitations Source: https://docs.bytebase.com/gitops/state-based-workflow/limitations State-based workflow (SDL) currently has these limitations: ## Database Support **Supported:** * PostgreSQL * MySQL (5.7 and 8.0) **Not yet supported:** * SQL Server * Oracle * MongoDB * Other databases ## Supported SQL Statements Only these PostgreSQL statements are supported: * `CREATE TABLE` * `CREATE INDEX` / `CREATE UNIQUE INDEX` * `CREATE VIEW` * `CREATE SEQUENCE` * `CREATE FUNCTION` * `ALTER SEQUENCE` (for OWNED BY) **Not supported:** * Complex stored procedures * Triggers * Row-level security policies * DML operations (INSERT, UPDATE, DELETE) * Transaction control * Database-level settings Only these MySQL statements are supported: * `CREATE TABLE` * `CREATE INDEX` / `CREATE UNIQUE INDEX` / `CREATE FULLTEXT INDEX` / `CREATE SPATIAL INDEX` * `CREATE VIEW` / `CREATE OR REPLACE VIEW` * `CREATE FUNCTION` * `CREATE PROCEDURE` * `CREATE TRIGGER` * `CREATE EVENT` **Not supported:** * `ALTER`, `DROP`, `RENAME`, `TRUNCATE` (Bytebase generates the migration DDL) * `CREATE TABLE ... AS SELECT` * DML operations (INSERT, UPDATE, DELETE) * Transaction control * Database-level settings (default charset/collation changes are not diffed) ## Strict Syntax Requirements PostgreSQL SDL requires strict adherence to conventions: 1. **All objects must use fully qualified names** (with schema prefix) ```sql theme={null} -- Required: public.users -- Not allowed: users ``` 2. **PRIMARY KEY, UNIQUE, FOREIGN KEY, CHECK must be table-level with explicit names** ```sql theme={null} -- Correct: CONSTRAINT users_pkey PRIMARY KEY (id) -- Not allowed: id INTEGER PRIMARY KEY ``` 3. **Only NOT NULL, DEFAULT, GENERATED allowed at column level** ```sql theme={null} -- Allowed: id SERIAL, name TEXT NOT NULL, created_at TIMESTAMP DEFAULT NOW() -- Not allowed: id INTEGER PRIMARY KEY ``` 4. **Foreign key references must be fully qualified** ```sql theme={null} -- Required: REFERENCES public.users(id) -- Not allowed: REFERENCES users(id) ``` 5. **All indexes must have explicit names** ```sql theme={null} -- Required: CREATE INDEX idx_users_email ON public.users(email) -- Not allowed: CREATE INDEX ON public.users(email) ``` MySQL SDL accepts standard `CREATE` syntax as produced by `SHOW CREATE` and the Bytebase schema export. Conventions: 1. **Use unqualified object names** - MySQL has no schema namespace; write `users`, not `mydb.users` 2. **Write routine, trigger, and event bodies directly** - no `DELIMITER` blocks 3. **Omit `DEFINER` clauses** - definers are not managed by SDL 4. **Omit `AUTO_INCREMENT=N` table options** - counter values are runtime state, not schema, and are ignored by the comparison ## MySQL Considerations * **CHECK constraints require MySQL 8.0.16+**: MySQL 5.7 parses and ignores CHECK constraints, so they are not compared on 5.7. * **DEFINER is not managed**: `DEFINER` clauses are omitted from the export and ignored in comparison. * **Session context is preserved for routines, triggers, and events only**: functions, procedures, triggers, and events keep their original `sql_mode` and character-set context when recreated; events also keep their `time_zone`. Views do not carry session context. * **Column renames are drop + add**: renaming a column in an SDL file generates `DROP COLUMN` + `ADD COLUMN`, which loses the column's data. Use migration-based workflow for renames. * **Partition changes regenerate the whole clause**: any change to a table's partitioning emits a full `ALTER TABLE ... PARTITION BY ...` (or `REMOVE PARTITIONING`), not incremental `ADD/DROP PARTITION` statements. * **Trigger ordering is not tracked**: `FOLLOWS` / `PRECEDES` clauses are not part of the comparison. * **Event start times are normalized**: MySQL stores an implicit `STARTS` timestamp on every recurring event, so a change that only modifies an explicit `STARTS` value is not detected as a diff. Because `DEFINER` is not managed, views, routines, triggers, and events are recreated with the deploying account as their definer. For `SQL SECURITY DEFINER` views and routines - and for triggers and events, which always execute as their definer - this changes the execution security context: a more privileged deploy account can broaden runtime privileges, and a less privileged one can break existing callers. Use a dedicated deploy account, or the migration-based workflow, when definer identity matters. ## No Direct Data Operations SDL only manages schema structure. For data operations, use migration-based workflow. **Not supported in SDL:** * INSERT statements * UPDATE statements * DELETE statements * Data transformation logic SDL rejects top-level DML only. MySQL function, procedure, trigger, and event **bodies** can still contain DML that runs when the object executes - for example, a scheduled event that purges rows. Review these bodies with the same care as data changes. **Solution:** Combine both workflows: ``` schema/ # SDL for schema structure ├── tables.sql └── indexes.sql migrations/ # Migration-based for data ├── 001__seed_roles_dml.sql └── 002__migrate_users_dml.sql ``` ## Destructive Operations SDL-generated DROP statements execute automatically when objects are removed from files: ```sql theme={null} -- Remove table from SDL file -- → Bytebase generates: DROP TABLE old_table; ``` Always backup data before deploying SDL changes that remove objects from schema files. Bytebase will automatically drop those objects. ## Limited Rollback SDL only moves forward to new desired states: * No automatic rollback generation * To rollback: revert SDL files to previous state and redeploy * Data in dropped objects is lost (backup required) Approaches for reversing schema changes ## Performance Considerations For large schemas: * Initial SDL adoption requires exporting complete schema * State comparison time increases with schema complexity * DDL generation uses topological sort (handles dependencies) **Mitigation:** * Organize schema into multiple files * Use database groups for fleet management * Test SDL workflow on staging first *** ## Next Steps Learn about the imperative alternative Production-ready workflow patterns Solutions for common issues Return to GitOps overview # Overview Source: https://docs.bytebase.com/gitops/state-based-workflow/overview The state-based workflow uses **SDL (Schema Definition Language)** - a declarative approach where you define the desired end-state of your database schema, and Bytebase automatically generates the migration DDL. This approach mirrors infrastructure-as-code principles used by Kubernetes, Terraform, and other modern DevOps tools. ## How It Works Instead of writing incremental changes, you maintain the complete schema definition: ```sql theme={null} -- schema/public.sql (complete desired state) CREATE TABLE public.users ( id INTEGER, name TEXT NOT NULL, email TEXT, CONSTRAINT users_pkey PRIMARY KEY (id), CONSTRAINT users_email_key UNIQUE (email) ); CREATE INDEX idx_users_email ON public.users(email); ``` ```sql theme={null} -- tables/users.sql (complete desired state) CREATE TABLE `users` ( `id` int NOT NULL, `name` varchar(255) NOT NULL, `email` varchar(255), PRIMARY KEY (`id`), UNIQUE KEY `uk_users_email` (`email`), KEY `idx_users_name` (`name`) ) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4; ``` When you update this file and deploy, Bytebase: 1. Compares your desired state with the current database 2. Automatically generates the necessary ALTER/DROP statements 3. Executes the generated migration to reach the desired state ## When to Use State-Based Workflow ✅ **Use state-based workflow when:** * You're managing pure schema changes (DDL only) * Your team embraces infrastructure-as-code principles * You want Git-friendly diffs showing schema evolution * You need simplified management without tracking migration order * You're working with PostgreSQL or MySQL databases (currently supported) * You prefer automatic dependency resolution over manual ordering ❌ **Don't use state-based workflow when:** * You need data migrations (INSERT, UPDATE, DELETE) * You require complex multi-step logic * You're working with SQL Server, Oracle, or other databases (not yet supported) **Hybrid Approach:** Use state-based workflow for schema structure and migration-based workflow for data operations in the same project. ## Complete Workflow The state-based workflow follows the same three stages: ``` 1. Develop → 2. SQL Review (PR/MR) → 3. Release (Bytebase) ``` ### Stage 1: Develop Maintain complete schema definition files representing desired state. ### Stage 2: SQL Review (PR/MR) Open a pull/merge request. SDL validation runs in CI/CD. ### Stage 3: Release (Bytebase) After merge, Bytebase compares states, generates DDL, and deploys. *** ## Next Steps Learn about SDL syntax and requirements Set up SDL validation in your CI/CD pipeline Deploy SDL changes to your databases Understand constraints and considerations # Release Source: https://docs.bytebase.com/gitops/state-based-workflow/release After PR/MR merge, Bytebase creates a release, compares the desired state with current database state, generates migration DDL, and deploys. ## How SDL Release Works 1. **State Comparison** - Bytebase compares your SDL files with current database schema 2. **DDL Generation** - Automatically generates ALTER/DROP statements using topological sort 3. **Migration Execution** - Applies generated DDL to reach desired state 4. **Revision Tracking** - Records new version for future comparisons ## CI/CD Integration Add `--declarative` flag to enable SDL mode: Use a compatible `bytebase-action` image. For Bytebase Cloud, use `bytebase/bytebase-action:cloud`. For self-hosted Bytebase, replace `:cloud` with your Bytebase server version, for example `:3.14.0`. The examples assume your SDL files live under a `schema/` directory - for example `schema/tables/users.sql` (MySQL) or `schema/schemas/public/tables/users.sql` (PostgreSQL). If your files live elsewhere, adjust the trigger paths and `--file-pattern`. The pattern supports `**` globs and brace alternation, for example `{schemas,tables,views,functions,procedures,triggers,events}/**/*.sql` for an export extracted at the repository root. ```yaml theme={null} # .github/workflows/release.yml name: SDL Release on: push: branches: [main] paths: - 'schema/**' jobs: release: runs-on: ubuntu-latest steps: - uses: actions/checkout@v4 - name: Deploy SDL uses: bytebase/sql-review-action@v1 with: command: rollout bytebase-url: ${{ secrets.BYTEBASE_URL }} bytebase-token: ${{ secrets.BYTEBASE_SERVICE_ACCOUNT_TOKEN }} file-pattern: 'schema/**/*.sql' project: projects/my-project targets: instances/prod/databases/mydb declarative: true ``` ```yaml theme={null} # .gitlab-ci.yml sdl-rollout: image: bytebase/bytebase-action:cloud stage: deploy only: - main script: - | bytebase-action rollout \ --url $BYTEBASE_URL \ --service-account $BYTEBASE_SERVICE_ACCOUNT \ --service-account-secret $BYTEBASE_SERVICE_ACCOUNT_SECRET \ --file-pattern "schema/**/*.sql" \ --project projects/my-project \ --targets instances/prod/databases/mydb \ --declarative ``` ```yaml theme={null} # azure-pipelines.yml jobs: - job: SDLRelease pool: vmImage: 'ubuntu-latest' steps: - task: Docker@2 inputs: command: run arguments: > bytebase/bytebase-action:cloud rollout --file-pattern "/workspace/schema/**/*.sql" --project projects/my-project --targets instances/prod/databases/mydb --declarative ``` ## Version Management SDL automatically generates versions using timestamp format `YYYYMMDD.HHMMSS`: ``` Release 2025-01-15 10:30:00 → Version: 20250115.103000 Release 2025-01-20 14:15:00 → Version: 20250120.141500 ``` All files in a release share the same version. Bytebase compares this version with the latest revision to determine if deployment is needed. ## Migration Generation Example **Current Database:** ```sql theme={null} CREATE TABLE public.users ( id INTEGER PRIMARY KEY, username TEXT NOT NULL ); ``` **New SDL:** ```sql theme={null} CREATE TABLE public.users ( id INTEGER, username TEXT NOT NULL, email TEXT NOT NULL, CONSTRAINT users_pkey PRIMARY KEY (id), CONSTRAINT users_email_key UNIQUE (email) ); CREATE INDEX idx_users_email ON public.users(email); ``` **Generated Migration:** ```sql theme={null} -- Bytebase automatically generates: ALTER TABLE public.users ADD COLUMN email TEXT NOT NULL; ALTER TABLE public.users ADD CONSTRAINT users_email_key UNIQUE (email); CREATE INDEX idx_users_email ON public.users(email); ``` **Current Database:** ```sql theme={null} CREATE TABLE `users` ( `id` int NOT NULL, `username` varchar(64) NOT NULL, PRIMARY KEY (`id`) ) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4; ``` **New SDL:** ```sql theme={null} CREATE TABLE `users` ( `id` int NOT NULL, `username` varchar(64) NOT NULL, `email` varchar(255) NOT NULL, PRIMARY KEY (`id`), UNIQUE KEY `uk_users_email` (`email`), KEY `idx_users_username` (`username`) ) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4; ``` **Generated Migration:** ```sql theme={null} -- Bytebase automatically generates: ALTER TABLE `users` ADD COLUMN `email` varchar(255) NOT NULL; ALTER TABLE `users` ADD UNIQUE KEY `uk_users_email` (`email`); ALTER TABLE `users` ADD KEY `idx_users_username` (`username`); ``` *** ## Next Steps Understand SDL constraints and considerations Production-ready workflow patterns # SQL Review CI Source: https://docs.bytebase.com/gitops/state-based-workflow/sql-review-ci SDL validation runs automatically during pull/merge requests to catch syntax and convention violations before merge. ## What Gets Validated **SDL-specific checks:** * Schema qualification on all objects * Table-level constraint placement * Constraint naming requirements * Foreign key type matching * Cross-file integrity validation * Unsupported statement detection * SQL syntax validation * SQL syntax validation * Unsupported statement detection (`ALTER`, `DROP`, DML, `CREATE TABLE ... AS SELECT`) * Dependency cycle detection across objects * Destructive-change warnings for dropped and redefined objects - these compare your SDL against the current database state, so pass `--project` and `--targets` for your real target databases to the `check` command **AI-powered validation with your team's standards (Optional):** * Define standards in natural language (any language) * Enforce team-specific conventions * Get context-aware recommendations * Validate naming conventions and best practices ## AI-Powered Validation with Your Standards Enhance SDL validation by teaching AI your team's SQL standards using natural language. Define your project's conventions, naming rules, and best practices in any language - AI will validate your schema against them. ### How It Works 1. **Write Your Standards**: Document your team's SQL conventions in natural language (supports any language) 2. **AI Validates**: Bytebase AI analyzes your schema files against your documented standards 3. **Get Feedback**: Receive specific, actionable feedback on any violations with exact file and line references ### Setup Requirements AI must be enabled and configured in your Bytebase instance to use this feature. Without AI setup, your standards file will be ignored during validation. **Step 1: Enable AI in Bytebase** (One-time setup) * Navigate to **Settings** → **General** → **AI Assistant** * Enable AI and choose your provider (OpenAI, Azure OpenAI, Gemini, or Claude) * Enter your API credentials and test the connection **Step 2: Document Your Standards** * Create `.bytebase/sql-review.md` in your repository * Write your team's SQL standards in natural language - no special syntax required **Step 3: Update Your CI/CD Pipeline** * Add the `--custom-rules` flag pointing to your standards file * See platform-specific examples in the [CI/CD Integration](#cicd-integration) section ### Example: SQL Review Standards Create a Markdown file with your SQL review standards. We recommend placing it in `.bytebase/sql-review.md`: ```markdown theme={null} # .bytebase/sql-review.md # SQL Review Standards ## 1. Table Naming Convention - All table names must be in snake_case - Table names should be plural nouns (e.g., users, orders, products) - Avoid abbreviations unless commonly understood ## 2. Column Standards - Every table must have a created_at timestamp column - Every table must have an updated_at timestamp column - Primary key columns should be named 'id' ## 3. Index Requirements - Foreign key columns must have indexes - Columns used in WHERE clauses frequently should be indexed ## 4. Comment Requirements - All tables must have descriptive comments - Complex columns should have comments explaining their purpose ## 5. Type Consistency - Use TEXT instead of VARCHAR for string columns - Use TIMESTAMP for datetime values - Use BIGINT for auto-incrementing primary keys ``` ### AI Validation Output When AI detects violations, you'll see detailed feedback in your PR/MR: ``` ❌ AI-powered validation errors found: File: schemas/public/tables/users.sql Rule: Table Naming Convention Message: Table name 'User' should be plural. Consider renaming to 'users'. Line: 1 File: schemas/public/tables/products.sql Rule: Column Standards Message: Table 'products' is missing required 'created_at' timestamp column. Line: 1 File: schemas/public/tables/orders.sql Rule: Index Requirements Message: Foreign key column 'user_id' should have an index for better query performance. Line: 5 ``` ### Best Practices for SQL Review Standards 1. **Be Specific and Actionable**: Clear, specific standards produce better AI analysis * ✅ "Primary key columns must be named 'id'" * ❌ "Use good naming conventions" 2. **Organize by Category**: Group related standards for clarity ```markdown theme={null} ## Naming Conventions - Tables: plural, snake_case - Columns: singular, snake_case ## Type Standards - Timestamps: use TIMESTAMP NOT NULL - Text fields: use TEXT not VARCHAR ``` 3. **Document the "Why"**: Explain the rationale behind each standard ```markdown theme={null} ## Primary Key Type Use BIGINT for primary keys (not INTEGER) to prevent ID exhaustion in high-volume tables and support future growth. ``` 4. **Start Small, Iterate**: Begin with a few critical standards and expand based on team needs 5. **Version Control Standards**: Commit `.bytebase/sql-review.md` to your repository alongside schema files for team collaboration ## CI/CD Integration Configure SDL validation in your CI/CD pipeline. Choose between: * **Basic SDL Validation**: Built-in checks for syntax, naming, and structure * **With Your Team's Standards**: AI validates against your custom standards in `.bytebase/sql-review.md` Use a compatible `bytebase-action` image. For Bytebase Cloud, use `bytebase/bytebase-action:cloud`. For self-hosted Bytebase, replace `:cloud` with your Bytebase server version, for example `:3.14.0`. The examples assume your SDL files live under a `schema/` directory - for example `schema/tables/users.sql` (MySQL) or `schema/schemas/public/tables/users.sql` (PostgreSQL). If your files live elsewhere, adjust the trigger paths and `--file-pattern`. The pattern supports `**` globs and brace alternation, for example `{schemas,tables,views,functions,procedures,triggers,events}/**/*.sql` for an export extracted at the repository root. ```yaml theme={null} # .github/workflows/sql-review.yml name: SDL Review on: pull_request: paths: - 'schema/**' jobs: sdl-review: runs-on: ubuntu-latest steps: - uses: actions/checkout@v4 - name: SDL Validation run: | docker run --rm \ -v ${{ github.workspace }}:/workspace \ bytebase/bytebase-action:cloud \ check \ --url ${{ secrets.BYTEBASE_URL }} \ --service-account ${{ secrets.BYTEBASE_SERVICE_ACCOUNT }} \ --service-account-secret ${{ secrets.BYTEBASE_SERVICE_ACCOUNT_SECRET }} \ --file-pattern "/workspace/schema/**/*.sql" \ --declarative ``` **Prerequisites:** AI must be enabled in Bytebase Settings → AI Assistant ```yaml theme={null} # .github/workflows/sql-review.yml name: SDL Review with Team Standards on: pull_request: paths: - 'schema/**' - '.bytebase/sql-review.md' jobs: sdl-review: runs-on: ubuntu-latest steps: - uses: actions/checkout@v4 - name: SDL Validation with Team Standards run: | docker run --rm \ -v ${{ github.workspace }}:/workspace \ bytebase/bytebase-action:cloud \ check \ --url ${{ secrets.BYTEBASE_URL }} \ --service-account ${{ secrets.BYTEBASE_SERVICE_ACCOUNT }} \ --service-account-secret ${{ secrets.BYTEBASE_SERVICE_ACCOUNT_SECRET }} \ --file-pattern "/workspace/schema/**/*.sql" \ --declarative \ --custom-rules "$(cat ${{ github.workspace }}/.bytebase/sql-review.md)" ``` ```yaml theme={null} # .gitlab-ci.yml sdl-review: image: bytebase/bytebase-action:cloud stage: test only: - merge_requests script: - | bytebase-action check \ --url $BYTEBASE_URL \ --service-account $BYTEBASE_SERVICE_ACCOUNT \ --service-account-secret $BYTEBASE_SERVICE_ACCOUNT_SECRET \ --file-pattern "schema/**/*.sql" \ --declarative ``` **Prerequisites:** AI must be enabled in Bytebase Settings → AI Assistant ```yaml theme={null} # .gitlab-ci.yml sdl-review: image: bytebase/bytebase-action:cloud stage: test only: - merge_requests script: - | bytebase-action check \ --url $BYTEBASE_URL \ --service-account $BYTEBASE_SERVICE_ACCOUNT \ --service-account-secret $BYTEBASE_SERVICE_ACCOUNT_SECRET \ --file-pattern "schema/**/*.sql" \ --declarative \ --custom-rules "$(cat .bytebase/sql-review.md)" ``` ```yaml theme={null} # azure-pipelines.yml jobs: - job: SDLReview pool: vmImage: 'ubuntu-latest' steps: - script: | docker run --rm \ -v $(Build.SourcesDirectory):/workspace \ bytebase/bytebase-action:cloud \ check \ --url $(BYTEBASE_URL) \ --service-account $(BYTEBASE_SERVICE_ACCOUNT) \ --service-account-secret $(BYTEBASE_SERVICE_ACCOUNT_SECRET) \ --file-pattern "/workspace/schema/**/*.sql" \ --declarative displayName: 'SDL Validation' ``` **Prerequisites:** AI must be enabled in Bytebase Settings → AI Assistant ```yaml theme={null} # azure-pipelines.yml jobs: - job: SDLReview pool: vmImage: 'ubuntu-latest' steps: - script: | docker run --rm \ -v $(Build.SourcesDirectory):/workspace \ bytebase/bytebase-action:cloud \ check \ --url $(BYTEBASE_URL) \ --service-account $(BYTEBASE_SERVICE_ACCOUNT) \ --service-account-secret $(BYTEBASE_SERVICE_ACCOUNT_SECRET) \ --file-pattern "/workspace/schema/**/*.sql" \ --declarative \ --custom-rules "$(cat $(Build.SourcesDirectory)/.bytebase/sql-review.md)" displayName: 'SDL Validation with Team Standards' ``` ```bash theme={null} bytebase-action check \ --url https://bytebase.example.com \ --service-account service@example.com \ --service-account-secret "${SERVICE_SECRET}" \ --file-pattern "schema/**/*.sql" \ --declarative ``` **Prerequisites:** AI must be enabled in Bytebase Settings → AI Assistant ```bash theme={null} bytebase-action check \ --url https://bytebase.example.com \ --service-account service@example.com \ --service-account-secret "${SERVICE_SECRET}" \ --file-pattern "schema/**/*.sql" \ --declarative \ --custom-rules "$(cat .bytebase/sql-review.md)" ``` ## Common Validation Errors **Error:** `Table 'users' must include schema name` **Fix:** Add schema prefix to all objects ```sql theme={null} -- Change: CREATE TABLE users -- To: CREATE TABLE public.users ``` **Error:** `PRIMARY KEY must be table-level constraint` **Fix:** Move constraint to table level ```sql theme={null} -- Change: id INTEGER PRIMARY KEY -- To: id INTEGER, CONSTRAINT users_pkey PRIMARY KEY (id) ``` **Error:** `All constraints must have explicit names` **Fix:** Add CONSTRAINT keyword with name ```sql theme={null} -- Change: UNIQUE (email) -- To: CONSTRAINT users_email_key UNIQUE (email) ``` **Error:** `Foreign key column 'user_id' (INTEGER) references 'users.id' (BIGINT)` **Fix:** Align column types ```sql theme={null} -- Ensure both are same type user_id BIGINT -- Match users.id type ``` **Error:** `SDL does not allow ALTER TABLE statements` **Fix:** Edit the `CREATE TABLE` to reflect the desired end state instead of writing imperative changes ```sql theme={null} -- Change: ALTER TABLE `users` ADD COLUMN `email` varchar(255) -- To: add the column to the CREATE TABLE definition ``` **Error:** `SDL does not allow CREATE TABLE ... AS SELECT statements` **Fix:** Write the table's full column definitions explicitly; backfill data with the migration-based workflow **Error:** `SDL dependency cycle detected` **Fix:** Break the circular reference between objects (for example, two views selecting from each other) See [SDL Issues](/gitops/troubleshooting/sdl-issues) for details on each error. *** ## Next Steps Deploy SDL changes after validation approval Explore all available SQL review rules # CI/CD Issues Source: https://docs.bytebase.com/gitops/troubleshooting/cicd-issues Troubleshoot problems with CI/CD integration, authentication, and automated workflows. ## GitHub Actions: API Call Fails **Error:** ``` curl: (6) Could not resolve host: bytebase.example.com ``` **Causes:** 1. Incorrect Bytebase URL 2. Network restrictions 3. DNS resolution issues **Solutions:** ```yaml theme={null} # .github/workflows/gitops.yml # 1. Verify BB_URL environment variable - name: Check Bytebase URL run: echo "Bytebase URL: $BB_URL" env: BB_URL: ${{ secrets.BYTEBASE_URL }} # 2. Test connectivity - name: Test Connection run: curl -f $BB_URL/v1/actuator/info # 3. Use correct authentication - name: Create Release run: | curl -X POST "$BB_URL/v1/projects/my-project/releases" \ -H "Authorization: Bearer $BB_TOKEN" \ -H "Content-Type: application/json" \ -d @release.json env: BB_URL: ${{ secrets.BYTEBASE_URL }} BB_TOKEN: ${{ secrets.BYTEBASE_TOKEN }} ``` ## Unauthorized Error **Error:** ``` 401 Unauthorized: Invalid token ``` **Causes:** 1. Token expired or invalid 2. Token not properly set in secrets 3. Wrong token format **Solutions:** Test token manually: ```bash theme={null} curl -H "Authorization: Bearer $BB_TOKEN" \ $BB_URL/v1/actuator/info ``` Should return server info, not 401. 1. Login to Bytebase 2. Navigate to Settings → Service Accounts 3. Find CI/CD service account 4. Generate new token 5. Update CI/CD secrets Token should be: * Format: `bb_xxxxxxxxxxxxxxxxxxxxxx` * No quotes or extra characters * Properly base64 encoded if required **In GitHub Actions:** ```yaml theme={null} env: BB_TOKEN: ${{ secrets.BYTEBASE_TOKEN }} # NOT: "${{ secrets.BYTEBASE_TOKEN }}" ``` ## SQL Review Fails in CI **Error:** ``` SQL review failed: Rule violation - table naming convention ``` **Cause:** Migration violates configured SQL review rules. **Solutions:** **Option 1: Fix the violation** ```sql theme={null} -- If error: Table name must be lowercase -- ❌ Wrong CREATE TABLE UserProfiles (...); -- ✅ Correct CREATE TABLE user_profiles (...); ``` **Option 2: Adjust SQL review policy** If rule is too strict: 1. Navigate to SQL Review settings 2. Adjust rule severity (ERROR → WARNING) 3. Or disable rule if not needed Complete reference of 200+ rules **Option 3: Override for specific case** Add comment justification in SQL: ```sql theme={null} -- bytebase:ignore-rule table.naming.convention -- Legacy table name required for compatibility CREATE TABLE UserProfiles (...); ``` Review failures prevent bad changes from reaching production. Fix violations rather than bypassing checks. ## CI Timeout **Error:** ``` Error: The operation was canceled. ``` **Causes:** 1. Long-running migration exceeds CI timeout 2. Network connectivity issues 3. Bytebase server overloaded **Solutions:** **GitHub Actions:** ```yaml theme={null} jobs: deploy: runs-on: ubuntu-latest timeout-minutes: 30 # Increase from default 360 steps: - name: Deploy timeout-minutes: 20 # Step-level timeout run: ... ``` **GitLab CI:** ```yaml theme={null} rollout: timeout: 30m script: - bytebase-action rollout ... ``` For long-running migrations: * Break into smaller batches * Use online schema migration for large tables * Consider running outside of CI Learn optimization techniques Verify Bytebase is responsive: ```bash theme={null} # Test API response time time curl -f $BB_URL/v1/actuator/info # Check Bytebase logs docker logs bytebase ``` ## Workflow Dispatch Not Triggering **Error:** Manual workflow doesn't start when triggered. **Causes:** 1. Wrong branch selected 2. Insufficient permissions 3. Workflow file syntax error **Solutions:** **Check workflow configuration:** ```yaml theme={null} # .github/workflows/deploy.yml name: Manual Deploy on: workflow_dispatch: inputs: environment: description: 'Environment to deploy' required: true type: choice options: - dev - staging - production jobs: deploy: runs-on: ubuntu-latest steps: - name: Deploy to ${{ inputs.environment }} run: echo "Deploying to ${{ inputs.environment }}" ``` **Verify permissions:** * Must have write access to repository * Workflow must be in default branch * Actions must be enabled for repository ## Secrets Not Available **Error:** ``` Error: Secret BYTEBASE_TOKEN is not set ``` **Solutions:** 1. **Add secrets to repository:** * Navigate to Settings → Secrets and variables → Actions * Click "New repository secret" * Add required secrets 2. **Environment-specific secrets:** ```yaml theme={null} jobs: deploy: runs-on: ubuntu-latest environment: production # Use environment secrets steps: - name: Deploy env: BB_TOKEN: ${{ secrets.BYTEBASE_TOKEN }} ``` 3. **Organization secrets:** * Can be shared across repositories * Set at organization level * Select which repositories can access *** ## Next Steps Step-by-step CI/CD setup guides Learn about service accounts CI/CD patterns and strategies Return to troubleshooting overview # Overview Source: https://docs.bytebase.com/gitops/troubleshooting/overview This troubleshooting guide helps you diagnose and resolve common issues in GitOps workflows with Bytebase. ## Common Issue Categories Invalid files, SHA256 mismatches, release not found, plan creation failures Skipped tasks, pending approvals, SQL errors, permissions, connectivity Schema qualification, constraints, foreign keys, validation errors GitHub Actions, authentication, SQL review failures in CI ## Quick Diagnostics ### Check System Status ```bash theme={null} # Verify Bytebase is accessible curl -f $BB_URL/v1/actuator/info # Test authentication curl -H "Authorization: Bearer $BB_TOKEN" \ $BB_URL/v1/actuator/info ``` ### View Recent Releases ```bash theme={null} # List releases in project curl -X GET "$BB_URL/v1/projects/my-project/releases" \ -H "Authorization: Bearer $BB_TOKEN" ``` ### Check Task Logs 1. Navigate to the rollout in Bytebase UI 2. Click on the failed task 3. View execution logs for error details ## Getting Help If you've tried these solutions and still need help: 1. **Check documentation:** * [API Reference](/integrations/api/overview) * [Error Codes](/sql-review/error-codes) * [GitOps Tutorials](/tutorials/gitops-github-workflow) 2. **Review logs:** * Task execution logs in Bytebase UI * CI/CD workflow logs * Database server logs 3. **Community support:** * GitHub Issues: [github.com/bytebase/bytebase/issues](https://github.com/bytebase/bytebase/issues) 4. **Enterprise support:** * Contact your Bytebase account team to schedule technical consultation ## Related Documentation Understand the complete GitOps workflow Production-ready patterns Complete API documentation SQL review error reference # Release and Plan Issues Source: https://docs.bytebase.com/gitops/troubleshooting/release-and-plan Troubleshoot problems with release creation and plan generation. ## Release Creation Fails with Invalid File **Error:** ``` Failed to create release: Invalid migration file '002__add_users.sql' ``` **Possible Causes:** 1. **Invalid filename format** * Missing version number * Invalid characters * Incorrect suffix 2. **SQL syntax errors** * Unparseable SQL * Database-specific syntax issues 3. **File encoding issues** * Non-UTF-8 encoding * Binary content in SQL file **Solutions:** Ensure filename follows pattern: ``` ___.sql ``` Valid examples: ``` ✅ 002__add_users.sql ✅ v1.0.0__init.sql ✅ 20250120__add_email_dml.sql ``` Invalid examples: ``` ❌ add_users.sql (missing version) ❌ 002-add-users.sql (wrong separator, use __) ❌ 002__add users.sql (space in filename) ``` Test SQL locally: ```bash theme={null} # PostgreSQL psql -h localhost -U user -d testdb -f 002__add_users.sql # MySQL mysql -h localhost -u user -p testdb < 002__add_users.sql ``` Common syntax issues: * Missing semicolons * Incorrect quote types * Database-specific keywords Ensure UTF-8 encoding: ```bash theme={null} file 002__add_users.sql # Should show: UTF-8 Unicode text # Convert if needed iconv -f ISO-8859-1 -t UTF-8 file.sql > file_utf8.sql ``` ## SHA256 Mismatch Warning **Warning:** ``` Warning: Version 005 already applied with different content Existing SHA256: abc123... New SHA256: xyz789... ``` **Cause:** Migration file content changed for an existing version. **Impact:** File is skipped (idempotent behavior), but indicates inconsistency. **Solutions:** **Option 1: Accept the difference (if intentional)** * Document why content differs * Verify both versions produce same schema * No action needed (file will be skipped) **Option 2: Fix the inconsistency (recommended)** 1. Revert file to original content 2. Create new migration with changes: ```sql theme={null} -- 006__fix_previous_migration.sql -- Corrects issue from migration 005 ALTER TABLE ... ``` Never modify deployed migrations. Always create new migrations for fixes. ## Release Not Found **Error:** ``` Release 'projects/my-project/releases/v1.0.0' not found ``` **Causes:** 1. Release was deleted 2. Wrong project name 3. Wrong release identifier **Solutions:** ```bash theme={null} # List all releases curl -X GET "$BB_URL/v1/projects/my-project/releases" \ -H "Authorization: Bearer $BB_TOKEN" # Search by digest curl -X GET "$BB_URL/v1/projects/my-project/releases:search?digest=v1.0.0" \ -H "Authorization: Bearer $BB_TOKEN" # Recreate release if deleted curl -X POST "$BB_URL/v1/projects/my-project/releases" \ -H "Authorization: Bearer $BB_TOKEN" \ -d @release-payload.json ``` ## Plan Creation Fails - Invalid Target **Error:** ``` Invalid target: instances/prod-mysql/databases/nonexistent ``` **Cause:** Specified database doesn't exist or is archived. **Solutions:** ```bash theme={null} # List databases in instance curl -X GET "$BB_URL/v1/instances/prod-mysql/databases" \ -H "Authorization: Bearer $BB_TOKEN" ``` Check if database: * Exists with correct name * Is not archived * Has correct instance path If using database group: ```bash theme={null} # List database groups curl -X GET "$BB_URL/v1/projects/my-project/databaseGroups" \ -H "Authorization: Bearer $BB_TOKEN" # Get specific group members curl -X GET "$BB_URL/v1/projects/my-project/databaseGroups/prod-fleet" \ -H "Authorization: Bearer $BB_TOKEN" ``` Verify group: * Exists * Contains expected databases * Has active members ## Plan References Wrong Release **Error:** ``` Release 'projects/my-project/releases/old-release' has no unapplied migrations ``` **Cause:** Plan references release where all migrations already applied. **Expected Behavior:** This is normal if all migrations are already deployed. **Solutions:** **If migrations should be unapplied:** 1. Check revision records on target database: ```bash theme={null} curl -X GET "$BB_URL/v1/instances/prod/databases/app_db/revisions" \ -H "Authorization: Bearer $BB_TOKEN" ``` 2. Verify versions in release match: ```bash theme={null} curl -X GET "$BB_URL/v1/projects/my-project/releases/old-release" \ -H "Authorization: Bearer $BB_TOKEN" ``` **If releasing new changes:** 1. Create new release with new version numbers 2. Update plan to reference new release *** ## Next Steps Troubleshoot deployment problems Complete API documentation # Rollout Issues Source: https://docs.bytebase.com/gitops/troubleshooting/rollout Troubleshoot problems during rollout execution and task failures. ## All Tasks Skipped **Behavior:** Rollout completes immediately with all tasks skipped. **Cause:** All migrations already applied (detected via revision records). **Verification:** ```bash theme={null} # Check revisions on target database curl -X GET "$BB_URL/v1/instances/prod/databases/app_db/revisions" ``` **This is normal behavior** indicating idempotent operation. **To deploy new changes:** 1. Create migrations with new version numbers 2. Create new release 3. Create new plan/rollout ## Task Stuck in PENDING\_APPROVAL **Behavior:** Task waiting for approval indefinitely. **Causes:** 1. Rollout policy requires manual approval 2. No authorized users have approved **Solutions:** ```bash theme={null} curl -X POST "$BB_URL/v1/projects/my-project/rollouts/rollout-123/stages/stage-1/tasks/task-1:approve" \ -H "Authorization: Bearer $BB_TOKEN" \ -d '{"comment": "Approved for deployment"}' ``` Verify rollout policy: ```bash theme={null} curl -X GET "$BB_URL/v1/projects/my-project/environments/prod" \ -H "Authorization: Bearer $BB_TOKEN" ``` Check `rollout_policy` field: ```json theme={null} { "rollout_policy": { "automatic": false, "issue_roles": ["roles/projectOwner", "roles/projectDBA"] } } ``` Change to automatic (use with caution): ```bash theme={null} curl -X PATCH "$BB_URL/v1/projects/my-project/environments/prod" \ -H "Authorization: Bearer $BB_TOKEN" \ -d '{ "rollout_policy": { "automatic": true } }' ``` Only enable automatic rollout in non-production environments or with proper safeguards. ## Task Fails with SQL Error **Error in task logs:** ``` ERROR: syntax error at or near "CRATE" ``` **Cause:** SQL syntax error in migration file. **Solutions:** 1. **Identify problematic file:** * Check task logs for error location * Identify which migration file failed 2. **Fix syntax error:** ```sql theme={null} -- Wrong: CRATE TABLE users (...); -- Correct: CREATE TABLE users (...); ``` 3. **Create new release:** * Don't modify original file if already released * Create new migration with fix: ```sql theme={null} -- 006__fix_users_table.sql CREATE TABLE IF NOT EXISTS users (...); ``` 4. **Retry or create new rollout:** ```bash theme={null} # Option 1: Retry failed task (if file was corrected before execution) curl -X POST "$BB_URL/v1/projects/my-project/rollouts/rollout-123/stages/stage-1/tasks:batchRun" \ -d '{"tasks": ["task-1"]}' # Option 2: Create new rollout with corrected release curl -X POST "$BB_URL/v1/projects/my-project/rollouts" \ -d '{"plan": "projects/my-project/plans/plan-new"}' ``` ## Permission Denied Error **Error:** ``` ERROR: permission denied to create table "users" ``` **Cause:** Bytebase database user lacks required permissions. **Solutions:** **For DDL operations (schema changes):** ```sql theme={null} -- PostgreSQL GRANT CREATE, ALTER, DROP ON DATABASE app_db TO bytebase_user; GRANT ALL ON SCHEMA public TO bytebase_user; -- MySQL GRANT CREATE, ALTER, DROP, INDEX ON app_db.* TO 'bytebase_user'@'%'; ``` **For DML operations (data changes):** ```sql theme={null} -- PostgreSQL GRANT SELECT, INSERT, UPDATE, DELETE ON ALL TABLES IN SCHEMA public TO bytebase_user; -- MySQL GRANT SELECT, INSERT, UPDATE, DELETE ON app_db.* TO 'bytebase_user'@'%'; ``` If using wrong user in Bytebase: 1. Navigate to Instance settings 2. Update connection username 3. Save changes Or via API: ```bash theme={null} curl -X PATCH "$BB_URL/v1/instances/prod-mysql" \ -d '{ "username": "bytebase_admin", "password": "new_password" }' ``` Configure database connections Test permissions: ```sql theme={null} -- PostgreSQL: Check grants SELECT grantee, privilege_type FROM information_schema.role_table_grants WHERE grantee = 'bytebase_user'; -- MySQL: Show grants SHOW GRANTS FOR 'bytebase_user'@'%'; ``` ## Connection Timeout **Error:** ``` Connection timeout after 30 seconds ``` **Causes:** 1. Database server down 2. Network connectivity issues 3. Firewall blocking connection 4. Incorrect connection parameters **Solutions:** Check if database is running: ```bash theme={null} # PostgreSQL pg_isready -h prod-mysql -p 5432 # MySQL mysqladmin -h prod-mysql -u user -p ping ``` ```bash theme={null} # Test connection telnet prod-mysql 5432 # Or with nc nc -zv prod-mysql 5432 # Test from Bytebase server docker exec bytebase nc -zv prod-mysql 5432 ``` Ensure firewall allows traffic: * From Bytebase server IP * To database port (3306/5432/etc.) * Bidirectional if stateful **Cloud providers:** * AWS: Security Groups * GCP: Firewall Rules * Azure: Network Security Groups Check instance configuration: ```bash theme={null} curl -X GET "$BB_URL/v1/instances/prod-mysql" ``` Verify: * Host address correct * Port correct * SSL/TLS settings if required Database connection setup *** ## Next Steps Troubleshoot SDL-specific problems Learn production-ready patterns # SDL Issues Source: https://docs.bytebase.com/gitops/troubleshooting/sdl-issues Troubleshoot State-Based Workflow (SDL) specific validation errors. SDL is currently supported for PostgreSQL and MySQL. Validation rules enforce strict conventions to ensure maintainable schema definitions. The rules differ per engine - see [PostgreSQL validation errors](#postgresql-validation-errors) and [MySQL validation errors](#mysql-validation-errors). ## PostgreSQL Validation Errors ### Missing Schema Qualification **Error:** ``` Table 'users' must include schema name ``` **Cause:** SDL requires all objects to have schema prefix. **Solution:** ```sql theme={null} -- ❌ Wrong CREATE TABLE users (id INTEGER PRIMARY KEY); -- ✅ Correct CREATE TABLE public.users (id INTEGER PRIMARY KEY); ``` Apply to all objects: * Tables: `CREATE TABLE public.users` * Indexes: `CREATE INDEX idx_name ON public.users(name)` * Views: `CREATE VIEW public.active_users AS ...` * Functions: `CREATE FUNCTION public.get_user() ...` ### Column-Level Constraint **Error:** ``` PRIMARY KEY must be table-level constraint ``` **Cause:** SDL requires constraints at table level (except NOT NULL, DEFAULT, GENERATED). **Solution:** ```sql theme={null} -- ❌ Wrong - Column-level constraint CREATE TABLE public.users ( id INTEGER PRIMARY KEY, email TEXT UNIQUE ); -- ✅ Correct - Table-level constraints CREATE TABLE public.users ( id INTEGER, email TEXT, CONSTRAINT users_pkey PRIMARY KEY (id), CONSTRAINT users_email_key UNIQUE (email) ); ``` **Allowed at column level:** ```sql theme={null} CREATE TABLE public.users ( id SERIAL, -- ✅ SERIAL allowed name TEXT NOT NULL, -- ✅ NOT NULL allowed created_at TIMESTAMP DEFAULT NOW(), -- ✅ DEFAULT allowed CONSTRAINT users_pkey PRIMARY KEY (id) ); ``` ### Unnamed Constraint **Error:** ``` All constraints must have explicit names using CONSTRAINT keyword ``` **Solution:** ```sql theme={null} -- ❌ Wrong - Unnamed constraints CREATE TABLE public.orders ( id INTEGER, user_id INTEGER, PRIMARY KEY (id), FOREIGN KEY (user_id) REFERENCES public.users(id) ); -- ✅ Correct - Named constraints CREATE TABLE public.orders ( id INTEGER, user_id INTEGER, CONSTRAINT orders_pkey PRIMARY KEY (id), CONSTRAINT fk_orders_users FOREIGN KEY (user_id) REFERENCES public.users(id) ); ``` **Naming conventions:** * Primary keys: `{table}_pkey` * Unique constraints: `{table}_{column}_key` * Foreign keys: `fk_{table}_{referenced_table}` * Check constraints: `check_{description}` ### Foreign Key Type Mismatch **Error:** ``` Foreign key column 'user_id' (INTEGER) references 'users.id' (BIGINT) Types must match exactly ``` **Cause:** Column types don't match between foreign key and referenced column. **Solution:** ```sql theme={null} -- Ensure types match CREATE TABLE public.users ( id BIGINT, CONSTRAINT users_pkey PRIMARY KEY (id) ); CREATE TABLE public.orders ( id BIGINT, user_id BIGINT, -- Must match users.id type CONSTRAINT orders_pkey PRIMARY KEY (id), CONSTRAINT fk_orders_users FOREIGN KEY (user_id) REFERENCES public.users(id) ); ``` ### Foreign Key Missing Schema **Error:** ``` Foreign key reference must include schema name ``` **Solution:** ```sql theme={null} -- ❌ Wrong FOREIGN KEY (user_id) REFERENCES users(id) -- ✅ Correct FOREIGN KEY (user_id) REFERENCES public.users(id) ``` ### CHECK Constraint References Other Table **Error:** ``` CHECK constraint cannot reference other tables ``` **Cause:** CHECK constraints can only reference columns in the same table. **Solution:** ```sql theme={null} -- ❌ Wrong - References other table CREATE TABLE public.orders ( user_id INTEGER, CONSTRAINT check_valid_user CHECK (user_id IN (SELECT id FROM public.users)) ); -- ✅ Correct - Use foreign key instead CREATE TABLE public.orders ( user_id INTEGER, CONSTRAINT fk_orders_users FOREIGN KEY (user_id) REFERENCES public.users(id) ); ``` ### Migration Generation Fails **Error:** ``` Failed to generate migration: duplicate object 'users_pkey' ``` **Causes:** 1. Duplicate constraint names across tables 2. Duplicate index names 3. Conflicting object names **Solution:** Ensure all object names are unique: ```sql theme={null} -- ❌ Wrong - Duplicate constraint name CREATE TABLE public.users ( id INTEGER, CONSTRAINT pkey PRIMARY KEY (id) ); CREATE TABLE public.orders ( id INTEGER, CONSTRAINT pkey PRIMARY KEY (id) -- Duplicate name! ); -- ✅ Correct - Unique names CREATE TABLE public.users ( id INTEGER, CONSTRAINT users_pkey PRIMARY KEY (id) ); CREATE TABLE public.orders ( id INTEGER, CONSTRAINT orders_pkey PRIMARY KEY (id) ); ``` ## MySQL Validation Errors ### Unsupported Statement **Error:** ``` SDL does not allow ALTER TABLE statements SDL does not allow DROP statements SDL does not allow INSERT statements ``` **Cause:** SDL files declare the desired end state; imperative statements (`ALTER`, `DROP`, `RENAME`, `TRUNCATE`, DML) are rejected because Bytebase generates the migration DDL itself. **Solution:** ```sql theme={null} -- ❌ Wrong - Imperative change ALTER TABLE `users` ADD COLUMN `email` varchar(255) NOT NULL; -- ✅ Correct - Edit the CREATE TABLE to reflect the desired state CREATE TABLE `users` ( `id` int NOT NULL, `email` varchar(255) NOT NULL, PRIMARY KEY (`id`) ) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4; ``` ### CREATE TABLE ... AS SELECT **Error:** ``` SDL does not allow CREATE TABLE ... AS SELECT statements ``` **Cause:** `CREATE TABLE ... AS SELECT` is imperative - its resulting schema depends on the query, so it cannot be represented as a schema snapshot. **Solution:** Write the table's full column definitions explicitly. To backfill data, use the migration-based workflow. ### Dependency Cycle **Error:** ``` SDL dependency cycle detected ``` **Cause:** Objects in the SDL files reference each other in a cycle (for example, two views selecting from each other), so no valid creation order exists. **Solution:** Break the cycle by removing or restructuring one of the circular references. *** ## Next Steps Learn SDL syntax requirements Understand SDL constraints Troubleshoot CI/CD problems # Model Context Protocol (MCP) Source: https://docs.bytebase.com/integrations/mcp Connect AI assistants to Bytebase using the Model Context Protocol. New to the Bytebase MCP server? Follow the [Connect an AI Assistant to Bytebase with MCP](/tutorials/connect-ai-assistant-with-mcp) tutorial for a hands-on walkthrough. ## Prerequisites * [Configure the external URL](/get-started/self-host/external-url) — the MCP server uses it to tell the AI assistant where to authenticate. A local address such as `http://localhost:8080` works for a local deployment; use the public address your AI assistant can reach for anything else. Without it set, the **Integration** > **MCP** page cannot show a usable endpoint URL. * OAuth authentication is required on first connection. Ensure you have appropriate Bytebase permissions. ## Configuration The **Integration** > **MCP** page in your Bytebase console shows the endpoint two ways: a JSON snippet for manual client configuration, and a ready-to-run CLI command for supported tools. ### JSON configuration Add this to your MCP client's configuration file: ```json theme={null} { "mcpServers": { "bytebase": { "type": "http", "url": "https:///mcp" } } } ``` ### CLI commands The console page provides a one-line command for Claude Code, Codex, Copilot CLI, Gemini CLI, and VS Code — copy the one for your client. ## Authentication Bytebase authenticates MCP clients via OAuth. On first connection, your browser opens to the Bytebase login page; sign in and authorize the client to access your account. Bytebase OAuth authorization screen for an MCP client Once authorized, the client stores the token and reuses it automatically until it expires. ## Usage Once connected, try asking your AI assistant: * "List all projects in my Bytebase instance" * "Show me the schema for database X" * "Generate a migration script to add a users table" ## Important notes Caution: Once connected, your AI assistant will inherit all your permissions in Bytebase, including making changes to database or updating settings (in the case of an admin). So be cautious when instructing AI assistant to apply any change via Bytebase. Audit logging: any action taken by the AI assistant via MCP will be audit-logged in Bytebase under your account. ## How changes work A few behaviors are worth knowing before you work through an AI assistant: * **One database per change.** A proposed change targets a single database. For changes across multiple databases, the assistant uses the batch workflow and the underlying APIs. * **Each change creates a new issue.** A proposed change always creates a fresh sheet, plan, and issue — it does not edit an existing one. Iterating on SQL across several turns produces multiple issues, so settle on the statement before proposing, and close any superseded issues. * **Masking is enforced.** Query results honor your data masking policies — masked values appear as `******` (full) or `**rn**` (partial). The assistant only ever sees what your policies allow. ## Limitations * **No standalone dry-run.** Plan checks (SQL review, DDL simulation) run as part of creating the plan; there is no way to validate SQL without creating a change. * **Query results are capped.** Queries return 100 rows by default and 1,000 at most, with a 30-second timeout. Larger result sets are truncated. * **Schema output is capped.** Schema inspection returns up to 200 tables per schema in detailed mode; narrow with a schema or table name. Column masking details are shown only when a specific table is requested. * **HTTP transport only.** The server is reached over the `/mcp` HTTP endpoint; stdio transport is not supported. # Terraform Provider Source: https://docs.bytebase.com/integrations/terraform/overview The Bytebase Provider itself is free to use. Some advanced resource operations require Pro or Enterprise Plan. Bytebase Terraform Provider handles control plane configuration such as settings, policies, access controls. It does not handle data plane operations such as database creation, schema migrations, DML execution, query. Bytebase provides the [Terraform Provider](https://registry.terraform.io/providers/bytebase/bytebase) to let you manage your Bytebase resources via Terraform. ```plain theme={null} Terraform Provider → Bytebase Configuration ├── Instances, Projects & Environments ├── Access Policies └── Settings such as Approval Workflows Bytebase UI/API → Database Operations ├── Database Creation ├── Schema Migrations (DDL) ├── Data Changes (DML) └── Query ``` ## Create Service Account The Terraform Provider uses [service account](/integrations/api/authentication/#service-account) to call the Bytebase API to manage the resources and settings. After creating the service account with the `Workspace Admin` role, you can copy the service key as `service_key` and the email as `service_account` to initialize the Terraform provider in the next step. service-account ### Optional: Separate Accounts for Plan and Apply If you prefer to use different service accounts for `terraform plan` and `terraform apply`, you can: * **For `terraform apply`**: Assign the `Workspace Admin` role. * **For `terraform plan`**: Create a **custom role** with only read permissions under **IAM & Admin > Custom Roles**, then assign it to the plan-only service account. The read-only role should include the following permissions: * All `.list` and `.get` permissions * `bb.databases.export` * `bb.databases.query` * `bb.databases.getSchema` * `bb.databases.getSecrets` * `bb.auditLogs.export` * `bb.auditLogs.search` * `bb.issues.subscribe` * `bb.projects.getIamPolicy` ## Tutorials **Resources** **Access Control** **Workspace Settings** ## Troubleshoot ### status: 404 This often happens when someone deletes the resources from the Bytebase UI and then attempts to run `terraform apply`. To recover, please run `terraform refresh` to refresh the state first. 404 # Supported Databases Source: https://docs.bytebase.com/introduction/supported-databases Bytebase officially supports the following major versions for each supported database engine. Bytebase usually works fine with older database versions, we just won't support features specific to those older versions. We may add other open source databases in the future. ## Supported Database Versions | Category | Database | Version | | :----------------- | :------------------------------------- | :------------- | | **RDBMS** | MySQL, Aurora MySQL | 5.7 and above | | | PostgreSQL, Aurora PostgreSQL, AlloyDB | 12.0 and above | | | Oracle | 11g and above | | | Microsoft SQL Server | 2019 and above | | | MariaDB | 10.7 and above | | | TiDB | 5.0 and above | | | OceanBase | 3.x and above | | | CockroachDB | 24.1 and above | | | Spanner | All versions | | **NoSQL** | MongoDB | 4.2 and above | | | Redis | 6.0 and above | | | Cassandra | 3.0 and above | | | DocumentDB | All versions | | | DynamoDB | All versions | | | Cosmos DB | All versions | | **Data Warehouse** | Snowflake | All versions | | | BigQuery | All versions | | | Redshift | All versions | | | Hive | All versions | | | ClickHouse | 21.0 and above | | | Databricks | All versions | | | StarRocks | All versions | | **Others** | Elasticsearch | All versions | ## Feature Matrix ### Change Workflow | Feature | [UI Workflow](/change-database/change-workflow/#ui-workflow) | [GitOps Workflow](/change-database/change-workflow/#gitops-workflow) | [Batch Change](/change-database/batch-change) | [SQL Lint](/sql-review/review-policy) | [Data Rollback\*](/change-database/rollback-data-changes) | | | :----------------- | :----------------------------------------------------------- | :------------------------------------------------------------------- | :-------------------------------------------- | :------------------------------------ | :-------------------------------------------------------- | - | | **RDMBS** | | | | | | | | MySQL | ✔️ | ✔️ | ✔️ | ✔️ | ✔️ | | | PostgreSQL | ✔️ | ✔️ | ✔️ | ✔️ | ✔️ | | | Oracle | ✔️ | ✔️ | ✔️ | ✔️ | ✔️ | | | SQL Server | ✔️ | ✔️ | ✔️ | ✔️ | ✔️ | | | MariaDB | ✔️ | ✔️ | ✔️ | ✔️ | | | | TiDB | ✔️ | ✔️ | ✔️ | ✔️ | | | | OceanBase | ✔️ | ✔️ | ✔️ | ✔️ | | | | CockroachDB | ✔️ | ✔️ | ✔️ | | | | | Spanner | ✔️ | ✔️ | ✔️ | | | | | **NoSQL** | | | | | | | | MongoDB | ✔️ | ✔️ | ✔️ | | | | | Redis | ✔️ | ✔️ | ✔️ | | | | | Cassandra | ✔️ | ✔️ | ✔️ | | | | | DocumentDB | ✔️ | ✔️ | ✔️ | | | | | DynamoDB | ✔️ | ✔️ | ✔️ | | | | | Cosmos DB | ✔️ | ✔️ | ✔️ | | | | | **Data Warehouse** | | | | | | | | Snowflake | ✔️ | ✔️ | ✔️ | ✔️ | | | | BigQuery | ✔️ | ✔️ | ✔️ | | | | | Redshift | ✔️ | ✔️ | ✔️ | | | | | Hive | ✔️ | ✔️ | ✔️ | | | | | ClickHouse | ✔️ | ✔️ | ✔️ | | | | | Databricks | ✔️ | ✔️ | ✔️ | | | | | StarRocks | ✔️ | ✔️ | ✔️ | | | | | **Others** | | | | | | | | Elasticsearch | ✔️ | ✔️ | ✔️ | | | | Data Rollback supports `UPDATE` and `DELETE` statements. ### SQL Editor | Feature | [Sheet Management](/sql-editor/manage-sql-scripts) | [Query Mode](/sql-editor/run-queries) | [Admin Mode](/sql-editor/admin-mode) | [Batch Mode](/sql-editor/run-queries/#batch-mode) | [AI Assistant](/sql-editor/ai-assistant) | | :----------------- | :------------------------------------------------- | :------------------------------------ | :----------------------------------- | :------------------------------------------------ | :--------------------------------------- | | **RDBMS** | | | | | | | MySQL | ✔️ | ✔️ | ✔️ | ✔️ | ✔️ | | PostgreSQL | ✔️ | ✔️ | ✔️ | ✔️ | ✔️ | | Oracle | ✔️ | ✔️ | ✔️ | ✔️ | ✔️ | | SQL Server | ✔️ | ✔️ | ✔️ | ✔️ | ✔️ | | MariaDB | ✔️ | ✔️ | ✔️ | ✔️ | ✔️ | | TiDB | ✔️ | ✔️ | ✔️ | ✔️ | ✔️ | | OceanBase | ✔️ | ✔️ | ✔️ | ✔️ | ✔️ | | CockroachDB | ✔️ | ✔️ | ✔️ | ✔️ | ✔️ | | Spanner | ✔️ | ✔️ | ✔️ | ✔️ | ✔️ | | **NoSQL** | | | | | | | MongoDB | ✔️ | ✔️ \* | ✔️ | ✔️ | ✔️ | | Redis | ✔️ | ✔️ \* | ✔️ | ✔️ | ✔️ | | Cassandra | ✔️ | ✔️ \* | ✔️ | ✔️ | ✔️ | | DocumentDB | ✔️ | ✔️ \* | ✔️ | ✔️ | ✔️ | | DynamoDB | ✔️ | ✔️ \* | ✔️ | ✔️ | ✔️ | | Cosmos DB | ✔️ | ✔️ \* | ✔️ | ✔️ | ✔️ | | **Data Warehouse** | | | | | | | Snowflake | ✔️ | ✔️ | ✔️ | ✔️ | ✔️ | | BigQuery | ✔️ | ✔️ | ✔️ | ✔️ | ✔️ | | Redshift | ✔️ | ✔️ | ✔️ | ✔️ | ✔️ | | Hive | ✔️ | ✔️ | ✔️ | ✔️ | ✔️ | | ClickHouse | ✔️ | ✔️ | ✔️ | ✔️ | ✔️ | | Databricks | ✔️ | ✔️ | ✔️ | ✔️ | ✔️ | | StarRocks | ✔️ | ✔️ | ✔️ | ✔️ | ✔️ | | **Others** | | | | | | | Elasticsearch | ✔️ | ✔️ | ✔️ | ✔️ | ✔️ | Normally, Bytebase will block non-readonly statements in Query Mode. This is not supported NoSQL Databases. You can work around this by configuring a [readonly connection](/get-started/instance/#read-only-connection). ### Data Security & Compliance | Feature | [RBAC](/administration/roles) | [Audit](/security/audit-log) | [Secret Manager](/get-started/instance/#use-secret-manager) | [Query Control](/security/database-permission/overview/) | [Data Masking](/security/data-masking/overview) | | :----------------- | :---------------------------- | :--------------------------- | :---------------------------------------------------------- | :------------------------------------------------------- | :---------------------------------------------- | | **RDBMS** | | | | | | | MySQL | ✔️ | ✔️ | ✔️ | ✔️ | ✔️ | | PostgreSQL | ✔️ | ✔️ | ✔️ | ✔️ | ✔️ | | Oracle | ✔️ | ✔️ | ✔️ | ✔️ | ✔️ | | SQL Server | ✔️ | ✔️ | ✔️ | ✔️ | ✔️ | | MariaDB | ✔️ | ✔️ | ✔️ | ✔️ | ✔️ | | TiDB | ✔️ | ✔️ | ✔️ | ✔️ | ✔️ | | OceanBase | ✔️ | ✔️ | ✔️ | ✔️ | ✔️ | | CockroachDB | ✔️ | ✔️ | ✔️ | ✔️ | | | Spanner | ✔️ | ✔️ | ✔️ | ✔️ | | | **NoSQL** | | | | | | | MongoDB | ✔️ | ✔️ | ✔️ | | ✔️ | | Redis | ✔️ | ✔️ | ✔️ | | | | Cassandra | ✔️ | ✔️ | ✔️ | | ✔️ | | DocumentDB | ✔️ | ✔️ | ✔️ | | | | DynamoDB | ✔️ | ✔️ | ✔️ | | | | Cosmos DB | ✔️ | ✔️ | ✔️ | | ✔️ | | **Data Warehouse** | | | | | | | Snowflake | ✔️ | ✔️ | ✔️ | ✔️ | ✔️ | | BigQuery | ✔️ | ✔️ | ✔️ | ✔️ | ✔️ | | Redshift | ✔️ | ✔️ | ✔️ | | | | Hive | ✔️ | ✔️ | ✔️ | | | | ClickHouse | ✔️ | ✔️ | ✔️ | ✔️ | | | Databricks | ✔️ | ✔️ | ✔️ | | | | StarRocks | ✔️ | ✔️ | ✔️ | | | | **Others** | | | | | | | Elasticsearch | ✔️ | ✔️ | ✔️ | | ✔️ | ### Schema Management Schema management is only applicable to RDBMS. If you are looking for a feature not supported in your database, please [contact us](/faq/#how-to-reach-us). | Feature | [Schema Synchronization](/change-database/synchronize-schema) | [Schema Editor](/change-database/schema-editor) | [Online Schema Change](/change-database/online-schema-migration-for-mysql) | [State-Based Migration (SDL)](/gitops/state-based-workflow/overview) | | :--------- | :------------------------------------------------------------ | :---------------------------------------------- | :------------------------------------------------------------------------- | :------------------------------------------------------------------- | | **RDBMS** | | | | | | MySQL | ✔️ | ✔️ | ✔️ | ✔️ | | PostgreSQL | ✔️ | ✔️ | | ✔️ | | Oracle | ✔️ | | | | | SQL Server | ✔️ | | | | | MariaDB | | | ✔️ | | | TiDB | ✔️ | ✔️ | | | # Use Cases Source: https://docs.bytebase.com/introduction/use-cases Bytebase is a unified database DevOps platform that transforms how teams collaborate on database development, security, and operations. From startups to enterprises, organizations use Bytebase to eliminate database bottlenecks and accelerate their development velocity. ## Enterprise Security & Compliance **Problem:** Data breaches cost millions, compliance violations risk business operations, and permanent database access creates security vulnerabilities while blocking operational needs. **Solution:** Bytebase provides military-grade database security with zero-trust principles, role-based access controls, data masking, and comprehensive audit trails that satisfy SOC 2, GDPR, and HIPAA requirements. **Perfect for:** Financial services, healthcare, regulated industries, and DevOps teams balancing security requirements with operational agility. **Key outcomes:** * Eliminate compliance violations and reduce audit preparation time by 80% * Prevent unauthorized data access with automated permission expiration * Maintain detailed audit trails for forensic analysis and regulatory reporting * Balance security requirements with operational efficiency **Key features:** * [Granular permission management](/security/database-permission/overview) with multi-level approval workflows * **Just-in-Time (JIT) Access:** Time-bound permissions that automatically expire after use * [Dynamic data masking](/security/data-masking/overview) for PII protection in non-production environments * [Watermarking](/security/watermark) to track and trace data access patterns * [Complete audit logs](/security/audit-log) for compliance reporting and forensic analysis ## Database Change Automation **Problem:** Manual database deployments cause delays, errors, and weekend emergencies. **Solution:** Bytebase automates the entire database change lifecycle with GitOps-style workflows, built-in rollback capabilities, automated testing, and seamless CI/CD integration. **Perfect for:** Engineering teams practicing continuous deployment who need database changes to match their code delivery speed. **Key outcomes:** * Deploy schema changes 10x faster with zero manual errors * Eliminate weekend deployments and emergency rollbacks * Achieve 99.9% deployment success rate with automated testing * Reduce database-related deployment delays by 90% **Key features:** * [VCS Integration](/gitops/overview) for GitOps workflows * [API-driven automation](/integrations/api/overview/) for custom CI/CD pipelines * Automated rollback capabilities * Multi-environment progression (dev → staging → production) ## Multi-Database & Multi-Cloud Operations **Problem:** Managing PostgreSQL, MySQL, MongoDB, and Snowflake across AWS, GCP, and Azure creates operational chaos. **Solution:** Bytebase provides a single control plane for all your databases, regardless of type or location. Manage 100+ database instances as easily as one. **Perfect for:** Organizations with heterogeneous database environments or multi-cloud strategies. **Key outcomes:** * Reduce database management overhead by 70% with unified operations * Eliminate vendor lock-in with consistent tooling across all platforms * Achieve consistent security and compliance across diverse database types * Streamline operations for both OLTP and OLAP workloads **Key features:** * [20+ database types](/introduction/supported-databases) including SQL and NoSQL * Cross-cloud management (AWS RDS, GCP Cloud SQL, Azure Database) * Unified operations for OLTP and OLAP workloads * Consistent security policies across all databases ## Multi-Tenant & Sharded Database Management **Problem:** Whether you have 1,000 tenant databases or 50 sharded partitions, deploying schema changes manually is error-prone, time-consuming, and risks data integrity. **Solution:** Bytebase treats multiple databases as a single logical unit, executing changes in parallel with automatic rollback on failure. Deploy to thousands of databases in minutes, not days. **Perfect for:** B2B SaaS companies with database-per-tenant architecture and high-scale applications with horizontally partitioned data. **Key outcomes:** * Deploy to 10,000 databases in under 1 hour (vs. days of manual work) * Achieve 100% consistency across all tenant databases * Reduce deployment errors by 95% with automated rollback * Scale database operations without scaling operations team **Key features:** * [Batch operations](/change-database/batch-change/) with parallel execution * Automatic rollback on any failure * Zero-downtime migrations with health checks * Progress tracking and detailed execution logs * Tenant/shard-aware deployment strategies ## Schema Quality & Standardization **Problem:** Inconsistent naming conventions and poor schema design lead to technical debt and production issues. **Solution:** Enforce schema standards automatically with 100+ customizable SQL review rules. Catch issues before they reach production. **Perfect for:** Platform teams establishing engineering excellence standards. **Key outcomes:** * Reduce schema-related incidents by 90% * Enforce consistent naming conventions across all teams * Prevent performance issues with automated detection of missing indexes and schema anti-patterns * Customize rules for different environments (stricter for production) **Key features:** * [100+ SQL review rules](/sql-review/review-policy) with customizable enforcement * Automated detection of anti-patterns and performance issues * Environment-specific rule configurations * Integration with development workflows ## Platform Engineering & IDP Integration **Problem:** Building database management capabilities into your Internal Development Platform requires months of engineering effort. **Solution:** Use Bytebase as a headless database platform via comprehensive APIs. Get enterprise database management without building it yourself. **Perfect for:** Platform teams building golden paths for developers. **Key outcomes:** * Reduce IDP development time from months to weeks * Provide developers with self-service database capabilities * Maintain centralized control while enabling team autonomy * Integrate seamlessly with existing developer workflows **Key features:** * [REST API](/integrations/api/overview/) for all operations * Webhook notifications for event-driven workflows * SSO integration with existing identity providers * Embeddable UI components for custom portals # What's Bytebase Source: https://docs.bytebase.com/introduction/what-is-bytebase **Bytebase: The standard for database governance** Bytebase is the database governance platform, built by the team that previously led the engineering of Google Cloud SQL and Google's API infrastructure. Databases hold your most sensitive data, yet how they get changed and queried is often the least governed part of the stack. **Database governance** closes that gap. Bytebase solves each problem with a dedicated pillar:
Problem Solution
Migrations applied by hand, approvals scattered over tickets and chat, no way back from a bad change Database Change Management
Credentials shared across SQL clients, no control over who can query what, standing access that never expires Database Access Control
Can't show the auditor who changed or accessed what, and when Database Compliance
house To deliver the three pillars, Bytebase acts as a single control plane between your users — humans and AI agents — and your databases. Every change and query routes through one place, where policies are enforced and every action is recorded. middleware AI agents are governed like human users: each connects with its own identity, over the API or an MCP server, and inherits the same access controls, data masking, and audit trail. ## Database Change Management Governs how schema and data changes reach your databases, eliminating the need for engineers to make direct changes against them. Every change becomes a reviewed, approved, and recorded rollout. Learn more in [Database CI/CD](/change-database/change-workflow). * **Automated SQL review**: Lint every proposed change against [100+ review rules](/sql-review/review-rules) before a human ever looks at it. * **Risk-based approval flows**: Route changes through [multi-step approvals](/change-database/approval) scaled to their risk and target environment. * **UI-driven or GitOps workflow**: Manage changes from the console, or entirely as code via [GitOps](/gitops/overview) with GitHub, GitLab, Bitbucket, and Azure DevOps. * **Fleet-wide batch changes**: Apply one change across a collection of databases in a single [coordinated workflow](/change-database/batch-change). * **1-click rollback**: Recover instantly from unintended UPDATE or DELETE statements with [data rollback](/change-database/rollback-data-changes). ## Database Access Control Governs who can query what data, replacing disparate SQL clients and shared credentials. Explore the full capabilities in the [SQL Editor](/sql-editor/overview) section. * **Centralized permissions**: Grant query and export access [by role](/security/database-permission/overview), not by handing out database passwords — one place to manage who can see what. * **Just-in-time access**: Let users [request time-bound access](/security/database-permission/just-in-time) that routes through the same risk-based approvals and expires automatically instead of holding standing privileges. * **Dynamic data masking**: [Mask sensitive columns](/security/data-masking/overview) in query results, so PII stays protected even for users who are allowed to query the table. ## Database Compliance Change management and access control only count if you can prove they happened. Bytebase records the evidence as a byproduct of the workflow — no separate bookkeeping. * **Complete audit log**: Every change, approval, permission grant, and query is captured in the [audit log](/security/audit-log) — filterable, streamable to external monitoring systems, and exportable as evidence for SOC 2, GDPR, and HIPAA reviews. * **Data classification**: [Classify columns by sensitivity](/security/data-masking/data-classification) and drive masking and access policies from the classification. * **Policy as code**: Codify approval flows, access and masking policies, and SQL review rules with the [Terraform provider](/integrations/terraform/overview), so the controls themselves are version-controlled and consistent across environments. In short, the three pillars consolidate what previously required stitching together migration tools, SQL clients, and ticketing systems — one platform governing how your databases are changed and accessed, with the records to prove it. venn ## Try Bytebase Sign up for a free workspace and start in seconds. Deploy with Docker on your own infrastructure. ## Compare ### Schema Migration Tools While Liquibase and Flyway are powerful command-line migration tools (think Git), Bytebase provides the complete platform experience (think GitHub/GitLab). It wraps these migration capabilities in a collaborative interface with approval workflows, audit trails, and enterprise controls. As the fastest-growing open source solution in this space, Bytebase has become the top choice for teams seeking a unified database DevSecOps platform. bytebase-vs-liquibase-flyway * [Bytebase vs. Liquibase](https://www.bytebase.com/blog/bytebase-vs-liquibase/) * [Bytebase vs. Flyway](https://www.bytebase.com/blog/bytebase-vs-flyway/) * [Bytebase vs. schemachange](https://www.bytebase.com/blog/snowflake-schema-change/) ### SQL Clients Traditional SQL clients like DBeaver, DataGrip, and Navicat excel at individual database access but weren't built for team collaboration or enterprise security. Bytebase transforms SQL access from a personal tool into an enterprise-ready platform—providing not just query capabilities, but also role-based permissions, data masking, audit logging, and centralized governance that eliminates the need to share database credentials. * [Bytebase vs. DBeaver](https://www.bytebase.com/blog/bytebase-vs-dbeaver/) * [Bytebase vs. DataGrip](https://www.bytebase.com/blog/bytebase-vs-datagrip/) * [Bytebase vs. Navicat](https://www.bytebase.com/blog/stop-using-navicat/) # Organize Resources Source: https://docs.bytebase.com/onboarding/organize-resources ## Workspace A `Workspace` represents your entire Bytebase installation, containing all users, projects, databases, and resources. Each Bytebase deployment corresponds to one workspace, making this concept largely transparent to users. To achieve workspace separation, you need to deploy multiple Bytebase installations. ## Project A `Project` groups databases, team members, and workflows together—similar to projects in Jira or GitHub. This is where all database activities happen: schema migrations, data changes, SQL queries, and more. **How to structure your projects:** * **Team-based organization** — Align projects with your organizational structure. Each team gets their own project, providing clear ownership and accountability for database changes. * **Application-centric approach** — Create projects around applications or services. This works well when multiple teams contribute to a single application or when applications share common database components. * **Schema-based grouping** — Perfect for multi-tenant architectures where you manage hundreds of databases with identical schemas. Group them in one project to roll out changes consistently across all tenants. * **Environment-specific projects** — Though Bytebase provides environment controls, some organizations create separate projects for production versus non-production databases to enforce stricter isolation and different workflows. Many enterprises layer these approaches. A common pattern is to first divide projects by application or service, then further split them by environment for critical systems that require additional safeguards. To manage project roles efficiently at scale, create [user groups](/administration/user-groups/) for teams and assign project roles to these groups rather than individual users. You can also integrate with [directory services](/administration/scim/overview) to automatically synchronize group membership with Bytebase. ## Database Instance A `Database Instance` represents a database server or cluster that you connect to using a host address and port. This includes MySQL servers, PostgreSQL clusters, AWS RDS instances, and similar database systems. A single instance can host multiple individual databases. Workspace administrators and DBAs are typically responsible for managing database instances. ## Database A `Database` is an individual database residing within a database instance. Each database belongs to exactly one project, establishing clear ownership and access boundaries. Project developers primarily interact with databases in their day-to-day work. ## Database Group A `Database Group` consists of multiple databases that share similar or identical schema structures. Common use cases include per-tenant databases in multi-tenant architectures or partitioned databases for scalability. Bytebase can execute [batch changes](/change-database/batch-change) consistently across all databases within a group, ensuring schema uniformity. ## Environment An `Environment` represents a specific stage in your development and deployment pipeline, such as development, testing, staging, or production. These typically correspond directly to your actual deployment environments. Workspace administrators can attach organizational policies to environments, enabling different rules and approval workflows for each stage. # Onboarding Plan Source: https://docs.bytebase.com/onboarding/plan For Enterprise customers, **we guarantee 100% implementation success in 90 days**. [Contact us](https://www.bytebase.com/contact-us/) to discuss your specific requirements. The ultimate goal is to make Bytebase the only tool to manage all human-to-db operations. It's fine if your team has existing tooling and process. Bytebase can be adopted in phases to improve the database operational practices gradually: ### Phase 1 - Standardize ad-hoc change process | | | | ------------ | ----------------------------------------------------------------------------------------------------------------------------------------------------- | | **Duration** | 2 weeks | | **Goals** | 1. Remove direct human write access to database.
2. Streamlined change process with review, check, rollout, rollback, notification, and history. | Manage one-off DML changes (e.g. INSERT/UPDATE/DELETE) and out-of-band schema changes (e.g. add an index on emergency). Related features: * [Change Workflow](/change-database/change-workflow/) * [Data Rollback](/change-database/rollback-data-changes/) * [Custom Approval](/change-database/approval/) * [Webhook](/change-database/webhook/) * [SQL Review](/sql-review/review-policy/) e.g. * [Alert when DML attempts to update more than 100 rows](/sql-review/review-rules/#limit-affected-row-limit) * [Specify explicit columns in INSERT](/sql-review/review-rules/#insert-statements-must-specify-columns) ### Phase 2 - Centralize data query access | | | | ------------ | ----------------------------------------------------------------------------------------------------------------------------------------------------------- | | **Duration** | 2 weeks | | **Goals** | 1. Remove direct human read access to database.
2. Access grants are reviewed and recorded.
3. Dynamic masking policy depending on the querier. | Bytebase provides a web-based SQL Editor. Related features: * [SQL Editor](/sql-editor/overview/) * [Database Permission](/security/database-permission/overview/) * [Request Query](/security/database-permission/request/) and [Export](/security/database-permission/export/) access flow * [Dynamic Data Masking](/security/data-masking/overview/) ### Phase 3 - Integrate SQL review into CI | | | | ------------ | --------------------------------------------------------------------------------------------- | | **Duration** | 1 week | | **Goals** | 1. Auto check SQL anti-patterns during CI.
2. Non-intrusive to the existing CI pipeline. | If your team has the existing database schema migration process in place. You can start by including the SQL Review check into the CI. Related features: * [SQL Review](/sql-review/review-policy/) * [SQL Review API](/integrations/api/sql-review/) * [Service Account](/integrations/api/authentication/#service-account) ### Phase 4 - Manage schema migration | | | | ------------ | ---------------------------------------------------------------------------------------------------------------------------------------------------- | | **Duration** | 4 \~ 8 weeks | | **Goals** | 1. Decouple schema migration from code release.
2. Streamlined change process with review, check, rollout, rollback, notification, and history. | You have achieved decent success in implementing Bytebase for your org. You may not need phase 4 if the application is OK to deploy schema migration together with the code release. On the other hand, it's beneficial to use Bytebase for schema migration if you have any of the following situations: * Multiple servers connecting to the same database. * Multi-region database deployment. * Multi-tenant service and each tenant has its own database. * Long-running schema migration. The implementation duration depends on how many existing schema migration workflows need to be moved to Bytebase. Related features (in addition to Phase 1): * [Schema Synchronization](/change-database/synchronize-schema/) * [GitOps](/gitops/overview/) * [Batch Change](/change-database/batch-change/) * [Online Schema Migration (MySQL only)](/change-database/online-schema-migration-for-mysql/) # POC Checklist Source: https://docs.bytebase.com/onboarding/poc The POC (Proof of Concept) checklist guides the potential buyers to evaluate the Bytebase Enterprise plan. The POC process usually takes 1 \~ 2 weeks. Please [contact us](https://www.bytebase.com/contact-us) to request an enterprise license. ## Install * [System Requirements](/get-started/self-host/production-setup#system-requirements) * [Docker](/get-started/self-host/deploy-with-docker), [Kubernetes](/get-started/self-host/deploy-with-kubernetes) * [Production Setup](/get-started/self-host/production-setup) ## Understand Permission Model Check out [Roles and Permissions](/administration/roles). ## Configure Environment * [What is Environment](/onboarding/organize-resources/#environment) * [Rollout Policy](/change-database/environment-policy/rollout-policy/) * [SQL Review Policy](/sql-review/review-policy/) ## Configure Instance * [What is Instance](/onboarding/organize-resources/#database-instance) * [Assign License](/administration/license/) * [Connect Instance](/get-started/instance/) ## Configure Project * [What is Project](/onboarding/organize-resources/#project) * [Create a Project and Transfer Database](/get-started/step-by-step/create-a-project/#option-a-transfer-an-existing-database-into-the-project) ## Database Change Workflow * [Database Change Workflow](/change-database/change-workflow/) * [Data Rollback](/change-database/rollback-data-changes/) * [Custom Approval](/change-database/approval/) * [Webhook Notification](/change-database/webhook/) * [Batch Change](/change-database/batch-change/) * [GitOps](/gitops/overview/) ## SQL Editor * [Query Mode (read-only)](/sql-editor/run-queries/) * [Admin Mode](/sql-editor/admin-mode/) ## Database Security * [Database Permission](/security/database-permission/overview/) * [Dynamic Data Masking](/security/data-masking/overview/) * [Audit Log](/security/audit-log/) * [Watermark](/security/watermark/) ## Login and SSO User can self self-signup or invited by the `Workspace Admin`. Alternatively, `Workspace Admin` can configure SSO. Bytebase supports all common providers such as Okta, Google, LDAP. * [SSO](/administration/sso/overview/) * [SCIM (Directory Sync)](/administration/scim/overview/) * [2FA](/administration/2fa/) * [Password Restriction](/administration/password/) * [Sign-in Restriction](/administration/sign-in-restriction/) ## Misc * [Customize Logo](/administration/customize-logo/) * [Announcement Banner](/administration/announcement/) # SQL Editor Data Access Control Source: https://docs.bytebase.com/onboarding/sql-editor-data-access-control ## Background Traditional local SQL clients like DBeaver and Navicat carries security limitations that Bytebase's centralized approach solves: | Traditional SQL Clients | Bytebase Solution | | --------------------------------------------------------------------------------------------- | --------------------------------------------------------------------------------------------------------- | | **Distributed Credentials**: Stored locally in plain text, manual revocation, multiple copies | **Centralized Credentials**: Never leave platform, SSO authentication, directory sync, instant revocation | | **No Fine-Grained Access**: All-or-nothing database access, no table/column restrictions | **Granular Control**: Database, schema, table-level restrictions with column masking | | **Missing Audit Trails**: Zero visibility, no centralized logging, compliance violations | **Complete Auditing**: Every query, change, and admin action logged with full context | | **Time-Based Access Gaps**: No temporary access, permanent until revoked | **Just-in-Time Access**: Request-based temporary access with approval workflows | Below we walk through a two-environment example, one for test and one for production. The principle is to provide maximum security for production while keeping developer productivity for test. Query access, just-in-time access, and audit logging apply to both environments alike; the only environment-specific choice is where DDL and DML statements may run directly — and that choice is made in the role grant itself. ## Fine-Grained Query Users need to be granted explicit permissions to query the data from SQL Editor. The most straightforward way is to grant the `SQL Editor User` role to the user inside the project. 1. Specify `SQL Editor User` as the role. 2. Specify a reason. 3. Grant the access to all databases in the project or fine-grained to specific databases, schemas, and tables. 4. Select the environments where DDL and DML statements may run directly — for this setup, test only. See [DDL and DML Execution](#ddl-and-dml-execution). 5. Specify an expiration date. grant `SQL Editor User` is a built-in role covering `EXPLAIN`, `SELECT`, and — in the environments selected in the grant — DDL/DML. For read-only access, grant the built-in `SQL Editor Read User` role instead, or create a [custom role](/administration/roles) with only the permissions you want (e.g. `bb.sql.explain`). ### DDL and DML Execution Whether a user can run DDL and DML statements directly in SQL Editor is part of their [database permissions](/security/database-permission/overview/) (`bb.sql.ddl`, `bb.sql.dml`), scoped to the environments selected when granting the role (step 4 above): | | Test | Prod | | ------------------------ | ---- | ---- | | Run DDL and DML directly | ✅ | ❌ | In production, where the grant does not cover DDL/DML, SQL Editor directs the user to create a database change plan and follow the [change workflow](/change-database/change-workflow) instead. ## Just-In-Time Access You may disallow any production access by default and only allow temporary access on-demand. Users can request temporary access via the SQL Editor. You will configure the [custom approval policy](/change-database/approval) to designate the approvers. ## Audit Logging For data privacy reasons, Bytebase does not log the actual data in the query result. Every query is [audit logged](/security/audit-log). From the audit logging view, you can see the actor, the database, the SQL statement, and the result metadata such as the returned columns and the row count. grant grant grant ## Summary Below is a summary of the access control settings for the test and production environments. | | Test | Prod | | ----------------------------------- | ---- | ---- | | Run DDL and DML directly | ✅ | ❌ | | Fine-grained EXPLAIN, Query, Export | ✅ | ✅ | | Just-in-Time Access | ✅ | ✅ | | Audit Logging | ✅ | ✅ | You can extend the solution by exploring the following features: * Create [user groups](/administration/user-groups) and enable [directory sync](/administration/scim/overview) to automate the role assignment. * Configure [data masking](/security/data-masking/overview) to mask sensitive data. * Use [Terraform](/integrations/terraform/overview) to codify all the settings. # Page Agent Source: https://docs.bytebase.com/page-agent The Page Agent is an AI assistant built into Bytebase. It lives in a floating chat window that's available on every page — ask it anything in plain language, and it helps you get it done. ## Why we built it Bytebase has a lot of features, and it's not always obvious how to accomplish what you're trying to do — even when the feature you need is already there. You might know what you want (archive old rows, grant a teammate access to a project, fix a failing plan check) but not which page to open, which buttons to click, or which settings control the behavior you're after. Or maybe you know exactly how to do it — you'd just rather hand it off and review the result instead of clicking through the flow yourself. The Page Agent is designed for both. It understands how Bytebase works end to end, so it can: * **Show you the way** — walk you through a workflow step by step when you want to do it yourself. * **Do it for you** — take the actions directly when you'd rather describe the outcome and check the result. You stay in control: the agent shows each step it takes so you can follow along, and you can stop or redirect it at any time. ## Setup The Page Agent uses the same AI provider configuration as the rest of Bytebase. If you haven't enabled it yet, follow the steps in [AI Assistant](/ai-assistant) to configure your OpenAI, Claude, Gemini, or Azure OpenAI credentials. Once AI is configured, the Page Agent is available from any page in the Bytebase console. Click the agent toggle in the dashboard header to open the chat window. ## How it works Page Agent Architecture The Page Agent has three layers: 1. **Chat window** — A floating panel available from any page in the console. It stays open as you move between pages. 2. **Agent loop** — Runs in your browser. When you ask a question, the agent thinks about what to do, takes an action (reading the page, calling an API, navigating somewhere), observes the result, and repeats until it has a complete answer. Each step is shown in the chat. 3. **AI backend** — Bytebase routes your conversation to the AI provider configured in your workspace settings. Your credentials never leave your server. ## Capabilities | Capability | Description | | ------------------------- | ------------------------------------------------------------------------------------------------------------------------------------- | | **Read the current page** | Understands which page you're on, what project, database, or issue you're viewing, and what state the UI is in | | **Call Bytebase APIs** | Queries and updates data directly — listing databases, creating issues, checking deployment status, and more | | **Navigate** | Goes to any page in the console on your behalf | | **Interact with the UI** | Clicks buttons, fills form fields, selects options — useful on pages with unsaved state like editors and creation forms | | **Discover APIs** | Browses the full Bytebase API catalog to find the right endpoint for a task | | **Load workflow guides** | Pulls in step-by-step instructions for common multi-step tasks like running queries, creating schema changes, or granting permissions | ## Controls and audit The Page Agent operates within Bytebase's existing safety mechanisms: * **Uses your permissions.** The agent can only perform actions your account is authorized for. If you can't drop a table, neither can the agent. * **Goes through the normal approval flow.** Schema changes the agent creates are regular Bytebase issues — they respect your project's approval policies, plan checks, and rollout gates. The agent cannot bypass these. * **Audit-logged under your account.** Every API call, issue creation, and permission change is recorded in the audit log with your user name. * **Controlled by workspace AI settings.** The Page Agent is available only when AI is enabled in your workspace. Disabling [AI Assistant](/ai-assistant) in **Settings > General** turns off the Page Agent along with other AI features. ## Use cases ### Fixing a plan check error Your issue failed a SQL review check and you want help understanding and fixing the violation. > **You:** Help me fix this plan check error.

> **Agent:** *(Reads the current page to inspect the issue and SQL review results)*

> **Agent:** The SQL review flagged a violation: **"Do not apply functions or perform calculations on indexed fields in the WHERE clause."** The problem is in your INSERT...SELECT statement: > > ```sql theme={null} > INSERT INTO orders_archive (id, customer_id, created_at) > SELECT id, customer_id, created_at > FROM orders > WHERE YEAR(created_at) = 2023; > ``` > > Wrapping `created_at` in `YEAR()` prevents the database from using the index on that column, forcing a full table scan. Here's a rewrite that keeps the index usable: > > ```sql theme={null} > INSERT INTO orders_archive (id, customer_id, created_at) > SELECT id, customer_id, created_at > FROM orders > WHERE created_at >= '2023-01-01' AND created_at < '2024-01-01'; > ``` > > Want me to update the SQL in the editor and re-run the check? Other things you can ask: * "Why did the SQL review fail on this issue?" * "The plan check says I have a naming convention error — can you fix it?" * "Rewrite my migration to pass all the review checks" ### Granting permissions A teammate needs access to a project, and you want to set it up without navigating to settings. > **You:** Give [alice@example.com](mailto:alice@example.com) developer access to the Payment project.

> **Agent:** I'll add Alice as a project developer. Let me look up the project members API. > *(Discovers the project IAM API → calls it to add the role binding)*

> **Agent:** Done! [alice@example.com](mailto:alice@example.com) now has the **Project Developer** role on the Payment project. She can view databases, create issues, and run queries in SQL Editor. Other things you can ask: * "Who has access to the Production environment?" * "Remove [bob@example.com](mailto:bob@example.com) from the HR project" * "What roles does [alice@example.com](mailto:alice@example.com) have?" ### Querying data across your workspace Get a quick overview of resources without clicking through multiple pages. > **You:** What databases are in the "Payment" project?

> **Agent:** Let me check. > *(Calls the Bytebase API to list databases in the Payment project)*

> **Agent:** The Payment project has 4 databases: > > | Database | Environment | Engine | > | ------------------ | ----------- | ------------- | > | payment\_prod | Production | PostgreSQL 15 | > | payment\_staging | Staging | PostgreSQL 15 | > | payment\_dev | Dev | PostgreSQL 15 | > | payment\_analytics | Production | ClickHouse | Other things you can ask: * "How many pending issues are there in the HR project?" * "Show me all PostgreSQL instances" * "What SQL review policies are configured for production?" ## Tips * **Be specific.** "Add a NOT NULL varchar(255) column called `display_name` to the `users` table in staging" works better than "change the users table." * **Ask follow-ups.** The agent remembers the full conversation, so "now do the same for production" works after a staging change. * **Let it navigate.** You don't need to be on the right page first — the agent will go where it needs to. * **Watch the tool calls.** The chat shows each API call or UI action the agent takes, so you always know what happened. ## Important notes The agent uses AI to plan and execute actions. Always review what it did — issue created, SQL written, permissions granted — before moving on. # Audit Log Source: https://docs.bytebase.com/security/audit-log Audit logging tracks all configuration changes and data operations within your Bytebase workspace. These logs help you identify who performed what action, when it occurred, and what resources were affected. This enables compliance monitoring, security analysis, and troubleshooting. Audit logging is available only for **Pro** and **Enterprise** plans. ## Accessing Audit Logs ### GUI **Workspace Admin** or **DBA** can access audit logs through the Bytebase console: 1. Navigate to the **Settings** page 2. Select **Audit Log** from the sidebar 3. View the chronological list of all audit events overview You can filter audit logs by: * User * Action type * Resource * Time range filter filter-date ### API Use the Bytebase API to programmatically access audit logs for integration with external systems and automated monitoring. For detailed API documentation and examples, see [Audit Log API](/integrations/api/audit-log/). ### Streaming Stream audit logs to external logging and monitoring systems for centralized management. It supports JSON format output in addition to the default text format. 1. Navigate to **Settings** → **General** → **Audit Log Export** 2. Enable **Enable audit logging to stdout** 3. Save the configuration Once enabled, all audit events stream to stdout of your Bytebase service. To enable **JSON format output**, start Bytebase with the `--enable-json-logging` flag: ```bash theme={null} docker run --rm --init \ --name bytebase \ --publish 8080:8080 --pull always \ --volume ~/.bytebase/data:/var/opt/bytebase \ bytebase/bytebase:latest \ --enable-json-logging ``` Supported integrations: * **SIEM Platforms**: e.g., Splunk, Datadog, Elastic Security * **Log Aggregators**: e.g., Fluentd, Logstash, Grafana Loki * **Cloud Services**: e.g., AWS CloudWatch, Google Cloud Logging, Azure Monitor ## Log Structure ### Example Entry ```json theme={null} { "parent": "projects/sample-project", "method": "/bytebase.v1.SQLService/Query", "resource": "instances/prod-postgres/databases/mydb", "user": "users/developer@example.com", "severity": "INFO", "request": "{\"name\":\"instances/prod-postgres/databases/mydb\",\"statement\":\"SELECT * FROM users LIMIT 10\",\"limit\":100}", "response": "{\"results\":[{\"columnNames\":[\"id\",\"name\"],\"columnTypeNames\":[\"int4\",\"text\"],\"rowsCount\":10}]}", "status": { "code": 0 }, "latency": { "seconds": 0, "nanos": 125000000 }, "requestMetadata": { "callerIp": "192.168.1.100", "callerSuppliedUserAgent": "Mozilla/5.0 Chrome/120.0" } } ``` ### Payload Fields | Field | Type | Description | | ----------------- | ------ | -------------------------------------------------------------------------------- | | `parent` | string | Scope of the audit log. Format: `projects/{project}` or `workspaces/{workspace}` | | `method` | string | Full API method name. Example: `/bytebase.v1.SQLService/Query` | | `resource` | string | The primary resource being acted upon | | `user` | string | User who performed the action. Format: `users/{email}` | | `severity` | string | Log severity level | | `request` | string | JSON-serialized request payload (sensitive fields redacted) | | `response` | string | JSON-serialized response payload (sensitive fields redacted) | | `status` | object | gRPC status with `code` and `message` fields | | `latency` | object | Operation duration with `seconds` and `nanos` fields | | `serviceData` | object | Service-specific metadata (e.g., IAM policy changes) | | `requestMetadata` | object | Client information including `callerIp` and `callerSuppliedUserAgent` | ### Severity Levels | Level | Value | Description | | ---------------------- | ----- | -------------------------------- | | `SEVERITY_UNSPECIFIED` | 0 | Unspecified | | `DEBUG` | 1 | Debug-level information | | `INFO` | 2 | Informational messages (default) | | `NOTICE` | 3 | Notable events | | `WARNING` | 4 | Warning conditions | | `ERROR` | 5 | Error conditions | | `CRITICAL` | 6 | Critical conditions | | `ALERT` | 7 | Action must be taken immediately | | `EMERGENCY` | 8 | System is unusable | ### Status Codes Status codes follow [gRPC conventions](https://grpc.io/docs/guides/status-codes/): | Code | Name | Description | | ---- | -------------------- | ------------------------------------------------------------------------ | | 0 | OK | Operation succeeded (no error) | | 1 | CANCELED | Operation was canceled, typically by the caller | | 2 | UNKNOWN | Operation failed for an unknown reason | | 3 | INVALID\_ARGUMENT | Client supplied an invalid argument | | 4 | DEADLINE\_EXCEEDED | Deadline expired before the operation could complete | | 5 | NOT\_FOUND | Requested entity (e.g., file, resource) was not found | | 6 | ALREADY\_EXISTS | Entity that client attempted to create already exists | | 7 | PERMISSION\_DENIED | Caller doesn't have permission to execute the operation | | 8 | RESOURCE\_EXHAUSTED | Some resource has been exhausted (e.g., quota, disk space) | | 9 | FAILED\_PRECONDITION | System is not in a state required for operation's execution | | 10 | ABORTED | Operation aborted due to concurrency issue (e.g., transaction abort) | | 11 | OUT\_OF\_RANGE | Operation attempted past the valid range (e.g., seek past EOF) | | 12 | UNIMPLEMENTED | Operation isn't implemented, supported, or enabled | | 13 | INTERNAL | Internal error - some invariants expected by the system have been broken | | 14 | UNAVAILABLE | Service is currently unavailable (usually temporary) | | 15 | DATA\_LOSS | Unrecoverable data loss or corruption | | 16 | UNAUTHENTICATED | Request does not have valid authentication credentials | ## Audited Events | Event | Method | Description | | -------------- | ---------------------------------------- | ---------------------------------------------------- | | User Login | `/bytebase.v1.AuthService/Login` | User authentication (password, SSO, MFA) | | User Logout | `/bytebase.v1.AuthService/Logout` | User session termination | | Token Exchange | `/bytebase.v1.AuthService/ExchangeToken` | Workload Identity token exchange for CI/CD pipelines | **Notes:** * Sensitive fields redacted: `password`, `otpCode`, `recoveryCode`, `mfaTempToken`, `idpContext` * MFA phase logs extract user email from MFA temp token when email is not in request | Event | Method | Description | | ------------ | --------------------------------------- | ---------------------------------- | | Create User | `/bytebase.v1.UserService/CreateUser` | Create new user account | | Update User | `/bytebase.v1.UserService/UpdateUser` | Modify user settings, MFA, profile | | Delete User | `/bytebase.v1.UserService/DeleteUser` | Soft-delete user account | | Restore User | `/bytebase.v1.UserService/UndeleteUser` | Restore deleted user | | Update Email | `/bytebase.v1.UserService/UpdateEmail` | Change user email address | **Notes:** * User responses redacted to only include: `name`, `email`, `title`, `userType` | Event | Method | Description | | ------------ | --------------------------------------- | -------------------------------- | | Create Group | `/bytebase.v1.GroupService/CreateGroup` | Create user group | | Update Group | `/bytebase.v1.GroupService/UpdateGroup` | Modify group membership/settings | | Delete Group | `/bytebase.v1.GroupService/DeleteGroup` | Delete user group | | Event | Method | Description | | ----------- | ------------------------------------- | ----------------------- | | Create Role | `/bytebase.v1.RoleService/CreateRole` | Create custom role | | Update Role | `/bytebase.v1.RoleService/UpdateRole` | Modify role permissions | | Delete Role | `/bytebase.v1.RoleService/DeleteRole` | Delete custom role | | Event | Method | Description | | ------------------------ | ------------------------------------------------------------- | -------------------------- | | Create Identity Provider | `/bytebase.v1.IdentityProviderService/CreateIdentityProvider` | Configure new SSO provider | | Update Identity Provider | `/bytebase.v1.IdentityProviderService/UpdateIdentityProvider` | Modify SSO configuration | | Delete Identity Provider | `/bytebase.v1.IdentityProviderService/DeleteIdentityProvider` | Remove SSO provider | | Event | Method | Description | | ---------------------- | ------------------------------------------------- | --------------------------------- | | Delete Project | `/bytebase.v1.ProjectService/DeleteProject` | Soft-delete project | | Restore Project | `/bytebase.v1.ProjectService/UndeleteProject` | Restore deleted project | | Batch Delete Projects | `/bytebase.v1.ProjectService/BatchDeleteProjects` | Delete multiple projects | | Set Project IAM Policy | `/bytebase.v1.ProjectService/SetIamPolicy` | Modify project member permissions | **Notes:** * IAM policy changes include `serviceData` with `PolicyDelta` showing added/removed bindings | Event | Method | Description | | ------------------------ | -------------------------------------------- | ---------------------------------- | | Set Workspace IAM Policy | `/bytebase.v1.WorkspaceService/SetIamPolicy` | Modify workspace-level permissions | **Notes:** * Includes `serviceData` with `PolicyDelta` showing added/removed bindings | Event | Method | Description | | ---------------------- | --------------------------------------------------- | ------------------------------ | | Create Instance | `/bytebase.v1.InstanceService/CreateInstance` | Register new database instance | | Update Instance | `/bytebase.v1.InstanceService/UpdateInstance` | Modify instance configuration | | Delete Instance | `/bytebase.v1.InstanceService/DeleteInstance` | Soft-delete instance | | Restore Instance | `/bytebase.v1.InstanceService/UndeleteInstance` | Restore deleted instance | | Batch Update Instances | `/bytebase.v1.InstanceService/BatchUpdateInstances` | Bulk update instances | | Add Data Source | `/bytebase.v1.InstanceService/AddDataSource` | Add connection to instance | | Update Data Source | `/bytebase.v1.InstanceService/UpdateDataSource` | Modify connection settings | | Remove Data Source | `/bytebase.v1.InstanceService/RemoveDataSource` | Remove connection | **Notes:** * DataSource sensitive fields redacted: `password`, `sslCa`, `sslCert`, `sslKey`, `sshPassword`, `sshPrivateKey`, `authenticationPrivateKey`, `externalSecret`, `saslConfig.krbConfig.keytab`, `masterPassword` | Event | Method | Description | | ----------------------- | ----------------------------------------------------------- | ------------------------------------ | | Update Database | `/bytebase.v1.DatabaseService/UpdateDatabase` | Modify database settings/labels | | Batch Update Databases | `/bytebase.v1.DatabaseService/BatchUpdateDatabases` | Bulk update databases | | Update Database Catalog | `/bytebase.v1.DatabaseCatalogService/UpdateDatabaseCatalog` | Modify schema catalog/classification | | Event | Method | Description | | --------------------- | ------------------------------------------------------- | ----------------------------- | | Create Database Group | `/bytebase.v1.DatabaseGroupService/CreateDatabaseGroup` | Create logical database group | | Update Database Group | `/bytebase.v1.DatabaseGroupService/UpdateDatabaseGroup` | Modify database group | | Delete Database Group | `/bytebase.v1.DatabaseGroupService/DeleteDatabaseGroup` | Delete database group | | Event | Method | Description | | ------------- | -------------------------------------- | --------------------------------------------- | | Execute Query | `/bytebase.v1.SQLService/Query` | Execute read-only SQL query | | Admin Execute | `/bytebase.v1.SQLService/AdminExecute` | Execute SQL with admin privileges (streaming) | | Export Data | `/bytebase.v1.SQLService/Export` | Export query results to file | **Notes:** * Response rows completely redacted - only metadata captured: `columnNames`, `columnTypeNames`, `rowsCount`, `error`, `latency`, `statement` * Each request/response pair in streaming operations generates a separate audit log * Export request `password` field redacted; response `content` not logged | Event | Method | Description | | -------------------------- | --------------------------------------------------- | ---------------------- | | Create Issue | `/bytebase.v1.IssueService/CreateIssue` | Create change request | | Update Issue | `/bytebase.v1.IssueService/UpdateIssue` | Modify issue details | | Create Issue Comment | `/bytebase.v1.IssueService/CreateIssueComment` | Add comment to issue | | Update Issue Comment | `/bytebase.v1.IssueService/UpdateIssueComment` | Modify issue comment | | Batch Update Issues Status | `/bytebase.v1.IssueService/BatchUpdateIssuesStatus` | Bulk status change | | Approve Issue | `/bytebase.v1.IssueService/ApproveIssue` | Approve change request | | Reject Issue | `/bytebase.v1.IssueService/RejectIssue` | Reject change request | | Request Issue | `/bytebase.v1.IssueService/RequestIssue` | Re-request approval | | Event | Method | Description | | ---------------- | ------------------------------------------------- | ------------------------- | | Create Plan | `/bytebase.v1.PlanService/CreatePlan` | Create deployment plan | | Update Plan | `/bytebase.v1.PlanService/UpdatePlan` | Modify deployment plan | | Create Rollout | `/bytebase.v1.RolloutService/CreateRollout` | Create deployment rollout | | Run Tasks | `/bytebase.v1.RolloutService/BatchRunTasks` | Execute deployment tasks | | Skip Tasks | `/bytebase.v1.RolloutService/BatchSkipTasks` | Skip deployment tasks | | Cancel Task Runs | `/bytebase.v1.RolloutService/BatchCancelTaskRuns` | Cancel running tasks | | Event | Method | Description | | -------------- | -------------------------------------------- | ---------------------------- | | Create Policy | `/bytebase.v1.OrgPolicyService/CreatePolicy` | Create organizational policy | | Update Policy | `/bytebase.v1.OrgPolicyService/UpdatePolicy` | Modify policy settings | | Delete Policy | `/bytebase.v1.OrgPolicyService/DeletePolicy` | Remove policy | | Update Setting | `/bytebase.v1.SettingService/UpdateSetting` | Modify system settings | ## Events Not Logged | Category | Operations | Reason | | ----------------------------- | -------------------------------------------------------------------- | ------------------------------------------------------- | | **Read-Only Operations** | All `Get*`, `List*`, `Search*` methods, `GetIamPolicy` | Low security impact - viewing data doesn't modify state | | **High-Frequency Operations** | `AuthService/Refresh`, `BatchSyncInstances`, Actuator health checks | Too frequent, would create excessive log volume | | **Validate-Only Requests** | Any request with `validate_only = true` | Dry-run operations that don't modify state | | **Utility Services** | `CELService/*`, `SQLService/AICompletion`, `SQLService/DiffMetadata` | Utility functions with no security implications | | **Review & Sheet Operations** | `ReviewConfigService/*`, `SheetService/*`, `WorksheetService/*` | Lower security impact configuration | | **Release & Revision** | `ReleaseService/*`, `RevisionService/*` | Schema tracking operations | | **Instance Role Operations** | `InstanceRoleService/*` | Database role management | ## Retention Bytebase does not automatically purge audit logs. You must implement periodic cleanup to prevent disk space exhaustion, which can cause Bytebase to crash. Audit logs are stored in the `audit_log` table in the Bytebase metadata database. **Recommended retention periods:** * **Minimum**: 90 days * **Compliance environments**: 6–12 months **Cleanup options:** * Manually delete old logs from the `audit_log` table * Set up a scheduled job using `pg_cron` or system cron * Export logs to external log management or SIEM systems for longer-term retention ## Limitations ### Privacy and Security * **Query results excluded**: Actual row data from queries is redacted * **Sensitive data masking**: Passwords, API keys, SSL certificates, SSH keys automatically replaced with masked values * **Response truncation**: Large response payloads may be omitted ### Technical Constraints * **Synchronous generation**: Audit logs are generated during request processing * **Pagination limit**: Maximum 5,000 entries per API call * **Filter complexity**: Searches must use valid CEL expressions * **Storage**: Logs stored in metadata database, following its retention policies # Masking Exemption Source: https://docs.bytebase.com/security/data-masking/access-unmasked-data Masking precedence: [Masking Exemption](/security/data-masking/access-unmasked-data) > [Global Masking Rule](/security/data-masking/global-masking-rule) > [Column Masking](/security/data-masking/column-masking). Certain roles can grant masking exemption to the users to access the unmasked data: * Built-in roles: `Workspace Admin`, `DBA`, `Project Owner`. * [Custom roles](/administration/roles): `bb.policies.createMaskingExemptionPolicy`, `bb.policies.updateMaskingExemptionPolicy`, `bb.policies.deleteMaskingExemptionPolicy`. To grant masking exemption: 1. Go to the project, click **Manage** > **Masking Exemptions**. 2. Click **Grant Exemption**. You can grant either `Export` or `Query` exemption. 3. Select the user/groups and the database/table, and click **Confirm**. bb-grant-exemption You can't grant masking exemption to the service account. Because the intended use is to exempt human users from the masking policy when they query from SQL Editor. Masking exemption is not supported for document databases (**MongoDB** and **Elasticsearch**). See [Document Database Masking](/security/data-masking/document-database-masking). # Column Masking Source: https://docs.bytebase.com/security/data-masking/column-masking Masking precedence: [Masking Exemption](/security/data-masking/access-unmasked-data) > [Global Masking Rule](/security/data-masking/global-masking-rule) > [Column Masking](/security/data-masking/column-masking). In the project level, besides `Workspace Admin` and `DBA`, `Project Owner` can set table columns semantic type to mask the data. This page covers relational databases. For document databases (**MongoDB** and **Elasticsearch**), field-level masking is configured through the collection/index Catalog instead — see [Document Database Masking](/security/data-masking/document-database-masking). Go to the **table detail** page, and click the **pen icon** and apply the semantic type. bb-column-masking Combined with [Semantic Types](/security/data-masking/semantic-types), here is the result you will get in **SQL Editor**. bb-sql-editor-partial-masking However, if the global masking rule is also applied, the result will be as follows. Because the global masking rule takes precedence over the column masking. bb-sql-editor-full-masking # Data Classification Source: https://docs.bytebase.com/security/data-masking/data-classification Data classification allows you to classify columns and apply masking to those columns via the [Global Masking Rule](/security/data-masking/global-masking-rule). This allows you to manage masking policy for many columns by controlling only a small number of classifications. overview In the above example, column `first_name` and `last_name` will be applied `Default Partial Masking`, because: * Column `first_name` and `last_name` are classified as `Contact Info`. * `Contact Info` corresponds to `security level 2`. * `Security level 2` applies semantic type `Default Partial Masking`. ## Step 1 - Define Classification definition You upload a JSON file containing the classification definition. The definition contains 2 sections: 1. Security levels. Usually you define 3 \~ 5 levels. 2. Classes. You can define multi-level classes. You assign a security level to each leaf class. ### Simple Classification This is a [simple classification](/content/docs/security/data-classification/classification-simple.json) showing the structure: 1. There are 2 security levels. 2. There are 2 top classes. Class 1 contains 4 sub-classes. Class 2 contains 2 sub-classes. Each subclass (leaf node) is assigned a security level. ```json theme={null} { "title": "Classification Example", "levels": [ { "id": "1", "title": "Level 1", "description": "" }, { "id": "2", "title": "Level 2", "description": "" } ], "classification": { "1": { "id": "1", "title": "Basic", "description": "" }, "1-1": { "id": "1-1", "title": "Basic", "description": "", "levelId": "1" }, "1-2": { "id": "1-2", "title": "Assert", "description": "", "levelId": "1" }, "1-3": { "id": "1-3", "title": "Contact", "description": "", "levelId": "2" }, "1-4": { "id": "1-4", "title": "Health", "description": "", "levelId": "2" }, "2": { "id": "2", "title": "Relationship", "description": "" }, "2-1": { "id": "2-1", "title": "Social", "description": "", "levelId": "1" }, "2-2": { "id": "2-2", "title": "Business", "description": "", "levelId": "1" } } } ``` ### Financial Industry Classification A comprehensive data classification ([English](/content/docs/security/data-classification/classification-financial-industry-en.json), [Chinese](/content/docs/security/data-classification/classification-financial-industry-zh.json)) for the financial industry. It contains: * 5 security levels. * 14 top-categories. * 300+ sub-categories. ## Step 2 - Configure Global Masking Policy global From the [Global Masking Policy](/security/data-masking/global-masking-rule), you can define the masking level for each classification level. ## Step 3 - Classify Column Classification is stored in Bytebase, not in the database schema. Go to the **table detail** page, click the **pen icon**, and set the classification for the column. Column detail dialog with a classification assigned from the classification tree ## API Integration Set a column's classification through `DatabaseCatalogService` in the [API reference](https://api.bytebase.com), or manage it as code with the [Terraform provider](/tutorials/manage-data-masking-with-terraform). # Document Database Masking Source: https://docs.bytebase.com/security/data-masking/document-database-masking For document databases (**MongoDB** and **Elasticsearch**), masking is configured **per-collection / per-index** through the **Catalog** using an `objectSchema`, not the column-based configuration used by relational databases. [Global Masking Rule](/security/data-masking/global-masking-rule) and [Masking Exemption](/security/data-masking/access-unmasked-data) are **not** supported for document databases at this time. [Dynamic Data Masking](/security/data-masking/overview) applies to document databases at the **field level**. When a user queries a collection or index through the [SQL Editor](/sql-editor/overview), fields tagged with a [Semantic Type](/security/data-masking/semantic-types) are masked in the result, while other fields are returned as-is. It works at the query layer — no changes to your application or data are required. Masking supports: * Fields in nested objects (e.g. `contact.phone`). * Elements within arrays. * Fields in joined collections via `$lookup` and `$graphLookup`. ## Configure field masking 1. Create the [Semantic Types](/security/data-masking/semantic-types) you want to apply. The masking algorithm is determined by the semantic type. For document databases, **Full mask** is the most common choice since it works with all field types (strings, numbers, objects, arrays). 2. Go to **Databases**, select your MongoDB or Elasticsearch database, and open a collection (MongoDB) or index (Elasticsearch). 3. In the collection (MongoDB) / index (Elasticsearch) detail, scroll to the **Catalog** section and edit the `objectSchema` JSON. Assign a semantic type to each field you want masked by setting its `semanticType` to the semantic type's ID (found on the **Data Access** > **Semantic Types** page). Example `objectSchema` for a `users` collection: ```json theme={null} { "name": "users", "objectSchema": { "type": "OBJECT", "structKind": { "properties": { "email": { "type": "STRING", "semanticType": "" }, "contact": { "type": "OBJECT", "structKind": { "properties": { "phone": { "type": "STRING", "semanticType": "" }, "city": { "type": "STRING" } } } }, "tags": { "type": "ARRAY", "arrayKind": { "kind": { "type": "STRING", "semanticType": "" } } }, "profile": { "type": "OBJECT", "semanticType": "", "structKind": { "properties": { "ssn": { "type": "STRING" }, "address": { "type": "STRING" } } } } } } } } ``` | Field | Description | | :---------------------- | :------------------------------------------------------------------------------------------------------- | | `name` | The collection or index name. | | `type` | Field type: `STRING`, `NUMBER`, `BOOLEAN`, `OBJECT`, or `ARRAY`. The root type is always `OBJECT`. | | `semanticType` | The ID of the [Semantic Type](/security/data-masking/semantic-types) to apply. Omit for unmasked fields. | | `structKind.properties` | A map of field names to their schema, for `OBJECT` types. | | `arrayKind.kind` | The schema of array elements, for `ARRAY` types. | When you assign a `semanticType` to an **object** field (like `profile` above), the entire subtree is replaced with a single masked value regardless of child field configurations. For **arrays**, you can mask at two levels: * **Item-level** — set `semanticType` on `arrayKind.kind`. Each element is masked individually (e.g. `["******", "******"]`). * **Array-level** — set `semanticType` on the array field itself. The entire array is replaced with a single masked value. ## Query behavior Masking is applied automatically to query results in the SQL Editor. Given the schema above, `db.users.find({ name: "Alice" })` returns `email`, `contact.phone`, and every element of `tags` masked, `contact.city` visible, and the whole `profile` object replaced with a single masked value. To prevent users from inferring masked values with targeted filters, **querying on a masked field in a filter predicate is rejected**: ``` using field "email" tagged by semantic type "" in query predicate is not allowed ``` ## Supported operations Masking only applies to operations that return documents while preserving their shape. Operations that reshape documents, return aggregate counts, or write data are **rejected** when masking is configured on the target collection or index. ### MongoDB | Operation | Supported | | :---------------------------------------------------------------------- | :-------------------------------------------- | | `find()`, `findOne()` | Yes | | `aggregate()` | Only with shape-preserving stages (see below) | | `countDocuments()`, `estimatedDocumentCount()`, `count()`, `distinct()` | No — return aggregate values, not documents | For `aggregate()`, **shape-preserving** stages such as `$match`, `$sort`, `$limit`, `$skip`, `$unwind`, `$addFields` / `$set`, `$unset`, and the basic form of `$lookup` / `$graphLookup` (with `localField` / `foreignField`) are supported. Stages that reshape documents — `$group`, `$project`, `$replaceRoot`, `$count`, `$facet`, `$bucket` — the pipeline form of `$lookup`, and stages that write to another collection (`$out`, `$merge`) are rejected: ``` MongoDB aggregate() with stage "$group" on collection "users" is not supported for dynamic masking. Supported operations are find(), findOne(), and aggregate() with shape-preserving stages only ``` ### Elasticsearch The `_search` / `_msearch` APIs are supported, with masking applied to `_source`, `fields`, `highlight`, `sort`, and `inner_hits`. Single- and multi-document retrieval (`GET //_doc/`, `GET //_source/`, `_mget`) and `_explain` are also supported. The following are **rejected** when masking is configured, because their results can bypass field-level masking: * **APIs** — `_async_search`, `_search/scroll`, `_search_template`, `_msearch/template`, `_sql`, `_eql/search`, `_esql/query`, `_terms_enum`, `_termvectors`. For OpenSearch: `_plugins/_asynchronous_search`, `_plugins/_sql`, `_plugins/_ppl`. * **`_search` request-body features** — `aggs` / `aggregations`, `suggest`, `script_fields`, `runtime_mappings`, `stored_fields`, `docvalue_fields`. ``` this Elasticsearch API is not supported when data masking is configured on the target index ``` ## Troubleshooting If masking is not applied to your results, check that: * The collection or index has an `objectSchema` configured in its **Catalog** (using `objectSchema`, not `columns`), with the correct field types and `semanticType` values. * At least one field has a `semanticType` assigned. * The instance has an **Enterprise** license assigned. Verify on **Settings** > **Subscription** and ensure the license toggle is enabled for the instance under **Instances**. # Global Masking Rule Source: https://docs.bytebase.com/security/data-masking/global-masking-rule Masking precedence: [Masking Exemption](/security/data-masking/access-unmasked-data) > [Global Masking Rule](/security/data-masking/global-masking-rule) > [Column Masking](/security/data-masking/column-masking). Admins may want to batch apply masking settings globally. e.g. * Mask all data in production * Mask all data for database under a specific project * Mask data conditionally according to [Data Classification](/security/data-masking/data-classification) Global masking rule along with [Semantic Types](/security/data-masking/semantic-types) allows you to do this. It's similar to the iptables where you configure an ordered rule list. The first matching rule will be applied. If no rule matches, no `Semantic Type` will be applied. Global masking rules are not supported for document databases (**MongoDB** and **Elasticsearch**). Configure masking per-collection/per-index instead — see [Document Database Masking](/security/data-masking/document-database-masking). Certain roles can set global masking rules to mask the data: * Built-in roles: `Workspace Admin`, `DBA`. * [Custom roles](/administration/roles): `bb.policies.createMaskingRulePolicy`, `bb.policies.updateMaskingRulePolicy`, `bb.policies.deleteMaskingRulePolicy`. 1. Go to **Data Access**>**Global Masking**. 2. Click **Add**. Click **+Add condition** or **+Add condition group**, set **Semantic Type** and then click **Confirm**. bb-global-masking 3. Repeat to add more rules. bb-global-masking-2 Combined with [Semantic Types](/security/data-masking/semantic-types), here is the result you will get in **SQL Editor**. bb-sql-editor-full-masking # Masking Algorithm Source: https://docs.bytebase.com/security/data-masking/masking-algorithm You may customize your data masking algorithm with the help of a predefined masking type while [creating a semantic type](/security/data-masking/semantic-types). ## Full mask Mask the entire value and show **Substitute** value. bb-algorithm-full ## Range mask Mask several characters of a string and replace them with **Substitute** value. bb-algorithm-range ## MD5 mask Use MD5 algorithm together with **Salt** value to hash. bb-algorithm-md5 ## Inner / Outer mask * Inner mask: Mask `123456` to format like `12**56`, `1***56`. * Outer mask: Mask `123456` to format like `**34**`, `*23***`. bb-algorithm-inner # Overview Source: https://docs.bytebase.com/security/data-masking/overview masking-overview Dynamic Data Masking (DDM) can mask sensitive data in the [SQL Editor](/sql-editor/overview) query result based on the context. It helps organizations to protect sensitive data from being exposed to unauthorized users. You can configure the masking policies from UI or via API. Check out [this GitOps example](https://github.com/bytebase/example-database-security) to see how to codify the masking policies. ## Configure Dynamic Data Masking * Workspace-level admins configure the [Global Masking Rule](/security/data-masking/global-masking-rule), [Semantic Types](/security/data-masking/semantic-types), and [Masking Algorithm](/security/data-masking/masking-algorithm). * Project-level owners configure the [Column Masking](/security/data-masking/column-masking) on the table column. This is only needed when the global masking rule is not applicable to a particular project. * Workspace-level admins or project-level owners grant [Masking Exemption](/security/data-masking/access-unmasked-data) to the users to access the unmasked data. * For document databases (**MongoDB** and **Elasticsearch**), masking is configured per-collection through the Catalog — see [Document Database Masking](/security/data-masking/document-database-masking). Global masking rules and masking exemption are not supported for document databases. ## Determine whether to mask data masking-detail ### Masking precedence 1. [Masking Exemption](/security/data-masking/access-unmasked-data). If user has been granted exemption, the data will not be masked. 2. [Global Masking Rule](/security/data-masking/global-masking-rule). If no exemption is granted, the global masking rule will be applied. 3. [Column Masking](/security/data-masking/column-masking). If no global masking rule is configured, the column masking will be applied. ### Masking algorithm The global masking rule and column masking are both mapped to the [Semantic Types](/security/data-masking/semantic-types). The semantic type determines the masking algorithm. ### Masking propagation When a column in a database table is masked, the masking effect is **infectious** in the sense that it propagates to any views or derived structures that depend on that column. This ensures that the protection applied to the underlying data is consistently enforced, even when accessed through alternative pathways like views. # Semantic Types Source: https://docs.bytebase.com/security/data-masking/semantic-types overview You may define semantic types and apply them to columns of different tables. Columns with the same semantic type will be masked with the same masking algorithm. This allows you to manage masking algorithms for many columns by controlling only a small number of semantic types. 1. Go to **Data Access** > **Semantic Types**. You'll see the default semantic type `Default` and `Default Partial`. 2. Click **Add**, and fill in the name and description. bb-add-semantic-type 3. Click the pencil under **Masking Algorithm** column to attach a masking algorithm. Click the checkmark to save. bb-add-algorithm *Check [Masking Algorithm](/security/data-masking/masking-algorithm) for more details.* 4. You may add other semantic types as needed. bb-semantic-type-list There're two ways to apply this semantic type to take effect: * [Global Masking](/security/data-masking/global-masking-rule) * [Column Masking](/security/data-masking/column-masking) # Export Data Source: https://docs.bytebase.com/security/database-permission/export Data can only be exported from **SQL Editor**. Which export path is available depends on two settings: | Workspace **Enable data export** | Project **Just-In-Time access** | How members export | | -------------------------------- | ------------------------------- | ------------------------------------------------------------------------------------------- | | On | — | [Directly](#direct-export), after running a query | | Off | On | [Just-in-time data export](#just-in-time-data-export) — via an access grant, after approval | | Off | Off | No export | ## Direct export With **Enable data export** on (**Settings > General**, the workspace default), anyone who can run a query can click **Export** on the result to download it — exporting is not a separate permission. See [Export data](/sql-editor/run-queries#export-data) for how Export behaves. ## Just-in-time data export To put every export behind an approval, a **Workspace Admin** turns off **Enable data export**; members can then no longer export directly. As long as **Just-In-Time access** is enabled on the project, they can still export through the [access grant](/security/database-permission/grant/) mechanism: 1. Check **Export** in the access grant request. request-data-export 2. The approver sees on the request issue that the grant allows exporting the result. export-request-issue 3. Once approved, run the approved statement and export its result before the grant expires. Every data-export request — pending, active, or expired — stays visible in the **Just-In-Time Access** list on the SQL Editor left sidebar (the shield icon). Add the **export** filter (`export: Yes`) to narrow the list to export grants; an expired grant can be re-requested from there. export-grant-list # Access Grants Source: https://docs.bytebase.com/security/database-permission/grant An **access grant** is a temporary, elevated privilege bound to specific resources: it authorizes exactly **one read-only statement** on selected databases, optionally with permission to see the result unmasked or export it, and it expires on its own. Where a [role](/security/database-permission/request/) gives standing capabilities, an access grant exists only through approval and only for a limited time — the mechanism behind [just-in-time data access](/security/database-permission/just-in-time/). ## Enable access grants In the project, click **Settings** on the left sidebar, find **Security & Policy**, and turn on **Just-In-Time access** — this lets project members request access grants. The indicator next to the toggle shows which approval flow requests will follow. enable-jit-setting ### Configure the approval flow Access grant requests are reviewed with [Custom Approval](/change-database/approval/) (**Workspace > CI/CD > Custom Approval**): add a rule under the **Request Just-In-Time Access** source to route them to the right approver. If no rule matches, the workspace **Fallback Rules** apply. custom-approval-jit Turning on the project setting requires the `bb.projects.update` permission — for example a **Project Owner** or **Workspace Admin**. Editing approval flows requires `bb.settings.set` — for example a **Workspace Admin**. ## Request an access grant When you don't have permission to query a database in **SQL Editor**, you can request an access grant for a single statement. In SQL Editor, choose the project with **Just-In-Time access** enabled, select the database, and run your query. Without permission, the result panel returns `permission_denied` along with a **Request just-in-time access** button. permission-denied-request Click it to open the **Request Data Access** drawer, pre-filled with the database, statement, and a default expiration. Review and complete: * **Databases** — the targets you need to access. * **Statement** — the SQL to run. Only read-only statements are allowed. * **Unmask** — see unmasked sensitive data in the result. * **Export** — also export the query result (see [just-in-time data export](/security/database-permission/export/#just-in-time-data-export)). * **Expiration** — how long the grant stays valid once approved: **1 hour**, **4 hours**, **1 day**, **7 days**, or a custom date. * **Reason** — the justification reviewers see; required. request-data-access-drawer Submit, and Bytebase opens the request issue in a new tab, routed through the **Request Just-In-Time Access** approval flow. You can also start a request from the masking reason popover (pre-selecting **Unmask**), or from the export flow when direct export is disabled. ## Approval The approver sees the requested databases, the exact statement, the granted permissions (including any **Unmask** or **Export**), the expiration, and the requester's reason. jit-access-request-issue Approval rules under the **Request Just-In-Time Access** source can match the request, for example: * `request.data_export == true` — the request includes export. * `request.unmask == true` — the request includes unmasking. Conditions can also match the target with `resource.database_name`, `resource.table_name`, and similar attributes. ## Use the grant Open the **Just-In-Time Access** tab — the shield icon on the **SQL Editor** left sidebar — to track your requests. Each shows its status (**Pending** while awaiting approval, **Active** once approved and usable), the databases, any **Export** or **Unmask** badge, the time left before expiration, and a link to the approval issue. jit-access-list Once a grant is **Active**, click **Run** to execute its approved statement; if the grant includes **Export**, you can export the result. The grant stays usable until it expires. jit-access-run-results Only a statement that **exactly** matches the one in the grant is allowed to run — leading and trailing whitespace is ignored, but any other edit to the SQL, even reformatting, is denied. Use the **Run** button on the Just-In-Time Access list to re-run the approved statement reliably. ## Review and revoke To review every access grant in a project, go to **Data Access > Access Grants** in the project sidebar. Revoking an active grant requires the `bb.accessGrants.revoke` permission — **Project Owner**, **Workspace Admin**, or **Workspace DBA**. ## Expiration An access grant expires automatically. For preset durations, the countdown starts when the request is **approved**, not when it is submitted. The expiration a member can request is bounded by the workspace **Maximum request expiration** setting (**Settings > General**, **Security** section) — "the maximum period a data access request can remain valid". This is a separate setting from the [maximum role expiration](/security/database-permission/request/#maximum-role-expiration) that caps role bindings. The [audit log](/security/audit-log/) records each query and export, including which access grant authorized it. # Just-in-Time Access Source: https://docs.bytebase.com/security/database-permission/just-in-time Just-in-Time (JIT) access grants database access only when it's needed and only for a limited time, instead of leaving standing permissions in place. A member requests access, an approver reviews it, and the grant expires automatically — so unused privileges never accumulate, every grant is tied to a stated reason, and each use is recorded for audit. Bytebase offers two ways to request access just-in-time, one per [permission model](/security/database-permission/overview/): * **Request a role** — a time-boxed [role](/security/database-permission/request/) (e.g. `SQL Editor Read User`) for a broad range of permissions on the selected databases. * **Request an access grant** — a limited, elevated privilege: run one approved read-only statement, optionally unmasked or exported. See [Access Grants](/security/database-permission/grant/). The two just-in-time approaches converge on expiring, audited access Either way, the request goes through an [approval flow](/change-database/approval/) and is recorded in the [audit log](/security/audit-log/). ## Choose an approach | | Time-boxed role | Access grant | | -------------------- | -------------------------------------------------------------------------------------------------------------------- | -------------------------------------------------------------------------------------------------------- | | **You get** | a project role on the selected databases | permission to run one approved read-only statement | | **Granularity** | the role's permissions, scoped to databases, schemas, and tables | a single statement, plus optional **Unmask** and **Export** | | **Duration presets** | 1 week – 1 year, capped by [maximum role expiration](/security/database-permission/request/#maximum-role-expiration) | 1 hour – 7 days, capped by [maximum request expiration](/security/database-permission/grant/#expiration) | | **Project setting** | **Allow request role** | **Just-In-Time access** | | **Approval source** | **Request Role** | **Request Just-In-Time Access** | | **Best for** | ongoing work that needs repeated access | one-off production reads, incident lookups, ad-hoc exports | Each approach has its own project setting and approval source — the setting turns on role requests or access grants, not the workflow itself. Set them up where each mechanism is documented: [Enable role requests](/security/database-permission/request/#enable-role-requests) and [Enable access grants](/security/database-permission/grant/#enable-access-grants). ## Expiration and audit Both approaches end the same way: the access expires on its own — a role binding at its expiration date, an access grant when its approved window (counted from approval) runs out — and the [audit log](/security/audit-log/) records every query and export along with the role or grant that authorized it. For the full request flows, see [Roles](/security/database-permission/request/) and [Access Grants](/security/database-permission/grant/). # Overview Source: https://docs.bytebase.com/security/database-permission/overview Bytebase sits between users and databases and checks every statement against fine-grained database permissions — directly in [SQL Editor](/sql-editor/overview/), and through the [change workflow](/change-database/change-workflow/) for statements the user cannot run directly. There are two permission models, and one workflow built on top of them: 1. **Roles** — standing access. A role bundles permissions (`bb.sql.select`, `bb.sql.ddl`, …) and is bound to a user or group, scoped to databases and environments, optionally expiring. Two ways to get one: [granted or requested](/security/database-permission/request/). 2. **Access grants** — temporary elevated access. An [access grant](/security/database-permission/grant/) authorizes a single read-only statement on specific databases — optionally unmasked or exported — for a limited time, after approval. 3. **Just-in-Time access** — the workflow that uses either model to keep standing privileges at zero: request access when needed, have it approved, let it expire. See [Just-in-Time Access](/security/database-permission/just-in-time/). The Bytebase access model: two permission models — roles and access grants — enforced per statement, with Just-in-Time access as the workflow on top ## Permissions SQL Editor classifies each statement you run and checks the matching permission: | Access Level | Operation | Permission | | -------------------------------- | ----------------------------- | ------------------ | | [Read](/sql-editor/run-queries/) | EXPLAIN | `bb.sql.explain` | | | Query | `bb.sql.select` | | | Metadata (information schema) | `bb.sql.info` | | Write | Mutation DML | `bb.sql.dml` | | | DDL | `bb.sql.ddl` | | [Admin](/sql-editor/admin-mode/) | Admin | `bb.sql.admin` | | Request change for review | Create Issue | `bb.issues.create` | For DDL and DML, the environment scope is part of the permission itself: when granting a role that carries `bb.sql.ddl` / `bb.sql.dml`, the granter selects the environments where those statements may run directly. On databases outside those environments, SQL Editor directs the user to create a database change plan and follow the change workflow instead. ## Roles The built-in roles carry these database permissions: | Role | Read | DML | DDL | Admin | | -------------------- | ---- | ----- | ----- | ----- | | Workspace Admin | ✅ | ✅ | ✅ | ✅ | | Workspace DBA | ✅ | ✅ | ✅ | ✅ | | Project Owner | ✅ | ✅ | ✅ | | | SQL Editor User | ✅ | ✅ \*1 | ✅ \*1 | | | SQL Editor Read User | ✅ | | | | | Project Developer | | | | | | Project Releaser | | | | | | Project Viewer | | | | | **Read** covers `bb.sql.select`, `bb.sql.explain`, and `bb.sql.info`. \*1 *In the environments selected when granting the role. It's still recommended to make schema and data changes through the change workflow for better traceability.* **Project Developer** and **Project Viewer** carry no SQL Editor permissions — they hold `bb.issues.create` and make changes through the [change workflow](/change-database/change-workflow/). A role reaches a member as a project-level binding that carries a **scope** — all databases in the project, or specific databases, schemas, and tables, plus the environments where DDL/DML run directly — and an optional **expiration**. A member gets a role either [granted by an admin or by requesting it](/security/database-permission/request/); role definitions and workspace IAM are covered in [Roles and Permissions](/administration/roles), where you can also build custom roles from individual permissions — e.g. a role that grants only `bb.sql.explain`. ## Access grants An [access grant](/security/database-permission/grant/) is the second model: a temporary, elevated privilege bound to specific resources rather than a standing capability. It authorizes exactly **one read-only statement** on selected databases, optionally with **Unmask** (see masked data in clear) or **Export** (download the result), and expires on its own — the clock starts at approval. Access grants exist only through approval: a member requests one from SQL Editor, an approver reviews, and every use is audited. ## Just-in-Time access To avoid standing privileges altogether, run access as a just-in-time workflow: members hold no default access and request it when needed — a time-boxed **role** for a broad range of permissions, or an **access grant** for a limited, elevated privilege. Both expire on their own and leave an audit trail. See [Just-in-Time Access](/security/database-permission/just-in-time/). # Roles Source: https://docs.bytebase.com/security/database-permission/request A [role](/security/database-permission/overview/#roles) is the standing permission model: it reaches a member as a project-level binding scoped to databases (and, for DDL/DML-capable roles, environments), with an optional expiration. There are **two ways to get a role**: * **Granted** — an admin assigns it from the Members page. See [Assign Roles](/administration/roles#assign-roles) for the mechanics at workspace and project level. * **Requested** — the member requests it, an approver reviews, and approval creates the same binding an admin would have made. Described below. Requesting a role is not limited to data access: any project role can be requested, for any purpose — **SQL Editor Read User** for query access, **Project Developer** to work on changes, or a [custom role](/administration/roles). It works independently of the project's [access grant](/security/database-permission/grant/) setting. ## Enable role requests In the project, click **Settings** on the left sidebar, find **Security & Policy**, and turn on **Allow request role**. The indicator next to the toggle shows which approval flow requests will follow. enable-request-role-setting Requesting requires the `bb.issues.create` permission in the project — the built-in **Project Viewer** and **Project Developer** roles carry it. ### Configure the approval flow Role requests are reviewed with [Custom Approval](/change-database/approval/) (**Workspace > CI/CD > Custom Approval**): add a rule under the **Request Role** source to route them to the right approver. If no rule matches, the workspace **Fallback Rules** apply. custom-approval-request-role Turning on the project setting requires the `bb.projects.update` permission — for example a **Project Owner** or **Workspace Admin**. Editing approval flows requires `bb.settings.set` — for example a **Workspace Admin**. ## Start a request * **Members** — in the project, go to **Manage > Members** and click **Request role**. (The button is hidden for members who already hold full project access.) * **SQL Editor** — on a permission-denied result, click **Request query**. The request opens pre-filled with the databases you tried to reach and the smallest built-in role covering the missing permissions. (If the project has [access grants](/security/database-permission/grant/) enabled and you only lack read permission, SQL Editor offers an access grant request instead.) * **Anywhere else** — a permission-denied page in Bytebase offers **Request role** directly. ## The request Pick the role, optionally narrow the scope to specific databases, schemas, and tables (for DDL/DML-capable roles, also the environments — the same scope as an [admin grant](/administration/roles#assign-roles)), set an [expiration](#expiration), and give a reason. After you submit, an approver reviews the request issue. Approval rules under the **Request Role** source can match the requested role (`request.role`) and duration (`request.expiration_days`). Once approved, the role is granted and shows up on the **Members** page like any other binding, until it expires. ## Expiration A role binding can carry an expiration; once it passes, Bytebase revokes the role automatically. When granting, pick a preset (**1 week**, **1 month**, **3 months**, **1 year**), a custom date, or **Never** if the workspace allows unlimited grants. ### Maximum role expiration A **Workspace Admin** can cap how long any role stays granted, so members can't hold long-lived access. Go to **Settings > General**, find **Maximum role expiration** in the **Security** section, and set the number of days. Select **Never expires** to remove the cap. The cap applies to project-level role bindings — both granted from the Members page and requested — with one exception: the **Project Owner** role. Once a cap is set, every new binding must carry an expiration within it; presets beyond the cap and the **Never** option disappear. Changing this setting requires the `bb.settings.setWorkspaceProfile` permission — for example a **Workspace Admin**. [Access grants](/security/database-permission/grant/) are capped by a separate setting, **Maximum request expiration**, in the same section. ### Expiration reminder Users holding a project role that expires within the next two days see a reminder dialog, which they can silence until the binding changes. # Watermark Source: https://docs.bytebase.com/security/watermark Workspace Admins can turn on / off watermark setting in **Settings/Workspace/General** watermark-setting.webp Then, watermark including current user's username, ID and email will be displayed on Bytebase console. watermark-displaying.webp # Admin Mode Source: https://docs.bytebase.com/sql-editor/admin-mode Be really careful when using Admin Mode, since you are taking full control of your database. If you are a `Workspace Admin` or `Workspace DBA` or granted a [custom role](/administration/roles) having the `bb.admin.sql` permission, you can break the glass and execute arbitrary statements. ## Run query in Admin Mode Click **Admin Mode** button to open a separate Admin Mode tab. Note that this is only available for DBAs and Workspace Admins. Admin Mode Button In Admin Mode, you are not limited to the SELECT, DDL, and DML statements your [database permissions](/security/database-permission/overview/) allow, but also allowed to run database administration SQL statements such as SHOW, SET, GRANT, etc. Admin Mode In Admin Mode, click the **Clear screen** button or use its keyboard shortcut ⇧ + ⌥ + C to clear the screen. You can also press ⌘ + Up / Down or Ctrl + Up / Down to explore history statements backward or forward. # AI Assistant Source: https://docs.bytebase.com/sql-editor/ai-assistant Bytebase sends the following info to the configured AI provider: * Tables and column names of the current database schema. Here is a video demoing this feature: