This is part of our database GitOps series with Bytebase:
This tutorial shows you how to build an database GitOps workflow using GitLab CI and Bytebase API. You’ll learn to create a streamlined database release workflow where you can:
main
While we use GitLab CI in this guide, you can apply these concepts to other CI platforms like GitHub Actions, Bitbucket Pipelines, or Azure DevOps using the Bytebase API.
While we use PostgreSQL with GitLab CI in this guide, you can apply these concepts to other SQL or NoSQL databases with any CI platforms like GitHub Actions, Bitbucket Pipelines, or Azure DevOps using the Bytebase API.
https://gitlab.com/bytebase-sample/gitops-example
ngrok is a reverse proxy tunnel, and in our case, we need it for a public network address to allow VCS to call the Bytebase API. ngrok we used here is for demonstration purposes.
Run Bytebase in Docker with the following command:
Bytebase is running successfully in Docker, and you can visit it via localhost:8080
. Register an admin account and it will be granted the workspace admin
role automatically.
Login to ngrok Dashboard and complete the Getting Started steps to install and configure. If you want to use the same domain each time you launch ngrok, go to Cloud Edge > Domains, where you’ll find the domain <<YOURS>>.ngrok-free.app
linked to your account.
Run the ngrok command ngrok http --domain=<<YOURS>>.ngrok-free.app 8080
to start ngrok with your specific domain, and you will see the output displayed below:
Log in Bytebase and click the gear icon (Settings) on the top right. Click General under Workspace. Paste <<YOURS>>.ngrok-free.app
as External URL under Network section and click Update.
Now you can access Bytebase via <<YOURS>>.ngrok-free.app
.
Log in as Workspace Admin
, and go to IAM & Admin > Users & Groups. Click + Add User, fill in with api-sample
, choose the Workspace DBA
role sufficient for this tutorial and click Confirm.
Find the newly created service account and Copy Service Key. We will use this token to authenticate the API calls.
If you have Enterprise Plan, you can create a Custom Role for the service account which require fewer permissions, and assign this role instead of DBA:
Since you will need to run SQL review on your PRs, you need to configure the SQL review in Bytebase.
Go to CI/CD > SQL Review, click Create SQL Review.
Select the Sample Template
and click Next.
Select Prod
environment as the attached resources and click Confirm. Now the SQL review is enabled for the Prod
environment.
Note: Usually we enable SQL review for Prod
environment as above. In this demo, we would switch to enable it for Test
to fit the following GitLab CI workflow.
Create a new repository and copy the configuration files from https://gitlab.com/bytebase-sample/gitops-example. There are two ymls in this repository:
.gitlab-ci.yml
: The CI pipeline for the repository which includes the SQL review and release creation.bytebase-review.yml
: Lint the SQL migration files after the MR is created.bytebase-rollout.yml
: Create a release in Bytebase after the MR is merged to the main
branch.Go into bytebase-review.yml
and bytebase-rollout.yml
. In the env
section, replace the variable values with your own and commit the changes.
api-example@service.bytebase.com
(the service account you created in the previous step)In bytebase-rollout.yml
, pay attention to BYTEBASE_TARGETS
in deploy-to-test
stage. You should put all the databases including both Test
and Prod
environments. NOT ONLY the Test
database.
To create migration files to trigger release creation, the files have to match the following pattern:
202505121650_create_table_t1.sql
.ddl
or dml
to indicate its change type. If it doesn’t end with any of the two, its change type is DDL by default.Within your forked repository, create the following migration files under migration
directory:
Commit to a new branch and create a merge request, the sql-review
pipeline will be triggered. There will be a warning in the SQL review result.
According to the SQL review result, you can do some changes to the SQL files and push to the branch. Then you should see the SQL review has passed. There are no warnings in the SQL review result.
When the SQL review is passed, you can merge the merge request. The release
pipeline will be triggered to create a release in Bytebase and then roll out automatically.
Click into the pipelines, you can see the release pipeline is triggered and passed. Click the number of the pipeline, you can see the stages.
If you click the deploy-to-test
and expand the logs, you can follow the links to Bytebase.
bytebase-action
in an Offline GitLab RunnerIf you are self-hosting GitLab in an internal network that has no access to the public internet, your CI/CD jobs may fail with the following error: Job failed: failed to pull image "bytebase/bytebase-action:latest"
. Since the image cannot be pulled directly from Docker Hub, you’ll need to download it from an external machine, then transfer and load it into your internal environment manually.
On an external (internet-accessible) machine:
Transfer the bytebase-action.tar
file to your internal server.
Use scp
, USB drive, or any method suitable for your setup.
On your internal (offline) machine:
Note: docker tag step is only needed if the loaded image doesn’t already have the correct tag.
If your GitLab instance uses an external URL but is hosted in an internal network, bytebase-action
may fail with:
This happens because GitLab redirects to the external URL, which isn’t accessible internally.
To resolve this, set the clone_url
in your GitLab Runner configuration to point to the internal GitLab address:
This forces the runner to clone from the internal URL and avoids redirection errors.
Now you have learned how to database GitOps with GitLab CI. If you want to trigger a release creation with other git providers (e.g. GitHub, Bitbucket, Azure DevOps), you may customize the workflow file.
This is part of our database GitOps series with Bytebase:
This tutorial shows you how to build an database GitOps workflow using GitLab CI and Bytebase API. You’ll learn to create a streamlined database release workflow where you can:
main
While we use GitLab CI in this guide, you can apply these concepts to other CI platforms like GitHub Actions, Bitbucket Pipelines, or Azure DevOps using the Bytebase API.
While we use PostgreSQL with GitLab CI in this guide, you can apply these concepts to other SQL or NoSQL databases with any CI platforms like GitHub Actions, Bitbucket Pipelines, or Azure DevOps using the Bytebase API.
https://gitlab.com/bytebase-sample/gitops-example
ngrok is a reverse proxy tunnel, and in our case, we need it for a public network address to allow VCS to call the Bytebase API. ngrok we used here is for demonstration purposes.
Run Bytebase in Docker with the following command:
Bytebase is running successfully in Docker, and you can visit it via localhost:8080
. Register an admin account and it will be granted the workspace admin
role automatically.
Login to ngrok Dashboard and complete the Getting Started steps to install and configure. If you want to use the same domain each time you launch ngrok, go to Cloud Edge > Domains, where you’ll find the domain <<YOURS>>.ngrok-free.app
linked to your account.
Run the ngrok command ngrok http --domain=<<YOURS>>.ngrok-free.app 8080
to start ngrok with your specific domain, and you will see the output displayed below:
Log in Bytebase and click the gear icon (Settings) on the top right. Click General under Workspace. Paste <<YOURS>>.ngrok-free.app
as External URL under Network section and click Update.
Now you can access Bytebase via <<YOURS>>.ngrok-free.app
.
Log in as Workspace Admin
, and go to IAM & Admin > Users & Groups. Click + Add User, fill in with api-sample
, choose the Workspace DBA
role sufficient for this tutorial and click Confirm.
Find the newly created service account and Copy Service Key. We will use this token to authenticate the API calls.
If you have Enterprise Plan, you can create a Custom Role for the service account which require fewer permissions, and assign this role instead of DBA:
Since you will need to run SQL review on your PRs, you need to configure the SQL review in Bytebase.
Go to CI/CD > SQL Review, click Create SQL Review.
Select the Sample Template
and click Next.
Select Prod
environment as the attached resources and click Confirm. Now the SQL review is enabled for the Prod
environment.
Note: Usually we enable SQL review for Prod
environment as above. In this demo, we would switch to enable it for Test
to fit the following GitLab CI workflow.
Create a new repository and copy the configuration files from https://gitlab.com/bytebase-sample/gitops-example. There are two ymls in this repository:
.gitlab-ci.yml
: The CI pipeline for the repository which includes the SQL review and release creation.bytebase-review.yml
: Lint the SQL migration files after the MR is created.bytebase-rollout.yml
: Create a release in Bytebase after the MR is merged to the main
branch.Go into bytebase-review.yml
and bytebase-rollout.yml
. In the env
section, replace the variable values with your own and commit the changes.
api-example@service.bytebase.com
(the service account you created in the previous step)In bytebase-rollout.yml
, pay attention to BYTEBASE_TARGETS
in deploy-to-test
stage. You should put all the databases including both Test
and Prod
environments. NOT ONLY the Test
database.
To create migration files to trigger release creation, the files have to match the following pattern:
202505121650_create_table_t1.sql
.ddl
or dml
to indicate its change type. If it doesn’t end with any of the two, its change type is DDL by default.Within your forked repository, create the following migration files under migration
directory:
Commit to a new branch and create a merge request, the sql-review
pipeline will be triggered. There will be a warning in the SQL review result.
According to the SQL review result, you can do some changes to the SQL files and push to the branch. Then you should see the SQL review has passed. There are no warnings in the SQL review result.
When the SQL review is passed, you can merge the merge request. The release
pipeline will be triggered to create a release in Bytebase and then roll out automatically.
Click into the pipelines, you can see the release pipeline is triggered and passed. Click the number of the pipeline, you can see the stages.
If you click the deploy-to-test
and expand the logs, you can follow the links to Bytebase.
bytebase-action
in an Offline GitLab RunnerIf you are self-hosting GitLab in an internal network that has no access to the public internet, your CI/CD jobs may fail with the following error: Job failed: failed to pull image "bytebase/bytebase-action:latest"
. Since the image cannot be pulled directly from Docker Hub, you’ll need to download it from an external machine, then transfer and load it into your internal environment manually.
On an external (internet-accessible) machine:
Transfer the bytebase-action.tar
file to your internal server.
Use scp
, USB drive, or any method suitable for your setup.
On your internal (offline) machine:
Note: docker tag step is only needed if the loaded image doesn’t already have the correct tag.
If your GitLab instance uses an external URL but is hosted in an internal network, bytebase-action
may fail with:
This happens because GitLab redirects to the external URL, which isn’t accessible internally.
To resolve this, set the clone_url
in your GitLab Runner configuration to point to the internal GitLab address:
This forces the runner to clone from the internal URL and avoids redirection errors.
Now you have learned how to database GitOps with GitLab CI. If you want to trigger a release creation with other git providers (e.g. GitHub, Bitbucket, Azure DevOps), you may customize the workflow file.