Bytebase is a database DevSecOps platform designed for developers, security, DBA, and platform engineering teams. While it offers an intuitive GUI for managing database schema changes and access control, some teams may want to integrate Bytebase into their existing DevOps platforms using the Bytebase API. Bytebase provides database dynamic data masking in the Enterprise Plan, which can mask sensitive data in the SQL Editor query result based on the context on the fly. It helps organizations to protect sensitive data from being exposed to unauthorized users. By using GitHub Actions with Bytebase API, you can implement policy-as-code to apply database masking policies via the GitOps workflow. This tutorial will guide you through the process.
This is Part 1 of our tutorial series on implementing automated database masking using GitHub Actions:

​
Overview

In this tutorial, you’ll learn how to automate database masking policies using GitHub Actions and the Bytebase API. This integration allows you to:
  • Manage data masking rules as code
  • Automatically apply masking policies when PRs are merged
Here is a merged pull request as an example, for this tutorial, only “Semantic Type and Global Masking Rule” is covered. The complete code for this tutorial is available at: example-database-security

​
Prerequisites

Before you begin, make sure you have:
  • Docker installed
  • A GitHub account
  • An ngrok account
  • Bytebase Enterprise Plan subscription (you can request a free trial)

​
Setup Instructions

​
Step 1 - Start Bytebase in Docker and set the External URL generated by ngrok

ngrok is a reverse proxy tunnel that provides a public network address to access Bytebase. We use ngrok here for demonstration purposes. ngrok-reverse-proxy
  1. Run Bytebase in Docker with the following command: 
    docker run --rm --init \
  --name bytebase \
  --publish 8080:8080 --pull always \
  --volume ~/.bytebase/data:/var/opt/bytebase \
  bytebase/bytebase:3.8.1
  2. Once Bytebase is running in Docker, you can access it at localhost:8080.
  3. Log in to the ngrok Dashboard and complete the Getting Started steps to install and configure ngrok. To use a consistent domain, navigate to Universal Gateway > Endpoints to find your assigned domain: <<YOURS>>.ngrok-free.app.
  4. Start ngrok with your domain by running: 
    ngrok http --url=<<YOURS>>.ngrok-free.app 8080
    You should see output similar to this: terminal-ngrok
  5. You can now access Bytebase at https://<<YOURS>>.ngrok-free.app.
  6. (Optional) To configure SSO (Entra/SCIM), log in to Bytebase, click Settings > General in the left sidebar. Scroll to the Network section, set https://<<YOURS>>.ngrok-free.app as the External URL and click Confirm and update. external-url

​
Step 2 - Create Service Account

  1. Log in as Workspace Admin, and go to IAM & Admin > Users & Groups. Click + Add User, fill in with api-sample, choose the Workspace DBA role sufficient for this tutorial and click Confirm. service-account-create
  2. Find the newly created service account and Copy Service Key. We will use this token to authenticate the API calls. service-account-key

​
Step 3 - Prepare Test Data

  1. Bytebase by default provides a project Sample Project with two database hr_test and hr_prod.
  2. Click IAM & Admin > Users & Groups on the left sidebar. Add users: dev@example.com, dev2@example.com and dev3@example.com with no roles.
  3. Add a group contractor@example.com with dev3@example.com as a member.
  4. Go to project Sample Project, click Manage > Members on the left sidebar.
  5. Click Grant Access and select users dev@example.com and dev2@example.com with Developer role and group contractor@example.com with Querier role.

​
Step 4 - Configure GitHub Actions

  1. Fork Database Security GitHub Actions Example.
  2. Click Settings and then click Secrets and variables > Actions. Add the following secrets:
    • BYTEBASE_URL: ngrok external URL
    • BYTEBASE_SERVICE_KEY: api-example@service.bytebase.com
    • BYTEBASE_SERVICE_SECRET: service key copied in the previous step

​
Step 5 - Understanding the GitHub Workflow

Let’s dig into the GitHub Actions workflow code:
  1. Trigger: Workflow runs when PRs are merged to main.
  2. Authentication: The step Login Bytebase will log in Bytebase using the official bytebase-login action. The variables you configured in the GitHub Secrets and variables are mapped to the variables in the action.
  3. File Detection: The step Get changed files will monitor the changed files in the pull request. For this workflow, we only care about semantic type and global masking rule. So masking/semantic-type.json and masking/global-masking-rule.json are filtered out.
  4. PR Feedback: The step Comment on PR will comment on the merged pull to notify the result.

​
Semantic Type

Masking algorithm is associated with Semantic Type. You define semantic types and apply them to global masking rule or table columns. For example, you may define a semantic type birth_date with a masking algorithm to mask month and day.

​
In Bytebase Console

Go to Data Access > Semantic Types and click Add. You can create a new semantic type with a name and description, and customize the masking algorithm. bb-add-semantic-type bb-add-algorithm

​
In GitHub Workflow

Find the step Apply semantic type, which will apply the semantic type to the database via API. All the masking algorithms should be defined in one file in the root directory as masking/semantic-type.json. 
response=$(curl -s -w "\n%{http_code}" --request PATCH "${BYTEBASE_API_URL}/settings/bb.workspace.semantic-types?allow_missing=true" \
   --header "Authorization: Bearer ${BYTEBASE_TOKEN}" \
   --header "Content-Type: application/json" \
   --data @"$CHANGED_FILE")
By changing file masking/semantic-type.json, creating a PR and merging, the semantic types will be applied. Go to Bytebase console, click Data Access > Semantic Types, you can see the applied semantic types.

​
Global Masking Rule

Global Masking Rule is configured by the admin.

​
In Bytebase Console

Go to Data Access > Global Masking and click Add. You can create a new global masking rule mapping condition to a semantic type. bb-global-masking

​
In GitHub Workflow

Find the step Apply global masking rule, which will apply the global masking rule to the database via API. All the global masking rules should be defined in one file in the root directory as masking/global-masking-rule.json. The code it calls Bytebase API is as follows: 
response=$(curl -s -w "\n%{http_code}" --request PATCH "${BYTEBASE_API_URL}/policies/masking_rule?allow_missing=true&update_mask=payload" \
   --header "Authorization: Bearer ${BYTEBASE_TOKEN}" \
   --header "Content-Type: application/json" \
   --data @"$CHANGED_FILE")
By changing file masking/global-masking-rule.json, creating a PR and merge, you can apply the global masking rules.

Next Step: Column Masking and Masking Exemption