Learn how to automate semantic type and global masking rule using GitHub Actions and Bytebase API
Bytebase is a database DevSecOps platform designed for developers, security, DBA, and platform engineering teams. While it offers an intuitive GUI for managing database schema changes and access control, some teams may want to integrate Bytebase into their existing DevOps platforms using the Bytebase API.
Bytebase provides database dynamic data masking in the Enterprise Plan, which can mask sensitive data in the SQL Editor query result based on the context on the fly. It helps organizations to protect sensitive data from being exposed to unauthorized users.
By using GitHub Actions with Bytebase API, you can implement policy-as-code to apply database masking policies via the GitOps workflow. This tutorial will guide you through the process.
This is Part 1 of our tutorial series on implementing automated database masking using GitHub Actions:
In this tutorial, you’ll learn how to automate database masking policies using GitHub Actions and the Bytebase API. This integration allows you to:
Here is a merged pull request as an example, for this tutorial, only “Semantic Type and Global Masking Rule” is covered.
The complete code for this tutorial is available at: example-database-security
Before you begin, make sure you have:
ngrok is a reverse proxy tunnel, and in our case, we need it for a public network address to allow VCS to call the Bytebase API. ngrok we used here is for demonstration purposes.
Run Bytebase in Docker with the following command:
Bytebase is running successfully in Docker, and you can visit it via localhost:8080
. Register an admin account and it will be granted the workspace admin
role automatically.
Login to ngrok Dashboard and complete the Getting Started steps to install and configure. If you want to use the same domain each time you launch ngrok, go to Cloud Edge > Domains, where you’ll find the domain <<YOURS>>.ngrok-free.app
linked to your account.
Run the ngrok command ngrok http --domain=<<YOURS>>.ngrok-free.app 8080
to start ngrok with your specific domain, and you will see the output displayed below:
Log in Bytebase and click the gear icon (Settings) on the top right. Click General under Workspace. Paste <<YOURS>>.ngrok-free.app
as External URL under Network section and click Update.
Now you can access Bytebase via <<YOURS>>.ngrok-free.app
.
Log in as Workspace Admin
, and go to IAM & Admin > Users & Groups. Click + Add User, fill in with api-sample
, choose the Workspace DBA
role sufficient for this tutorial and click Confirm.
Find the newly created service account and Copy Service Key. We will use this token to authenticate the API calls.
Sample Project
with two database hr_test
and hr_prod
.dev@example.com
, dev2@example.com
and dev3@example.com
with no roles.contractor@example.com
with dev3@example.com
as a member.Sample Project
, click Manage > Members on the left sidebar.dev@example.com
and dev2@example.com
with Developer
role and group contractor@example.com
with Querier
role.Click Settings and then click Secrets and variables > Actions. Add the following secrets:
BYTEBASE_URL
: ngrok external URLBYTEBASE_SERVICE_KEY
: api-example@service.bytebase.com
BYTEBASE_SERVICE_SECRET
: service key copied in the previous stepLet’s dig into the GitHub Actions workflow code:
Trigger: Workflow runs when PRs are merged to main
.
Authentication: The step Login Bytebase
will log in Bytebase using the official bytebase-login action. The variables you configured in the GitHub Secrets and variables are mapped to the variables in the action.
File Detection: The step Get changed files
will monitor the changed files in the pull request. For this workflow, we only care about semantic type and global masking rule. So masking/semantic-type.json
and masking/global-masking-rule.json
are filtered out.
PR Feedback: The step Comment on PR
will comment on the merged pull to notify the result.
Masking algorithm is associated with Semantic Type. You define semantic types and apply them to global masking rule or table columns. For example, you may define a semantic type birth_date
with a masking algorithm to mask month and day.
Go to Data Access > Semantic Types and click Add. You can create a new semantic type with a name and description, and customize the masking algorithm.
Find the step Apply semantic type
, which will apply the semantic type to the database via API. All the masking algorithms should be defined in one file in the root directory as masking/semantic-type.json
.
By changing file masking/semantic-type.json
, creating a PR and merging, the semantic types will be applied. Go to Bytebase console, click Data Access > Semantic Types, you can see the applied semantic types.
Global Masking Rule is configured by the admin.
Go to Data Access > Global Masking and click Add. You can create a new global masking rule mapping condition to a semantic type.
Find the step Apply global masking rule
, which will apply the global masking rule to the database via API. All the global masking rules should be defined in one file in the root directory as masking/global-masking-rule.json
. The code it calls Bytebase API is as follows:
By changing file masking/global-masking-rule.json
, creating a PR and merge, you can apply the global masking rules.
Learn how to automate semantic type and global masking rule using GitHub Actions and Bytebase API
Bytebase is a database DevSecOps platform designed for developers, security, DBA, and platform engineering teams. While it offers an intuitive GUI for managing database schema changes and access control, some teams may want to integrate Bytebase into their existing DevOps platforms using the Bytebase API.
Bytebase provides database dynamic data masking in the Enterprise Plan, which can mask sensitive data in the SQL Editor query result based on the context on the fly. It helps organizations to protect sensitive data from being exposed to unauthorized users.
By using GitHub Actions with Bytebase API, you can implement policy-as-code to apply database masking policies via the GitOps workflow. This tutorial will guide you through the process.
This is Part 1 of our tutorial series on implementing automated database masking using GitHub Actions:
In this tutorial, you’ll learn how to automate database masking policies using GitHub Actions and the Bytebase API. This integration allows you to:
Here is a merged pull request as an example, for this tutorial, only “Semantic Type and Global Masking Rule” is covered.
The complete code for this tutorial is available at: example-database-security
Before you begin, make sure you have:
ngrok is a reverse proxy tunnel, and in our case, we need it for a public network address to allow VCS to call the Bytebase API. ngrok we used here is for demonstration purposes.
Run Bytebase in Docker with the following command:
Bytebase is running successfully in Docker, and you can visit it via localhost:8080
. Register an admin account and it will be granted the workspace admin
role automatically.
Login to ngrok Dashboard and complete the Getting Started steps to install and configure. If you want to use the same domain each time you launch ngrok, go to Cloud Edge > Domains, where you’ll find the domain <<YOURS>>.ngrok-free.app
linked to your account.
Run the ngrok command ngrok http --domain=<<YOURS>>.ngrok-free.app 8080
to start ngrok with your specific domain, and you will see the output displayed below:
Log in Bytebase and click the gear icon (Settings) on the top right. Click General under Workspace. Paste <<YOURS>>.ngrok-free.app
as External URL under Network section and click Update.
Now you can access Bytebase via <<YOURS>>.ngrok-free.app
.
Log in as Workspace Admin
, and go to IAM & Admin > Users & Groups. Click + Add User, fill in with api-sample
, choose the Workspace DBA
role sufficient for this tutorial and click Confirm.
Find the newly created service account and Copy Service Key. We will use this token to authenticate the API calls.
Sample Project
with two database hr_test
and hr_prod
.dev@example.com
, dev2@example.com
and dev3@example.com
with no roles.contractor@example.com
with dev3@example.com
as a member.Sample Project
, click Manage > Members on the left sidebar.dev@example.com
and dev2@example.com
with Developer
role and group contractor@example.com
with Querier
role.Click Settings and then click Secrets and variables > Actions. Add the following secrets:
BYTEBASE_URL
: ngrok external URLBYTEBASE_SERVICE_KEY
: api-example@service.bytebase.com
BYTEBASE_SERVICE_SECRET
: service key copied in the previous stepLet’s dig into the GitHub Actions workflow code:
Trigger: Workflow runs when PRs are merged to main
.
Authentication: The step Login Bytebase
will log in Bytebase using the official bytebase-login action. The variables you configured in the GitHub Secrets and variables are mapped to the variables in the action.
File Detection: The step Get changed files
will monitor the changed files in the pull request. For this workflow, we only care about semantic type and global masking rule. So masking/semantic-type.json
and masking/global-masking-rule.json
are filtered out.
PR Feedback: The step Comment on PR
will comment on the merged pull to notify the result.
Masking algorithm is associated with Semantic Type. You define semantic types and apply them to global masking rule or table columns. For example, you may define a semantic type birth_date
with a masking algorithm to mask month and day.
Go to Data Access > Semantic Types and click Add. You can create a new semantic type with a name and description, and customize the masking algorithm.
Find the step Apply semantic type
, which will apply the semantic type to the database via API. All the masking algorithms should be defined in one file in the root directory as masking/semantic-type.json
.
By changing file masking/semantic-type.json
, creating a PR and merging, the semantic types will be applied. Go to Bytebase console, click Data Access > Semantic Types, you can see the applied semantic types.
Global Masking Rule is configured by the admin.
Go to Data Access > Global Masking and click Add. You can create a new global masking rule mapping condition to a semantic type.
Find the step Apply global masking rule
, which will apply the global masking rule to the database via API. All the global masking rules should be defined in one file in the root directory as masking/global-masking-rule.json
. The code it calls Bytebase API is as follows:
By changing file masking/global-masking-rule.json
, creating a PR and merge, you can apply the global masking rules.