MySQL

Follow these steps to use MySQL as your source.

Prerequisite - Configure MySQL for CDC

Upsolver supports ingesting CDC data from relational databases including MySQL. Upsolver provides CDC capabilities by running a Debezium Engine under the hood. Connectors detect and ingest changes automatically.

Before you start, please .

Step 1 - Connect to MySQL

Create a new connection

Click Create a new connection, if it is not already selected.

Enter your connection string in the following format:

jdbc:mysql://HOST:PORT

where:

HOST - hostname (e.g. database instance endpoint in AWS RDS)
PORT - port number

Provide the Username and Password to authenticate to the database.

In the Name your connection field, type in the name for this connection. Please note this connection will be available to other users in your organization.

Use an existing connection

By default, if you have already created a connection, Upsolver selects Use an existing connection, and your MySQL connection is populated in the list.

For organizations with multiple connections, select the source connection you want to use.

Step 2 - Select schemas, tables, and columns to ingest

There are two ways to replicate objects to the target:

Manually select the schemas, tables, and columns.
Use regular expressions to specify which tables are included and which columns are excluded.

The following examples show how to use a regular expression to include specific tables:

Expression	Results
db_name.*	Select all tables in the db_name database
db_name.users, db_name.items	Select users and items tables in the db_name database
db1.items_.*	Select all tables in the db1 database that start with items_

Expression

Results

db_name.*

Select all tables in the db_name database

db_name.users, db_name.items

Select users and items tables in the db_name database

db1.items_.*

Select all tables in the db1 database that start with items_

By default, Upsolver ingests all columns in the tables that match the regular expression. Below are examples of regular expressions to exclude columns from included tables:

Expression	Results
db.users.address_.*	Select all columns starting with address_ in the users table in the db database.
db..(._pii)	Select all columns ending in _pii across all tables in the db database.

Expression

Results

db.users.address_.*

Select all columns starting with address_ in the users table in the db database.

db.*.(.*_pii)

Select all columns ending in _pii across all tables in the db database.

Schema evolution

By default, new schemas, tables, and columns added after the job is launched, will be replicated to the target.

You can overwrite the default behavior and set specific behavior for newly created schemas, tables in a specific schema, or columns in a specific table.

Last updated 5 months ago

Step 2 - Select schemas, tables, and columns to ingest

There are two ways to replicate objects to the target:

Manually select the schemas, tables, and columns.

Use regular expressions to specify which tables are included and which columns are excluded.

The following examples show how to use a regular expression to include specific tables:

Expression	Results
db_name.*	Select all tables in the db_name database
db_name.users, db_name.items	Select users and items tables in the db_name database
db1.items_.*	Select all tables in the db1 database that start with items_

Expression

Results

db_name.*

Select all tables in the db_name database

db_name.users, db_name.items

Select users and items tables in the db_name database

db1.items_.*

Select all tables in the db1 database that start with items_

By default, Upsolver ingests all columns in the tables that match the regular expression. Below are examples of regular expressions to exclude columns from included tables:

Expression	Results
db.users.address_.*	Select all columns starting with address_ in the users table in the db database.
db..(._pii)	Select all columns ending in _pii across all tables in the db database.

Expression

Results

db.users.address_.*

Select all columns starting with address_ in the users table in the db database.

db.*.(.*_pii)

Select all columns ending in _pii across all tables in the db database.

Schema evolution

By default, new schemas, tables, and columns added after the job is launched, will be replicated to the target.

You can overwrite the default behavior and set specific behavior for newly created schemas, tables in a specific schema, or columns in a specific table.