Upsolver supports uploading data from the following JDBC sources:
Microsoft SQL Server
Other databases can also be added. Please contact Upsolver.
The JDBC data source has three operating modes for determining changes.
By a timestamp column
By an auto-incrementing key
Capture the entire table periodically (can be used for loading reference data periodically)
Mode 1 and 2 require the table containing a column that Upsolver can query from to detect new changes. The column must be updated every time Upsolver reads the change to the given row. A combination of both mode 1 and mode 2 can be used together as well.
The JDBC data source default behavior performs a full initial load followed by incremental updates.
1. From the Data Sources page, click New.
2. Select JDBC.
3. Name this data source.
4. From the dropdown, select a compute cluster (or create a new one) to run the calculation on.
5. Select a target storage connection (or create a new one) where the data read will be stored (output storage).
6. Under Parallelism, enter in the number of independent shards to parse data, increase parallelism, and reduce latency.
7. Enter the JDBC connection string.
8. (Optional) Configure any additional properties necessary to make the connection.
9. Enter in the connection username.
10. Enter in the corresponding password.
11. Enter the name of the table to connect to.
12. (Optional) Identify an incrementing column.
13. (Optional) Identify any timestamp columns.
If you have two columns
(time1, time2) and a specific row only has an entry in
time2, this will be used.
If there is an entry in both
time2, it uses the timestamp from
14. (Optional) Set a read delay (in terms of seconds) to provide a buffer for how far back in time to look in the data.
15. (Optional) Enter the full load interval.
16. Click Continue to preview your data and review the following metrics:
# of Errors
17. (Optional) If there are any errors, click Back to change the settings as required.
18. Click Create.