Amazon Kinesis
This page describes how to create and maintain connections to your Amazon Kinesis stream.
To read and work with your Amazon Kinesis data in Upsolver, you should first create a connection to your Kinesis stream that provides Upsolver with the necessary credentials to access your data. This guide shows you how.
Create a Kinesis connection
Simple example
A Kinesis connection can be created as simply as follows:
Note that the connection in this example is created based on the default credentials derived from Upsolver's integration with your AWS account.
Full example
The following example also creates a Kinesis connection but additionally configures credentials by providing a specific role as well as establishes the connection as read-only:
To establish a connection with specific permissions, you can configure the AWS_ROLE
and EXTERNAL_ID
options like in the example above or you can configure the AWS_ACCESS_KEY_ID
and AWS_SECRET_ACCESS_KEY
options to provide the credentials to read from your stream.
Additionally, all connections have read and write permissions by default but you can easily create a connection with only read access by setting READ_ONLY
to true
.
You can also limit the list of streams displayed within your catalog by providing the list of streams to display using STREAM_DISPLAY_FILTER[S]
.
After you've created your connection, you are ready to move on to the next step of building your data pipeline: reading your data into Upsolver with an ingestion job.
Alter a Kinesis connection
Some connection options are considered mutable, meaning that in some cases, you can run a SQL command to alter an existing Kinesis connection rather than create a new one.
For example, take the Kinesis connection we created previously based on default credentials:
To change the connection's permissions and keep everything else the same without having to create an entirely new connection, you can run the following command:
Note that some options such as READ_ONLY
and REGION
cannot be altered once the connection has been created.
Drop a Kinesis connection
If you no longer need a connection, you can easily drop it with the following SQL command:
However, note that if there are existing tables or jobs that are dependent upon the connection in question, the connection cannot be deleted.
Learn more
For the full list of connection options with syntax and detailed descriptions, please see the SQL command reference for Amazon Kinesis.
Last updated