Configure access to Amazon Kinesis

This section covers how to configure an Amazon Kinesis Connection in SQLake to read from an Amazon Kinesis stream managed by another AWS account.

In order to create an IAM role and a trust relationship, please visit the Role Based AWS Credentials documentation, and use the following documentation to create the IAM policy with the required Amazon Kinesis permissions.

Create an IAM policy with required Amazon Kinesis permissions

SQLake requires the following permissions:

ListStreams
ListShards
GetShardIterator
GetRecords
DescribeStream

When creating an Amazon Kinesis connection in SQLake, you can include the STREAM_DISPLAY_FILTERS property, which allows you to restrict the Amazon Kinesis streams that users can see in the SQLake navigation tree. However, this does not limit the user’s ability to read objects; that is still managed by the permissions in the IAM role attached to the connection. This property is not to be used to restrict access to data.

If the STREAMS_DISPLAY_FILTERS property is omitted, SQLake attempts to list all streams in the account. The available streams are listed in the SQLake navigation tree to make it easier for users to discover datasets. For this to function correctly, SQLake requires the IAM policy to include kinesis:ListStreams.

If STREAMS_DISPLAY_FILTERS is included when creating the Amazon Kinesis connection, you do not need to add the kinesis:ListStreams permission.

When creating the IAM policy, add the policy statements that allow SQLake to access the data in your Amazon Kinesis:

{
	"Version": "2012-10-17",
	"Statement": [
    	{
        	"Effect": "Allow",
        	"Action": [
            	"kinesis:ListStreams"
        	],
        	"Resource": "*"
    	},
    	{
        	"Effect": "Allow",
        	"Action": [
                "kinesis:Get*",
                "kinesis:DescribeStream"
            ],
            "Resource": [
                "arn:aws:kinesis:us-east-1:111122223333:stream/stream1"
            ]
    	}
	]
}

To learn more about setting permissions for Amazon Kinesis, see Policies and Permissions in Amazon Kinesis.