AWS Glue Data Catalog
In order to create and work with data lake tables and Apache Iceberg tables within Upsolver, you first need to establish a connection with a metadata store such as AWS Glue Data Catalog.
The tables created from your AWS Glue Data Catalog connection can also be queried from Athena and from within the Upsolver UI.
Note that when you integrate Upsolver with your AWS account, there is a Glue Data Catalog connection created by default, but you may still wish to create your own connection for specific access configurations.
Syntax
Jump to
Connection options
AWS_ROLE
— editable
AWS_ROLE
— editableType: text
(Optional) The AWS IAM role ARN. Used in conjunction with EXTERNAL_ID
.
If omitted, the role created when integrating Upsolver with the AWS account is used.
EXTERNAL_ID
— editable
EXTERNAL_ID
— editableType: text
(Optional) The external ID of the role to assume. Used in conjunction with AWS_ROLE
.
If omitted, the role created when integrating Upsolver with the AWS account is used.
AWS_ACCESS_KEY_ID
— editable
AWS_ACCESS_KEY_ID
— editableType: text
(Optional) The AWS access key ID. Used in conjunction with AWS_SECRET_ACCESS_KEY
.
If omitted, the role created when integrating Upsolver with the AWS account is used.
AWS_SECRET_ACCESS_KEY
— editable
AWS_SECRET_ACCESS_KEY
— editableType: text
(Optional) The AWS secret key corresponding to the provided AWS_ACCESS_KEY_ID
.
If omitted, the role created when integrating Upsolver with the AWS account is used.
DEFAULT_STORAGE_CONNECTION
DEFAULT_STORAGE_CONNECTION
Type: identifier
An Amazon S3 connection with the appropriate credentials to write to the DEFAULT_STORAGE_LOCATION
provided.
DEFAULT_STORAGE_LOCATION
DEFAULT_STORAGE_LOCATION
Type: text
The Amazon S3 path that serves as the default storage location for the underlying files associated with tables created under this metastore connection.
REGION
REGION
Type: text
Default: Region in which Upsolver is deployed within your AWS account
(Optional) The region your Glue Catalog is in.
DATABASE_DISPLAY_FILTER[S]
— editable
DATABASE_DISPLAY_FILTER[S]
— editableType: text | list
(Optional) A single database or the list of databases to show. If left empty, all databases are visible.
COMMENT
— editable
COMMENT
— editableType: text
(Optional) A description or comment regarding this connection.
Minimum example
This example uses the default credentials from Upsolver's integration with AWS.
Additionally, this example assumes that you have created the Amazon S3 connection my_s3_storage_connection
with proper write permissions to the specified storage location.
Refer to Amazon S3 for more information on creating a connection using SQL.
Full example
Last updated