LogoLogo
OverviewQuickstartsHow To GuidesReferenceArticlesSupport
How To Guides
How To Guides
  • How To Guides
  • SETUP
    • Deploy Upsolver on AWS
      • Deployment Guide
      • AWS Role Permissions
      • VPC Peering Guide
      • Role-Based AWS Credentials
    • Enable API Integration
    • Install the Upsolver CLI
  • CONNECTORS
    • Create Connections
      • Amazon Kinesis
      • Amazon Redshift
      • Amazon S3
      • Apache Kafka
      • AWS Glue Data Catalog
      • ClickHouse
      • Confluent Cloud
      • Elasticsearch
      • Microsoft SQL Server
      • MongoDB
      • MySQL
      • PostgreSQL
      • Snowflake
      • Tabular
    • Configure Access
      • Amazon Kinesis
      • Amazon S3
      • Apache Kafka
      • AWS Glue Data Catalog
      • Confluent Kafka
    • Enable CDC
      • Microsoft SQL Server
      • MongoDB
      • MySQL
      • PostgreSQL
  • JOBS
    • Basics
      • Real-time Data Ingestion — Amazon Kinesis to ClickHouse
      • Real-time Data Ingestion — Amazon S3 to Amazon Athena
      • Real-time Data Ingestion — Apache Kafka to Amazon Athena
      • Real-time Data Ingestion — Apache Kafka to Snowflake
    • Advanced Use Cases
      • Build a Data Lakehouse
      • Enriching Data - Amazon S3 to ClickHouse
      • Joining Data — Amazon S3 to Amazon Athena
      • Upserting Data — Amazon S3 to Amazon Athena
      • Aggregating Data — Amazon S3 to Amazon Athena
      • Managing Data Quality - Ingesting Data with Expectations
    • Database Replication
      • Replicate CDC Data into Snowflake
      • Replicate CDC Data to Multiple Targets in Snowflake
      • Ingest Your Microsoft SQL Server CDC Data to Snowflake
      • Ingest Your MongoDB CDC Data to Snowflake
      • Handle PostgreSQL TOAST Values
    • VPC Flow Logs
      • Data Ingestion — VPC Flow Logs
      • Data Analytics — VPC Flow Logs
    • Job Monitoring
      • Export Metrics to a Third-Party System
    • Data Observability
      • Observe Data with Datasets
  • DATA
    • Query Upsolver Iceberg Tables from Snowflake
  • APACHE ICEBERG
    • Analyze Your Iceberg Tables Using the Upsolver CLI
    • Optimize Your Iceberg Tables
Powered by GitBook
On this page
  • AWS Glue Data Catalog Access
  • Creating an IAM Role for AWS Glue Access
  • Configuring AWS Glue Connection in Upsolver
  1. CONNECTORS
  2. Configure Access

AWS Glue Data Catalog

AWS Glue Data Catalog Access

This section outlines how to configure AWS Glue Data Catalog access in Upsolver. Setting up AWS Glue access in Upsolver facilitates effective data management and processing.

Creating an IAM Role for AWS Glue Access

To set up AWS Glue Data Catalog in Upsolver, an IAM role with the appropriate permissions is required. Follow the AWS documentation to create an IAM role and establish a trust relationship.

Required IAM Permissions for AWS Glue

Upsolver needs these permissions for integrating with AWS Glue:

  • glue:GetDatabase

  • glue:GetTable

  • glue:GetPartition

  • glue:GetPartitions

  • glue:GetUserDefinedFunction

  • glue:CreateDatabase: Create new databases in the AWS Glue Catalog

  • glue:UpdateDatabase: Update existing databases

  • glue:DeleteDatabase: Delete databases

  • glue:CreateTable: Create new tables

  • glue:UpdateTable: Update existing tables

  • glue:DeleteTable: Delete tables

  • glue:BatchCreatePartition: Create multiple partitions simultaneously

  • glue:BatchDeletePartition: Delete multiple partitions simultaneously

Configuring AWS Glue Connection in Upsolver

When creating an AWS Glue connection in Upsolver, specify the AWS Glue Data Catalog and database for Upsolver to access and manage data structures.

IAM Policy for AWS Glue Access

Create an IAM policy with the following statements to grant Upsolver access to AWS Glue resources. Replace <database_name> with your actual database name:

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Action": [
                "glue:GetDatabase",
                "glue:GetTable",
                "glue:GetPartition",
                "glue:GetPartitions",
                "glue:GetUserDefinedFunction"
                "glue:CreateDatabase",
                "glue:UpdateDatabase",
                "glue:DeleteDatabase",
                "glue:CreateTable",
                "glue:UpdateTable",
                "glue:DeleteTable",
                "glue:BatchCreatePartition",
                "glue:BatchDeletePartition"
            ],
            "Resource": [
                "arn:aws:glue:<region>:<account-id>:database/<database_name>",
                "arn:aws:glue:<region>:<account-id>:table/<database_name>/*",
                "arn:aws:glue:<region>:<account-id>:catalog"
            ]
        }     
    ]
}

This policy allows Upsolver to perform read and write operations in the specified AWS Glue Data Catalog.

Learn More

Last updated 12 months ago

For detailed information on AWS Glue permissions and integration with Upsolver, refer to Amazon's .

AWS Glue documentation