LogoLogo
OverviewQuickstartsHow To GuidesReferenceArticlesSupport
How To Guides
How To Guides
  • How To Guides
  • SETUP
    • Deploy Upsolver on AWS
      • Deployment Guide
      • AWS Role Permissions
      • VPC Peering Guide
      • Role-Based AWS Credentials
    • Enable API Integration
    • Install the Upsolver CLI
  • CONNECTORS
    • Create Connections
      • Amazon Kinesis
      • Amazon Redshift
      • Amazon S3
      • Apache Kafka
      • AWS Glue Data Catalog
      • ClickHouse
      • Confluent Cloud
      • Elasticsearch
      • Microsoft SQL Server
      • MongoDB
      • MySQL
      • PostgreSQL
      • Snowflake
      • Tabular
    • Configure Access
      • Amazon Kinesis
      • Amazon S3
      • Apache Kafka
      • AWS Glue Data Catalog
      • Confluent Kafka
    • Enable CDC
      • Microsoft SQL Server
      • MongoDB
      • MySQL
      • PostgreSQL
  • JOBS
    • Basics
      • Real-time Data Ingestion — Amazon Kinesis to ClickHouse
      • Real-time Data Ingestion — Amazon S3 to Amazon Athena
      • Real-time Data Ingestion — Apache Kafka to Amazon Athena
      • Real-time Data Ingestion — Apache Kafka to Snowflake
    • Advanced Use Cases
      • Build a Data Lakehouse
      • Enriching Data - Amazon S3 to ClickHouse
      • Joining Data — Amazon S3 to Amazon Athena
      • Upserting Data — Amazon S3 to Amazon Athena
      • Aggregating Data — Amazon S3 to Amazon Athena
      • Managing Data Quality - Ingesting Data with Expectations
    • Database Replication
      • Replicate CDC Data into Snowflake
      • Replicate CDC Data to Multiple Targets in Snowflake
      • Ingest Your Microsoft SQL Server CDC Data to Snowflake
      • Ingest Your MongoDB CDC Data to Snowflake
      • Handle PostgreSQL TOAST Values
    • VPC Flow Logs
      • Data Ingestion — VPC Flow Logs
      • Data Analytics — VPC Flow Logs
    • Job Monitoring
      • Export Metrics to a Third-Party System
    • Data Observability
      • Observe Data with Datasets
  • DATA
    • Query Upsolver Iceberg Tables from Snowflake
  • APACHE ICEBERG
    • Analyze Your Iceberg Tables Using the Upsolver CLI
    • Optimize Your Iceberg Tables
Powered by GitBook
On this page
  • Permissions
  • UpsolverServerRole
  • Policies
  • UpsolverManagementRole
  • Server Role
  1. SETUP
  2. Deploy Upsolver on AWS

AWS Role Permissions

This page describes the AWS role permissions needed to integrate your account with Upsolver.

When integrating with AWS, one or two managed roles are created in your account to give Upsolver the required access.

There are three role types. The role types in your account depend on the type of integration.

  • With Private VPC integration, two roles are created:

    • UpsolverManagementRole

    • UpsolverServerRole

  • With Upsolver Cloud integration, one role includes the permissions of both the UpsolverManagementRole and the UpsolverServerRole roles, unless otherwise stated.

Permissions

UpsolverServerRole

This is the role that the Upsolver servers running in your VPC use to access the data in your account. The permissions given to this role are:

Permission

Description

s3:ListAllMyBuckets

Allows the servers to view which buckets you have so Upsolver's UI can suggest them for your convenience.

kinesis:ListStreams

Allows the servers to identify your Kinesis Streams so that Upsolver's UI can suggest them for your convenience.

arn:aws:iam::aws:policy/ AmazonAthenaFullAccess

Allows the servers to manage your Athena tables. Athena does not allow partial permissions; full access is required.

Additional data read/write permissions

When adding data sources or creating data outputs, you may need to add read/write permissions.

Policies

UpsolverManagementRole

Managed Policies

Policy

Description

AWSCloudFormationReadOnlyAccess

Permission is required for Upsolver to identify when the initial integration completed successfully.

Custom Policies

Policies

Description

ec2:RunInstances, ec2:StartInstances, ec2:TerminateInstances, ec2:RequestSpotInstances, ec2:CancelSpotInstanceRequests, ec2:CreateVolume, ec2:AttachVolume, ec2:DeleteVolume

Allows running and stopping Upsolver EC2 instances.

ec2:DescribeInstances, ec2:DescribeSpotInstanceRequests, ec2:DescribeInstanceStatus, ec2:CreateTags, ec2:DescribeTags

Allows monitoring Upsolver EC2 clusters.

ec2:DescribeSecurityGroups, ec2:DescribeImages, ec2:DescribeImageAttribute

Required for Spotinst validation.

ec2:AssociateAddress, ec2:DisassociateAddress, ec2:AllocateAddress, ec2:ReleaseAddress, ec2:DescribeAddresses

Allows Upsolver to use static IP addresses for discoverability.

cloudwatch:PutMetricData, cloudwatch:GetMetricStatistics, cloudwatch:ListMetrics, cloudwatch:DescribeAlarmHistory, cloudwatch:DescribeAlarmsForMetric, cloudwatch:DescribeAlarms

Auto Scaling against CloudWatch statistics and alarms.

iam:ListPolicies, iam:GetPolicyVersion, iam:GetPolicy, iam:ListRoles, iam:ListInstanceProfiles, iam:AddRoleToInstanceProfile, iam:ListInstanceProfilesForRole, iam:ListAttachedRolePolicies, iam:ListAccountAliases, iam:PassRole

Required for Spotinst for policy validation.

{
    "Statement": [
    "Action": [
        "ec2:DescribeSpotInstanceRequests",
        "ec2:DescribeAddresses",
        "ec2:DescribeInstances",
        "ec2:DescribeSecurityGroups",
        "ec2:DescribeInstanceStatus",
        "ec2:DescribeTags",
        "ec2:DescribeImages",
        "ec2:DescribeImageAttribute",
         "ec2:DescribeSpotPriceHistory",
        "cloudwatch:PutMetricData",
        "cloudwatch:GetMetricStatistics",
        "cloudwatch:ListMetrics",
        "cloudwatch:DescribeAlarmHistory",
        "cloudwatch:DescribeAlarmsForMetric",
        "cloudwatch:DescribeAlarms",
        "iam:ListPolicies",
        "iam:GetPolicyVersion",
        "iam:GetPolicy",
        "iam:ListRoles",
        "iam:ListInstanceProfiles",
        "iam:AddRoleToInstanceProfile",
        "iam:ListInstanceProfilesForRole",
        "iam:ListAttachedRolePolicies",
        "iam:ListAccountAliases",
        "iam:PassRole"
    ],
    "Resource": [
        "*"
    ],
    "Effect": "Allow"
        },
        {
    "Action": [
        "ec2:RequestSpotInstances",
        "ec2:CancelSpotInstanceRequests",
        "ec2:CreateTags",
        "ec2:AssociateAddress",
        "ec2:DisassociateAddress",
        "ec2:AllocateAddress",
        "ec2:ReleaseAddress"
    ],
    "Resource": [
        "*"
    ],
    "Effect": "Allow"
        },
        {
    "Condition": {
        "StringLike": {
            "aws:RequestTag/Name": "*upsolver*"
        }
    },
    "Action": [
        "ec2:CreateVolume",
        "ec2:RunInstances"
    ],
    "Resource": [
        "*"
    ],
    "Effect": "Allow"
        },
        {
    "Condition": {
        "StringLike": {
            "ec2:ResourceTag/Name": "*upsolver*"
        }
    },
    "Action": [
        "ec2:TerminateInstances",
        "ec2:StartInstances",
        "ec2:AttachVolume",
        "ec2:DeleteVolume",
        "ec2:RunInstances"
    ],
    "Resource": [
        "*"
    ],
    "Effect": "Allow"
        },
        {
    "Action": [
        "ec2:RunInstances"
    ],
    "Resource": [
        "arn:aws:ec2:*:*:network-interface/*",
        "arn:aws:ec2:*:*:subnet/*",
        "arn:aws:ec2:*::image/*"
    ],
    "Effect": "Allow"
        }
    ]
}

Server Role

Managed Policies

Policy

Description

AmazonAthenaFullAccess

Allows the servers to manage your Athena tables. Athena does not allow partial permissions; full access is required.

To configure finely grained permissions, use AWS Lake Formation.

AWSCloudFormationReadOnlyAccess

Required for Upsolver to identify when potential follow-up integrations have completed successfully.

Custom Policy:

{
    "Statement": [
        {
    "Sid": "upsolverBucketAccess",
    "Action": [
        "s3:*"
    ],
    "Resource": [
        "arn:aws:s3:::us-east-1-upsolver-UPSOLVER_ORG_ID",
        "arn:aws:s3:::us-east-1-upsolver-UPSOLVER_ORG_ID/*"
    ],
    "Effect": "Allow"
        },
        {
    "Sid": "listStreams",
    "Action": [
        "kinesis:ListStreams"
    ],
    "Resource": [
        "*"
    ],
    "Effect": "Allow"
        },
        {
    "Sid": "upsolverManagedStream",
    "Action": [
        "kinesis:*"
    ],
    "Resource": [
        "arn:aws:kinesis:*:*:stream/upsolver_*"
    ],
    "Effect": "Allow"
        },
        {
    "Sid": "sendScalingMetrics",
    "Action": [
        "cloudwatch:PutMetricData"
    ],
    "Resource": [
        "*"
    ],
    "Effect": "Allow"
        }
    ]
}

Last updated 11 months ago