Amazon AWS data sources

This page provides an introduction to the different types of AWS data sources that can be created on Upsolver.

AWS data sources guides

• S3

• Kinesis

• S3 over SQS

• AppFlow

Amazon S3

What is Amazon S3?

Amazon Simple Storage Service (or Amazon S3) is object storage built to store and retrieve any amount of data from anywhere on the Internet. It’s a simple storage service that offers a highly durable, available, and scalable data storage infrastructure at low costs.

Amazon S3 has a web services interface that can be used to store and retrieve data at any time, from anywhere on the web. It gives developers access to the same scalable, reliable, fast, and inexpensive data storage infrastructure that Amazon uses to run its own global network of web sites.

Upsolver can read events directly from your Amazon S3 bucket, assuming events are partitioned by event date (this partitioning defines the folder structure in the cloud storage).

Amazon S3 data source

Amazon Kinesis Stream

What is Amazon Kinesis Stream?

Amazon Kinesis Data Streams (KDS) is a real-time data streaming service.

KDS can continuously capture gigabytes of data per second from hundreds of thousands of sources such as website clickstreams, database event streams, financial transactions, social media feeds, IT logs, and location-tracking events. The data collected is available in milliseconds to enable real-time analytics use cases such as real-time dashboards, real-time anomaly detection, dynamic pricing, and more.

Amazon Kinesis Stream data source

Amazon S3 over SQS

What is Amazon SQS?

Amazon Simple Queue Service (Amazon SQS) offers a secure, durable, and available hosted queue that lets you integrate and decouple distributed software systems and components.

Amazon SQS offers common constructs such as dead-letter queues and cost allocation tags. It provides a generic web services API and it can be accessed by any programming language that the AWS SDK supports.

Amazon SQS supports both standard and FIFO queues.

Amazon AppFlow

What is Amazon AppFlow?

Amazon AppFlow is a fully managed integration service that enables you to securely transfer data between Software-as-a-Service (SaaS) applications like Salesforce, Marketo, Slack, and ServiceNow, and AWS services like Amazon S3 and Amazon Redshift. With AppFlow, you can run data flows at nearly any scale at the frequency you choose - on a schedule, in response to a business event, or on demand.

Amazon S3 over SQS data source