Apache Kafka

Learn the steps for ingesting your Apache Kafka data into your target.

Steps

1. Configure access to your source Apache Kafka stream.

2. Create a connection to your source Apache Kafka stream.

3. Create a connection to your target (see the Supported Targets list below for links).

4. Create the entities for your target per the list in the Job Type & Steps column below. Note that there are multiple options for ingesting to Snowflake:

Job Type & Steps	Supported Targets
Ingestion Job Create a staging table in the data lake Create an ingestion job to load data into the staging table Create a transformation job for each target	Amazon Redshift Amazon S3 AWS Glue Data Catalog ClickHouse Elasticsearch Polaris Catalog PostgreSQL Snowflake
Direct Ingestion Job Create an ingestion job to load directly into the target	Snowflake

No-code Wizard

Alternatively, you can use the Upsolver no-code Wizard to get your pipelines started in super-quick time. The Wizard generates the code for you so you can optionally customize it for your requirements, and create advanced use cases.

---

Related Content

Quickstarts

• Create an Apache Kafka Connection

• Ingest Your Data from Apache Kafka

How To Guides

• Create and Maintain Connections to Your Apache Kafka Stream

• Configure Access to Apache Kafka

• Real-time Data Ingestion — Apache Kafka to Amazon Athena

• Real-time Data Ingestion — Apache Kafka to Snowflake

Reference

• Connection Options for Apache Kafka

• Ingestion Job Options for Apache Kafka