Data formats
This page goes over the different data formats available in Upsolver and, if applicable, their corresponding configuration options.
When creating a data source, the content format is typically auto detected, but you can manually select a format and specify additional details, as required.
The following content formats are supported:
For Amazon S3, Azure Blob Storage, Google Cloud Storage or File Upload data sources.
For Amazon Kinesis, Kafka, S3 Over SQS.
Avro
The schema is auto detected.
Parquet
The schema is auto detected.
ORC
The schema is auto detected.
JSON
The schema is auto detected. The body should contain the message itself, which should not be url-encoded.
(Optional) Whether to store the JSON in native format in a separate field.
CSV
(Optional) Select whether to auto detect the types. If not selected, Upsolver will read all fields as strings.
TSV
(Optional) Whether to auto detect the types.
x-www-form-urlencoded
The body should contain the message itself, which should not be url-encoded.
Protobuf
(Optional) Click Select to choose the required schema files.
Avro-record
For Amazon S3, Azure Blob Storage, Google Cloud Storage or File Upload data sources.
(Optional) The Avro record schema.
Avro-record
For Amazon Kinesis, Kafka, S3 Over SQS.
The Avro record schema.
Avro with Schema Registry
(Optional) The URL to the schema registry.
XML
The schema is auto-detected.
Last updated