Content format of ingested data source (Avro, Parquet, ORC, JSON, CSV, TSV, x-www-form-urlencoded, Protobuf, Avro-record, Avro with Schema Registry, or XML).
Compute cluster this data source is running on.
Where to store the data read (the output storage).
The retention policy in Upsolver in minutes, hours, or days. The data is deleted permanently after this period elapses (unless Soft Retention is set to Yes). By default, the data is kept forever.
A setting that prevents data deletion when the retention policy in Upsolver activates. When enabled, the metadata is purged but the underlying data (e.g. S3 object) is not deleted.
The connection string that represents a resource and its credentials.
Whether the stream was read from the start.
The data folder (e.g. billing data). If this is not specified, the data is assumed to be in the top-level of the hierarchy.
The date pattern of the files ingested.