Modify a data source
This article provides a guide on how to modify data sources using an API call.
This API enables you to modify an existing data source. All API calls require an API token.
Toggle soft retention
Fields
Field
Name
Type
Description
Optional
softRetention
Soft Retention
Boolean
This setting prevents data deletion when the retention policy in Upsolver activates. When enabled, the metadata is purged but the underlying data (e.g. S3 object) is not deleted.
Example
Update retention
Fields
Field
Name
Type
Description
Optional
retention
Retention
Int (Minutes)
The retention period for the data in Upsolver.
+
Example
Update end execution at
Fields
Field
Name
Type
Description
Optional
endExecutionAt
End Read At
String (ISO-8601)
Stop reading after this date.
+
Example
Set ingestion environment
Fields
Field
Name
Type
Description
Optional
environment
Ingestion Cluster
String
The compute cluster to run the calculation on. See: Compute cluster
Example
Set compute environment
Fields
Field
Name
Type
Description
Optional
Example
Rename
Fields
Field
Name
Type
Description
Optional
name
Name
String
The data source name.
Example
Update parallelism
Fields
Field
Name
Type
Description
Optional
parallelism
Parallelism
Int
The number of independent shards to parse data, to increase parallelism and reduce latency. This should remain 1 in most cases.
Example
Update shard parallelism
Fields
Field
Name
Type
Description
Optional
parallelism
Parallelism
Int
The number of independent shards to parse data, to increase parallelism and reduce latency. This should remain 1 in most cases.
Example
Update content type
Fields
Field
Name
Type
Description
Optional
contentType
Content Type
ContentType
The format of the messages. Supported formats are: JSON, AVRO, CSV, TSV, ORC, Protobuf and x-www-form-urlencoded.
For self-describing formats like JSON, the schema is auto-detected. The body should contain of the message should contain the message itself, which should not be url-encoded.
Messages can be compressed, Upsolver automatically detects the compression type.
Supported compression types are: Zip, GZip, Snappy and None.
See: Content formats
Example
Update description
Fields
Field
Name
Type
Description
Optional
description
Description
String
The description of the data source.
Example
Attach workspace
Fields
Field
Name
Type
Description
Optional
workspace
Workspace
String
The workspace to attach to this data source.
Example
Detach workspace
Fields
Field
Name
Type
Description
Optional
workspace
Workspace
String
The workspace to detach the data source from.
Example
Attach many workspaces
Fields
Field
Name
Type
Description
Optional
workspaces
Workspaces
String[]
The workspaces to attach this data source to.
Example
Update connection
Fields
Field
Name
Type
Description
Optional
connection
Connection
String
Updating the storage location will cause new objects to be written to the new location.
moveData
Move existing objects
Boolean
Copy existing objects to the new storage location and then delete from the original location. Dependencies will continue working normally during this process.
Example
Stop
Example
Update reporting tags
Fields
Field
Name
Type
Description
Optional
reportingTags
Reporting Tags
String
Update the tags that can be sent to external monitoring systems.
Example
Last updated