This API enables you to modify an existing data source. All API calls require an API token.
PATCH https://api.upsolver.com/inputs/DATA-SOURCE-ID
Field | Name | Type | Description | Optional |
softRetention | Soft Retention | Boolean | This setting prevents data deletion when the retention policy in Upsolver activates. When enabled, the metadata is purged but the underlying data (e.g. S3 object) is not deleted. | ​ |
curl -X POST -H "content-type: application/json" -H"Authorization: YOUR_TOKEN" \-d '{"clazz" : "ToggleSoftRetention","softRetention" : true}' "https://api.upsolver.com/inputs/DATA-SOURCE-ID"
Field | Name | Type | Description | Optional |
retention | Retention | Int (Minutes) | The retention period for the data in Upsolver. | + |
curl -X POST -H "content-type: application/json" -H "Authorization:YOUR_TOKEN" \-d '{"clazz" : "UpdateRetention","retention" : 1440}' "https://api.upsolver.com/inputs/DATA-SOURCE-ID"
Field | Name | Type | Description | Optional |
endExecutionAt | End Read At | String (ISO-8601) | Stop reading after this date. | + |
curl -X POST -H "content-type: application/json" -H"Authorization: YOUR_TOKEN" \-d '{"clazz" : "UpdateEndExecutionAt","endExecutionAt" : "2018-11-04T08:00:00Z"}' "https://api.upsolver.com/inputs/DATA-SOURCE-ID"
Field | Name | Type | Description | Optional |
environment | Ingestion Cluster | String | The compute cluster to run the calculation on. See: Compute cluster​ | ​ |
curl -X POST -H "content-type: application/json" -H"Authorization: YOUR_TOKEN" \-d '{"clazz" : "SetIngestionEnvironment","environment" : "09267786-8d84-4644-adac-52601b9609bb"}' "https://api.upsolver.com/inputs/DATA-SOURCE-ID"
Field | Name | Type | Description | Optional |
environment | Environment | String | The compute cluster to run the calculation on. See: Compute cluster​ | ​ |
curl -X POST -H "content-type: application/json" -H"Authorization: YOUR_TOKEN" \-d '{"clazz" : "SetComputeEnvironment","environment" : "12ba7222-5764-4d37-8581-95ca5be4b3f7"}' "https://api.upsolver.com/inputs/DATA-SOURCE-ID"
Field | Name | Type | Description | Optional |
name | Name | String | The data source name. | ​ |
curl -X POST -H "content-type: application/json" -H"Authorization: YOUR_TOKEN" \-d '{"clazz" : "Rename","name" : "NEW_NAME"}' "https://api.upsolver.com/inputs/DATA-SOURCE-ID"
Field | Name | Type | Description | Optional |
parallelism | Parallelism | Int | The number of independent shards to parse data, to increase parallelism and reduce latency. This should remain 1 in most cases. | ​ |
curl -X POST -H "content-type: application/json" -H"Authorization: YOUR_TOKEN" \-d '{"clazz" : "UpdateParallelism","parallelism" : 1}' "https://api.upsolver.com/inputs/DATA-SOURCE-ID"
Field | Name | Type | Description | Optional |
parallelism | Parallelism | Int | The number of independent shards to parse data, to increase parallelism and reduce latency. This should remain 1 in most cases. | ​ |
curl -X POST -H "content-type: application/json" -H"Authorization: YOUR_TOKEN" \-d '{"clazz" : "UpdateShardParallelism","shards" : 1}' "https://api.upsolver.com/inputs/DATA-SOURCE-ID"
Field | Name | Type | Description | Optional |
contentType | Content Type | ContentType | The format of the messages. Supported formats are: JSON, AVRO, CSV, TSV, ORC, Protobuf and x-www-form-urlencoded. For self-describing formats like JSON, the schema is auto-detected. The body should contain of the message should contain the message itself, which should not be url-encoded. Messages can be compressed, Upsolver automatically detects the compression type. Supported compression types are: Zip, GZip, Snappy and None. See: Content formats​ | ​ |
curl -X POST -H "content-type: application/json" -H"Authorization: YOUR_TOKEN" \-d '{"clazz" : "UpdateContentType","contentType" : {"type" : "JsonContentType"}}' "https://api.upsolver.com/inputs/DATA-SOURCE-ID"
Field | Name | Type | Description | Optional |
description | Description | String | The description of the data source. | ​ |
curl -X POST -H "content-type: application/json" -H"Authorization: YOUR_TOKEN" \-d '{"clazz" : "UpdateDescription","description" : "description"}' "https://api.upsolver.com/inputs/DATA-SOURCE-ID"
Field | Name | Type | Description | Optional |
workspace | Workspace | String | The workspace to attach to this data source. | ​ |
curl -X POST -H "content-type: application/json" -H"Authorization: YOUR_TOKEN" \-d '{"clazz" : "AttachWorkspace","workspace" : "workspace"}' "https://api.upsolver.com/inputs/DATA-SOURCE-ID"
Field | Name | Type | Description | Optional |
workspace | Workspace | String | The workspace to detach the data source from. | ​ |
curl -X POST -H "content-type: application/json"-H "Authorization: YOUR_TOKEN" \-d '{"clazz" : "DetachWorkspace","workspace" : "workspace"}' "https://api.upsolver.com/inputs/DATA-SOURCE-ID"
Field | Name | Type | Description | Optional |
workspaces | Workspaces | String[] | The workspaces to attach this data source to. | ​ |
curl -X POST -H "content-type: application/json" -H"Authorization: YOUR_TOKEN" \-d '{"clazz" : "AttachManyWorkspaces","workspaces" : "workspaces"}' "https://api.upsolver.com/inputs/DATA-SOURCE-ID"
Field | Name | Type | Description | Optional |
connection | Connection | String | Updating the storage location will cause new objects to be written to the new location. | ​ |
moveData | Move existing objects | Boolean | Copy existing objects to the new storage location and then delete from the original location. Dependencies will continue working normally during this process. | ​ |
curl -X POST -H "content-type: application/json" -H"Authorization: YOUR_TOKEN" \-d '{"clazz" : "UpdateConnection","connection" : "00dbf0e2-7c11-47f8-a672-cacb537af190","moveData" : true}' "https://api.upsolver.com/inputs/DATA-SOURCE-ID"
curl -X POST -H "content-type: application/json" -H"Authorization: YOUR_TOKEN" \-d '{"clazz" : "Stop"}' "https://api.upsolver.com/inputs/DATA-SOURCE-ID"
Field | Name | Type | Description | Optional |
reportingTags | Reporting Tags | String | Update the tags that can be sent to external monitoring systems. | ​ |
curl -X POST -H "content-type: application/json" -H"Authorization: YOUR_TOKEN" \-d '{"clazz" : "UpdateReportingTags","reportingTags" : "reportingTags"}' "https://api.upsolver.com/inputs/DATA-SOURCE-ID"