Upsolver
Search…
Getting Started
Connecting data sources
Data outputs and data transformation
Guide for developers
Release notes
Support
Modify a data source
This article provides a guide on how to modify data sources using an API call.
1
PATCH https://api.upsolver.com/inputs/DATA-SOURCE-ID
Copied!
Toggle soft retention
Fields
Field
Name
Type
Description
Optional
softRetention
Soft Retention
Boolean
This setting prevents data deletion when the retention policy in Upsolver activates. When enabled, the metadata is purged but the underlying data (e.g. S3 object) is not deleted.
​
Example
1
curl -X POST -H "content-type: application/json" -H
2
"Authorization: YOUR_TOKEN" \
3
-d '{
4
"clazz" : "ToggleSoftRetention",
5
"softRetention" : true
6
}' "https://api.upsolver.com/inputs/DATA-SOURCE-ID"
Copied!
Update retention
Fields
Field
Name
Type
Description
Optional
retention
Retention
Int (Minutes)
The retention period for the data in Upsolver.
+
Example
1
curl -X POST -H "content-type: application/json" -H "Authorization:
2
YOUR_TOKEN" \
3
-d '{
4
"clazz" : "UpdateRetention",
5
"retention" : 1440
6
}' "https://api.upsolver.com/inputs/DATA-SOURCE-ID"
Copied!
Update end execution at
Fields
Field
Name
Type
Description
Optional
endExecutionAt
End Read At
String (ISO-8601)
Stop reading after this date.
+
Example
1
curl -X POST -H "content-type: application/json" -H
2
"Authorization: YOUR_TOKEN" \
3
-d '{
4
"clazz" : "UpdateEndExecutionAt",
5
"endExecutionAt" : "2018-11-04T08:00:00Z"
6
}' "https://api.upsolver.com/inputs/DATA-SOURCE-ID"
Copied!
Set ingestion environment
Fields
Field
Name
Type
Description
Optional
environment
Ingestion Cluster
String
​
Example
1
curl -X POST -H "content-type: application/json" -H
2
"Authorization: YOUR_TOKEN" \
3
-d '{
4
"clazz" : "SetIngestionEnvironment",
5
"environment" : "09267786-8d84-4644-adac-52601b9609bb"
6
}' "https://api.upsolver.com/inputs/DATA-SOURCE-ID"
Copied!
Set compute environment
Fields
Field
Name
Type
Description
Optional
environment
Environment
String
​
Example
1
curl -X POST -H "content-type: application/json" -H
2
"Authorization: YOUR_TOKEN" \
3
-d '{
4
"clazz" : "SetComputeEnvironment",
5
"environment" : "12ba7222-5764-4d37-8581-95ca5be4b3f7"
6
}' "https://api.upsolver.com/inputs/DATA-SOURCE-ID"
Copied!
Rename
Fields
Field
Name
Type
Description
Optional
name
Name
String
The data source name.
​
Example
1
curl -X POST -H "content-type: application/json" -H
2
"Authorization: YOUR_TOKEN" \
3
-d '{
4
"clazz" : "Rename",
5
"name" : "NEW_NAME"
6
}' "https://api.upsolver.com/inputs/DATA-SOURCE-ID"
Copied!
Update parallelism
Fields
Field
Name
Type
Description
Optional
parallelism
Parallelism
Int
The number of independent shards to parse data, to increase parallelism and reduce latency.
This should remain 1 in most cases.
​
Example
1
curl -X POST -H "content-type: application/json" -H
2
"Authorization: YOUR_TOKEN" \
3
-d '{
4
"clazz" : "UpdateParallelism",
5
"parallelism" : 1
6
}' "https://api.upsolver.com/inputs/DATA-SOURCE-ID"
Copied!
Update shard parallelism
Fields
Field
Name
Type
Description
Optional
parallelism
Parallelism
Int
The number of independent shards to parse data, to increase parallelism and reduce latency.
This should remain 1 in most cases.
​
Example
1
curl -X POST -H "content-type: application/json" -H
2
"Authorization: YOUR_TOKEN" \
3
-d '{
4
"clazz" : "UpdateShardParallelism",
5
"shards" : 1
6
}' "https://api.upsolver.com/inputs/DATA-SOURCE-ID"
Copied!
Update content type
Fields
Field
Name
Type
Description
Optional
contentType
Content Type
ContentType
The format of the messages. Supported formats are: JSON, AVRO, CSV, TSV, ORC, Protobuf and x-www-form-urlencoded.
For self-describing formats like JSON, the schema is auto-detected. The body should contain of the message should contain the message itself, which should not be url-encoded.
Messages can be compressed, Upsolver automatically detects the compression type.
Supported compression types are: Zip, GZip, Snappy and None.
​
Example
1
curl -X POST -H "content-type: application/json" -H
2
"Authorization: YOUR_TOKEN" \
3
-d '{
4
"clazz" : "UpdateContentType",
5
"contentType" : {
6
"type" : "JsonContentType"
7
}
8
}' "https://api.upsolver.com/inputs/DATA-SOURCE-ID"
Copied!
Update description
Fields
Field
Name
Type
Description
Optional
description
Description
String
The description of the data source.
​
Example
1
curl -X POST -H "content-type: application/json" -H
2
"Authorization: YOUR_TOKEN" \
3
-d '{
4
"clazz" : "UpdateDescription",
5
"description" : "description"
6
}' "https://api.upsolver.com/inputs/DATA-SOURCE-ID"
Copied!
Attach workspace
Fields
Field
Name
Type
Description
Optional
workspace
Workspace
String
The workspace to attach to this data source.
​
Example
1
curl -X POST -H "content-type: application/json" -H
2
"Authorization: YOUR_TOKEN" \
3
-d '{
4
"clazz" : "AttachWorkspace",
5
"workspace" : "workspace"
6
}' "https://api.upsolver.com/inputs/DATA-SOURCE-ID"
Copied!
Detach workspace
Fields
Field
Name
Type
Description
Optional
workspace
Workspace
String
The workspace to detach the data source from.
​
Example
1
curl -X POST -H "content-type: application/json"
2
-H "Authorization: YOUR_TOKEN" \
3
-d '{
4
"clazz" : "DetachWorkspace",
5
"workspace" : "workspace"
6
}' "https://api.upsolver.com/inputs/DATA-SOURCE-ID"
Copied!
Attach many workspaces
Fields
Field
Name
Type
Description
Optional
workspaces
Workspaces
String[]
The workspaces to attach this data source to.
​
Example
1
curl -X POST -H "content-type: application/json" -H
2
"Authorization: YOUR_TOKEN" \
3
-d '{
4
"clazz" : "AttachManyWorkspaces",
5
"workspaces" : "workspaces"
6
}' "https://api.upsolver.com/inputs/DATA-SOURCE-ID"
Copied!
Update connection
Fields
Field
Name
Type
Description
Optional
connection
Connection
String
Updating the storage location will cause new objects to be written to the new location.
​
moveData
Move existing objects
Boolean
Copy existing objects to the new storage location and then delete from the original location. Dependencies will continue working normally during this process.
​
Example
1
curl -X POST -H "content-type: application/json" -H
2
"Authorization: YOUR_TOKEN" \
3
-d '{
4
"clazz" : "UpdateConnection",
5
"connection" : "00dbf0e2-7c11-47f8-a672-
6
cacb537af190",
7
"moveData" : true
8
}' "https://api.upsolver.com/inputs/DATA-SOURCE-ID"
Copied!
Stop
Example
1
curl -X POST -H "content-type: application/json" -H
2
"Authorization: YOUR_TOKEN" \
3
-d '{
4
"clazz" : "Stop"
5
}' "https://api.upsolver.com/inputs/DATA-SOURCE-ID"
Copied!
Update reporting tags
Fields
Field
Name
Type
Description
Optional
reportingTags
Reporting Tags
String
Update the tags that can be sent to external monitoring systems.
​
Example
1
curl -X POST -H "content-type: application/json" -H
2
"Authorization: YOUR_TOKEN" \
3
-d '{
4
"clazz" : "UpdateReportingTags",
5
"reportingTags" : "reportingTags"
6
}' "https://api.upsolver.com/inputs/DATA-SOURCE-ID"
Copied!
Last modified 1yr ago
Copy link