This API enables you to create a new data source. All API calls require an API token.
POST https://api.upsolver.com/inputs
Amazon S3 (Quick)
Connect to your AWS S3 Bucket.
In order for Upsolver to read events directly from your cloud storage, files should be partitioned by date and time (which defines the folder structure in the cloud storage).
A prerequisite for defining a cloud storage data source is providing Upsolver with the appropriate credentials for reading from your cloud storage.
See:S3 connection
curl-XPOST-H"content-type: application/json"-H"Authorization: token"-d'{"clazz":"QuickS3StorageInputRequest", "bucket": "upsolver-tutorials-orders", "globFilePattern": "*", "datePattern": "yyyy/MM/dd/HH", "prefix": "data/", "contentType": { "clazz": "JsonContentType" }, "compression": { "clazz": "AutoDetectCompression" }, "displayData": { "name": "API test data source", "description": "Description of first Amazon S3 data source"}, "softRetention":false }'"https://your-api.upsolver.com/inputs/"
Amazon S3 (Advanced)
Connect to your AWS S3 Bucket.
In order for Upsolver to read events directly from your cloud storage, files should be partitioned by date and time (which defines the folder structure in the cloud storage).
A prerequisite for defining a cloud storage data source is providing Upsolver with the appropriate credentials for reading from your cloud storage.
See:S3 connection
Connect to your Amazon Kinesis. Upsolver can read events from your Amazon Kinesis, according to the stream you define.
A prerequisite for defining an Amazon Kinesis stream connection is providing Upsolver with the appropriate credentials for reading from your Amazon Kinesis stream.
See:Kinesis connection
Connect to your Amazon Kinesis. Upsolver can read events from your Amazon Kinesis, according to the stream you define.
A prerequisite for defining an Amazon Kinesis stream connection is providing Upsolver with the appropriate credentials for reading from your Amazon Kinesis stream.
See:Kinesis connection
Connect to your AWS S3 Bucket using SQS Notifications.
You will need to configure SQS Notifications from your S3 Bucket and open permissions to read and delete messages from the SQS Queue to the same access key and secret key you entered to give Upsolver permissions to read from the S3 Bucket.
See:S3 over SQS connection
Fields
Example
curl-XPOST-H"content-type: application/json"-H"Authorization: YOUR_TOKEN" \-d '{ "clazz" : "S3OverSQSInputRequest", "displayData" : { "name" : "First S3 Over SQS Data Source", "description" : "Description of first S3 Over SQS data source" }, "sourceStorage" : "6ee598b6-4928-4b07-b532-83a79464e5bb", "contentType" : { "type" : "JsonContentType" }, "computeEnvironment" : "46c2945e-5a46-4d93-9e21-5a85653c28c5", "destinationStorage" : "2a08601c-52d2-425e-8d2c-324dbcea5858", "compression" : { "clazz" : "AutoDetectCompression" }, "softRetention" : true, "executionParallelism" : 1}'"https://api.upsolver.com/api/v1/data-source/"
Apache Kafka (Quick)
Connect to any topic on your Kafka Servers. Upsolver can read events from your Kafka cluster from the specified Kafka topic.
A prerequisite for defining a Kafka stream connection is providing Upsolver with the appropriate credentials for reading from your Kafka cluster.
See:Kafka connection
Connect to any topic on your Kafka Servers. Upsolver can read events from your Kafka cluster from the specified Kafka topic.
A prerequisite for defining a Kafka stream connection is providing Upsolver with the appropriate credentials for reading from your Kafka cluster.
See:Kafka connection
In order for Upsolver to read events directly from your cloud storage, files should be partitioned by date and time (which defines the folder structure in the cloud storage).
A prerequisite for defining a cloud storage data source is providing Upsolver with the appropriate credentials for reading from your cloud storage.
See:Azure Blob storage connection
In order for Upsolver to read events directly from your cloud strage, files should be partitioned by date and time (which defines the folder structure in the cloud storage)
A prerequisite for defining a cloud storage data source is providing Upsolver with the appropriate credentials for reading from your cloud storage.
See:Google Storage connection
Connect your stream using HTTP requests from any source.
Once you create the connection, you will be provided with an HTTP endpoint. Upsolver receives the data as POST, with the data in the body, and the data is stored in a Kinesis Stream until processed by Upsolver.
Headers sent with the request are also ingested as part of the stream, so metadata can be added to the request header.
