Output properties

This page provides a comprehensive list of output properties along with their corresponding descriptions

Note: Not all properties can be modified.

Properties

Note: Properties listed below may not apply to all outputs.

General

See the full list here.

Hive Metastore

Property

Description

Connection

Connection type (e.g. Athena).

Database Name

Output database name.

Table Name

Output table name.

Resources

Property

Description

Compute Cluster

The compute cluster running the calculation.

Scheduling

Property

Description

Output Interval

The output interval in minutes, hours or days.

Ending At

Whether to continue processing indefinitely, stop processing as soon as possible, or stop on a specific date and time.

Soft Retention

A setting that prevents data deletion when the retention policy in Upsolver activates. When enabled, the metadata is purged but the underlying data (e.g. S3 object) is not deleted.

Retention

The retention policy in Upsolver in minutes, hours, or days. The data is deleted permanently after this period elapses (unless Soft Retention is set to Yes). By default, the data is kept forever.

Start Execution From

When execution started.

Advanced

Property

Description

Shards #

The number of independent shards to write, to increase parallelism and reduce latency. This should remain 1 in most cases, and should never be larger than the number of shards of the data sources.

Output Shards #

Set the number of files to be created each interval in the output. This applies only to aggregated outputs (for non-aggregated outputs, the Shards configuration has this effect).

Upsert Partition Fields

Determines weather or not the partitions fields will be included as part of the Upsert Key. Enable this for improved performance if updated records always reside in the same partition.

Allow Not Partitioned Table

Enable this feature to allow writing to a table that is not partitioned time. It is highly recommended to partition by time unless you are always going to be querying across all partitions.

Compaction Shards #

The number of files that can be compacted in parallel during a compaction.

Display Data

Property

Description

Name

Output name in Upsolver.

Description

Description of output.

Creation Time

Time this output was created.

Created By

Which user this output was created by.

Modified Time

If applicable, time this output was last modified.

Modified By

Which user last modified this output.

Property

Description

Aggregated

Whether or not this output is aggregated.

Aliases

The lookup table alias.

Clusters

Click to edit the output clusters.

Attached Workspaces

The workspaces attached to this output.

Last updated