The Presto Utilization dashboard show you your recent and historical use of Presto resources. You can use the dashboard to determine if you risk exceeding your monthly plan limits. 

Access

From the console, Arm Treasure Data owners and administrators, click Admin > Presto Utilization to access the dashboard.

Presto Terms and Concepts

Understanding Presto concepts and terminology is useful when assessing metrics that are presented in the dashboard.

The following terms are used in more than one metric:

Splits

Sections of a larger data set, including tables and intermediate data during query processing. A split is useful for SLA critical, such as for separating "development" and "production" environments, or when dividing up resources between multiple teams.

Anytime Splits

The total number of splits that can be processed at any moment.

Splits Hours

Refers to the time required to process splits.

Presto Query Signatures

For queries that don’t have a name, the query signature is a concise representation that summarizes the operations performed by the query. The notation used for query signatures is described in Presto Query Signatures.

Metrics

The dashboard includes the following charts:

Your Current Presto Units

The total number of presto units assigned to your account, based on your minimum price plan. 

Max Anytime Split Limit

The total number of split resources you can use at a given moment. Calculated by Presto Units * 300 Anytime Splits.

Monthly Split Hour Limit

The total number of split-hours that your account can use within one month, based on your price plan. Calculated as Presto Units * 15,000 Split-Hrs. 

Max Query Memory Limit

The maximum total memory that can be used on a per query basis, based on your price plan.

Additional Presto Units Needed

The number of additional presto units needed to support either the number of split hours you’re using beyond your limit, or the total memory/query you need for your top memory consuming queries. Calculated as Presto Units * 1 Hr.

This Month’s and Last Month’s Split Hours

Gray Line - Your accumulative split-hours usage within the month.
Blue Line - Represent your daily split-hour usage within the month.
Green Line - Your account’s maximum allowed split-hour usage, based on price plan.

Monthly Total Presto Jobs Ran

Number of Presto Jobs run, by month, over the past 12 months.

Total Presto Hours

Number of Presto Split-Hours consumed, by month, over the past 12 months.

Monthly Over Memory Jobs

Number of jobs that have exceeded your account’s Max Memory Limit.

Presto Query Details

The Presto Query Details table, at the bottom of the dashboard, shows a per-query usage data. Use the table to target your top consuming queries, queries that are unusually resource-intensive.

You can then work to optimize resource usage. Consider whether these queries can be run less frequently or modified to process less data. Or perhaps you can identify inefficient operations like unnecessary sorting or unnecessary columns.

The Presto Query Details table, includes the following columns:

Month

Month and Year that the query ran.

Query Name or Signature

If the query is a saved query, this field shows the query name. Otherwise, it shows a “query hash” created to identify unique ad-hoc, script, or Treasure Workflow based queries.

Last Month Total Split Hours

Total number of split-hours this query consumed last month.

Current Month Split Hours

Total number of split-hours this query consumed up to the present day this month.

Current Month Total Executions

Total number of times that this query or query signature ran up to the present day this month.

Average Split Hours Per Run

Average number of split-hours consumed, per run of that query.

Median Query Duration in Minutes

Median duration, in minutes, the query takes to complete.

Num of Jobs over Allotted Memory

Number of times this query hit the account’s maximum per-query memory limit.

Max Memory Used in MB

This field shows how much memory a query needs to succeed. It is only shown for queries that are exceeding the account’s per-query max memory limit.
The memory requirement is determined by re-running queries that fail due to memory limits. The queries are rerun in a pool that gives the queries much higher memory limits.

Last Job ID

The most recent Treasure Data Job ID run from this query. This is helpful if you want to find out more about the query in question, and the query is from an external system. There is no Treasure Data Saved Query ID but you can use the most recent (last executed) Job ID.