The Presto Utilization dashboard show you your recent and historical use of Presto resources. You can use the dashboard to determine if you risk exceeding your monthly plan limits.
|The Presto Utilization page is available upon request for most customers. However, availability for customers in Japan is still pending and will occur soon.|
From the console, Arm Treasure Data owners and administrators, click Admin > Presto Utilization to access the dashboard.
Presto Terms and Concepts
Understanding Presto concepts and terminology is useful when assessing metrics that are presented in the dashboard.
The following terms are used in more than one metric:
- Sections of a larger data set, including tables and intermediate data during query processing. A split is useful for SLA critical, such as for separating "development" and "production" environments, or when dividing up resources between multiple teams.
- Anytime Splits
- The total number of splits that can be processed at any moment.
- Splits Hours
- Refers to the time required to process splits.
- Presto Query Signatures
- For queries that don’t have a name, the query signature is a concise representation that summarizes the operations performed by the query. The notation used for query signatures is described in Presto Query Signatures.
The dashboard includes the following charts:
- Your Current Presto Units
- The total number of presto units assigned to your account, based on your minimum price plan.
- Max Anytime Split Limit
- The total number of split resources you can use at a given moment. Calculated by Presto Units * 300 Anytime Splits.
- Monthly Split Hour Limit
- The total number of split-hours that your account can use within one month, based on your price plan. Calculated as Presto Units * 15,000 Split-Hrs.
- Max Query Memory Limit
- The maximum total memory that can be used on a per query basis, based on your price plan.
- Additional Presto Units Needed
- The number of additional presto units needed to support either the number of split hours you’re using beyond your limit, or the total memory/query you need for your top memory consuming queries. Calculated as Presto Units * 1 Hr.
- This Month’s and Last Month’s Split Hours
- Gray Line - Your accumulative split-hours usage within the month.
Blue Line - Represent your daily split-hour usage within the month.
Green Line - Your account’s maximum allowed split-hour usage, based on price plan.
- Monthly Total Presto Jobs Ran
- Number of Presto Jobs run, by month, over the past 12 months.
- Total Presto Hours
- Number of Presto Split-Hours consumed, by month, over the past 12 months.
- Monthly Over Memory Jobs
- Number of jobs that have exceeded your account’s Max Memory Limit.
Presto Query Details
The Presto Query Details table, at the bottom of the dashboard, shows a per-query usage data. Use the table to target your top consuming queries, queries that are unusually resource-intensive.
You can then work to optimize resource usage. Consider whether these queries can be run less frequently or modified to process less data. Or perhaps you can identify inefficient operations like unnecessary sorting or unnecessary columns.
The Presto Query Details table, includes the following columns:
- Month and Year that the query ran.
- Query Name or Signature
- If the query is a saved query, this field shows the query name. Otherwise, it shows a “query hash” created to identify unique ad-hoc, script, or Treasure Workflow based queries.
- Last Month Total Split Hours
- Total number of split-hours this query consumed last month.
- Current Month Split Hours
- Total number of split-hours this query consumed up to the present day this month.
- Current Month Total Executions
- Total number of times that this query or query signature ran up to the present day this month.
- Average Split Hours Per Run
- Average number of split-hours consumed, per run of that query.
- Median Query Duration in Minutes
- Median duration, in minutes, the query takes to complete.
- Num of Jobs over Allotted Memory
- Number of times this query hit the account’s maximum per-query memory limit.
- Max Memory Used in MB
- This field shows how much memory a query needs to succeed. It is only shown for queries that are exceeding the account’s per-query max memory limit.
The memory requirement is determined by re-running queries that fail due to memory limits. The queries are rerun in a pool that gives the queries much higher memory limits.
- Last Job ID
- The most recent Treasure Data Job ID run from this query. This is helpful if you want to find out more about the query in question, and the query is from an external system. There is no Treasure Data Saved Query ID but you can use the most recent (last executed) Job ID.