Insights metrics glossary
Overview
This document provides definitions for all the metrics in CircleCI Insights. You can review metrics in the Insights dashboard as well as the Insights API.
General metrics
General metrics appear across the Insights experience and can refer to different entities, depending on the context. For example, the Runs
metric may refer to a count of workflow executions or jobs, depending on the context.
Term | Definition |
---|---|
Runs | The count of executions in a selected time window. |
Total Duration | The sum of execution times for all workflows in a project. |
P95 Duration | The 95th percentile of execution times for an entity in a selected time window (i.e. 95% of runs completed in the same or less amount of this time). The 95th percentile is a standard measure used to interpret performance data. It provides a measure of max value when outlier or transient values are excluded. |
P50 (median) Duration | The median execution time. Medians are a better measure of central tendency than arithmetic means because they are more resilient to skewness in datasets. |
Total Credits | The sum of credits consumed during execution. |
Success Rate | The percentage of runs that completed successfully, calculated by 100 x (Successful Runs / All Runs)
|
Organization-level metrics
Organization-level metrics allow you to analyze your organization’s performance.
Term | Definition |
---|---|
Workflow Runs | The count of all workflows executions for an organization for the selected projects in the selected time frame. |
Total Workflow Duration | The total execution time of all workflows runs for an organization within the selected projects and time frame. |
Total Credits | The total credits consumed across all selected projects in an organization. |
Overall Success Rate | The percentage of runs that completed successfully across all runs in the selected projects and workflows. (calculated by 100 x (All Successful Runs/ All Runs) ) |
Workflow Metrics
Term | Definition |
---|---|
Time to Recovery | Total time spent in a failed state before a successful execution. |
Mean Time to Recovery (MTTR) | The average time it takes to get back to a successful execution from a failed one (Calculated by total recovery time / # of failures ). If there are no failed builds (and thus no recoveries) in the current time window, the MTTR value will be empty. Empty MTTR values will be treated as 0 when calculating Trends. |
Throughput | The average (mean) number of builds of any status per day. Calculated by Sum of builds/ # of days.
|
Trends
Raw metric values are often difficult to interpret on their own. Trends provide additional context to Insights metrics by presenting a relative benchmark against previous performance. For instance, on the last 7-day view, trends will display the change in value or delta compared to the prior 7-day window.
Please note that trends are available only for 24-hour, 7-day, and 30-day time windows.
Trends are calculated as 100 * (current value - previous value) / prior-value
.
Approximate Trends
For percentile metrics like duration, approximation methods are used to find the prior window benchmarks.
Duration
P95 Duration
For P95 duration, trends data is calculated by using the worst-case performance of the previous time window as a benchmark.
For example, to calculate the last 7-day trend information, Insights selects the worst/largest daily P95 duration of the previous 7 days as a benchmark. For the last 24-hour trends, Insights selects the worst hourly P95 of the previous 24 hours as a benchmark.
P50 Duration
For P50 duration, Trends compares current duration with the median performance of the prior time window. For example, the last 30-day trends for P50 duration are calculated by taking the median of the daily P50 values as a benchmark.
Representations of trend data
This section describes how your trend data may appear across various metrics.
- Green: The metric is trending in the right direction.
- Red: The metric is trending in the wrong direction.
- Grey: A trend in either direction is not right or wrong.
Red and Green are used when describing the Success Rate
, Throughput
and MTTR
metrics. Grey arrows are used when describing Runs
, Duration
and Total Credits
Percentages
Percentages are used to indicates the relative percentage change for a metric in the selected time window compared to the prior window. For instance, if the success rate of a workflow in the last 7 days has increased to 60% from 40% in the prior 7 days, Trends displays the +50% change in the current time window.
Multiples
Multiples are used to indicate large swings in relative change over the selected period.
Arrows
Trend arrows with no values are used to indicate that the previous time window contains zero executions or no data.
No Trend Data
Empty trend values indicate that there has been no change in data between the two periods.
Help make this document better
This guide, as well as the rest of our docs, are open source and available on GitHub. We welcome your contributions.
- Suggest an edit to this page (please read the contributing guide first).
- To report a problem in the documentation, or to submit feedback and comments, please open an issue on GitHub.
- CircleCI is always seeking ways to improve your experience with our platform. If you would like to share feedback, please join our research community.
Need support?
Our support engineers are available to help with service issues, billing, or account related questions, and can help troubleshoot build configurations. Contact our support engineers by opening a ticket.
You can also visit our support site to find support articles, community forums, and training resources.
CircleCI Documentation by CircleCI is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License.