i

Description

Tableau de bord d’un petit avion, avec ses cadrans, ses manettes et ses rangées de commutateurs.

Source

U.S. Fish and Wildlife Service · original page

Licence

PD Public domain

Original photo, unretouched. The background is a version of it stylised by the site.

Project Metrics and Reporting

You will learn how to calculate and interpret velocity, burndown/burnup, cumulative flow diagram, lead time, cycle time, throughput, WIP, and how they relate (Little's Law).

In brief

• You will know how to calculate and interpret velocity, burndown/burnup, cumulative flow diagram, lead time, cycle time, throughput, WIP, and how they relate (Little's Law).
• You will understand DORA metrics and their limits outside a purely software context.
• You will know how to measure estimation quality and backlog health.
• You will have metrics specific to data projects: freshness, data incidents, rejection rate, acceptance testing gaps.
• You will know how to avoid pitfalls from Goodhart's Law, and build a Power BI dashboard on Azure DevOps.

Rating context: team of 5 to 15 people, data/BI. Ratings /5 are assessments (see RATING).

On this page

1. 🟢 Novice — Why measure, and what
   1. 🟢 Basic definitions
2. 🟡 Hands-on — Velocity, burndown, burnup
   1. Velocity
   2. Sprint burndown
   3. Burnup (preferred)
3. 🟡 Hands-on — Flow: CFD, lead/cycle time, throughput, WIP
   1. Cumulative flow diagram (CFD)
   2. Little's Law
   3. Lead time and cycle time: percentiles, not average
   4. Age of in-progress items (aging WIP)
   5. Probabilistic forecasting (Monte Carlo)
4. 🟡 Hands-on — DORA and delivery metrics
5. 🟡 Practice — Quality of Estimates
6. 🟡 Practice — Backlog Health
7. 🔴 Expert — Metrics Specific to Data Projects
8. 🔴 Expert — Goodhart's Law and Perverse Uses
9. 🔴 Expert — Power BI Dashboards on Azure DevOps
10. Metrics — Which Metrics Are Worth Their Cost?
11. 🤖 AI Contribution
12. See also

Level: 🟢→🔴

🟢 Novice — Why measure, and what#

Measuring is for deciding and learning, not monitoring. A good metric answers a question the team is actually asking: “are we delivering predictably?”, “where is work waiting?”, “is quality dropping?”. Before choosing an indicator, write down the question and the decision the answer can change; without a decision, don't measure.

Three categories:

🟢 Basic definitions#

🟡 Hands-on — Velocity, burndown, burnup#

Velocity#

Sprint n velocity = Σ story points “Done” at close of sprint n
Reference velocity = average (or median) of the last 3 to 5 sprints
Range = [min ; max] of recent sprints, more meaningful than the average

Velocity is a local planning tool for the team, not a performance indicator, nor a way to compare two teams (points don't have the same unit from one team to another). It becomes reliable after 4 to 6 stable sprints (same team, same duration).

Sprint burndown#

Curve of remaining work (hours or points) by sprint day, compared with the ideal line. Read it as: a cliff at the end of the sprint = work not broken down or late updates; a long flat line = blockage or tasks that are too large; an upward move = scope increase.

Burnup (preferred)#

Two curves: cumulative completed work and total scope. Unlike burndown, scope is visible: a rising scope line explains why the finish moves out.

        Points
  200 ┤                          ·· scope (rising: added)
  150 ┤                    ····
  100 ┤            ·····  /──── completed (cumulative)
   50 ┤      /─────
    0 ┼──┴───────────────────────────→ sprints

🟡 Hands-on — Flow: CFD, lead/cycle time, throughput, WIP#

Cumulative flow diagram (CFD)#

Each band represents a state (Backlog, In Progress, In Acceptance Testing, Done); a band's height at a date = number of items in that state. How to read it:

Little's Law#

Average lead time = Average WIP / Average throughput        (steady state)
Example: WIP = 12 items, throughput = 3 items/week → average lead time = 4 weeks
To divide lead time by 2 at constant throughput: divide WIP by 2.

This is the core argument for WIP limits: reducing work in progress speeds up delivery without anyone working faster. See Agile and Lean Methods (Manifesto, Kanban, XP, Lean, Shape Up, DSDM, SAFe, DevOps…).

Lead time and cycle time: percentiles, not average#

Durations are asymmetrical (heavy tail); the average is misleading. Use percentiles:

The scatterplot diagram (scatterplot, one point per completed item) makes these thresholds visible.

Age of in-progress items (aging WIP)#

For each in-progress item: age = today − start date. Compare with the 85% range: anything above it deserves action now (more useful than lead time, which only exists afterward).

Probabilistic forecasting (Monte Carlo)#

Rather than extrapolating an average, resample historical throughput to estimate a date range. Mini PowerShell script for illustration:

# Forecast: how many weeks to complete 40 items? (historical weekly throughput)
$debits = 3,5,2,4,6,3,2,5,4,3          # items completed per week, 10 weeks
$reste = 40; $essais = 10000; $res = @()
$rnd = New-Object System.Random 42
1..$essais | ForEach-Object {
    $faits = 0; $sem = 0
    while ($faits -lt $reste) { $faits += $debits[$rnd.Next(0,$debits.Count)]; $sem++ }
    $res += $sem
}
$tri = $res | Sort-Object
"P50 = {0} wk ; P85 = {1} wk ; P95 = {2} wk" -f $tri[[int]($essais*0.50)], $tri[[int]($essais*0.85)], $tri[[int]($essais*0.95)]

🟡 Hands-on — DORA and delivery metrics#

DORA metrics (the DevOps Research and Assessment research program, popularized by the book Accelerate by Nicole Forsgren, Jez Humble, and Gene Kim, 2018) measure software delivery performance. The four historical metrics:

DORA has since evolved this metric set: in 2023, recovery time was redefined (recovery after a failed deployment, not after any outage); since the 2024 report, DORA includes five metrics, with deployment rework rate (deployment rework rate: share of unplanned deployments following a production incident), split between throughput (lead time, frequency, recovery) and stability (failure rate, rework rate). DORA also warns against using these metrics as targets and against comparisons across very different applications.

🟡 Practice — Quality of Estimates#

An estimate is a bet with a margin. Measuring its quality helps improve it without turning it into absolute truth.

🟡 Practice — Backlog Health#

🔴 Expert — Metrics Specific to Data Projects#

A datamart or reporting project has measurement objects that software metrics ignore. See also data project management (matrix).

T-SQL example for measuring freshness and rejection rate from a log table (conventional, adapt to actual names):

-- Fraîcheur par table cible (en heures) et dépassement du seuil
SELECT  j.Table_Cible,
        MAX(j.Date_Fin_Chargement)                                   AS Dernier_Chargement,
        DATEDIFF(HOUR, MAX(j.Date_Fin_Chargement), SYSDATETIME())    AS Age_Heures,
        CASE WHEN DATEDIFF(HOUR, MAX(j.Date_Fin_Chargement), SYSDATETIME()) > s.Seuil_Heures
             THEN 1 ELSE 0 END                                        AS Hors_Seuil
FROM    [dbo].[Journal_Chargement]  AS j WITH (NOLOCK)
JOIN    [dbo].[Seuil_Fraicheur]     AS s WITH (NOLOCK) ON s.Table_Cible = j.Table_Cible
WHERE   j.Statut = 'OK'
GROUP BY j.Table_Cible, s.Seuil_Heures;

-- Taux de rejet par source et par jour
SELECT  CAST(j.Date_Debut AS date)                         AS Jour,
        j.Source,
        SUM(j.Lignes_Rejetees) * 1.0 / NULLIF(SUM(j.Lignes_Lues), 0) AS Taux_Rejet
FROM    [dbo].[Journal_Chargement] AS j WITH (NOLOCK)
GROUP BY CAST(j.Date_Debut AS date), j.Source;

🔴 Expert — Goodhart's Law and Perverse Uses#

Defense rules: (1) measure in pairs (speed + quality), (2) display the trend, (3) keep metrics for the team first, (4) review regularly: remove any metric that no longer supports a decision, (5) distinguish diagnostic metrics (explored) and objectives (rare, few in number).

🔴 Expert — Power BI Dashboards on Azure DevOps#

Source: Analytics OData feed (see Azure DevOps Server (Formerly TFS) In Depth). Recommended model: WorkItemSnapshot (daily flow fact), WorkItem (current), Iteration, Area, User, Date.

Useful pages: (1) Current sprint (burndown, capacity, blocked), (2) Flow (CFD, WIP, item age, cycle time scatterplot), (3) Quality (open bugs, aging, escaped defects, test cases), (4) Release / burnup (scope vs completed), (5) Backlog health.

Typical DAX measures:

WIP actuel =
CALCULATE ( DISTINCTCOUNT ( WorkItem[WorkItemId] ),
            WorkItem[State] IN { "Active", "In Progress", "Committed" } )

Throughput hebdo =
CALCULATE ( DISTINCTCOUNT ( WorkItem[WorkItemId] ),
            WorkItem[CompletedDate] <> BLANK(),
            DATESINPERIOD ( 'Date'[Date], MAX ( 'Date'[Date] ), -7, DAY ) )

Cycle time P85 (jours) =
PERCENTILEX.INC ( FILTER ( WorkItem, NOT ISBLANK ( WorkItem[CycleTimeDays] ) ),
                  WorkItem[CycleTimeDays], 0.85 )

Taux d'engagement tenu =
DIVIDE ( [Points terminés du sprint], [Points engagés au démarrage] )

Design best practices, detailed in Cross-Cutting Power BI Best Practices: a star schema model, no import of all revisions, explicit measures, a clean theme, event annotations (team change, vacations) to explain breaks.

Metrics — Which Metrics Are Worth Their Cost?#

🤖 AI Contribution#

See 11 — AI in Planning and Version Control Tools.

See also#

• Overview · Azure DevOps Server (Formerly TFS) In Depth · Jira In Depth and Alternatives · Comparison
• Scrum in Practice · Scrum and data · CI/CD
• Data Quality (matrix) · Governance and Steering (GOU) (matrix) · Glossary

✻ Written by Claude Sonnet 5.5