i

Description

Niveau à bulle circulaire dont la bulle centrée dans le cercle rouge indique que tout est d'aplomb.

Source

Dmeranda · original page

Licence

PD Public domain

Original photo, unretouched. The background is a version of it stylised by the site.

Self-Assessment with Maturity Matrices

You will be able to run a team self-assessment, step by step, on one of the four matrices: prepare, collect evidence, rate, calibrate, weight, report back.

In brief

• You will be able to run, step by step, a team self-assessment on one of the four matrices: prepare, collect evidence, rate, calibrate, weight, report back.
• You will have a reusable grid template (Markdown tables) for each matrix, with the real theme codes.
• You will be able to facilitate a 2-hour workshop, avoid score drift (Goodhart's law), and turn the gap between current state and target into a progress plan.
• A complete numerical example based on a fictional six-person BI/datamart team illustrates every calculation, all the way through to text-described radar reporting.

On this page

1. Principles and preparation
2. The step-by-step method
3. Grid templates
   1. Milestone-level grid (single template)
   2. Theme-level grid — PI, IT project
   3. Grid by theme — SD, software development
   4. Grid by theme — DP, data project management
   5. Grid by theme — BI, BI development
4. 2-hour workshop
5. Calibration Between Evaluators
6. Weighting
7. Progress Plan
8. Measurement Gaming and Goodhart’s Law
9. Complete numerical example
   1. Step 1 — One theme in detail: BI-TST (testing and metrics reliability)
   2. Step 2 — The twelve themes
   3. Step 3 — Matrix scores
   4. Step 4 — Targets and gaps (12 months)
   5. Step 5 — What this score does not say
10. Radar reporting described in text
11. AI Contribution
12. See also

Level: 🟢→🔴

Principles and preparation#

🟢 Novice. A self-assessment answers one question only: “where are we really, and which two or three milestones do we want to achieve before next time?” It does not rank people, and it does not produce a grade for management: it is used to decide where to invest. If you have not read the principle of matrices and the rating rule, start there.

Four principles govern the method:

1. Evidence, not statements. A milestone only reaches 1 if the evidence was seen during the workshop. Without evidence, it is capped at 0.5.
2. A precise scope. One team, one product, one period. “All of IT” is not a scope.
3. Compare to yourselves. The assessment is repeated every 6 to 12 months, always in the same way.
4. Assess each fact only once, in the matrix that owns it (see overlaps).

🟡 Practice — choosing your matrix and scope.

Preparation (one week before). Appoint a facilitator who does not rate their own scope (or a peer from another team), assemble an evidence package (links to pipelines, test reports, logs, repositories, capacity dashboards), and invite 4 to 8 people covering the roles (development, operations, business, steering). Send the selected theme pages in advance.

The step-by-step method#

🟡 Practice. Seven steps, from preparation to follow-up.

Theme calculation, in pseudo-algorithm (faithful to the common rule):

for N from 1 to 5:
    J(1..N) = all milestones from levels 1 to N
    if average(progress of J(1..N)) >= 0.8 AND min(progress of J(1..N)) > 0 :
        N_acquired = N
    else: stop
theme_score = N_acquired + average(progress of milestones at level N_acquired+1) × 0.99

The same calculation in PowerShell, useful for automating consolidation of a grid exported to CSV (one row per milestone, columns Theme, Level, Progress):

function Get-ScoreTheme {
    param([Parameter(Mandatory)][object[]]$Jalons)   # objets avec Niveau (1..5) et Avancement (0, 0.5, 1)
    $acquis = 0
    foreach ($n in 1..5) {
        $jusquaN = $Jalons | Where-Object { [int]$_.Niveau -le $n }
        if (-not $jusquaN) { break }
        $moy = ($jusquaN | Measure-Object -Property Avancement -Average).Average
        $min = ($jusquaN | Measure-Object -Property Avancement -Minimum).Minimum
        if ($moy -ge 0.8 -and $min -gt 0) { $acquis = $n } else { break }
    }
    $suivants = $Jalons | Where-Object { [int]$_.Niveau -eq ($acquis + 1) }
    $partiel  = if ($suivants) { ($suivants | Measure-Object -Property Avancement -Average).Average * 0.99 } else { 0 }
    [math]::Round($acquis + $partiel, 2)
}
# Avancement doit être lu en décimal invariant : [double]::Parse($txt, [cultureinfo]::InvariantCulture)

Grid templates#

🟡 Practice. Two templates: the milestone-level grid (the workshop work), identical for all matrices, and the theme-level grid (the summary), with each matrix's codes. Copy them into a workbook or a Markdown file tracked in your repository, alongside the assessment history.

Milestone-level grid (single template)#

IDs and labels can be read on the theme page (section “Milestones by level”); only the last row for each theme below must be filled in.

Theme result, to be completed below the table: cumulative average at levels 1…5, milestone(s) at 0, N_acquired, average at next level, score.

Theme-level grid — PI, IT project#

Grid by theme — SD, software development#

Grid by theme — DP, data project management#

Grid by theme — BI, BI development#

2-hour workshop#

🟡 Hands-on. Framework for a team of 4 to 8 people, on a 12-theme matrix (only the 4 to 6 themes selected during the scoping step are rated in detail; the others are rated “roughly” in one minute).

Facilitation rules:

• Simultaneous voting (cards, blind round-robin) before discussion, to avoid anchoring on the most senior person’s opinion.
• Disagreements are data: a half-point gap on a milestone is resolved with evidence; a one-point gap signals that two people are not talking about the same scope.
• Parking lot: anything that looks like a remediation effort is recorded for the “Decide” step, not handled during the session.
• Prohibited: linking scores to individual evaluation, projecting scores from other teams, discussing tools before rating (milestones focus on outcomes, not products).

Calibration Between Evaluators#

🔴 Expert. Two honest evaluators differ by half a point on a milestone: this is the residual subjectivity stated in each matrix README. Calibration makes it stable and comparable from one cycle to the next.

1. Anchor benchmark. Before the first evaluation, jointly choose three milestones for which you agree on what 0, 0.5, and 1 mean, with a concrete example of each (“0: no evidence; 0.5: the practice exists but has not been applied to the last two batches; 1: applied to every delivery of the quarter, evidence observed”). Record them; they are the reference for subsequent cycles.
2. Double rating. Two people rate the same theme separately (one from the team, the other from a neighboring team), then compare. The average gap observed should narrow from one cycle to the next.
3. Evidence sampling. For each theme, thoroughly verify one random piece of evidence (trace back from the dashboard to the event). One false piece of evidence downgrades all milestones in the theme rated on that same basis.
4. Doubt rule. In case of persistent disagreement, keep the lower score and note the doubt; do not average two opposing opinions.
5. Traceability. Keep the grid for each cycle (date, participants, evidence): history is the only legitimate basis for comparison.

Useful indicator: the gap rate — share of milestones for which two evaluators differ by at least 0.5. A fair order of magnitude for a first cycle is one milestone out of four or five (expert benchmark, not measured by a survey); after two calibration cycles, the reasonable goal is to cut it in half.

Weighting#

🟡 Practical. The matrix score is a weighted average: default weight is 1, editable. Weighting is used to reflect the risk and stakes of the context, not to make the result look better.

Good usage rules:

• Set weights before rating, never after: changing a weight afterward to improve the average is a form of cheating yourself.
• Document each weight with one sentence of justification (“Weight” column of the grid + comment).
• Compare with constant weights from one cycle to the next; if weights change, publish both calculations.
• Weight 0 = out of scope, must be justified, not used to hide a weakness.

The formula: score_matrice = Σ(poids × score_thème) / Σ(poids).

Progress Plan#

🟡 Practical. Evaluation has value only if it leads to a few well-chosen decisions. Five-step method.

1. Gap = target − score, theme by theme. Sort by descending gap, then weight by the weight.
2. Filter by dependencies. Do not address a theme whose prerequisite theme is insufficient (see dependency graphs in each README: for example tests before continuous delivery). Prioritize the upstream link.
3. Choose milestones, not themes. Keep 2 to 3 milestones from level N_acquired+1 of 2 to 3 themes. A milestone is a verifiable deliverable; a theme is a domain.
4. Estimate cost and benefit using the difficulty and effort ratings /5 given on the theme page (expert assessments, to be adapted). Favor milestones with low effort and high impact.
5. Commit: one owner, one deadline, expected evidence, one mid-point review date.

The “possible means” listed on each theme page are optional: a milestone is satisfied by any means that produces the evidence. For the practices themselves, see for example CI/CD: Continuous Integration and Delivery, CI/CD for Data: Databases, Datamarts, Pipelines, and Reports, Branching Strategies and Project Metrics and Reporting.

Measurement Gaming and Goodhart’s Law#

🔴 Expert. Goodhart’s law (when a measure becomes a target, it stops being a good measure) is the main threat to a maturity framework. Each matrix mentions it; here are typical forms of gaming and their countermeasures.

Defense principles:

1. Evidence before score, evidence sampling at each cycle.
2. Outcome before practice: when a practice is checked but the outcome remains poor (incidents, disputed numbers), the outcome prevails and the milestone is reopened.
3. Adverse indicators: pair a volume indicator with a quality indicator (deployment frequency with failure rate; number of tests with escaped defects).
4. Rotation of evaluators and samples.
5. Score governance: the score is never a contractual objective; only the milestone plan is binding.

Complete numerical example#

🟡 Practical. Fictitious and anonymous context: “the Atlas team,” six people (four BI/datamart developers, one operator, one business point person), maintaining a SQL Server datamart fed by a field sales force application and about fifteen multilingual Power BI reports, versioned in TFVC and delivered as installation kits to about a dozen clients. Selected matrix: BI (12 themes). Period: the last twelve months. 2-hour workshop, with a peer from another team as observer.

Step 1 — One theme in detail: BI-TST (testing and metrics reliability)#

The milestones are those from the official theme page (levels 1 to 3; levels 4 and 5 are not included in the calculation as long as level 3 has not been achieved). Only the progress ratings and the observed evidence are fictitious.

Calculation:

• Levels 1 and 2: sum = 3 + (1 + 0,5 + 0,5 + 1) = 6 for 7 milestones → average = 0,857 ≥ 0,8, no zeroes → level 2 achieved.
• Levels 1 to 3: sum = 6 + (0 + 0,5 + 0,5 + 0,5 + 0,5) = 8 for 12 milestones → average = 0,667 < 0,8 and one milestone at 0 → level 3 not achieved. N_acquired = 2.
• Average of level 3 milestones = 2 / 5 = 0,4 ; × 0,99 = 0,396.
• BI-TST score = 2 + 0,396 ≈ 2,40.

In what follows, the scores for the other themes are given as calculated by the workshop in the same way (illustrative values, rounded to 0,1).

Step 2 — The twelve themes#

The weight 2 on RLS and TST is decided before rating: reports are delivered to clients whose commercial data is confidential, and an incorrect metric is costly.

Step 3 — Matrix scores#

• Simple sum of the twelve scores (using 2,4 for TST): 3,4 + 3,1 + 2,2 + 2,6 + 2,8 + 2,5 + 1,7 + 2,4 + 1,5 + 1,4 + 2,1 + 1,0 = 26,7.
• Unweighted average: 26,7 / 12 = 2,23.
• Weighted average: add one more instance of RLS (2,5) and TST (2,4): weighted sum = 26,7 + 2,5 + 2,4 = 31,6 ; sum of weights = 12 + 2 = 14 ; score = 31,6 / 14 = 2,26.

Reading with the “2 – 3” scale of the BI matrix: common practices within the team, repeatable deliveries, initial controls; quality still depends on vigilance. More useful than the average: the gap of 2,4 points between MOD (3,4) and CST (1,0), and downstream weakness (IND 1,7, OPS 1,4) despite a rather solid upstream.

Step 4 — Targets and gaps (12 months)#

Targets decided in workshop based on risk (kits delivered to about a dozen clients):

Selected milestones (three, no more): (a) put test DAX queries in repository with independent SQL expected values (BI-TST-3.1, 3.2), then run them for every delivery (BI-TST-4.1), (b) deploy by pipeline to a test workspace (BI-IND-3.x), (c) an operations dashboard for refreshes (BI-OPS-3.x). Expert estimate: about three months at two days per week for one person; real capacity is the team's, not the spreadsheet's.

Projection if TST (3,2), IND (3,0), OPS (2,5), and RLS (3,0) reach their targets: gains are +0,8, +1,3, +1,1, and +0,5. Simple sum = 26,7 + 3,7 = 30,4, for an average of 2,53. Weighted sum = 31,6 + 0,8 × 2 + 0,5 × 2 + 1,3 + 1,1 = 36,6, divided by 14: 2,61. Expected weighted gain: 2,61 − 2,26 = +0,35. An overall score moving by 0,3 to 0,5 per year is a realistic team pace (expert benchmark, to be adjusted).

Step 5 — What this score does not say#

• The score does not tell you whether the metric is correct today: reconciliation (milestones in BI-TST, PD-REC) tells you that.
• The team has a CST of 1,0 and a weight of 1: if capacity becomes a financial issue, the weight will move to 1,5 in the next cycle (publishing both calculations).
• The profile is consistent: weak TST slows IND, which slows OPS. It's the upstream theme to address.

Radar reporting described in text#

🟡 Practical. A radar chart (spider diagram) shows the team's profile: one spoke per theme, a distance from center proportional to the score. It can be described without an image, making it readable in a report, a wiki, or a pipeline stage.

Reading the Atlas team radar (12 spokes, from 0 at center to 5 at edge, two blocks per score point):

MOD  ■■■■■■■  3,4   ← high point
ETL  ■■■■■■   3,1
DAX  ■■■■■■   2,8
VIZ  ■■■■■    2,6
RLS  ■■■■■    2,5   (weight 2)
TST  ■■■■■    2,4   (weight 2)
PRF  ■■■■     2,2
ACC  ■■■■     2,1
IND  ■■■      1,7
GOV  ■■■      1,5
OPS  ■■■      1,4
CST  ■■       1,0   ← low point

Reporting phrases, from general to specific:

1. Shape: a “pear-shaped” polygon, bulging on the modeling side (MOD, ETL, DAX) and narrowed on the operations side (IND, GOV, OPS, CST). Typical of a team that invested in building the product before investing in delivery and operations.
2. Average: 2,23 unweighted, 2,26 weighted; to be read in the “2 – 3” range.
3. Links to address: TST (upstream of IND) then IND then OPS.
4. Commitments: three milestones, one owner each, mid-course review.
5. Caution reminder: team assessment based on evidence, comparable to itself in twelve months, not comparable to another team.

To produce the image itself, any charting tool works (workbook, radar visual in a Power BI report): the input needed is a two-column table (theme, score). The text description remains the reference version of the report anyway.

AI Contribution#

Expert assessments, context: team of 5 to 10 people.

Example prompt for the report:

Here is the team's consolidated grid (topic, code, score, weight, comment).
Write a 5-sentence report: profile shape, weighted average, three links
to address while respecting upstream/downstream dependencies, commitments. Do not cite any practice
absent from the grid. Flag any inconsistency between scores of dependent topics.

Governance framework (evidence confidentiality, traceability, sign-off): Cross-Cutting Overview — AI Contribution.

See also#

• Matrices: Maturity Matrices — Principles, Anatomy, and Comparison · PI · DL · PD · BI.
• Methods: Project Management Fundamentals · Scrum in Practice · Project Management Pitfalls and Best Practices (Cross-Cutting Summary).
• Tools: Project Metrics and Reporting · Azure DevOps / TFS.
• Reporting: Power BI Reporting Project Checklists · Cross-Cutting Power BI Best Practices.
• Framework: RATING · Glossary · Bibliography and Resources.

✻ Written by Claude Sonnet 5.5