i

Description

Poste d'aiguillage ferroviaire à côté d'un passage à niveau : un point de supervision du trafic.

Source

Hassocks5489 · original page

Licence

CC0 Public domain

Original photo, unretouched. The background is a version of it stylised by the site.

Operations, Monitoring, and Capacity

This theme assesses the ability to keep the BI product running: scheduling and tracking refreshes, failure alerts, capacity monitoring (Capacity Metrics app…

In brief

This theme assesses the ability to keep the BI product running: scheduling and tracking refreshes, failure alerts, capacity monitoring (the Microsoft Fabric/Power BI Capacity Metrics app, activity logs, query logs), runbooks, incident management, service level objectives (SLOs), capacity planning. It complements IND (deliver) and comes before CST (pay fairly).

On this page

1. 1. Why this theme
2. 2. What the theme measures
3. 3. The 5 levels
4. 4. Milestones by level
5. 5. Detailed reading by level
   1. 🟢 Level 1 — Blind
   2. 🟢 Level 2 — Failure alerts
   3. 🟡 Level 3 — Dashboard
   4. 🟡 Level 4 — SLOs and runbooks
   5. 🔴 Level 5 — Anticipated
6. 6. Partial achievement and rating
7. 7. Moving to the next level
8. 8. See also

Level: 🟢→🔴 · Theme code: OPS

1. Why this theme#

A refresh that fails overnight means that in the morning, yesterday's numbers are presented as today's. A saturated capacity slows down or blocks all reports sharing it, including those not responsible (“noisy neighbor”). On Fabric/Premium capacity, consumption is smoothed over time; an unmanaged spike eventually triggers progressive throttling: first 20-second delays on new interactive operations, then their rejection, and finally rejection of all requests (Microsoft policy may change). Without monitoring, incidents are discovered through users.

2. What the theme measures#

• Axis: from users discovering outages to proactive operations with SLOs.
• Key indicators: refresh success rate; time to detect failure (MTTD); time to restore service (MTTR); average and peak capacity usage; observed vs promised freshness.
• Units: %, minutes, % of capacity units (CU).

3. The 5 levels#

4. Milestones by level#

5. Detailed reading by level#

🟢 Level 1 — Blind#

• Signs: “it didn't refresh this morning,” and the email comes from sales leadership.
• Difficulty: 1/5 · Effort: 1/5.

🟢 Level 2 — Failure alerts#

• Signs: notifications sent to a group; history is checked when in doubt. Refreshes are scheduled outside peak hours.
• Benefit: MTTD reduced from several hours to less than one hour.
• Limit: a successful refresh can still load incomplete data (source not finished): failure alerts do not detect false success (TST).
• Difficulty: 2/5 · Effort: 2/5.

🟡 Level 3 — Dashboard#

• Signs: an operations report (model based on activity logs, refresh history, Capacity Metrics) answers questions: how many refreshes failed this week? Which models consume the most? Which hours are saturated?

// Example (Log Analytics, if service logs are sent there — verify based on configuration)
PowerBIDatasetsWorkspace
| where OperationName == "QueryEnd"
| summarize P95_ms = percentile(DurationMs, 95), Nb = count() by ArtifactName, bin(TimeGenerated, 1h)
| order by P95_ms desc

• Benefit: decisions grounded in numbers.
• Limit: more reactive than proactive.
• Difficulty: 3/5 · Effort: 3/5.

🟡 Level 4 — SLOs and runbooks#

• Signs: “the sales report is fresh by 7:00 AM on weekdays, 99% of business days”; the runbook states what to do if the gateway stops responding or a source token expires. After an incident, a blameless review is written.
• Benefit: clear commitment, faster resolution.
• Limit: overly ambitious SLOs without resources = ignored SLOs.
• Difficulty: 4/5 · Effort: 4/5.

🔴 Level 5 — Anticipated#

• Signs: trend alerts (usage growth of +30% in three months), sizing reviewed every quarter, planned capacity pause/resume or resizing (Fabric F capacities; e.g., nighttime/weekend reduction when the business model allows), failure exercises.
• Benefit: no capacity surprises, better cost fit (CST).
• Limit: capacity automation requires fine-grained understanding of smoothing and billing; pausing capacity makes content unavailable.
• Difficulty: 5/5 · Effort: 5/5.

6. Partial achievement and rating#

Common rule. “A partial 5”: trend alerts set on capacity but not on per-report performance (BI-OPS-5.1 = 0.5), quarterly review actually performed (5.2 = 1), manual capacity (5.3 = 0), failure exercise done once (5.4 = 0.5).

7. Moving to the next level#

Possible means (optional):

Pitfalls:

• False positive: 100% refresh success with empty data.
• Goodhart: “short MTTR” achieved by closing tickets without root cause.
• Alert fatigue: too many notifications → no one reads them. Select, route, remove noise.
• Relying on a single point-in-time capacity measure: peaks are short; look at the distribution.

8. See also#

• Related themes: PRF · ETL · IND · CST · GOV
• Other matrices: DL — Observability and Reliability · DL — Deployment and Operations · PD — Operations and Support
• Content: OneLake + SaaS Mobile Apps: architecture · OneLake + SaaS Mobile Apps: Difficulties and Pitfalls · SQL Server in SaaS — architecture · Power BI Reporting Project Checklists

✻ Written by Claude Sonnet 5.5