Matrix: Back office and tools

How the product is administered: accounts and rights, settings, per-customer customization, logs, monitoring, support, automation.

≈ 11 min readReading

In brief

8 dimensions, levels 0 to 5, 112 criteria.
Tool: run an assessment by choosing "Back office and tools".
Levels and criteria: original synthesis, references cited at the bottom of the page.

On this page

Overview
Accounts and rights
1. Accounts and roles
2. Entitlement review
Settings and customization
1. Application settings
2. Customization by customer or group
Traceability and support
1. Administration and audit logs
2. Monitoring and administration dashboards
Administration tooling
1. Diagnostic and support tools
2. Administration automation
Sources and verification

Scope

The product's administration and support tools: account and role management, entitlement review, application settings, customization by customer or group, administration logs, monitoring, diagnostics, administration scripts and interfaces. Deploying these tools belongs to the "Deployments" theme, day-to-day running to the "Operations" theme.

8 dimensions, six levels (0 to 5) and 112 observable criteria. To place your product: open the assessment for this theme. The scoring rules are here.

Overview#

Each cell is the level's signature for the dimension.

Dimension	0 · Handcrafted	1 · Locally tooled	2 · Shared	3 · Industrialized	4 · Integrated and governed	5 · Autonomous
Accounts and roles	Accounts created by hand, rights assigned case by case, shared accounts possible.	Named accounts.	Rights by role.	Roles governed.	Rights tied to the organization.	Rights automated.
Entitlement review	No review: rights are never removed.	One-off review.	Periodic review.	Review by managers.	Review supported by tools.	Continuous review.
Application settings	Settings written in the code or in files specific to each workstation.	Settings grouped.	Configuration separated from code.	Configuration per environment.	Settings administrable.	Configuration steered.
Customization by customer or group	One copy of the product per customer, modified by hand.	Variants identified.	Standard and specific separated.	Inheritance and layers.	Customization governed.	Customization steered by the customer.
Administration and audit logs	No trace of administration actions.	Partial traces.	Action log.	Protected log.	Log put to use.	Anomaly detection.
Monitoring and administration dashboards	The state of the product is found out by querying the databases directly.	Manual checks.	Status view.	View by customer and by environment.	Usage and health indicators.	Predictive monitoring.
Diagnostic and support tools	Diagnosis requires the author of the code and full access to the database.	Diagnostics shared.	Diagnostic tools.	Tiered support.	Resolution tooling.	Proactive support.
Administration automation	Operations done by hand in interfaces, from memory.	Personal scripts.	Shared, versioned scripts.	Replayable scripts.	Administration interface.	Declarative administration.

Link with the overall model

Dimensions of the overall model that are most relevant: Access and entitlements, Security and compliance, Support and observability and Project steering and roadmap.

Accounts and rights#

Accounts and roles#

How are accounts created and rights assigned?

Level	Signature	Criteria to meet
0 · Handcrafted	Accounts created by hand, rights assigned case by case, shared accounts possible.	Starting point: no criteria.
1 · Locally tooled	Named accounts.	Each person has a named account. The list of accounts and their rights can be consulted.
2 · Shared	Rights by role.	Rights are assigned by role, not person by person. Roles are described (scope, rights, owner). An administrator can create an account and assign it a role without technical intervention.
3 · Industrialized	Roles governed.	A person can hold several roles, and incompatible combinations are forbidden. Rights requests follow a workflow with approval by the manager. Changes to an account's rights are traced.
4 · Integrated and governed	Rights tied to the organization.	Rights derive from the organization (function, team, scope) and are updated along with it. The accounts of a person who leaves or changes position are handled within a defined time. Technical accounts and service accounts are distinguished and have an owner.
5 · Autonomous	Rights automated.	Accounts and rights are created and removed automatically from the people directory. Temporary rights expire on their own. An unusual assignment of rights is detected and flagged.

Effect on the user

Everyone has exactly the rights of their function, and assigning them does not require a developer.

Entitlement review#

Is it checked that the rights granted are still justified?

Level	Signature	Criteria to meet
0 · Handcrafted	No review: rights are never removed.	Starting point: no criteria.
1 · Locally tooled	One-off review.	A review of rights is carried out on the occasion of an incident or an audit. Unused accounts are spotted.
2 · Shared	Periodic review.	A review of rights is carried out at fixed intervals. The result of the review is kept with its date. Unjustified rights are removed at the end of the review.
3 · Industrialized	Review by managers.	Each manager confirms the rights within their scope. The review also covers the broadest rights (administration, access to all data). Accounts inactive for a defined period are deactivated.
4 · Integrated and governed	Review supported by tools.	A report presents each person's effective rights, including inherited rights. The review is run in a tool that keeps evidence of each decision. Gaps between theoretical rights and effective rights are looked for.
5 · Autonomous	Continuous review.	Rights not used for a defined period are automatically proposed for removal. Breaches of segregation-of-duties rules are detected continuously. Review results feed a tracked indicator.

Effect on the user

Rights do not pile up over time: those that are no longer needed disappear.

Settings and customization#

Application settings#

Where are the product's settings, and can they be changed without modifying the product?

Level	Signature	Criteria to meet
0 · Handcrafted	Settings written in the code or in files specific to each workstation.	Starting point: no criteria.
1 · Locally tooled	Settings grouped.	Settings are grouped in a limited number of files or tables. The list of settings is known.
2 · Shared	Configuration separated from code.	Configuration is separated from the code and can change without a new release. Each setting is described (role, permitted values, default value). Secrets (passwords, keys) are stored separately, never in the versioned configuration.
3 · Industrialized	Configuration per environment.	Each environment has its own configuration, declared and versioned. An invalid setting is rejected with a clear message. Changing a setting is traced: who, when, old and new value.
4 · Integrated and governed	Settings administrable.	A screen or an interface lets authorized administrators adjust settings. Sensitive settings require approval before being applied. The actual configuration of an environment can be compared with the expected one.
5 · Autonomous	Configuration steered.	Configuration is deployed automatically from a versioned description. Configuration drift is detected and corrected or flagged automatically. The effect of a setting is simulated before it is applied.

Effect on the user

Adapting the product to a context is a matter of adjusting settings, not development; and it is known who adjusted what.

Customization by customer or group#

Can the product be adapted to a customer or a group without duplicating it?

Level	Signature	Criteria to meet
0 · Handcrafted	One copy of the product per customer, modified by hand.	Starting point: no criteria.
1 · Locally tooled	Variants identified.	Differences between customers are listed. Variants are isolated from the rest of the code.
2 · Shared	Standard and specific separated.	Standard and specific are separated: an element is one or the other. A customer's specific elements are never overwritten by an update of the standard. Specific elements are described in a single place per customer.
3 · Industrialized	Inheritance and layers.	A specific value replaces a standard value according to a written priority rule (standard, customized, project). Differences between a customer and the standard can be listed automatically. A customer's customizations are replayed after each version upgrade.
4 · Integrated and governed	Customization governed.	Each customization has an owner and a reason. A frequent customization is considered for inclusion in the standard. The effect of a version upgrade on customizations is assessed before installation.
5 · Autonomous	Customization steered by the customer.	A customer's administrators can adapt for themselves what is open to them, within defined limits. Customizations are tested automatically with each new version. Each customer's differences from the standard are tracked as a debt to be reduced.

Effect on the user

Each customer has what is specific to them, without losing the improvements of the standard product.

Traceability and support#

Administration and audit logs#

Can it be traced who did what on the product, and are these traces reliable?

Level	Signature	Criteria to meet
0 · Handcrafted	No trace of administration actions.	Starting point: no criteria.
1 · Locally tooled	Partial traces.	Some administration actions leave a trace that can be consulted. Each trace carries at least the date and the author.
2 · Shared	Action log.	All administration actions (account creation, rights changes, setting changes) are logged. Failed logins and access denials are logged. The log contains no password, no secret and no unnecessary personal data.
3 · Industrialized	Protected log.	The log is protected against modification and deletion. Access to the log is restricted and itself recorded. The log retention period is set and enforced.
4 · Integrated and governed	Log put to use.	The log can be queried by criteria (person, object, period). Important events (access to sensitive data, privileged actions) are flagged. The log is centralized with those of the other components.
5 · Autonomous	Anomaly detection.	Unusual behavior (hours, volumes, locations) is detected automatically. An alert triggers a documented response procedure. The integrity of the log is verified automatically.

Effect on the user

After an incident or a question, the facts can be retrieved; there is no need to guess.

Monitoring and administration dashboards#

Do administrators see the state of the product without querying the database?

Level	Signature	Criteria to meet
0 · Handcrafted	The state of the product is found out by querying the databases directly.	Starting point: no criteria.
1 · Locally tooled	Manual checks.	Check queries or scripts exist and are shared. The state of the processes can be read without going through the process's author.
2 · Shared	Status view.	A screen or a report gives the state of processes, accounts and space used. The main anomalies are highlighted. Administrators are notified of an incident without having to consult the screen.
3 · Industrialized	View by customer and by environment.	The status view distinguishes customers and environments. The view is accessible to support teams with appropriate rights. The status history allows comparison with previous days.
4 · Integrated and governed	Usage and health indicators.	Usage indicators (logins, reports viewed, active accounts) are available. Alert thresholds are defined for health indicators. The status view also covers the components of the environment (services, platforms, capacities).
5 · Autonomous	Predictive monitoring.	A likely drift (saturation, delay, failure) is announced before it occurs. Simple corrective actions are triggered automatically from monitoring. Health indicators feed the planning of changes.

Effect on the user

The state of the product can be read at a glance, and a drift is seen before it is felt.

Administration tooling#

Diagnostic and support tools#

Does support have the means to understand a problem without unlimited access?

Level	Signature	Criteria to meet
0 · Handcrafted	Diagnosis requires the author of the code and full access to the database.	Starting point: no criteria.
1 · Locally tooled	Diagnostics shared.	Diagnostic procedures are written for the most frequent incidents. Diagnostic queries are kept and shared.
2 · Shared	Diagnostic tools.	Tools make it possible to reproduce a view or a calculation as a given user sees it. Diagnostic access is read-only by default. A support request is recorded with its context (version, customer, environment).
3 · Industrialized	Tiered support.	Support is organized in tiers, with rules for moving from one to the next. Exceptional access to data is time-limited and logged. A knowledge base gathers problems that have already been solved.
4 · Integrated and governed	Resolution tooling.	Tools fix common cases (recalculating a period, resynchronizing an account) in a safe and traced way. An automatic diagnosis of an environment produces a report attached to the support request. The causes of requests are classified, and the most frequent ones feed the list of changes.
5 · Autonomous	Proactive support.	A problem is detected and handled before a user reports it. Users get answers to common questions from an assistant that draws on the knowledge base. Resolution time and satisfaction are tracked and steered.

Effect on the user

A reported problem is understood quickly, without exposing more data than necessary.

Administration automation#

Are administration operations scripted, replayable and safe?

Level	Signature	Criteria to meet
0 · Handcrafted	Operations done by hand in interfaces, from memory.	Starting point: no criteria.
1 · Locally tooled	Personal scripts.	Common operations are described in a written procedure. Scripts exist for some operations.
2 · Shared	Shared, versioned scripts.	Administration scripts are versioned and shared in a common repository. Each script states what it does, the parameters expected and the target environment. A destructive script requires explicit confirmation or a simulation mode.
3 · Industrialized	Replayable scripts.	Scripts give the same result when they are run again (replayable). Scripts record their execution in the administration log. The credentials that scripts need are supplied by a vault, not written in clear text.
4 · Integrated and governed	Administration interface.	Common operations are also available through a programmable interface (API) or a console. Scripts are tested in a test environment before use in production. The rights needed for each operation are defined as narrowly as possible.
5 · Autonomous	Declarative administration.	The desired state (accounts, rights, settings) is described in a versioned file and applied automatically. Operations are triggered by events or approved requests, with no manual intervention. Differences between the described state and the actual state are corrected or flagged automatically.

Effect on the user

An operation performed a hundred times gives the same result a hundred times, without depending on one person's memory.

Origin of the criteria

The dimensions are inspired by the references cited below; the levels and criteria are an original synthesis, not a reproduction of a framework. They are offered for discussion and adaptation, not as authoritative.

Sources and verification#

NIST — Role Based Access Control — users-roles-permissions model, separation of duties; became the ANSI INCITS 359-2004 standard, revised in 2012
OWASP — Logging Cheat Sheet — events to log (authentication, access control, administration actions), data never to log (passwords, tokens, secrets), protection of logs
The Twelve-Factor App — III. Config — strictly separate configuration from code; whatever varies from one deployment to another (credentials, addresses) is configuration
OWASP — Application Security Verification Standard — verifiable security requirements, at three levels of rigor
Microsoft Learn — Optimization guide for Power BI — role of administrators: capacities, gateways, performance monitoring