# Reports and scorecards

## Purpose
Turn a campaign's submissions into scored results, change how they are calculated, and
get them out as a file.

## Who this is for
Account administrators, campaign managers, and anyone with reporting permissions.

## How scoring is organised

A **report definition** is the recipe; a **scorecard** is the result of running that
recipe over one campaign's completed submissions.

Definitions come in two kinds:

- **Global built-ins**: supplied by the platform, marked with a globe icon, read-only.
  You can run them but not edit them.
- **Account definitions**: yours. Create, edit, delete.

Three built-ins ship with the platform:

| Built-in | Scope | What it reports |
| --- | --- | --- |
| Assessment Performance | Campaign | Answers recorded, distinct participants, and how many answers an auditor has verified. |
| Participant Progress | Participant | Answers recorded, and **Verified %**, the share of that person's answers an auditor has verified. |
| BIMMaS Maturity Profile | Campaign | Current-vs-target maturity per domain. A different engine; see the maturity panel on the campaign's reporting page. |

**"Verified %" reads verified answers over all answers given.** It used to divide the
verified answers by themselves and therefore sat at 100% on every account, whatever
anybody had reviewed. It moves now: a participant with four answers of which an auditor
has verified one reads 25%. If you are comparing against a figure you noted before this
changed, expect it to be lower and more useful.

Every definition has a **scope** that decides what it is computed over: *campaign*,
*assessment* or *participant*. A participant-scoped definition needs you to pick a person
before it can compute anything. (An older *account* scope is no longer offered and is
refused if sent; a definition still carrying it says "Account-wide reports are not
supported yet." instead of drawing a chart.)

## Find them

**Reports** in the sidebar is a picker, not a dashboard: scorecards are always computed
inside a campaign, so this page lists your campaigns and tells you how many definitions
are visible to the account. Choose a campaign to open its reporting page.

You can also get there directly from a campaign header via **Scorecards**.

## Run a scorecard

1. Open a campaign's reporting page.
2. If the page shows tabs, stay on **Scorecards**. The **Maturity Profile** tab beside it
   is a different report on a different engine (see below).
3. Pick a definition from the left-hand list.
4. For a participant-scoped definition, choose the person in the **Participant**
   dropdown. Until you do, the page says so rather than showing an empty chart.
5. The result renders on the right.

Here the built-in **Assessment Performance** is selected and shown as KPI cards:

![The reporting page of a campaign, on the Scorecards tab. On the left, the Reports list with New report, an Export as menu set to XLSX with Export all, and three built-in definitions, each marked with a globe and a padlock: BIMMaS Maturity Profile, Participant Progress and Assessment Performance, which is selected. On the right, Assessment Performance with a Refresh button, a strip reading 150 submissions, 86 participants, 0 reviews, a date range and Cached, the six chart buttons, and three KPI cards: Answers recorded 868, Participants 86 and Verified answers 0.](/assets/public_doc_images/guides/scorecards-assessment-performance.webp)

Above the chart, a strip of metadata tells you how much data went in (submissions,
participants, reviews, the date range), and a coloured dot says whether you are looking
at a **Cached** result or a **Freshly computed** one, with how long ago it was produced.
**Refresh** recomputes when the cached snapshot has gone stale.

Snapshots are invalidated automatically when the underlying data changes: a new public
submission, a saved review, a reopened submission, a claimed anonymous submission. You
rarely need to force a refresh.

A snapshot is also treated as stale when the scoring engine itself has changed since it
was computed, so a release that corrects how a number is calculated recomputes the
result on the next view rather than serving the old one. The exception is a **closed**
campaign: its numbers are frozen deliberately as the record of what was published, and
they do not move when the engine does.

## Change the chart

Six renderings are available from the toggle above the chart: **KPI cards**, **Bar**,
**Line**, **Pie**, **Heatmap**, **Table**. The definition's own default is selected when
you open it; switching is a per-view choice and is not saved.

## Nothing is listed?

If the left-hand list is empty, the environment has never had the built-in definitions
seeded. The empty state offers two ways forward: **New report** to define your own, and a
seeding button that is only visible to platform superadmins. If you are not a superadmin
and you see no definitions, ask for the built-ins to be seeded.

## Create your own definition

From the reporting page, choose **New report** above the definition list (the empty
state offers the same button).

- **Title** and **Code**: the code is a stable identifier, suggested from the title
  until you edit it.
- **Description**: what this report measures.
- **Scope** and **Report type**.
- **Default chart**, **Sort by** (definition order, label or value), **Sort order**, and
  whether to show a legend.
- **Aggregation rules**: at least one is required, and each needs a **key** and a
  **label**. A rule either counts across all answer keys or across a comma-separated
  list you supply, and can be restricted to answers with a given verification status
  (verified, modified, unverified).
- **Advanced**: an optional JSON escape hatch for `scoringModel`, `filterPredicates`
  and chart `colorBands`, for the cases the form does not cover.

Save, and the definition appears in the list ready to run. Validation happens on the
server, so any rejection message you see is the authoritative reason.

## Start from a built-in

Built-in definitions carry a **Clone to account** button, with one exception: the
BIMMaS Maturity Profile, which is seeded content rather than a definition and carries no
action at all. Cloning makes an editable copy
in your account named `<original> (copy)` and selects it so you can adjust it
immediately. This is the supported way to "edit" a built-in.

Your own definitions carry **Edit** and **Delete** instead. Editing clears the cached
snapshots for that definition, so the next view recomputes.

## How a group of questions becomes one number

Two aggregations are available to a rule, and the competency panel below picks between
them for you. They are in **different units**, so read the unit before the number.

| The questions in the group | What is reported | Unit |
|---|---|---|
| All on the same numeric ladder whose choices are `1, 2, … N` (N of 3 or more) | The **mean level**: the average of the answers | The ladder, e.g. 1–5 |
| Anything else | The **count-of-'1' percentage**: the share of answers equal to `1` | % |

Why the ladder rule is narrow. A five-level maturity ladder scored as a percentage
reports *the share of answers at the lowest level* and presents it as a score: a red
bar for a cohort averaging 3.2 out of 5. But the reverse mistake is worse, so the mean
is only used when the choice values are exactly `1..N` and nothing else:

- A list such as `1, 2, 0, 00` (common in the assessment bank, where `0` is "Not sure"
  and `00` is "Not applicable") stays a percentage. Averaging it would score Not
  Applicable as zero and silently drag the group below its own floor.
- A `0`-based ladder (`0, 1, 2, 3, 4`) stays a percentage too. It may well be a real
  ladder, but from the outside it is indistinguishable from a scale whose `0` is a
  sentinel, and that is not a guess a report should make on your behalf.
- A group that **mixes** a ladder with anything else, or two ladders of different
  heights, stays a percentage: two units cannot be averaged into one number.

If you want a ladder averaged and it is not being, re-author its choice values as
`1..N`, or split the group so every question in it is on one ladder.

Where the unit is shown: the competency export carries a `unit` column on every row
(`%` or `level 1-5`), and the panel's footnote says how many topics are on each.

**Matrix questions are scored.** A matrix answer is one cell per row, and each cell is
addressed as `<question>::<row>`, the same name the answer export uses for that
column, so a spreadsheet and a scorecard always mean the same cell. A rule that names
the matrix question scores all of its rows; a rule may also name a single row. Counts
that run across *all* answer keys (such as "Answers recorded") still count a matrix as
one answer, not one per row. `matrixdropdown`, `matrixdynamic` and `multipletext`
questions are still not scored (they have no single scale) and are reported in the
coverage chips rather than counted as zeros.

## Filtering to a cohort

A `field` filter predicate (available through the **Advanced** JSON box) selects
**respondents**, not answers. `{"type":"field","key":"province","operator":"equals","value":"ON"}`
keeps every answer from the submissions that answered `ON` to `province`. It does not
keep only the province answers. A submission that never answered the qualifier is not in
the cohort.

## Competency scorecards

Below the definition-driven scorecards sits a separate **Competency scorecards** panel.
These are the native BIMe Competency Set and Topic scores, computed from the campaign's
template tags against the canonical taxonomy in the BIMei Ontology (ONT). They are not report definitions and
cannot be edited. Each group is scored by the rule in the table above.

The panel can be scoped to one participant or left across all of them, and exported as
XLSX, CSV or JSON. Coverage chips report how much of the template it could actually read.

If you see **No competency tags**, the campaign's templates do not tag their questions
with a Competency Topic id. Only templates that carry those tags produce these
scorecards. Topics still marked *interim-proposed* in ONT are called out by code in the
coverage strip at the top of the panel.

## The Maturity Profile tab

Where a maturity definition is seeded (it is a global built-in, so in every account), the
reporting page carries a second tab, **Maturity Profile**. It is a different report on a
different engine, and this guide's rules about definitions, scopes and cloning do not
apply to it.

## Letting respondents see their own results

Some reports can be shown back to the people who answered: a "your results" page on the
participant's own landing (see [Your results](/use/participants/participant-your-results)
for what they see). Two independent things decide whether that door is open for a report
on a campaign:

1. **The report's own visibility flag**, set on the report definition and shared by every
   campaign in the account. Your own definitions carry this in the New/Edit dialog. It is
   **off by default**, so a report you have never touched is never shown to respondents.
2. **A per-campaign list**, set on the campaign's reporting page below the reports (**Respondent
   results visibility**). A report reaches a respondent when EITHER of the two is true: the flag
   OR the campaign's list.

**Why there is a second way in.** The BIMMaS Maturity Profile is a platform built-in (the same
"global, read-only, seeded content" the rest of this guide describes), and its visibility flag
cannot be set from an account at all: there is nothing to edit here for it. The
**Respondent results visibility** checklist is how you open the door for it on ONE campaign,
without asking for a platform-wide change and without it turning on anywhere else. Tick the
report, **Save visibility**, and it takes effect on that campaign alone. A report that already
carries the flag shows checked and locked in the checklist: unchecking it there would do
nothing, since the flag already opens the door for every campaign in the account.

**This checklist only says WHICH reports can be seen. WHEN is a separate setting.** It has its
own panel beside this one, **When respondents see their results**: the release timing (on
submission / at the wave's close / released manually by a coordinator), the **Release results
now** action, and the comparison floor with the rule that fixes it. Step-by-step in
[Releasing results to respondents](/operate/tenant-admin/releasing-results); the stored shape is
in the API contract (`openapi/v1/specs/account-campaigns.yaml`, the `results` block under
`CampaignSettings`).

It shows the campaign's **Profile** radar, a **Ladder and gap** view, **Themes**, a
**Segments** heatmap with its own **Segment by** selector, and a **Measure composition**
table across MoP / MoE / MoO (renamed from "Level of evidence": that name now belongs to
the account-configurable review-level list on answer reviews; see
[Review answers and share feedback](/operate/tenant-admin/auditor-reviews-and-feedback)).
Its own buttons sit with it: **Export** (XLSX, CSV or
JSON), **PDF**, **Set targets** and **Recompute**, none of which is the **Export all**
below, which covers scorecards only.

Segments with too few respondents are suppressed rather than drawn. That floor is
report-definition configuration, not an account setting, and it is stricter again on a
published page (see
[Publish results to a public link](/operate/tenant-admin/publish-results-to-a-public-link)).

### The PDF is not a tagged PDF

**The file the PDF button downloads is not tagged, so it does not meet PDF/UA.** It carries
no structure tree, which is the part of the format a screen reader uses to move through a
document by heading, and the part that gives an image its alternative text. We say so here
rather than leave it to be discovered.

What it does carry: a document **title** and a **language**, both set from the report, so a
reader announces the document by name in the right language rather than by filename; the
**account name printed under the cover mark**, which is the logo's text equivalent, the same
sentence the file prints *instead* of the mark for an account with no logo; and a
**one-sentence description beside every chart**, in the document's language, saying what the
figure shows. Every chart is also a **table on the same page**, with the same numbers: the
radar and the ladder are drawn from the domain table printed beneath them, and the segments
and level-of-evidence views are tables to begin with. So nothing in the document is available
only as a picture.

If a procurement or a policy requires the PDF/UA claim itself, that is a change to the PDF
renderer we build the file with, not a change to the report: it would have to mark up every
line of text and every image as it draws them. Tell us and we will scope it; we will not put
the claim on a file that cannot support it.

## Export

- **Export all**, above the definition list, downloads every visible scorecard for the
  campaign in one file: XLSX, CSV or JSON.
- The competency panel has its own **Export** with the same three formats.
- A single participant's results can be downloaded from the participants list
  (**⋮ → Download XLSX Report**).
- Where an auditor left a note on an answer, both files carry a **Notes** table: the
  note, its hashtags, and an identity-folded key for the auditor (never an email
  address). A note whose *include in export* box is unticked is not in the file at
  all, and a campaign with no notes gets no Notes table.

### For integrators pulling these files over the API

- Both export endpoints answer two ways. By default they return the file inside a JSON
  envelope; add `?download=1` and the same file is streamed as an attachment. Both arms
  are written from one model, so the bytes are identical.
- **The JSON format changed on 2026-09-10.** It used to be pretty-printed with a
  two-space indent and it is now compact. Nothing else moved: the same keys, in the
  same order, with the same values. If you diff stored copies of these files you will
  see one whitespace-only change and then nothing.
- The competency file's `reviewStatus` column is the **taxonomy term's** editorial
  status, not an auditor's verdict on an answer. The two are unrelated, and neither of
  these files carries the per-answer `rev:` columns the submission matrix does:
  every row in them is an aggregate, not an answer.

## Limits and gotchas
- In-progress drafts are excluded from scoring. Only completed submissions count.
- Deleting a definition is permanent and is confirmed in a dialog.
- Global built-ins cannot be deleted or renamed from an account, only cloned, except the
  BIMMaS Maturity Profile, which is seed-only and offers no row action whatsoever.
- The **Export** menu in the *campaign header* is a different thing from **Export
  all** here: it downloads the campaign's raw submission matrix, not the scorecards
  (see [Review campaign analytics and logs](/operate/tenant-admin/review-campaign-analytics-and-logs)).
- Both files apply the auditor's corrections, so the numbers in a downloaded workbook
  and the numbers on this page agree.

## Related pages
- [Review campaign analytics and logs](/operate/tenant-admin/review-campaign-analytics-and-logs)
- [Publish results to a public link](/operate/tenant-admin/publish-results-to-a-public-link)
- [Review answers and share feedback](/operate/tenant-admin/auditor-reviews-and-feedback)
