Reports and scorecards

Purpose

Turn a campaign's submissions into scored results, change how they are calculated, and get them out as a file.

Who this is for

Account administrators, campaign managers, and anyone with reporting permissions.

How scoring is organised

A report definition is the recipe; a scorecard is the result of running that recipe over one campaign's completed submissions.

Definitions come in two kinds:

• Global built-ins — supplied by the platform, marked with a globe icon, read-only. You can run them but not edit them.
• Account definitions — yours. Create, edit, delete.

Three built-ins ship with the platform:

Built-in	Scope	What it reports
Assessment Performance	Campaign	Answers recorded, distinct participants, and how many answers an auditor has verified.
Participant Progress	Participant	Answers recorded, and Verified % — the share of that person's answers an auditor has verified.
BIMmaas Maturity Profile	Campaign	Current-vs-target maturity per domain. A different engine; see the maturity panel on the campaign's reporting page.

"Verified %" reads verified answers over all answers given. It used to divide the verified answers by themselves and therefore sat at 100% on every account, whatever anybody had reviewed. It moves now: a participant with four answers of which an auditor has verified one reads 25%. If you are comparing against a figure you noted before this changed, expect it to be lower and more useful.

Every definition has a scope that decides what it is computed over: campaign, assessment or participant. A participant-scoped definition needs you to pick a person before it can compute anything. (An older account scope is no longer offered and is refused if sent; a definition still carrying it says "Account-wide reports are not supported yet." instead of drawing a chart.)

Find them

Reports in the sidebar is a picker, not a dashboard: scorecards are always computed inside a campaign, so this page lists your campaigns and tells you how many definitions are visible to the account. Choose a campaign to open its reporting page.

You can also get there directly from a campaign header via Scorecards.

Run a scorecard

1. Open a campaign's reporting page.
2. If the page shows tabs, stay on Scorecards. The Maturity Profile tab beside it is a different report on a different engine — see below.
3. Pick a definition from the left-hand list.
4. For a participant-scoped definition, choose the person in the Participant dropdown. Until you do, the page says so rather than showing an empty chart.
5. The result renders on the right.

Above the chart, a strip of metadata tells you how much data went in — submissions, participants, reviews, the date range — and a coloured dot says whether you are looking at a Cached result or a Freshly computed one, with how long ago it was produced. Refresh recomputes when the cached snapshot has gone stale.

Snapshots are invalidated automatically when the underlying data changes: a new public submission, a saved review, a reopened submission, a claimed anonymous submission. You rarely need to force a refresh.

A snapshot is also treated as stale when the scoring engine itself has changed since it was computed, so a release that corrects how a number is calculated recomputes the result on the next view rather than serving the old one. The exception is a closed campaign: its numbers are frozen deliberately as the record of what was published, and they do not move when the engine does.

Change the chart

Six renderings are available from the toggle above the chart: KPI cards, Bar, Line, Pie, Heatmap, Table. The definition's own default is selected when you open it; switching is a per-view choice and is not saved.

Nothing is listed?

If the left-hand list is empty, the environment has never had the built-in definitions seeded. The empty state offers two ways forward: New report to define your own, and a seeding button that is only visible to platform superadmins. If you are not a superadmin and you see no definitions, ask for the built-ins to be seeded.

Create your own definition

From the reporting page, choose New above the definition list (in the empty state the same button reads New report).

• Title and Code — the code is a stable identifier, suggested from the title until you edit it.
• Description — what this report measures.
• Scope and Report type.
• Default chart, Sort by (definition order, label or value), Sort order, and whether to show a legend.
• Aggregation rules — at least one is required, and each needs a key and a label. A rule either counts across all answer keys or across a comma-separated list you supply, and can be restricted to answers with a given verification status (verified, modified, unverified).
• Advanced — an optional JSON escape hatch for scoringModel, filterPredicates and chart colorBands, for the cases the form does not cover.

Save, and the definition appears in the list ready to run. Validation happens on the server, so any rejection message you see is the authoritative reason.

Start from a built-in

Built-in definitions carry a Clone to account button — with one exception, the BIMmaas Maturity Profile, which is seeded content rather than a definition and carries no action at all. Cloning makes an editable copy in your account named <original> (copy) and selects it so you can adjust it immediately. This is the supported way to "edit" a built-in.

Your own definitions carry Edit and Delete instead. Editing clears the cached snapshots for that definition, so the next view recomputes.

How a group of questions becomes one number

Two aggregations are available to a rule, and the competency panel below picks between them for you. They are in different units, so read the unit before the number.

The questions in the group	What is reported	Unit
All on the same numeric ladder whose choices are 1, 2, … N (N of 3 or more)	The mean level — the average of the answers	The ladder, e.g. 1–5
Anything else	The count-of-'1' percentage — the share of answers equal to 1	%

Why the ladder rule is narrow. A five-level maturity ladder scored as a percentage reports the share of answers at the lowest level and presents it as a score — a red bar for a cohort averaging 3.2 out of 5. But the reverse mistake is worse, so the mean is only used when the choice values are exactly 1..N and nothing else:

• A list such as 1, 2, 0, 00 — common in the assessment bank, where 0 is "Not sure" and 00 is "Not applicable" — stays a percentage. Averaging it would score Not Applicable as zero and silently drag the group below its own floor.
• A 0-based ladder (0, 1, 2, 3, 4) stays a percentage too. It may well be a real ladder, but from the outside it is indistinguishable from a scale whose 0 is a sentinel, and that is not a guess a report should make on your behalf.
• A group that mixes a ladder with anything else, or two ladders of different heights, stays a percentage: two units cannot be averaged into one number.

If you want a ladder averaged and it is not being, re-author its choice values as 1..N, or split the group so every question in it is on one ladder.

Where the unit is shown: the competency export carries a unit column on every row (% or level 1-5), and the panel's footnote says how many topics are on each.

Matrix questions are scored. A matrix answer is one cell per row, and each cell is addressed as <question>::<row> — the same name the answer export uses for that column, so a spreadsheet and a scorecard always mean the same cell. A rule that names the matrix question scores all of its rows; a rule may also name a single row. Counts that run across all answer keys (such as "Answers recorded") still count a matrix as one answer, not one per row. matrixdropdown, matrixdynamic and multipletext questions are still not scored — they have no single scale — and are reported in the coverage chips rather than counted as zeros.

Filtering to a cohort

A field filter predicate (available through the Advanced JSON box) selects respondents, not answers. {"type":"field","key":"province","operator":"equals","value":"ON"} keeps every answer from the submissions that answered ON to province — it does not keep only the province answers. A submission that never answered the qualifier is not in the cohort.

Competency scorecards

Below the definition-driven scorecards sits a separate Competency scorecards panel. These are the native BIMe Competency Set and Topic scores, computed from the campaign's template tags against the canonical ONT taxonomy — they are not report definitions and cannot be edited. Each group is scored by the rule in the table above.

The panel can be scoped to one participant or left across all of them, and exported as XLSX, CSV or JSON. Coverage chips report how much of the template it could actually read.

If you see No competency tags, the campaign's templates do not tag their questions with a Competency Topic id. Only templates that carry those tags produce these scorecards. Topics still marked interim-proposed in ONT are called out by code in the coverage strip at the top of the panel.

The Maturity Profile tab

Where a maturity definition is seeded — it is a global built-in, so in every account — the reporting page carries a second tab, Maturity Profile. It is a different report on a different engine, and this guide's rules about definitions, scopes and cloning do not apply to it.

Letting respondents see their own results

Some reports can be shown back to the people who answered — a "your results" page on the participant's own landing (see Your results for what they see). Two independent things decide whether that door is open for a report on a campaign:

1. The report's own visibility flag, set on the report definition and shared by every campaign in the account. Your own definitions carry this in the New/Edit dialog. It is off by default, so a report you have never touched is never shown to respondents.
2. A per-campaign list, set on the campaign's reporting page below the reports (Respondent results visibility). A report reaches a respondent when EITHER of the two is true — the flag OR the campaign's list.

Why there is a second way in. The BIMmaas Maturity Profile is a platform built-in — the same "global, read-only, seeded content" the rest of this guide describes — and its visibility flag cannot be set from an account at all: there is nothing to edit here for it. The Respondent results visibility checklist is how you open the door for it on ONE campaign, without asking for a platform-wide change and without it turning on anywhere else. Tick the report, Save visibility, and it takes effect on that campaign alone. A report that already carries the flag shows checked and locked in the checklist — unchecking it there would do nothing, since the flag already opens the door for every campaign in the account.

This checklist only says WHICH reports can be seen — WHEN is a separate setting. It has its own panel beside this one, When respondents see their results: the release timing (on submission / at the wave's close / released manually by a coordinator), the Release results now action, and the comparison floor with the rule that fixes it. Step-by-step in Releasing results to respondents; the stored shape is in the API contract (openapi/v1/specs/account-campaigns.yaml, the results block under CampaignSettings).

It shows the campaign's Profile radar, a Ladder and gap view, Themes, a Segments heatmap with its own Segment by selector, and a Measure composition table across MoP / MoE / MoO (renamed from "Level of evidence" — that name now belongs to the account-configurable review-level list on answer reviews; see Review answers and share feedback). Its own buttons sit with it: Export (XLSX, CSV or JSON), PDF, Set targets and Recompute — none of which is the Export all below, which covers scorecards only.

Segments with too few respondents are suppressed rather than drawn. That floor is report-definition configuration, not an account setting, and it is stricter again on a published page — see Publish results to a public link.

The PDF is not a tagged PDF

The file the PDF button downloads is not tagged, so it does not meet PDF/UA. It carries no structure tree, which is the part of the format a screen reader uses to move through a document by heading, and the part that gives an image its alternative text. We say so here rather than leave it to be discovered.

What it does carry: a document title and a language, both set from the report, so a reader announces the document by name in the right language rather than by filename; the account name printed under the cover mark, which is the logo's text equivalent — the same sentence the file prints instead of the mark for an account with no logo; and a one-sentence description beside every chart, in the document's language, saying what the figure shows. Every chart is also a table on the same page, with the same numbers: the radar and the ladder are drawn from the domain table printed beneath them, and the segments and level-of-evidence views are tables to begin with. So nothing in the document is available only as a picture.

If a procurement or a policy requires the PDF/UA claim itself, that is a change to the PDF renderer we build the file with, not a change to the report — it would have to mark up every line of text and every image as it draws them. Tell us and we will scope it; we will not put the claim on a file that cannot support it.

Export

• Export all, above the definition list, downloads every visible scorecard for the campaign in one file — XLSX, CSV or JSON.
• The competency panel has its own Export with the same three formats.
• A single participant's results can be downloaded from the participants list (⋮ → Download XLSX Report).
• Where an auditor left a note on an answer, both files carry a Notes table — the note, its hashtags, and an identity-folded key for the auditor (never an email address). A note whose include in export box is unticked is not in the file at all, and a campaign with no notes gets no Notes table.

For integrators pulling these files over the API

• Both export endpoints answer two ways. By default they return the file inside a JSON envelope; add ?download=1 and the same file is streamed as an attachment. Both arms are written from one model, so the bytes are identical.
• The JSON format changed on 2026-09-10. It used to be pretty-printed with a two-space indent and it is now compact. Nothing else moved: the same keys, in the same order, with the same values. If you diff stored copies of these files you will see one whitespace-only change and then nothing.
• The competency file's reviewStatus column is the taxonomy term's editorial status, not an auditor's verdict on an answer. The two are unrelated, and neither of these files carries the per-answer rev: columns the submission matrix does — every row in them is an aggregate, not an answer.

Limits and gotchas

• In-progress drafts are excluded from scoring. Only completed submissions count.
• Deleting a definition is permanent and is confirmed in a dialog.
• Global built-ins cannot be deleted or renamed from an account, only cloned — except the BIMmaas Maturity Profile, which is seed-only and offers no row action whatsoever.
• The Export button in the campaign header is a different thing from Export all here: it downloads the campaign's raw submission matrix, not the scorecards — see Review campaign analytics and logs.
• Both files apply the auditor's corrections, so the numbers in a downloaded workbook and the numbers on this page agree.

Related pages

• Review campaign analytics and logs
• Publish results to a public link
• Review answers and share feedback