Measurement

The principle

Measure changes in people, institutions, livelihoods and communities — not only activities delivered.

Activity is easy to count and easy to mistake for effect. A programme that reports workshops run, people attended and prototypes built has described its own effort, not its results. The whole apparatus below exists to keep those two things apart.

Quality is anchored to recognised frameworks rather than invented: the principles of asking why, designing at scale for all, empowering teachers, engaging the ecosystem and being data-driven[54]; and the pillars of safety first, impact on learning, designed for children, accessibility, and inclusivity and equity[53]. Assessment in maker programmes is feasible and precedented — most surveyed makerspaces already do it[21].

The longer progression the organisation is testing runs: access → learning → capability → innovation → livelihood or enterprise → community impact → system change. That is a hypothesis to test, not a claim being made — and it is not compulsory for any participant. Employment, research, further education, community leadership, teaching and venture creation are all valid destinations.

Six results levels, never interchangeable

Activities, attendances, prototypes, innovations, ventures and impacts are not interchangeable. A workshop delivered is not a skill gained. A skill gained is not a job. A prototype is not a solved problem. Nothing about this is subtle, and it is broken constantly — usually by moving one line up the table when the numbers on the correct line are disappointing.

The innovation ladder

Within the innovation story specifically, four different things get reported as one. They are separated, and the drop-off between them is measured — because the drop-off is where the information is.

The eight instruments that make it testable

These come before any expansion of the indicator set. Additional indicators are introduced only when they support a defined programme, management or funding decision — and a indicators added before these are in place produce numbers without the baseline needed to interpret them.

The five questions the mission is tested against

1. Was the community demonstrably underserved on a defined dimension?
2. Did the community participate meaningfully in decisions?
3. Did participants acquire demonstrated capability?
4. Did they apply it after our direct involvement ended?
5. Did they keep solving without remaining dependent on us?

Where we currently stand: we answer the first partially, the third weakly, and the fourth and fifth mainly through individual cases rather than an organisation-wide system. That is the gap the eight instruments are being built to close.

When measurement happens

Brokerage — the least evidenced claim, made countable

Connections and brokerage is repeatedly named as this organisation’s differentiator and has essentially no external research literature supporting it as a mechanism. Our own record is therefore the only evidence that will exist, which makes these indicators load-bearing.

• Jobs and paid assignments brokered
• Partnerships that produced an observable opportunity
• Local organisations that gained funding or visibility through a connection
• Research placements and university connections
• Partner organisations independently adopting the methodology
• Former participants becoming trainers, managers and directors — and the share of delivery led by local alumni

Two reporting rules apply. A partnership counts as an outcome only when it produced an observable result — delivery, funding, placement, employment, research, infrastructure, adoption or policy change. And one-way brokerage does not count: a relationship where value flows only inward is sourcing, and the return leg is measured too.

The rules we report under

1. Count unique people separately from attendances.
2. Record leavers and reasons at every stage.
3. Report ventures as active, dormant, closed or transformed — separately.
4. Do not infer community impact from a prototype or an award.
5. Do not infer income improvement without an intake baseline and follow-up.
6. Report stopped projects as well as successful ones.
7. Separate self-report, observation, administrative record and independent verification.
8. Measure retention and drop-off, not only immediate post-programme gain.
9. Adjust causal claims for attribution, deadweight, displacement and drop-off where feasible.
10. Collect only data that informs a material decision and can be protected responsibly.

How a result is presented

baseline → target → actual result → difference → explanation → corrective action

Targets are set after a reliable baseline exists, not before. Every indicator declares its result level and causal link, its exact numerator and denominator, whether it counts people or attendances, its baseline and target, its data source and instrument, who collects it and how often, the disaggregation required, its consent and privacy rules, its verification method, and its known limitations.

Disaggregation is by gender, disability, geography and socioeconomic condition. Age group is not a field — the model does not use age, and an age band entering through an intake form is how a retired band re-enters a model.

Generating the evidence the field is missing

The income link is a hypothesis, and a sceptical search confirmed it is thin: no clean rural-makerspace-to-income result exists anywhere, and the most rigorous skills trial fades[55, 56]. Education, creativity and community-impact outcomes are well evidenced[24, 29]. Income is the thing that has to be measured rather than asserted.

That is an opportunity rather than a weakness, and it shapes what we ask funders for.