- Fig 01 / Research Synthesis
Decision: Put getting trustworthy data out ahead of form features
Result: Researchers could audit every result back to the variant that produced it
Civic Tech
Improving Access to Government Benefits and Services
Evaluating digital forms to enhance public access to critical services like healthcare, housing, and financial aid.
Overview
- 01 / Problem
- GSA asked its Office of Evaluation Sciences whether A/B testing could be built into federal form digitization, to cut paperwork burden and widen access to benefits. The users were behavioral scientists who would audit whatever the tool produced.
- 02 / Design Challenge
- Russ led design research and product definition for GSA TTS: what the product was, which features came first, and how OES researchers would run tests.
- 03 / Approach
- He had OES score 27 features on two axes, research value and autonomy; getting trustworthy data out ranked highest. He drew a self-service product and a contractor-run service side by side for the same researcher.
- 04 / Outcome
- A defined product, a ranked feature set and an instrumented proof of concept, after which OES preregistered a real trial. The trial results in the case study aren't in the archive and need OES's published report.
Process + Evidence Plates
Axis two — autonomy, and the sharper of the two. "Now sort the same items along a spectrum defining the range of direct control you need to have over their implementation/configuration in order to evaluate feasibility," anchored by the status quo written out as a sentence: "I communicate my needs for an experiment to a developer and the developer implements a solution." The whole product argument is in that anchor — the researcher's dependence on a developer is the problem being measured. - Fig 02 / Information Architecture
The variant taxonomy — how a researcher would generate N variants, listed as the properties that distinguish them: open versus closed choices, conditional logic, pre-populated answers, question properties, question order within a given subset, question language, question content, question order at the form level. The bridge between the form-structure sketch and the requirements list. Decision: Group testable variables by what they belong to: question, form or session
Result: Researchers could see at once what could be varied and where
- Fig 03 / Personas
The participant's side of the experiment, and the only artifact that speaks for them. In the same user-story grammar but from the other chair: "As a person participating in an experiment... I want to complete a form, end a session with the form and return to the same spot later... so that I can access government services, benefits, and programs, and adapt my interaction with the form in response to other things competing for my attention." That last clause is the most humane sentence in the archive — it assumes the person filling in a federal form is interrupted. Decision: Write the participant's story alongside the researcher's
Result: People filling in test forms could still pause, return and finish