What sustains daily self-reflection: what the measures couldn't ask
DESIGN RESEARCH LEAD | LOTIC | SUMMER 2023
The Well-Being Science team ran a fourteen-day study testing whether daily self-reflection improves well-being. Their measures could show whether it worked. I built the instrument that asked why.
Twenty-one participants at CalArts spent fourteen days straight with a lesson, a prompt, a voice recording, and a daily survey. The science team tracked order effects, dose response, and individual differences in the numbers.
The study was already designed when I came into it, and I couldn't change it. Checking in partway through would have registered in the dose-response measure as extra reflection, so I had one instrument at the end and one pass at twenty-one people.
Approach
01
Asked what changed, not whether they liked it
02
Built for one look back
03
Asked about each activity on its own
Rating questions would have given us a thinner copy of what the science team already had. Mine went after what shifted, when it shifted, and what it felt like, so the answers could sit beside their numbers instead of repeating them.
With no contact during the two weeks, the instrument had to cover all fourteen days in one sitting. So it asked about specific moments and specific activities. People can't summarize two weeks accurately, but they can tell you about Tuesday.
The lesson, prompt, recording, and survey each got their own questions rather than sitting inside one question about the experience. That's what revealed the survey problem. Nobody tells you the measurement was the worst part when you ask them how it went.
What the measures couldn't ask
SAYING IT OUT LOUD CHANGES THE THOUGHT
Twelve of twenty-one valued the recording, and they described a mechanism rather than a preference. Leo said speaking his goals gave him a real plan instead of leaving them “in the aether” of his mind. Jarred heard himself describe goals he'd already met and recognized them as achievements for the first time.
Dose response can establish that more reflection produces more change. It can't tell you that getting the thought out of your head is what does the work.
“Things settle differently in the body once they flow into the air and are held in an additional space.”
Lena, end-line responseAND IT SHUT SOME PEOPLE OUT
Two wouldn't record at all. Michelle wanted typing instead because she gets nervous recording. Jonah: “I don't like recording my voice, honestly.” The build made it worse by deleting anything under fifteen seconds and forcing a re-record, which hit hardest on the people already reluctant to speak.
THE MEASUREMENT WAS COSTING THE PARTICIPANTS
Seven of twenty-one disliked the daily survey against five who liked it: repetitive, tedious, hard work. Jude said answering it every day made her “kind of sad” because she “often had more negative answers.” A study measuring well-being was running something that lowered it daily.
IT WAS GOOD IN THE MOMENT AND GONE BY THE NEXT
Zara: “It felt like having a very brief conversation with a friend every day, in that it was nice in the moment, but didn't stick with me.” Nothing was revisitable. The dose-response measure would have counted her fourteen days as fourteen doses either way.
THE VOICE DIDN’T SOUND LIKE A PERSON
Three participants reached independently for the same kind of comparison.
“Corporate life coaching”
Zara“Therapy-speak”
Dakota“A pamphlet that you read at a doctor's office”
SashaAvery named what was doing it, calling out “social media buzzwords like 'mental well-being' or 'empowerment', which get used so often that they have entirely lost their meaning.”
THE CONTENT WORKED WHERE IT MADE A BIG IDEA SMALL
Six people said breaking purpose into parts made it approachable. Zara said the lessons helped her “let go of framing my existence around a need to have a grand life purpose.”
What participants asked for
Delete and re-record
A written option
Drop the fifteen-second minimum
Somewhere to look back
Their own data, shown back
Choice
Olivia “would make mistakes recording and I didn't know how to go back.”
Jonah wanted it “more usable in public.”
Monali: “some answers were not required to be so long.”
Zara wanted past prompts. Kira wanted her own recordings returned.
Three asked to see their survey responses charted across the two weeks.
Three wanted to pick their lessons and prompts instead of being assigned one.
Where it landed
The findings named the specific things wearing on people: a daily survey that had become a chore, nothing they could go back to, prompts that pushed toward the positive, and a voice that read as corporate. Those went to content and product as the list to work through.
The recording barrier is the one I can trace. Two people here wouldn't record themselves. It came up again a year later with an entirely different group, adults with disabilities in an Arkansas clinic, and that time the typed-response option got built. It took two studies and two audiences before the same finding landed.
Lotic ran out of funding and Talk2Me never launched.
Reflection
I came in after the design was set, and that decided what I could ask. One instrument, one look back, two weeks after the fact. It found real things, but the survey problem was fixable on day three and I didn't surface it until day fifteen. Getting into the design conversation early is the difference between explaining a result and changing one, and I push for that now.
That constraint left me with a habit I've kept. I ask for stories now. The first time something clicked. The last time they opened the app. The moment they gave up on it. Ask someone how they typically use a product and you get a summary they assembled on the spot, and you can't build anything from a summary. The details worth acting on only show up when someone walks you through something that happened.
Lastly, if you measure an experience every day, the measurement becomes part of the experience. Our survey was there to observe well-being and was quietly taxing it. I treat participant burden as something to design for now rather than something to note at the end.