Bennett King-Nyberg (University of Victoria)
January 30 @ 3:00 pm - 4:30 pm
Presenter: Bennett King-Nyberg (University of Victoria)
Title: Stimulus Sampling Reimagined: Diagnosing Item-Level Heterogeneity with Stimulus Plots
Abstract:
Psychological experiments routinely average over stimuli, implicitly assuming that effects generalize homogeneously across the items used to instantiate a manipulation. However, identical mean effects can arise from qualitatively different patterns of item-level behavior, including sign reversals, extreme heterogeneity, or dependence on a small subset of stimuli. In this talk, I discuss recent work by Simonsohn, Montealegre, and Evangelidis (2025), which reframes stimulus sampling as an inference problem rather than a nuisance and introduces stimulus plots as a diagnostic tool for examining how effects are distributed across items.
I then apply this framework to my own work on the photo-truthiness effect, demonstrating how identical mean effects across a between-subjects factor can mask theoretically important item-level heterogeneity. I conclude by illustrating why stimulus plots remain informative even when stimuli are selected based on prior evidence of reliable effects.
The talk will be in person in the Psychology Reading room, COR A228, from 3:00 – 4:30.