Central Question
When we measure preferences using different methods — Likert scales, pairwise comparisons, rankings, Q-sorts — are we measuring the same construct?
Overview
Different measurement paradigms exist in psychology, each producing different types of data from the same underlying preferences:
| Paradigm | Example | Response Format |
|---|
| Likert scale | "Rate your agreement 1–7" | Rating on a fixed scale |
| Pairwise comparison (comparative judgment) | "Which do you prefer, A or B?" | Binary choice between two options |
| Ordinal comparative judgment | "How much do you prefer A over B? (1–5)" | Graded preference between two options |
| Ranking | "Rank these 5 items" | Ordering of all items |
| Q-sort | "Sort cards into piles" | Sorting into a forced distribution |
My research investigates the theoretical conditions under which these methods yield equivalent conclusions, and develops joint models that combine multiple data types for more precise inference. The PhD dissertation (Cheng, 2025) synthesizes this line of work.
Theoretical Framework: Joint Thurstonian Models
Rather than relying on a single measurement format, my work proposes a family of Joint Thurstonian models that simultaneously analyze Likert scale and comparative judgment data within a unified framework. By combining both formats, these models address the scale-origin problem in comparative judgment while correcting response biases in Likert scales:
The first diagram shows how latent utilities give rise to different measurement paradigms. The second illustrates how Joint Thurstonian models integrate LS and CJ data:
Key insight: Prior researchers have noted risks in relying solely on either LS or CJ. LS and CJ can complement each other: LS helps CJ identify the scale origin, while LS-specific response biases can be partialled out given that the comparative judgment results are provided. Combining the two provides a more accurate representation of individuals' preferences.
Selected Publications
Dissertation
Cheng, C. (2025). Paradigmatic decisions for measuring choices and preferences: Likert scales, comparative judgments, or combined models? using academic attribution as an example [Doctoral dissertation, National Taiwan University].
Journal Articles
Cheng, C., Lay, K.-L., Hsu, Y.-F., & Tsai, Y.-M. (2021). Can likert scales predict choices?: Testing the congruence between using likert scale and comparative judgment on measuring attribution. Methods in Psychology, 5, Article 100081. https://doi.org/10.1016/j.metip.2021.100081 Conference Presentations
Cheng, C., Yang, H.-H., Hsu, Y.-F., & Lay, K.-L. (2025, September 5–7). Paradigmatic decisions for measuring choices and preferences: Likert scales, comparative judgments, or combined models? [Poster presentation]. The 89th Annual Convention of the Japanese Psychological Association, Sendai, Japan. https://pub.confit.atlas.jp/ja/event/jpa2025/ Cheng, C., Lay, K.-L., Yang, H.-H., & Hsu, Y.-F. (2025, May 1–3). A methodology for addressing the biases resulted in ipsative placement of Q-sort items [Poster presentation]. SRCD 2025 Biennial Meeting, Minneapolis, Minnesota, USA. https://www.srcd.org/event/srcd-2025-biennial-meeting/ Cheng, C., Yang, H.-H., Hsu, Y.-F., & Lay, K.-L. (2024, October 26–27). Two are better than one [Oral presentation]. 2024 Taiwanese Psychology Association Convention, Taipei, Taiwan. https://www.tpa-tw.org/2024main/ Cheng, C., Lay, K.-L., & Hsu, Y.-F. (2020, November 18). Can Likert scales predict choices? [Poster presentation]. The 8th Annual Academic Conference on Psychology at NTU, Taipei, Taiwan. Second Place Outstanding Poster Award
Cheng, C., Lay, K.-L., & Hsu, Y.-F. (2018, May 21–25). Do chinese really prioritize effort? Using ordinal paired comparison to investigate students' ideal self and ought self [Oral presentation]. 2018 Academic Conference of Psychology Departments from National Taiwan University, Peking University, and The Chinese University of Hong Kong, Hong Kong, China.
Research Grants
Cheng, C., Lay, K.-L., & Hsu, Y.-F. (2024). 學業歸因種類與向度對於學習適應之預測力探討 (The National Science and Technology Council Award for Doctoral Candidates in the Humanities and Social Sciences for Writing Doctoral Dissertations Research Grant 113-2424-H-002-002-DR). National Science and Technology Council, Taiwan.
Related fields: APA Division 5 (Quantitative and Qualitative Methods)