Interformat equivalence between paper and electronic administrations of the self-assessment anhedonia scale in a substance use disorder clinical sample

The increasing reliance on digital tools in healthcare and research necessitates a thorough understanding of their psychometric properties. A recent study published in Frontiers in Psychology has rigorously evaluated the interformat reliability and psychometric equivalence of the Self-Assessment Anhedonia Scale (SAAS), comparing its traditional paper-and-pencil (SAAS-p) format with a newly developed electronic version (SAAS-e). The findings, based on a sample of 55 adults diagnosed with substance use disorders (SUDs), indicate that the electronic SAAS is a reliable and valid measure, mirroring the psychometric performance of its paper-based counterpart.

Background: The Digital Shift in Mental Health Assessment

The integration of technology into clinical practice has accelerated the adoption of electronically administered questionnaires. This shift offers numerous advantages, including enhanced data integrity, reduced ambiguity, streamlined data management, and potentially lower costs. However, a critical consideration is whether digitizing a well-established instrument preserves its original psychometric properties. Without this assurance, data collected through electronic means may not be directly comparable to historical data, potentially impacting clinical decision-making and research findings. The International Test Commission emphasizes the importance of evaluating measurement equivalence when adapting psychological instruments to new formats, a principle central to ensuring data validity and reliability.

Anhedonia, characterized by the inability to experience pleasure, is a significant transdiagnostic construct observed across various psychiatric conditions, including depression, schizophrenia, and substance use disorders. In individuals with SUDs, anhedonia is linked to earlier onset of substance use, increased addiction severity, and a greater risk of relapse. Its presence can also predict intensified cravings and poorer treatment outcomes, making its accurate assessment crucial for optimizing treatment planning and monitoring recovery progress.

The SAAS, originally developed as a 27-item visual analogue scale, measures physical, intellectual, and social anhedonia across three dimensions: intensity, frequency, and change over time. This inclusion of a "change over time" dimension offers a unique perspective compared to other anhedonia scales. While the SAAS has demonstrated robust psychometric properties in its paper-based format, the manual scoring procedure can be labor-intensive. The development of an electronic version (SAAS-e) aims to address this limitation by enabling automated scoring and facilitating broader clinical and research applications.

Study Design and Methodology

To assess the interformat reliability and psychometric equivalence of the SAAS-e, researchers employed a fixed-order crossover design. Fifty-five adults diagnosed with SUDs were recruited from two Addiction Behavior Units during the initial phase of their psychotherapeutic treatment (within the first two weeks of admission). This timing was chosen to capture anhedonia levels before significant treatment-induced changes could occur.

Participants were required to have a diagnosis of SUD according to the International Classification of Diseases, 10th Revision (ICD-10), and could have comorbid psychiatric disorders. Exclusion criteria included illiteracy, intellectual disability, or any condition preventing comprehension of the assessment protocol. The study received ethical approval from the Ethics Committee of Pontevedra-Vigo-Ourense, and all participants provided written informed consent.

The sample comprised predominantly male participants (78.2%), with an average age of 41.2 years. The primary substances of use varied, with cocaine (50.9%) being the most common, followed by cannabis (23.6%) and heroin (12.7%). A significant portion of the sample reported polysubstance use (57.4% reporting two or more substances) and regular tobacco use (87%). Notably, at least 66% of participants had a diagnosed mental health comorbidity, highlighting the clinical heterogeneity of the sample. Educational backgrounds were diverse, with 54.5% reporting basic education.

In a controlled setting, participants completed both the SAAS-p and SAAS-e during the same session. To minimize potential technological anxiety and unfamiliarity with digital interfaces, the paper-based version was administered first, followed by the electronic version after a brief interval of 3-5 minutes. This fixed-order design, while not allowing for the assessment of order effects, prioritized participant comfort and reduced potential confounding factors. The electronic version was administered via a secure digital platform developed by the Translational Neuroscience Group at the Galicia Sur Health Research Institute, ensuring data confidentiality.

Rigorous Psychometric Evaluation

The study meticulously analyzed several key psychometric indicators for both formats:

• Internal Consistency: Assessed using Cronbach’s alpha and McDonald’s omega coefficients. These measures indicate how well the items within a scale or subscale measure the same underlying construct.
• Interformat Reliability: Evaluated using intraclass correlation coefficients (ICCs), specifically a two-way mixed-effects model with a consistency-based single measurement. ICCs are preferred over Pearson correlations for assessing agreement between two measurements as they account for systematic differences.
• Mean Score Differences: Analyzed using Wilcoxon signed-rank tests to detect statistically significant differences between the scores obtained from the paper and electronic versions. Effect sizes were calculated using Cohen’s d to quantify the magnitude of any observed differences.

Key Findings: High Equivalence Across Formats

The results of the study strongly support the psychometric equivalence of the SAAS-e and SAAS-p in individuals with SUDs.

Internal Consistency: Both versions demonstrated excellent internal consistency across all subscales and dimensions. Cronbach’s alpha values for the SAAS-p ranged from 0.87 to 0.92, and for the SAAS-e, from 0.88 to 0.93. McDonald’s omega coefficients were similarly high, ranging from 0.88 to 0.93 for both formats. These figures align with and even surpass the reliability estimates reported during the original validation of the SAAS-p (Cronbach’s alpha between 0.90 and 0.92). The Global Scale and the Physical, Intellectual, and Social dimensions also exhibited strong internal consistency, with coefficients consistently exceeding 0.80 and often reaching above 0.90 for both SAAS-p and SAAS-e. This indicates that the electronic adaptation effectively preserves the internal structure and reliability of the scale.

Central Tendency and Dispersion: Mean scores and standard deviations between the SAAS-p and SAAS-e were remarkably similar. Absolute differences in mean scores ranged from a mere 0.3 to 2.0 points on scales that can reach several hundred points, representing a very small percentage of the total possible range. Variability differences were also minimal, further supporting the distributional similarity of scores across formats.

Intraclass Correlation Coefficients (ICCs): The ICC values were exceptionally high, consistently exceeding 0.97 for the Global Score, subscales (Intensity, Frequency, Change), and dimensions (Physical, Intellectual, Social). An ICC value above 0.90 is generally considered excellent, indicating a very high degree of agreement between the two administration formats. The Global Score achieved an ICC of 0.993, and the Intensity subscale registered an ICC of 0.992, underscoring the robust concordance between the paper and electronic versions.

Paired Sample Tests: Wilcoxon signed-rank tests revealed no statistically significant differences between the SAAS-p and SAAS-e for the Global Scale, or for the Physical, Intellectual, and Social dimensions. Similarly, the Frequency and Change subscales did not show significant differences. The only exception was the Intensity subscale, which exhibited a statistically significant but clinically negligible difference, with slightly higher scores on the electronic version (mean difference of −1.900). This difference, accounting for less than 2 points on a 270-point scale, was deemed to be well within acceptable equivalence thresholds and lacked clinical significance, especially given the exceptionally high ICC for this subscale.

Discussion and Implications

The study’s findings provide compelling evidence that the SAAS-e is a psychometrically sound and reliable alternative to the SAAS-p for use in clinical populations with substance use disorders. The high internal consistency, minimal mean differences, and exceptionally strong interformat reliability (ICCs exceeding 0.97) confirm that the electronic version accurately captures anhedonia levels comparable to its paper-based predecessor.

This research aligns with the growing trend of digital transformation in mental health assessment, offering a scalable and efficient tool for measuring a critical construct in SUDs. The ability to administer and score the SAAS electronically can significantly reduce administrative burden, accelerate data collection, and facilitate real-time monitoring of patient progress. This is particularly valuable in busy clinical settings and for large-scale research projects.

The study adhered to methodological recommendations for evaluating computer-based test equivalence, strengthening the validity of its conclusions. While a fixed-order design was employed to enhance participant comfort, the authors acknowledge that this approach precludes formal assessment of order effects. Future research employing counterbalanced designs and exploring different intervals between administrations could further solidify these findings.

Broader Impact and Future Directions

The successful validation of the SAAS-e has several important implications. For clinicians, it offers a reliable digital tool to assess anhedonia, a key factor in understanding addiction severity and predicting treatment outcomes. For researchers, it provides a means to collect anhedonia data more efficiently, potentially enabling larger and more longitudinal studies.

The study’s authors emphasize the need for future research to:

• Evaluate the SAAS-e in more ecologically diverse and unsupervised settings to assess its performance across different levels of digital literacy and technological access.
• Incorporate formal equivalence testing and explore absolute agreement models for ICC calculations to provide even stronger statistical evidence of equivalence.
• Conduct within-format test-retest reliability studies to further assess the temporal stability of the SAAS-e.
• Expand validation efforts to other clinical populations, such as those with mood disorders or psychosis, where anhedonia is also a prominent symptom.
• Leverage the newly adapted digital instructions to develop fully independent electronic patient-reported outcome (ePRO) systems and integrate the SAAS-e into mobile platforms or ecological momentary assessment frameworks.

Conclusion

The study concludes that the electronic version of the Self-Assessment Anhedonia Scale (SAAS-e) demonstrates strong psychometric equivalence to its paper-and-pencil counterpart (SAAS-p) in individuals with substance use disorders. The SAAS-e preserves the scale’s excellent internal consistency and exhibits high interformat reliability, making it a valid and reliable tool for assessing anhedonia in this clinical population. This development contributes to the growing body of evidence supporting the use of digital instruments in mental health assessment, paving the way for more efficient, scalable, and accessible clinical and research practices. While the findings are robust, continued research into real-world application and diverse settings will further solidify the utility of the SAAS-e in contemporary healthcare.