Measuring the subjective: revisiting the psychometric properties of three rating scales that assess the acute effects of hallucinogens

Bouso and colleagues frame the problem by noting that classic hallucinogens produce dramatic alterations in consciousness but that measuring those subjective effects remains challenging. The introduction reviews commonly used instruments in contemporary psychedelic research — including the Hallucinogen Rating Scale (HRS), the Mystical Experience Questionnaire (MEQ), and the Addiction Research Center Inventory (ARCI) — and argues that, while neurobiological and pharmacological methods have advanced, many psychometric tools have not been revisited with modern statistical techniques. The authors highlight that only a few instruments (notably some versions of the OAV/5D-ASC and the MEQ in limited contexts) have undergone rigorous confirmatory psychometric evaluation, whereas factorial structure, item loadings and reliability of HRS and ARCI have been incompletely assessed or analysed with outdated methods. This study therefore set out to re-examine the factor structure and internal consistency of the MEQ, HRS and the 49-item ARCI using data gathered after naturalistic ayahuasca ceremonies in Spain. The investigators aimed to apply confirmatory and exploratory factor-analytic methods appropriate to ordinal and dichotomous data (polychoric and tetrachoric correlations) and to propose tentative alternative structures or reduced versions when the original theoretical models did not fit the empirical data. The authors also translated the MEQ into Spanish and treated the work as a first transcultural psychometric effort in a sample of ayahuasca users.

Participants were recruited opportunistically at ayahuasca ceremonies in different parts of Spain. Researchers attended ceremonies conducted by a Spanish practitioner for personal-growth (non-religious) purposes and invited attendees to participate at the end of each session. After informed consent, participants completed sociodemographic questions and the Spanish versions of the ARCI (49-item short form), the 71-item HRS (Spanish adaptation), and a Spanish translation of the 30-item MEQ. The Research Ethics Committee of the Autonomous University of Madrid approved the procedures. The MEQ translation used back-translation and multiple bilingual clinicians to prioritise conceptual equivalence. The MEQ items were scored on a six-point Likert scale and results were expressed as proportions of the maximum possible score. The HRS used 71 computable Likert items (0–4) and 28 qualitative items in the Spanish adaptation. The ARCI short form consisted of 49 true/false items across five established subscales (MBG, PCAG, LSD, BG, A); score ranges for each subscale were reported. For the psychometric analyses, the investigators first attempted confirmatory factor analysis (CFA) for each instrument using AMOS 18 and Unweighted Least Squares when multivariate normality could not be assumed. When the theoretical models failed empirical identification or fit, they performed unrestricted exploratory factor analysis (EFA) based on polychoric correlations for ordinal items and tetrachoric correlations for dichotomous items. Multivariate normality was tested with Mardia’s criterion. Parallel analysis (minimum rank factor analysis) determined the number of factors to retain, and a Simplimax rotation (with Promin start and multiple random starts) was applied. Kelley’s criterion assessed residuals. Internal consistency was estimated with multiple indices recommended for non-linear data: standardized Cronbach’s alpha, Carmine’s theta, and McDonald’s omega for whole scales, and a multivariate reliability formula plus standardized alpha for subscales. The FACTOR 9.2 package was used for EFA steps. Pearson correlations among derived factor scores were calculated with Bonferroni correction to control type I error.

Sample: Of 167 completed booklets, 158 were usable (100 men). Mean age was 39 years (range 20–60) and average education approximated three years of university. Most participants (134, 84%) had prior psychedelic experience (mean reported 21 lifetime uses, range 0–350). Mean ingested ayahuasca dose was 113 ml (range 50–265); concentrations of DMT and beta-carbolines were unknown. Self-rated perceived dose intensity was reported as low by 19 (12%), medium by 120 (76%), and high by 19 (12%). MEQ: A CFA testing the four-factor model previously proposed failed to identify with these data, so the authors performed an EFA on a polychoric matrix (KMO = 0.91; Bartlett p < 0.0001). Optimised parallel analysis suggested two factors, and the bifactorial solution explained 59.11% of the common variance (Factor 1 eigenvalue = 11.3, 47.75%; Factor 2 eigenvalue = 2.7, 11.37%). Estimated reliabilities were high for the two factors (0.95 and 0.92) and for the whole test (θ = 0.94; ω = 0.94; standardized Cronbach’s α = 0.94). The solution produced few residuals (RMSR = 0.062; expected acceptable RMSR = 0.080) and strong simplicity indices. A Schmid–Leiman transformation identified a second-order factor with high loadings, and the authors note the possibility that the MEQ could be treated as essentially unidimensional in this sample. HRS: The CFA of the original 71-item, six-factor HRS produced suboptimal fit (GFI = 0.82; AGFI = 0.81; PGFI = 0.77; NFI = 0.76). The investigators removed 12 poorly discriminating items and re-ran EFA on 59 items. A polychoric matrix (KMO = 0.83; Bartlett p < 0.001) and parallel analysis supported retaining six factors. The six-factor solution accounted for 56.5% of the common variance. Multivariate consistency for factors ranged from 0.87 to 0.94; whole-scale indices were θ = 0.94, ω = 0.93 and αs = 0.93. Inter-factor correlations ranged from 0.20 to 0.60. Residuals were low (RMSR = 0.051) and simplicity indices were strong. All items showed acceptable discriminative capacity (>0.30) except for one reverse-scored item (#44), which was retained. ARCI: The ARCI 49-item structure could not be modelled confirmatorily owing to cross-loading items in the theoretical structure. A tetrachoric correlation matrix was used but required smoothing because it was not positive definite; Mardia’s statistic was reported as 0.08 and KMO = 0.74 (Bartlett p < 0.001). Parallel analysis initially suggested three factors, but this solution explained only 26.0% of common variance and was empirically unsatisfactory: only 18 of 49 items loaded significantly (>0.20). Using those 18 items, fit indices were acceptable (GFI = 0.96; AGFI = 0.95; NFI = 0.93; RFI = 0.92), and three factors showed good multivariate consistency (θ = 0.92, 0.89, 0.86; αs = 0.85, 0.81, 0.79). Inter-factor correlations were near zero. However, the 18-item solution explained 50.5% of common variance and the whole-scale internal consistency was reported as precarious (θ = 0.75; ω = 0.51; αs = 0.73). Correlations among derived subscales: All MEQ subscales correlated positively with each other (p < 0.0004, Bonferroni corrected). Within the HRS, Agitation showed no correlation with Sensitive Distortion, Security/Control, and Visual Distortion. Within ARCI, Activation correlated positively with Euphoria and negatively with Sedation; Euphoria and Sedation did not correlate. Cross-instrument correlations showed a general pattern of association between MEQ and HRS subscales (except Agitation), and Euphoria (ARCI) correlated with MEQ total and MEQ Mystical Ecstasy and with certain HRS subscales (Sensitive Distortion and Security/Control). The authors summarise that Mystical Ecstasy did not correlate with Agitation, Activation or Sedation, whereas Transdimensionality correlated with all HRS subscales but with none of the ARCI factors.

Bouso and colleagues interpret their findings as a partial validation and partial revision of the three questionnaires in the context of ayahuasca use. For the MEQ, the authors did not replicate the previously proposed four-factor model; instead they found a robust two-factor solution that explained a similar proportion of variance to prior reports and exhibited excellent internal consistency. They further show that a higher-order factor may account for both components, suggesting the MEQ could function as a unidimensional measure of mystical experience in this sample. The differences from earlier studies are attributed to sample size, assessment setting (in situ versus retrospective or clinical), substance differences (psilocybin versus ayahuasca), and possible cultural effects; the authors recommend further cross-cultural and cross-substance replication. Regarding the HRS, the original theoretical structure was not supported by CFA. After excluding poorly discriminating items, the investigators derived a six-factor solution from 59 items that accounted for 56.5% of variance and showed good multivariate and univariate reliability. The authors propose tentative labels for those factors (Sensorial distortion, Cognitive distortion, Agitation, Security/Control, Visual distortion, and Quality of the experience) and suggest their reconfiguration may better represent ayahuasca effects than the original model, which was initially based on a very small sample and reported limited psychometric detail. For the ARCI 49-item short form, the authors conclude that the original five-dimension formulation does not hold cleanly in this dataset. They identified an 18-item, three-factor solution (Euphoria, Activation, Sedation) with acceptable subscale reliabilities but weak whole-scale consistency and limited explained variance. Consequently, Bouso and colleagues consider ARCI-49 to be of limited feasibility for psychedelic research with ayahuasca and recommend further psychometric work across diverse substances before any definitive structure is accepted. The authors discuss the pattern of correlations across instruments and infer that mystical aspects of the experience (MEQ) are closely related to psychedelic phenomenology as measured by the HRS, typically occurring in the absence of agitation, activation or sedation, and that euphoria (ARCI) may be an important component of the mystical–psychedelic experience. Key limitations acknowledged include the single-substance design, unknown alkaloid concentrations in the ayahuasca preparations, and the need for larger and more diverse samples and for replication with other psychedelics and altered states. The paper concludes that modern psychometric techniques based on polychoric and tetrachoric correlations should be applied more widely in psychopharmacology and offers these preliminary factor solutions as a provisional basis for more rigorous measurement of subjective psychedelic effects.