Serotonin and serotonin receptors in hallucinogen action

Hallucinogenic drugs have a long history of human use for ceremonial, medicinal and recreational purposes. Classic serotonergic hallucinogens such as mescaline, psilocybin and lysergic acid diethylamide (LSD) produce profound alterations of perception, mood and thought, whereas the related compound 3,4-methylenedioxymethamphetamine (MDMA) produces marked mood elevation and increased interpersonal empathy with relatively little sensory distortion. Early pharmacological observations and subsequent receptor studies linked the actions of many hallucinogens to the central serotonin (5-HT) system, but the precise receptor mechanisms and their behavioural consequences remained incompletely understood. Halberstadt and colleagues set out to review and synthesize evidence from preclinical and human studies on the receptor interactions that underlie the behavioural effects of serotonergic hallucinogens and MDMA. The chapter focuses on receptor-level pharmacology (notably 5-HT2A), behavioural assays in animals (drug discrimination, head-twitch response, locomotor and investigatory paradigms, prepulse inhibition), structure–activity relationships, and human pharmacological blockade studies, and it highlights areas of therapeutic interest and unresolved questions about secondary receptor contributions.

A large body of preclinical evidence identifies the 5-HT2A receptor as the principal mediator of the behavioural effects of classic hallucinogens. Members of both the indoleamine and phenylalkylamine classes bind 5-HT2A with high affinity, and selective 5-HT2A antagonists (for example M100907, ketanserin, pirenperone) reliably block hallucinogen-evoked behaviours. In drug discrimination paradigms, a robust linear relationship (r = 0.938 for a phenylalkylamine series) exists between 5-HT2 affinity and substitution potency, and antagonist potency to block hallucinogen stimulus generalisation correlates with 5-HT2A affinity (antagonist correlation r = 0.88 for mescaline HTR blockade). The head-twitch response (HTR), a well-characterised hallucinogen-induced stereotypy in rodents, is abolished in 5-HT2A knockout mice and is blocked by selective 5-HT2A antagonists, supporting a necessary role for this receptor in that behaviour. Behavioural pattern-monitoring studies indicate that hallucinogens produce a characteristic profile in novel environments: decreased locomotion and investigatory behaviours with increased centre avoidance, effects that are largely 5-HT2A-dependent for phenylalkylamines. Indoleamines show more complex pharmacology: 5-HT1A receptor activity contributes to early locomotor suppression for drugs such as LSD and 5-MeO-DMT, whereas later hyperactivity and perceptual effects involve 5-HT2A receptors. Hallucinogens including LSD and DOI disrupt prepulse inhibition (PPI) in rodents, an effect blocked by M100907 and implicating 5-HT2A receptors; the mechanism for 5-MeO-DMT on PPI appears to involve 5-HT1A and 5-HT2C receptors as well. Although many hallucinogens bind 5-HT2C receptors with high affinity, the preponderance of behavioural evidence indicates that 5-HT2C is not the primary mediator of classic hallucinogenic effects. Correlations between 5-HT2C affinity and potency exist, but these may reflect parallel structure–affinity relationships between 5-HT2A and 5-HT2C sites; selective 5-HT2C antagonists typically fail to block hallucinogen-induced behaviours. The 5-HT1A receptor is important particularly for indoleamine compounds: indoleamines act at 5-HT1A autoreceptors to inhibit dorsal raphe firing, and 5-HT1A agonist actions contribute to several behavioural readouts and can attenuate certain 5-HT2A-mediated responses (for example 8-OH-DPAT reduces DOI-induced HTR). Other serotonin receptors (5-HT5, 5-HT6, 5-HT7) may modulate indoleamine effects, but available data and knockout studies suggest they are unlikely to be primary mediators. Dopaminergic systems contribute in restricted ways: LSD, but not most other hallucinogens, binds to multiple dopamine receptor subtypes and can show a dopaminergic component in its discriminative stimulus, which may be time-dependent and involve D4 receptors under some conditions. For DOM and other phenylalkylamines, discrimination and HTR are primarily 5-HT2A-driven. Structure–activity analyses and receptor mutagenesis begin to explain why chemically diverse hallucinogens engage the 5-HT2A site. Tryptamines and ergolines resemble serotonin’s core and tolerate tertiary amines, whereas phenethylamines show stringent steric and electronic requirements (for example the importance of 2,5-dimethoxy substitution and a free primary amine), with key receptor contacts proposed at residues such as Ser239, Asp155 and Ser242 in transmembrane helices. LSD’s rigid ergoline scaffold shows stereospecific binding constraints and limited tolerability for modification. MDMA differs pharmacologically from classic hallucinogens: it is a potent monoamine releaser, producing carrier-mediated release of serotonin (and to lesser extents dopamine and norepinephrine), and its behavioural and subjective profile is distinct. In animal drug discrimination MDMA generalises to compounds that increase 5-HT (norfenfluramine, p-methoxyamphetamine) and to psychostimulants (amphetamine, cocaine), indicating a compound stimulus with a dominant 5-HT-release component and a secondary dopaminergic/stimulant component. MDMA-induced locomotor activation depends on SERT-mediated release of 5-HT and involves 5-HT1B and 5-HT2A receptors as well as dopaminergic D1/D2 contributions and noradrenergic α1 mechanisms; MDMA disrupts PPI via 5-HT2A receptors. Human pharmacological blockade studies corroborate several preclinical conclusions. Psilocybin's subjective effects across dimensions of Oceanic Boundlessness, Anxious Ego Dissolution and Visionary Restructuralization are reduced or nearly abolished by ketanserin (5-HT2A antagonist) and by risperidone (mixed D2/5-HT2A antagonist), whereas haloperidol has more limited and sometimes dysphoria-enhancing effects; these findings support a dominant role for 5-HT2A in mediating psilocybin effects, with a minor dopaminergic contribution. Neuroimaging with [18F]FDG PET showed psilocybin increases glucose metabolism in prefrontal and temporomedial regions and the putamen, and these metabolic increases correlated with subjective measures. Clinical studies of MDMA show that selective serotonin reuptake inhibitor pretreatment (citalopram) markedly attenuates most subjective effects and cardiovascular responses, consistent with carrier-mediated 5-HT release as the principal mechanism. Ketanserin partially attenuates perceptual (visionary) effects, while haloperidol reduces euphoria but can increase dysphoric responses. Overall, the human data indicate MDMA’s subjective effects are largely driven by increased 5-HT release, with 5-HT2A and D2 actions contributing to specific perceptual and mood components respectively. Finally, several human reports noted enduring positive changes after psilocybin administrations in normal volunteers, with effects on mood and behaviour persisting at 14-month follow-up in one study; clinical trials exploring therapeutic applications (for anxiety and obsessive–compulsive symptoms, and for distress in terminal illness) are under way or planned.

Halberstadt and colleagues interpret the collective evidence as placing the 5-HT2A receptor at the centre of classic hallucinogen action, supported by converging lines of preclinical (drug discrimination, head-twitch response, prepulse inhibition, receptor knockout and antagonist studies) and human pharmacological blockade data. They emphasise that other receptor systems shape or modulate the phenomenology of these drugs: 5-HT1A activity contributes substantially to the effects of indoleamines and can functionally oppose some 5-HT2A-mediated behaviours; 5-HT2C, 5-HT5/6/7 and dopaminergic receptors may influence certain endpoints but are not primary mediators for most classic hallucinogens. The investigators argue that MDMA is pharmacologically distinct from hallucinogenic amphetamines, being chiefly an entactogen whose core subjective effects arise from carrier-mediated serotonin release, with secondary dopaminergic and noradrenergic contributions and some 5-HT2A-mediated perceptual effects. The authors acknowledge several complexities and limitations. Correlations between receptor affinities and behavioural potency must be interpreted with caution because parallel affinity at multiple 5-HT receptor subtypes can confound attribution. Early hypotheses such as the presynaptic 5-HT1A-mediated ‘‘presynaptic hypothesis’’ do not account for differences between phenylalkylamines and indoleamines, and selective 5-HT1A agonists are not hallucinogenic despite their presynaptic actions. Some human antagonist studies are limited by possible inadequate antagonist dosing or nonselectivity (for example cyproheptadine), and receptor contributions can be time- or context-dependent, as illustrated by the time-dependent dopaminergic component in LSD discrimination. In terms of implications, the chapter highlights that renewed human research has validated mechanistic hypotheses derived from animal studies and that structure–activity studies combined with receptor mutagenesis are clarifying ligand–receptor interactions at the molecular level. The authors note growing interest in therapeutic applications, citing preliminary clinical trials of psilocybin and MDMA-assisted psychotherapy and reporting persisting positive psychological effects in some volunteer studies. They recommend continued clinical investigation to further characterise therapeutic potential and to improve understanding of serotonergic modulation of behaviour.