Molecular Biology of Serotonin Receptors

Jean Chen Shih, Kevin J.-S. Chen, and Timothy K. Gallaher

For decades it has been suggested that the factors involved in mediating mental states, including mental illness, are neurochemical. Successful drug therapies for mental illnesses have been developed which reinforce the causative nature of the neurochemical environment and the role it plays in mental states. Such empirical knowledge has preceded the physiological basis for the actions of both therapeutic drugs and injurious drugs of abuse. Recently, data have been emerging from the field of molecular biology to provide a molecular basis for the neurochemical workings of the brain. Specifically, the receptors for numerous neurotransmitters and drugs have been cloned and their molecular structures have been determined. The number of different receptors identified by molecular cloning has provided insight into the complexity of the nervous system and allows us to begin devising novel methods for studying the brain and nervous system.

This chapter will discuss key concepts of our knowledge base concerning serotonin [5-hydroxytryptamine (5-HT)] receptors and present new approaches to study the brain and nervous system. The approaches discussed here for 5-HT receptors are also applicable to other neurotransmitter systems, and, indeed, an integrative approach encompassing all neurotransmitter systems will be necessary to further our understanding of the brain and nervous system (see also Cholinergic Transduction, Molecular Biology of the Dopamine Receptor Subtypes, Dopamine Receptor Expression in the Central Nervous System, Signal Transduction Pathways for Catecholamine Receptors, and Histamine).

Molecular cloning has provided solid molecular evidence concerning the structures of receptors and has provided cDNA which is of use in many ways for studying the brain. Many receptors have been cloned, and the data conclusively confirm the identification of multiple 5-HT receptors. The knowledge of the primary structures of the receptors is of great use for studying the function and architecture of the brain and how it may relate to mental states (see also Serotonin Receptor Subtypes and Ligands and Molecular Biology of Serotonin Receptors: A Basis for Understanding and Addressing Brain Function).

Classification schemes for 5-HT receptors have been used for more than 30 years. Initially, Gaddum and Picarelli (13) described D- and M-type receptors. Later pharmacological studies defined 5-HT1a, 1b, 1c, 1d, and 1e receptors and a 5-HT2 receptor. A 5-HT3 classification was later added. Each of these nomenclature schemes was based upon the ligand specificities of the individual receptors. Such approaches have proven to be valuable in our gaining knowledge of the 5-HT receptors, but also have inherent limitations. A new basis for their classification has arisen with the advent of molecular cloning techniques which make possible the determination of primary structures and gene structures. These structures provide a solid foundation on which to base 5-HT receptor classification. Using the molecular information presents the best approach to this task in that the data provided for the basis of the classification is the most unequivocal available. The goal of an objective and unambiguous classification scheme can hopefully be met using this data. Eventually the limitations of this molecular-based scheme may become apparent, but currently the classifications as will be described here may be the best we can do. Refinements to the classification nomenclature presented here might be in order, but the general basis and classification should hold, at least for awhile, until our understanding and knowledge increases to a point where this system also becomes too tenuous. It may be time for receptorologists to devise a nomenclature along the lines of enzyme nomenclature where the receptors can be classified as objectively as possible. Granted that the nature of the numerous receptor families makes this task more difficult than with enzymes, nonetheless we should approach the task wholeheartedly.

Two types of 5-HT receptors have been determined by molecular cloning: G-protein-coupled 5-HT receptors (42) and ligand-gated ion channels (18). Both types of 5-HT receptors fall into distinct supergene families of receptors which are related to each other and which are defined by their structure and function.

5-HT3 receptors are members of the ion-gated family of receptors (27). The model protein for this family is the nicotinic acetylcholine receptor, and this family includes cation channels (the nAchR, 5-HT3 receptor) and anion channels (GABAa receptor, glutamate receptor). The 5-HT3 receptor primary sequence indicates that it contains four transmembrane domains as determined by hydrophobicity analysis of its primary structure. It also contains the putative ligand binding domain which is found in the extracellular region of the protein, and it is characteristically seen in the primary structure of a receptor of this class in a disulfide loop region near the amino terminal of the protein. Also, the M4 helix is seen to contain numerous serine residues which may form an aqueous channel for the ion transport. The 5-HT3 receptor is clearly of this class, yet only one subunit has been cloned. Most members of this family are pentameric and consist of heterologous subunits. The 5-HT3 receptor subunit which has been cloned is able to form a 5-HT-gated ion channel, so it is possible that the 5-HT3 receptor is a homomeric pentamer, but given the diversity of the different subunits found in this family (e.g., the GABA receptor has upwards of 10 different subunits determined by molecular cloning) it is likely that more 5-HT3 subunits will be found by molecular cloning. The 5-HT3 receptor was designated 5-HT3 to keep in accord with the nomenclature scheme of designating the 5-HT receptor class with a number. The 5-HT1 and 5-HT2 classes had been defined pharmacologically, so the clearly unique 5-HT ligand-gated ion channel receptor was deemed the 5-HT3 receptor. Molecular cloning confirmed this assignment.

All known 5-HT receptors except for 5-HT3 receptors are members of the G-protein-coupled (GPR) family of signal transducing receptors. Four different classes of 5-HT GPR receptors have been cloned: 5-HT1, 5-HT2, 5-HT5, and 5-HT6. Another class, the 5-HT4, has been identified but has yet to have had a member cloned. Multiple subtypes of 5-HT GPR receptors in each class have been observed as well. The nomenclature used to describe the receptors is based upon the pharmacological classification of Peroutka and Snyder (36), but is taken a step further to consistently integrate the classification of receptors based upon the data obtained from cloning with the pre-cloning pharmacologically based classification scheme; that is, new subclasses are numbered sequentially. Originally, two types of 5-HT receptors (designated 5-HT1 and 5-HT2) were defined (36) based upon their differing affinities for 5-HT and the neuroleptic antagonist spiperone. Currently, classification is based upon primary sequence homologies (and, to a lesser degree, second messenger responses). The strength of the pharmacological assignments has been demonstrated by the cloning data in that receptors classified as members of the 5-HT1 family (1a, 1b, 1c, 1d, 1e, and 1f) share greater sequence similarity with each other, with 30–50% sequence identity among the 5-HT1 receptors compared to the 5-HT2 receptors, which share less than 30% identity with the 5-HT1 receptors but greater than 30% with other 5-HT2 class receptors. The one exception here is the 5-HT1c receptor, which is clearly a member of the 5-HT2 receptor class (20, 38) but was originally classified in the 5-HT1 class due to its high affinity for 5-HT compared to the 5-HT2 receptor. Consequently, a revised nomenclature has been presented to more appropriately reflect the relationships between the receptors determined by molecular cloning.

Another subclass has been defined and called the 5-HT4 receptor (4, 8). These receptors demonstrated distinct pharmacological profiles and were positively linked to cyclic AMP production. The uniqueness of these observations merited the definition of the new 5-HT4 class of 5-HT receptors. This 5-HT4 class is the only defined subclass not to have a member cloned, so the primary sequences of any of this class are not known, and consequently the validity of the designation at a molecular level cannot be assessed. Because pharmacological and physiological definitions have been accurate, it is likely that the 5-HT4 receptors will stand as a class once the molecular data have been determined. Newer receptor clones (5-HT5 and 5-HT6 subtypes) have been obtained which do not fit the criteria for any pharmacologically defined 5-HT receptor subtype in either their pharmacological profile or, more importantly, their primary sequences. To continue using the nomenclature for the 5-HT receptors, the newest receptor was deemed the 5-HT5 receptor (37). Since this initial 5-HT5 receptor was cloned, another 5-HT5 receptor was cloned and so these two receptors are designated 5-HT5a and 5-HT5b (29). Another GPR 5-HT receptor has been cloned which is unique in primary sequence, pharmacological profile, and physiological response and has been deemed the 5-HT6 receptor (32). This receptor activates adenylate cyclase, but its pharmacological profile indicates that it is not a 5-HT4 receptor, and its primary sequence cannot place it in the 5-HT1, 5-HT2, or 5-HT5 class of receptors. Whether or not a 5-HT6b receptor or 5-HT7 subtype will be found is not known, but chances are likely that newer subtypes will continue to be cloned. It is also possible that 5-HT6 receptors may turn out to be 5-HT4 receptors based upon sequence similarities because the pharmacological profiles can be misleading in indicating primary structure.

The classifications of the 5-HT receptors that have so far been cloned are presented in . These include four classes of mammalian GPR 5-HT receptors, including 11 individual receptors. Also included in are the three 5-HT receptors which have been cloned in drosophila, namely, the cloned 5-HT4 subtype and the ligand-gated ion channel 5-HT3 receptor. With this inundation of molecular data, and likely more coming, it is time to examine what these data tell us and how we can use these data to determine how the brains works.

Now that we have such a wealth of primary sequence data, can we use these data to determine the molecular workings of the receptors? The answer is yes, and these types of studies have just begun. The comparisons of the primary structures have provided clues as to which amino acid residues are necessary for the functioning of the specific receptors. Also, experiments on other receptor and GPR proteins have contributed greatly to the understanding of 5-HT receptors. The initial cloning of the b-adrenergic receptor (bAR) (6) and the discovery that it is homologous to rhodopsin provided the first key in understanding the potential ligand binding sites for the GPR receptors. Deletion analysis of the bAR provided the initial evidence that the ligand binding site resides in the intramembrane regions of the GPR receptor (7). This observation established an analogy for receptor agonist-induced allosteric activation with the light-induced allosteric activation of rhodopsin that is mediated by the retinal chromophore known to be located in the intramembrane domains of the protein. Perusal of catecholamine receptor primary sequences pointed to potential sites on the protein for ligand interactions. Conserved aspartic acid residues in the second and third transmembrane domains of the receptors for biogenic amine agonists were obvious candidates as ligand binding residues. Site-directed mutagenesis served to support these ideas and indicate that aspartic acid in the third transmembrane acts as a counterion for the agonist amine group (44, 49). The 5-HT GPR receptors all contain the analogous TMD3 aspartic acid, and mutagenesis of this residue in the 5-HT2 receptor (Asp155) to asparagine results in profound losses of affinity of the ligand (agonists and agonists) to the receptor (50).

Thus data from the amino acid sequences derived from molecular cloning pointed to potential residues important for ligand recognition. This sort of comparative examination takes into account evolutionary considerations in that it is considered that the receptors have arisen from divergent evolution from a common ancestral structure. The unique properties of the myriad receptors stems from the accumulated differences in primary sequences. The more similar the binding properties, the more similar may be the primary sequences. Or at least certain functionally important residues may be present at analogous sites in the various primary sequences.

Support for the role of the TMD3 aspartic acid in ligand binding is found in the examination of primary sequences of all the known GPR. Only in GPR where there is an aliphatic amine in the agonist structure is this TMD3 aspartic acid found. In receptors for ligands which do not contain aliphatic amine groups, such as the adenosine receptor (43) and cAMP receptor (39), no Asp appears in the third TMD. Conversely, an aspartic acid in the second transmembrane domain is conserved in all GPR receptors without respect to ligand structure. The results of mutagenesis studies of this residue in catecholamine (44, 49), muscarinic (12), and 5-HT (50) receptors indicates that it is not as necessary for agonist binding as the TMD3 aspartic acid and that it is necessary for the activation of the second messenger response. The second messenger response is allosterically mediated, and the Asp in TMD 2 does play a role in allosterism (17), but its absolute necessity for second messenger response in all GPR receptors is unclear (46). These observations are indicative of how comparison of the primary sequences of these structurally related receptors in an evolutionary perspective can provide clues to their molecular workings.

An asparagine residue at the interface of the third extracellular loop and the seventh transmembrane domain has been shown to be necessary for the binding of aryloxyalkylamine antagonists (e.g., pindolol, propanolol, and alprenolol) to 5-HT1a receptors and rat 5-HT1b receptors. When this asparagine (Asn385 in 5-HT1a) is mutated to valine, the binding of aryloxyalkyamines to the 5-HT1a receptor is decreased 40- to 150-fold, but other agonists and antagonists demonstrate only minor changes in binding affinity (15). Also, the rat 5-HT1b receptor binds aryloxyalkylamine antagonists, but the human 5-HT1b receptor (also known as 5-HT1Db) has almost a thousandfold lower affinity for the aryloxyalkylamines. Rat 5-HT1b receptors contain asparagine at the cognate position, whereas the human receptor contains threonine. When the threonine of the human receptor (THR355) is mutated to asparagine, high-affinity binding of the aryloxyalkylamine antagonists is conferred (31, 33, 34). These experiments are excellent examples of how comparison of primary structures and ligand binding profiles can indicate functionally important amino acid residues.

Another example of this approach for exploring the structure of 5-HT receptor binding sites is seen in the 5-HT2 receptor and the differences in the human and rat primary structures. The primary sequences of human, rat, and mouse () share great homology, especially in the transmembrane regions where only three amino acid differences are observed. One of the differences is at position 242 in the primary sequence that is found in the putative fifth transmembrane domain. In rat and mouse this residue is an alanine, but in the human 5-HT2 receptor a serine is found in this position (5, 52). An analysis of position 242 and its proximity or contribution to the binding site of 5-HT2 receptors has been carried out using tryptamine agonists (14). Two naturally occurring compounds, bufotenin (5-hydroxy-N,N-dimethyltryptamine, 5-OH-DMT) and psilocin (4-hydroxy-N,N-dimethyltryptamine, 4-OH-DMT), were examined for their binding to human and rat 5-HT2 receptors. These two compounds differ only in their hydroxy substituent at the indole ring. Psilocin is at the four position and bufotenin is at the five position, which is the analogous position to the endogenous agonist 5-HT. It was observed that bufotenin binds with near equal affinity to both human and rat 5-HT2 receptors, whereas psilocin exhibited 15-fold higher affinity for the human receptor than for the rat receptor and that the binding of psilocin to the human receptor was comparable in affinity to the binding of bufotenin to either the rat or human 5-HT2 receptor. These results indicate that a determinant for psilocin binding is present in the human receptor but not in the rat receptor. Our knowledge of the binding mechanism of GPR agonists suggests that Ser242 of the human 5-HT2 receptor is the psilocin-specific determinant.

The results of bAR mutagenesis studies provide a strong analogy with which to analyze the psilocin binding to rat and human receptors (45). bAR agonists have two hydroxyl groups on their aromatic rings. bAR receptors also have serines at positions in their primary structure analogous to positions 239 and 242 in the 5-HT2 receptors. Site-directed mutagenesis has shown that these two serines serve as agonist binding residues. The evolutionary consideration that GPR receptors will conserve their basic functional mechanisms suggests that the 5-HT receptor agonists will bind analogously. The examination of numerous TMD5 sequences support the notion that residues of particular importance for specific ligand binding have been conserved. All receptors with dihydroxy-substituted aromatic rings contain the Ser-X-X-Ser sequence motif, whereas 5-HT receptors, which contain only one hydroxy substituent on the aromatic indole ring, contain only one serine or threonine in the fifth TMD that corresponds to the first serine of the S-X-X-S motif (Ser239 in the 5-HT2 receptor). Only the human 5-HT2 receptor contains the full S-X-X-S motif. Furthermore, the receptors for muscarinic receptors do not contain any analogous serines in the fifth TMD. The loss of these serines in the muscarinic receptors during evolution (or, conversely, the gain of the serines in the receptors for hydroxy-substituted agonists) is consistent with the evolutionary analysis of these GPR receptors. In the human 5-HT2 receptors the presence of the serine at position 242 provides a binding site for the 4-hydroxy group of psilocin. The geometries of the two tryptamine agonists and the helical fifth transmembrane domain peptide structure indicate that if 4-OH-DMT and 5-OH-DMT bind analogously, then 242 can serve as a hydrogen bond site for psilocin and 239 can serve as the site for the 5-hydroxy-substituted bufotenin. Based on a normal a-helical structure the 239 and 242 serines would be 60° apart and 4.5Å would separate them in the membrane. Correspondingly, 4- and 5-substituted indoles differ by 60° with respect to the planar aromatic ring. The pharmacological properties of psilocin binding to rat and human 5-HT2 receptors support a binding site structure where the 4-or 5-hydroxy groups of the indole ring can interact with either Ser242 or Ser239, respectively. Such a prediction is in accord with the proposed binding site of the bAR receptor (26).

Another study has demonstrated the importance of position 242 in the 5-HT receptor in ligand binding. It has long been recognized that human and porcine 5-HT2 receptors differed in pharmacological profile with respect to one antagonist, mesulergine (35), which is seen to have a much lower affinity for the human and pig receptors than for the rat receptors. Site-directed mutagenesis of the human receptor where Ser242 was mutated to alanine indicated that this residue is the determinant for the difference in mesulergine affinity in rat and human receptors. Substitution of serine with alanine caused the mutant receptor to exhibit much greater affinity for mesulergine than the wild type, and the Ki value for this binding was nearly identical to that of the wild-type rat 5-HT2 receptor (21). The results are interesting because, while they do not indicate a binding site epitope, they do indicate that position 242 is in the binding site region.

Suggestions for ligand binding sites in 5-HT receptors based upon primary structure data in combination with pharmacological data serve to provide solid bases on which to build an accurate model of 5-HT receptor structure and function. These data must be applied to any computer modeling of the receptors as valuable constraints. The modeling, in turn, can reflect back on the feasibility of the proposed interactions of the ligands with the receptors. The data from the psilocin experiments, when examined in molecular model building, indicate the existence of a binding site where the 4 or 5 hydroxy can hydrogen bond with serine 242 or 239, respectively (unpublished observations). Furthermore, when the agonist is placed in this position, such a binding geometry of the indole nucleus reflects the basis for the differing affinities of mesulergine for the rat and human receptor. Serine 242 of human receptor is in a position to interact with the nonpolar saturated carbon of the mesulergine, which would cause a destabilization in its binding site. On the other hand, the nonpolar alanine does not present the same electrostatic destabilization for the saturated carbon of mesulergine that is in the vicinity of position 242. Such hypothetical model building will serve us until more precise physical methods such as x-ray crystallography or nuclear magnetic resonance are made possible with the availability of sufficient amounts of purified receptor.

The study of the genes for 5-HT receptors has also provided insights into their relationships with each other and with other GPR family members. 5-HT receptors fall into one of two categories with respect to their gene structures: either intronless genes or intron-containing genes. The 5-HT1a receptor was the first 5-HT receptor gene to be isolated due to its sequence similarity to the bAR and was seen to be intronless (9, 22. Numerous 5-HT1 receptor subtypes (5-HT1a, 1b, 1d, 1e, 1f) are coded for by intronless genes. This is an important finding because it provides one more basis to categorize these receptors as a distinct subclass. To the classification of these receptors into the 5-HT1 class based upon pharmacological, physiological, and molecular structures is added their intronless nature.

5-HT2 receptor subclass members contain introns. The 5-HT2 receptor was the first 5-HT receptor to be shown to contain introns (5, 52). The gene consists of three exons separated by two introns (). The other members of the 5-HT2 family, the 5-HT1c (also known as 5-HT2c) and the 5-HT2F (also known as 5-HT2b), now have also been shown to be intron-containing genes (11). Their evolutionary closeness is indicated by a partial conservation of intron–exon structure in all three of these 5-HT2 receptor subtypes. Two intron–exon junction structures are shared in each of the three 5-HT2 receptors and indicate that, as for the 5-HT1 subclass, the pharmacologically, physiologically, and molecularly defined class of 5-HT2 receptors is also defined by gene structure.

These findings reflect the divergent and convergent nature of the evolutionary processes that have resulted in the 5-HT subclass of GPR family receptors. Presumably all GPR family members have a common ancestral source, and divergent evolution has resulted in the receptors that bind the different ligands. As for the 5-HT receptors, we can see how receptors converged during evolution to use the same ligand as their activator. This is seen in the close relationship between the 5-HT1 receptors and the bAR class of receptors. Despite the fact that the 5-HT1 and 5-HT2 receptors share the same activating ligand, the 5-HT1 receptors are more closely evolutionarily related to the bAR receptors as determined by their primary sequence homologies. 5-HT1 receptors all share 50–60% sequence identity with each other but only approximately 30% identity with 5-HT2 receptors. The genetic data where the 5-HT1 receptor genes are all intronless whereas the 5-HT2 receptor genes contain a shared intronic structure further supports this assessment.

The most recently found members of the 5-HT receptor subgroup of GPR receptors contain introns. The 5-HT5a and 5-HT5b receptors each have one intron found in the region of the protein that constitutes the third intracellular loop (29). Furthermore, a cDNA for the 5-HT6 receptor was isolated that contained an unspliced intron (32), indicating that the 5-HT6 receptor has at least one intron. The presence of introns in the gene structures of these two 5-HT receptors, along with the 30–40% sequence identity shared with both 5-HT1 and 5-HT2 receptors, supports the conclusion that they reflect new 5-HT receptor subclasses and are not members of the 5-HT1 subclass.

The data garnered from, and the techniques made available by, molecular biological methods for studying 5-HT receptors provide new avenues to approach fundamental questions in neurobiology. One clear benefit is the ability to pharmacologically assess a pure receptor population. Pharmacological assays using brain tissue preparations have proven to be of great benefit in characterizing receptors and in developing specific ligands for receptors but have built-in limitations. Tissue preparations always contain multiple receptors and receptor subtypes. The ability to separate the individual receptors is overcome by the availability of cDNA for individual receptors and the ability to express the receptor in a mammalian cell. Such a "clean" environment can be used to screen newly synthesized drugs and can identify receptors in the brain that are being affected by a particular drug. This approach can also be of use in determining the basis of actions for therapeutic treatments. For example, tricyclic antidepressants used therapeutically are successful in treating certain mental disorders. The newly cloned 5-HT6 receptor is seen to have high affinity for this class of compounds and may indicate at least one of the sites in the brain where the tricyclic antidepressants exert their effects.

This same ability to express receptors in mammalian cell lines can also be of use in physiological studies where second messenger responses generated by a receptor, and the basis of the receptor-mediated response, will be examined. For example, the 5-HT1a receptor initiates different responses due to 5-HT stimulation depending on the cell line in which the cDNA is expressed (25). One can envision future "mixing and matching" experiments where receptors are transfected into cell lines that have known a G protein being expressed to do an inventory of, and obtain rank orders of potency for, the interactions of the receptors and the various G proteins.

The availability of the cDNA also makes it possible to express the receptors in a high yield expression system where large amounts of receptor can be produced. Such a technique can provide protein sufficient for reconstitution studies where the receptors can be reconstituted with various G proteins. Such a method would not only provide information concerning interactions between specific receptors and specific G proteins, but would also provide an environment where kinetic analysis of the GTP hydrolysis reaction which takes place in GPR-mediated signal transduction could be examined. A rigorous biochemical analysis of the signal transduction event will be of great use for computer-based simulations of neural function in providing accurate constants upon which to base the simulation's algorithms.

We also can foresee that the above-mentioned approaches can be seen as part of a comprehensive whole where a multidisciplinary examination of brain function is done based upon a neuromolecular anatomical mapping of the brain. The brain and nervous system are made up of billions of cells providing a myriad of neural connections between cells. Each cellular connection serves to communicate with and regulate its connected neighbor. The patterns of these connections and the effect of the neural networks on brain activity are believed to account for all behavioral and psychological activity of the organism. Drugs work by binding to a specific receptor and altering the endogenous regulation of the neurotransmitter-regulated brain activity. Until recently it was impossible to try to identify the components of these systems at more than an unrefined basic level. With the availability of specific genes that code for 5-HT receptors, the distribution of individual 5-HT receptors can be examined in nervous tissue. This method makes use of already existing techniques for mapping anatomical structure using the method of thin brain slicing and in situ hybridization. Using the computer-assisted technique to store and analyze this topographical information, a map of all known 5-HT-receptor-containing neurons in terms of their anatomical location can be achieved. Computer-aided visualization of brain based upon these data would provide an amazing anatomical insight at a higher level than mere structural topology. The methods of anatomy, combined with the molecular biological methods of identifying nucleic acids by hybridization, provide a greater analysis of the organization of the brain than ever before.

published 2000