Vaginal cytology is the microscopic examination of cells shed from the lining of the vagina or cervix, used to screen for cancer, gauge hormonal status, identify infections, and time ovulation in animals. Most people know it through its most famous application, the Pap smear, but the technique reaches far beyond routine cervical screening. It shows up in veterinary breeding programs, laboratory research on rodents, endangered-species conservation, and even forensic investigations of sexual assault.
How Papanicolaou Changed the Game
The story of vaginal cytology in human medicine starts with George Papanicolaou, a Greek physician working at Cornell in the early twentieth century. He began by studying vaginal smears in guinea pigs, charting how the cells changed across the reproductive cycle, and published those findings in 1917. When he turned the same technique on human samples, he noticed something unexpected: malignant cells could be spotted in smears taken from women with cervical cancer. He presented this finding in 1928, but the medical community was skeptical, insisting that only a tissue biopsy could diagnose cancer. It took more than a decade of collaboration with gynecologic pathologist Herbert Traut before their landmark 1943 paper, “Diagnosis of Uterine Cancer by the Vaginal Smear,” convinced clinicians that a simple scraping could catch cancer early.1PubMed Central. George Papanicolaou (1883-1962): Discoverer of the Pap Smear
The staining method that bears Papanicolaou’s name also evolved in phases. Between the 1910s and early 1930s, he used simple hematoxylin and eosin, which offered limited color contrast. By the late 1930s he introduced alcohol-ether fixation and aqueous waterblue staining, which made cells more transparent and helped distinguish cancerous cells from normal ones. A second refinement phase through the 1940s and 1950s added orange G and a cocktail called EA, producing the wide range of color reactions that pathologists still rely on to differentiate cell types at a glance.2Acta Cytologica. Inception and Development of the Papanicolaou Stain Method
The Cells and What They Reveal
Whether the sample comes from a human, a cow, or a rat, vaginal cytology revolves around recognizing the same basic cast of characters: parabasal cells, intermediate cells, and superficial cells. Parabasal cells are the smallest and roundest, with a large nucleus relative to the amount of surrounding cytoplasm. They sit deep in the vaginal lining and appear in smears mainly when estrogen levels are low or the tissue is thin. Intermediate cells are about two to three times larger, with more variable shapes. Superficial cells are the largest, flat and angular, and their nuclei often shrink to tiny specks or disappear entirely, a feature called pyknosis.3PubMed Central. Determining Proportion of Exfoliative Vaginal Cell during Various Stages of Estrus Cycle Using Vaginal Cytology Techniques in Aceh Cattle
The ratio of these cell types is what makes vaginal cytology useful. A smear dominated by superficial cells signals strong estrogen influence. One full of parabasal cells points to low estrogen, as you would see after menopause or during certain phases of an animal’s cycle. Intermediate cells fall somewhere between. By reading these proportions, a clinician or veterinarian can infer where in the reproductive cycle a patient or animal stands, without drawing blood for hormone assays.
How Estrogen Shapes the Vaginal Lining
Estrogen drives the vaginal epithelium to thicken, stratify, and eventually shed those large superficial cells. But the mechanism is not as simple as estrogen binding directly to the epithelial cells themselves. Research using tissue recombinants, where stromal (connective tissue) and epithelial layers from different genetic backgrounds were grafted together, showed that estrogen-driven cell proliferation works indirectly: it requires estrogen receptors in the stroma, not necessarily in the epithelium. When stromal receptors were present, epithelial cells multiplied regardless of whether they carried their own receptors. When stromal receptors were missing, nothing happened, even if the epithelial cells had working receptors.4PubMed. Role of stromal and epithelial estrogen receptors in vaginal epithelial proliferation, stratification, and cornification
Full cornification, the process by which the top layer of cells hardens and flattens into those distinctive superficial cells, required both stromal and epithelial estrogen receptors working together. This dual requirement helps explain why drugs designed to mimic estrogen can affect vaginal tissue differently depending on how they interact with each receptor population. Selective estrogen receptor modulators like ospemifene, for instance, produce a clear estrogenic shift in vaginal cytology in postmenopausal women, pushing the maturation index away from parabasal dominance toward a pattern that looks more like premenopausal tissue.5PubMed. Effects of ospemifene (FC-1271a) on uterine endometrium, vaginal maturation index, and hormonal status in healthy postmenopausal women
Atrophic Changes After Menopause
When estrogen drops after menopause, the vaginal lining thins and the cytology shifts toward parabasal and intermediate cells. A Pap smear may come back labeled “atrophic vaginitis” or just “atrophic pattern,” and the distinction matters more than it sounds. Research comparing those two read-outs found that “atrophic vaginitis” on a Pap report reliably indicates the presence of increased neutrophils, meaning actual inflammation in the tissue. An “atrophic pattern,” on the other hand, signals low estrogen without significant inflammation.6PubMed. Does a diagnosis of atrophic vaginitis on Papanicolaou test signify the presence of inflammation? So if your result says “atrophic vaginitis” and you have symptoms like dryness, burning, or discharge, the cytology is consistent with an inflammatory process, not just aging tissue.
Detecting Infections on a Pap Smear
The Pap smear was designed for cancer screening, but cytologists routinely spot infections along the way. In one study of women with urinary and genital complaints, Pap-stained cervical smears identified bacterial vaginosis in about a quarter of samples, Candida in roughly one in seven, and Trichomonas vaginalis in about one in eight.7PubMed Central. Determination of Trichomonas vaginalis, certain bacterial and fungal positivity, and risk factors in female patients with urinary and genital complaints Bacterial vaginosis is typically flagged by “clue cells,” epithelial cells coated with a stippled layer of bacteria that obscures the cell border. In a review of nearly 3,000 Pap smears, though, clue cells appeared in only about 3% of samples, and even when present they sometimes coexisted with lactobacilli or other organisms that complicated the diagnosis.8PubMed. Review of cytologic criteria of bacterial vaginosis: examination of 2,841 Papanicolaou-stained vaginal smears
One diagnostic trap that has gained attention is cytolytic vaginosis, a condition where lactobacilli overgrow and break down the vaginal epithelial cells, producing symptoms that mimic a yeast infection: lumpy discharge, irritation, and redness. Under the microscope, the giveaway is an abundance of lactobacilli alongside bare nuclei and cell fragments, but hardly any white blood cells and no yeast forms. A yeast infection, by contrast, shows candidal spores, hyphae, and a more prominent inflammatory response.9Journal of Lower Genital Tract Disease. Identification of Cytolytic Vaginosis Versus Vulvovaginal Candidiasis A scoping review of the cytolytic vaginosis literature found that microscopic features were more reliable than clinical symptoms alone for telling the two apart.10PLOS ONE. Scoping review of cytolytic vaginosis literature This matters because the standard treatment for yeast, antifungals, will do nothing for cytolytic vaginosis, which is typically managed by reducing vaginal acidity.
Vault Smears After Hysterectomy
If a woman has had her uterus and cervix removed, should she still get vaginal smears? The answer depends heavily on why the hysterectomy was performed. For women who had the surgery for benign conditions, a systematic review found that the value of routine vault smears is “largely unproven,” and the practice causes unnecessary anxiety while consuming healthcare resources.11PubMed. Vaginal vault smears after hysterectomy for reasons other than malignancy: a systematic review of the literature Another study echoed this, noting that while vault cytology can accurately detect local recurrence or vaginal intraepithelial neoplasia (VAIN) in women treated for cervical cancer, the prevalence of VAIN or vaginal cancer is so low in women who had a hysterectomy for benign reasons that routine screening is hard to justify, especially in resource-limited settings.12PubMed. Role of vault cytology in follow-up of hysterectomized women: results and inferences from a low resource setting
Even among women who had early-stage cervical cancer, one single-center study found that vault cytology did not change how recurrences were detected or managed, suggesting that other surveillance tools may be more useful.13International Journal of Gynecological Cancer. Vaginal vault smear cytology in detection of recurrence after hysterectomy for early cervical cancer The broader trend in clinical guidelines has been to scale back routine vault smears for women with no prior cervical disease.
Conventional Pap Versus Liquid-Based Cytology
The traditional Pap smear involves spreading cells directly onto a glass slide, while liquid-based cytology rinses the collection brush into a vial of preservative fluid, from which a thin, uniform layer is prepared in the lab. Proponents of liquid-based methods argue the cleaner background makes abnormalities easier to spot, and one comparison found far fewer unsatisfactory smears with liquid-based prep, along with higher sensitivity and diagnostic accuracy for detecting precancerous and cancerous lesions.14PubMed Central. Comparative Analysis of Conventional Cytology and Liquid-Based Cytology in the Detection of Carcinoma Cervix and its Precursor Lesions
Not everyone agrees that liquid-based prep is genuinely better at finding disease, though. A large randomized controlled trial concluded that liquid-based cytology did not outperform conventional Pap smears in sensitivity or positive predictive value for detecting cervical cancer precursors.15JAMA. Comparison of Liquid-Based Cytology With Conventional Cytology for Detection of Cervical Cancer Precursors: A Randomized Controlled Trial The practical upside of liquid-based methods is that the leftover fluid can be used for HPV DNA testing without recalling the patient for a second sample, which is why many labs have adopted the technique regardless of the detection debate.
Timing Breeding in Dogs and Cats
In veterinary practice, vaginal cytology is one of the most accessible tools for pinpointing the right moment to breed. As a female dog approaches ovulation, estrogen causes the vaginal epithelium to thicken and shed increasing numbers of superficial, cornified cells. A study of American Bully dogs found strong correlations between the cyclic changes visible on vaginal cytology and serum progesterone levels, confirming that the two methods track each other well.16PubMed. Timing of fertile period for successful pregnancy in American Bully dogs Because progesterone assays require a blood draw and lab turnaround, many breeders use vaginal cytology as a low-cost first screen and reserve hormone testing for confirmation.
Cats present a trickier picture. A study of intact queens found that oestrus was marked by 90% or more keratinised epithelial cells, but cellularity was poor in the majority of smears collected, including eight out of nine smears from queens known to be in oestrus.17PubMed Central. Vaginal cytology in the queen: clinical and cytological correlations The sparse cell yield means feline vaginal cytology requires a bit more interpretive skill and is generally less straightforward than in dogs.
Estrous Cycle Staging in Laboratory Rodents
Rats and mice cycle rapidly, roughly every four to five days, through four stages: proestrus, estrus, metestrus, and diestrus. Each stage has a distinctive cytological fingerprint. Proestrus smears are dominated by small, round, nucleated epithelial cells, often in clusters. Estrus brings large, flat, anucleated cornified cells. Metestrus is marked by a flood of neutrophils mingling with remaining cornified cells, and diestrus shows predominantly neutrophils with scattered nucleated epithelial cells.18PubMed Central. Vaginal cytology of the laboratory rat and mouse – review and criteria for the staging of the estrous cycle using stained vaginal smears
Staging the estrous cycle matters because hormone fluctuations can influence everything from drug metabolism to pain sensitivity. For a long time, researchers avoided using female rodents out of concern that cycling hormones would add too much variability to experiments. That assumption has been challenged. An analysis of 142 physiological traits measured across thousands of rats from over 50 inbred strains found no overall difference in variability between males and females. In fact, when sex differences in variance did appear, males were more variable nearly 60% of the time.19PubMed Central. Breaking the Cycle: Estrous Variation Does Not Require Increased Sample Size in the Study of Female Rats The takeaway has been influential: excluding females from studies or requiring larger sample sizes to “account for” their cycles is not supported by the data, and vaginal cytology can simply be recorded alongside other measurements as an informational covariate.
Even the staging process itself is getting a technology upgrade. A machine-learning model trained on 730 vaginal cytology images from mice achieved about 80% accuracy in classifying cycle stages, comparable to human experts and faster by a wide margin, processing 100 images in under three minutes.20PubMed Central. A novel method for estrous cycle staging using supervised object detection Automating this step could make it far more practical for large-scale studies to include estrous stage as a variable without adding hours of manual microscopy.
Conservation and Endangered Species
When you have only a handful of breeding animals left in a species, getting the timing right is everything. Vaginal cytology has been applied to giant pandas to correlate cell changes with other indicators of estrus and help predict ovulation for artificial insemination programs.21Zoo Biology. Vaginal cytology and vulvar swelling as indicators of impending estrus and ovulation in the giant panda In Hawaiian monk seals, one of the world’s most endangered marine mammals, the periodic appearance of cornified vaginal cells tracked with estrogen and progesterone fluctuations and helped researchers map out an estrous cycle of about 35 days.22Marine Mammal Science. Concentrations of Estrone Sulfate and Progesterone in Plasma and Saliva, Vaginal Cytology, and Bioelectric Impedance During the Estrous Cycle of the Hawaiian Monk Seal
The technique proved especially valuable in the recovery of the black-footed ferret, once considered extinct in the wild. Researchers compared vaginal cytology across black-footed ferrets, Siberian polecats (a close relative), and domestic ferrets, and found the method extremely useful for managing breeding in the captive population.23PubMed. Comparative vaginal cytology of the estrous cycle of black-footed ferrets, Siberian polecats, and domestic ferrets In all these cases, vaginal cytology offered a noninvasive, inexpensive complement to hormone assays that might require sedation, specialized equipment, or both.
Vaginal Cytology in Forensic Investigations
An emerging application sits at the intersection of cell biology and criminal justice. In sexual assault cases involving digital penetration, traditional DNA evidence may be absent or insufficient. Researchers have explored whether vaginal epithelial cells transferred to a suspect’s hand can be distinguished from the skin cells already there. Using morphological and autofluorescence properties, one study showed that samples collected after digital penetration contained markedly higher numbers of cells classifiable as vaginal tissue, with high probability scores, compared to samples from hands that had not contacted vaginal tissue.24PubMed Central. Differentiation of vaginal cells from epidermal cells using morphological and autofluorescence properties: Implications for sexual assault casework involving digital penetration The work is still relatively early-stage, but it points toward a scenario where cytological evidence could corroborate allegations that currently rest on testimony alone.
Staining Choices and Practical Constraints
Not every lab uses the classic Papanicolaou stain, especially in veterinary and field settings where access to the full five-dye protocol is impractical. A comparison of six common stains for exfoliative vaginal cytology found that Giemsa, Leishman, and Wright stains held up well over 35 days, maintaining good color differentiation, while gentian violet, nigrosin, and eosin faded more quickly.25Sahel Journal of Veterinary Sciences. Comparative Durability of Common Stains Used for Exfoliative Vaginal Cytology For field work or low-resource clinics where slides might sit before being read, stain durability can determine whether the sample is still interpretable weeks later. Some researchers have even demonstrated that unstained vaginal smears in rodents can be staged reliably using phase-contrast or simple bright-field microscopy, further lowering the cost barrier.26PubMed Central. Laboratory Rat Vaginal Cytology: A Quick Method of Staging Unstained Cells
In veterinary oncology, cytology also plays a diagnostic role beyond reproductive staging. Canine transmissible venereal tumor, a sexually transmitted cancer unique to dogs, shows a characteristic look on vaginal or genital cytology: round to ovoid neoplastic cells with coarse chromatin, a prominent nucleolus, and distinctive cytoplasmic vacuoles. Cytology is often the first step in diagnosis and can also track how well chemotherapy is working, since neoplastic cells may disappear from smears before the tumor mass visibly shrinks.27PubMed Central. Canine Transmissible Venereal Tumor: Anatomical Locations, Chemotherapy Response, and Epidemiological Aspects at a Veterinary Teaching Hospital in Brazil (2012–2022)28Colloquium Agrariae. Histopathological and Cytological Analysis of Transmissible Venereal Tumor in Dogs After Two Treatment Protocols