Perennial ryegrass (Lolium perenne L.) is a popular cool-season forage and turfgrass in temperate regions. Due to its self-incompatible and out-crossing nature, perennial ryegrass may show a high degree of heterozygosity. Perennial ryegrass generally is susceptible to drought stress, but variations of drought response of individual genotypes within a particular accession or cultivar are not well understood. The objective of this study was to characterize phenotypic diversity of drought tolerance within and among accessions in relation to genetic diversity in perennial ryegrass. Five individual genotypes from each of six accessions varying in origin and growth habits were subjected to drought stress in a greenhouse. Leaf wilting, plant height, chlorophyll fluorescence (Fv/Fm) and leaf water content (LWC) differed significantly among accessions as well as among genotypes within each accession under well-watered control and drought stress conditions. Fv/Fm was highly correlated with LWC under drought stress. Genetic diversity among and within accessions were identified by using previously characterized 23 simple sequence repeat markers. Across accessions, the mean major allele frequency, gene diversity, and heterozygosity values were 0.66, 0.43, and 0.66, respectively. Accessions with closer genetic distance generally had similar drought responses, while accessions with greater genetic distance showed distinct drought tolerance. Significant differences in drought tolerance among and within accessions, especially for individual genotypes within one accession, indicated that variations of drought response could be used for enhancing breeding programs and studying molecular mechanisms of stress tolerance in perennial ryegrass.