Abstract: This study investigates the effects of prior austenite grain size (PAGS) ranging from 22 to 62 μm on the crystallographic characteristics and hierarchical structures of bainite and martensite in a high-strength low-alloy steel. Bainite dominates when the PAGS ≤ 52 μm, whereas martensite prevails at a PAGS of 62 μm owing to elevated martensite start temperature. Using electron backscatter diffraction analysis and prior austenite orientation reconstruction, the crystallographic characteristics (variant selection and variant pairing) were analyzed, and more than 100 prior austenite grains (PAGs) were examined per specimen. As the PAGS increased, the total average number of variant types per PAG in the bainite specimens were 14.6, 14.1, and 18.3, thus indicating weakened variant selection at larger PAGSs. Smaller PAGs tend to form fewer variant types owing to insufficient autocatalytic nucleation. Variant pairing in the bainite specimens shows a decrease in the fractions of variant 1 (V1)/V2 and V1/V4 pairs and an increase in the fraction of V1/V8 pairs as the PAGS increases. By contrast, the martensitic specimen favors the formation of variants within the same close-packed plane (CP) group, particularly the self-accommodating V1/V2 pair. This is attributed to the enhanced austenite matrix strength and the reduction in transformation temperatures at finer PAGSs, which promotes the self-accommodation of transformation strain. Regarding hierarchical structures, for PAGS measuring ≤ 52 μm, the number of packets, Bain grains, blocks, and sub-blocks per PAG decrease and their mean area decreases with their size primarily due to spatial constraints. A PAGS of 62 μm results in packet coarsening, whereas other hierarchical structures undergo refinement. This study provides a statistical framework for analyzing the crystallographic characteristics and hierarchical structure of steels, thus offering insights into the optimization of bainitic and martensitic transformations through PAGS regulation.