Impaired musical rhythm abilities and developmental speech-language related disorders are biologically and clinically intertwined. Prior work examining their relationship has primarily used small samples; here, we studied associations at population-scale by conducting the largest systematic epidemiological investigation to date (total N = 39,358). Based on existing theoretical frameworks, we predicted that rhythm impairment would be a significant risk factor for speech-language disorders in the general adult population. Findings were consistent across multiple independent datasets and rhythm subskills (including beat synchronization and rhythm discrimination), and aggregate meta-analyzed data showed that non-linguistic rhythm impairment is a modest but consistent risk factor for developmental speech, language, and reading disorders (OR = 1.33 [1.18 – 1.49]; p < .0001). Further, cross-trait polygenic score analyses (total N = 7180) indicated shared genetic architecture between musical rhythm and reading abilities, suggesting genetic pleiotropy between musicality and language-related phenotypes.