Humans are keen pattern-seekers and take advantage of regularities present in their environment. In the temporal domain, we may call these patterns rhythms, but what is rhythm? Definitions vary, but all presuppose a categorical distinction between rhythm and randomness. Here, we challenge this view and show that two types of random sound sequences—classically considered arrhythmic by experimenters—differ in the amount of regularity humans reconstruct from them. When asked to synchronize to randomly timed sounds, participants leverage statistics to estimate the underlying tempo of the sequence, similar to linear statistical estimators. Theoretically, our results challenge current definitions of rhythm by showing that rhythmicity and randomness are instances of a continuum. Methodologically, our data and mathematical model show that a common method for creating random timing, namely the jittering of event onsets, introduces an undesirable regularity that humans readily exploit. New experiments should aim to maximize temporal randomness, and past experiments’ outcomes require reconsideration.