The pleasurable desire to move to music, also known as groove, is modulated by rhythmic complexity. How the sensation of groove is influenced by other musical features, such as the harmonic complexity of individual chords, is less clear. To address this, we asked people with a range of musical experience to rate stimuli that varied in both rhythmic and harmonic complexity. Rhythm showed an inverted U-shaped relationship with ratings of pleasure and wanting to move, whereas medium and low complexity chords were rated similarly. Pleasure mediated the effect of harmony on wanting to move and high complexity chords attenuated the effect of rhythm on pleasure. We suggest that while rhythmic complexity is the primary driver, harmony, by altering emotional valence, modulates the attentional and temporal prediction processes that underlie rhythm perception. Investigation of the effects of musical training with both regression and group comparison showed that training increased the inverted U effect for harmony and rhythm, respectively. Taken together, this work provides important new information about how the prediction and entrainment processes involved in rhythm perception interact with musical pleasure.