Congenital amusia is a neurodevelopmental disorder that is characterized primarily by difficulties in the pitch domain. The aim of the present study was to investigate the perception of musical timbre in a group of individuals with congenital amusia by probing discrimination and short-term memory for real-world timbral stimuli as well as examining the ability of these individuals to sort instrumental tones according to their timbral similarity. Thirteen amusic individuals were matched with thirteen non-amusic controls on a range of background variables. The discrimination task included stimuli of two different durations and pairings of instrumental tones that reflected varying distances in a perceptual timbre space. Performance in the discrimination task was at ceiling for both groups. In contrast, amusic individuals scored lower than controls on the short-term timbral memory task. Amusic individuals also performed worse than controls on the sorting task, suggesting differences in the higher-order representation of musical timbre. These findings add to the emerging picture of amusia as a disorder that has consequences for the perception and memory of musical timbre, as well as pitch.