Effective population size (N e) is a central parameter in population and conservation genetics. It measures the magnitude of genetic drift, rates of accumulation of inbreeding in a population, and it conditions the efficacy of selection. It is often assumed that a single N e can account for the evolution of genomes. However, recent work provides indirect evidence for heterogeneity in N e throughout the genome. We study this by examining genome-wide diversity in the Danish Holstein cattle breed. Using the differences in allele frequencies over a single generation, we directly estimated N e among autosomes and smaller windows within autosomes. We found statistically significant variation in N e at both scales. However, no correlation was found between the detected regional variability in N e, and proxies for the intensity of linked selection (local recombination rate, gene density), or the presence of either past strong selection or current artificial selection on traits of economic value. Our findings call for further caution regarding the wide applicability of the N e concept for understanding quantitatively processes such as genetic drift and accumulation of consanguinity in both natural and managed populations.