Gathering representative data using mobile sensing to answer research questions is becoming increasingly popular, driven by growing ubiquity and sensing capabilities of mobile devices. However, there are pitfalls along this path, which introduce heterogeneity in the gathered data, and which are rooted in the diversity of the involved device platforms, hardware, software versions and participants. Thus, we, as a research community, need to establish good practices and methodologies for addressing this issue in order to help ensure that, e.g., scientific results and policy changes based on collective, mobile sensed data are valid. In this paper, we aim to inform researchers and developers about mobile sensing data heterogeneity and ways to combat it. We do so via distilling a vocabulary of underlying causes, and via describing their effects on mobile sensing---building on experiences from three projects within citizen science, crowd awareness and trajectory tracking.