Agrivoltaics is a dual land-use approach, combining food and energy production. It is a yet underexplored innovation with high potential to address land-use conflicts. Understanding the basis on which stakeholders judge and decide on such innovations is crucial to understanding perception and adoption, especially when the potential value of an innovation is not solely on an individual level but also on a societal level. Therefore, we combine two theoretical lenses, the innovation diffusion theory for an individual and the social acceptance perspective for a societal lens. Through 27 semi-structured stakeholder interviews, we explore perceptions of agrivoltaics by different stakeholder types in three countries (Germany, Belgium, and Denmark) and different agrivoltaics system designs (vertical, horizontal, and as replacement of cover installations). We categorize our emerging themes into drivers and barriers of agrivoltaic diffusion in five subdimensions based on the known characteristics of innovation diffusion (Relative Advantage, Compatibility, Complexity, Trialability, and Communicability) and find social acceptance is the overarching dimension that embraces the five subdimensions by either strengthening or weakening acceptance on the micro, meso or macro level. Based on this categorization, we develop a conceptual model to highlight the need to address perceived drivers of, and barriers to, innovation adoption on different social acceptance levels. Our findings contribute to a better understanding of which perceptions play an essential role to whom. First, such a more holistic perspective can support policymakers' decisions on how to boost agrivoltaics as a potentially valuable innovation. Second, it can help researchers decide what to focus on when designing pilot studies, and third, it can support product and project developers decide on how to design agrivoltaic projects with better acceptance rates from all the involved stakeholder groups.