Background: To limit the rapid spread of COVID-19, countries have asked their citizens to stay at home. As a result, demographic and cultural factors related to home life have become especially relevant to predict population well-being during isolation. This pre-registered worldwide study analyses the relationship between the number of adults and children in a household, marital status, age, gender, education level, COVID-19 severity, individualism–collectivism, and perceived stress. Methods: We used the COVIDiSTRESS Global Survey data of 53,524 online participants from 26 countries and areas. The data were collected between 30 March and 6 April 2020. Results: Higher levels of stress were associated with younger age, being a woman, lower level of education, being single, staying with more children, and living in a country or area with a more severe COVID-19 situation. Conclusions: The COVID-19 pandemic revealed that certain people may be more susceptible to experience elevated levels of stress. Our findings highlight the need for public health to be attentive to both the physical and the psychological well-being of these groups.