Based on high-frequency data for more than fifty commodities, currencies, equity indices, and fixed-income instruments spanning more than two decades, we document strong similarities in realized volatility patterns within and across asset classes. Exploiting these similarities through panel-based estimation of new realized volatility models results in superior out-of-sample risk forecasts, compared to forecasts from existing models and conventional procedures that do not incorporate the similarities in volatilities. We develop a utility-based framework for evaluating risk models that shows significant economic gains from our new risk model. Lastly, we evaluate the effects of transaction costs and trading speed in implementing different risk models.Received March 7, 2016; editorial decision February 3, 2018 by Editor Andrew Karolyi. Authors have furnished an Internet Appendix, which is available on the Oxford University Press Web site next to the link to the final published paper online.