We introduce a simple and intuitive approach of modeling and forecasting correlations for use in portfolio optimization. The model is composite in nature and consists of elements based on a bivariate realized volatility model. Importantly, our framework allows for volatility spill-overs between assets which provide an edge compared to competing models when forming portfolios. We apply the model to high-frequency data for commodity markets and demonstrate significant economic gains for an investor basing portfolio decisions on our modeling framework. This gain is significant in economic terms, even after imposing realistic constraints on short selling and portfolio turnover.