Biological systems need to react to stimuli over a broad spectrum of timescales. If and how this ability can emerge without external fine-tuning is a puzzle. We consider here this problem in discrete Markovian systems, where we can leverage results from random matrix theory. Indeed, generic large transition matrices are governed by universal results, which predict the absence of long timescales unless fine-tuned. We consider an ensemble of transition matrices and motivate a temperature-like variable that controls the dynamic range of matrix elements, which we show plays a crucial role in the applicability of the large matrix limit: as the dynamic range increases, a phase transition occurs whereby the random matrix theory result is avoided, and long relaxation times ensue, in the entire "ordered"phase. We furthermore show that this phase transition is accompanied by a drop in the entropy rate and a peak in complexity, as measured by predictive information [Bialek, Nemenman, and Tishby Neural Comput. 13, 2409 (2001)NEUCEB0899-766710.1162/089976601753195969]. Extending the Markov model to a Hidden Markov model (HMM), we show that observable sequences inherit properties of the hidden sequences, allowing HMMs to be understood in terms of more accessible Markov models. We then apply our findings to fMRI data from 820 human subjects scanned at wakeful rest. We show that the data can be quantitatively understood in terms of the random model, and that brain activity lies close to the phase transition when engaged in unconstrained, task-free cognition - supporting the brain criticality hypothesis in this context.