We review and extend the results of [1] that gives a condition for reducibility of quantum representations of mapping class groups constructed from Reshetikhin-Turaev type topological quantum field theories based on modular categories. This criterion is derived using methods developed to describe rational conformal field theories, making use of Frobenius algebras and their representations in modular categories. Given a modular category C, a rational conformal field theory can be constructed from a Frobenius algebra A in C. We show that if C contains a symmetric special Frobenius algebra A such that the torus partition function Z(A) of the corresponding conformal field theory is non-trivial, implying reducibility of the genus 1 representation of the modular group, then the representation of the genus g mapping class group constructed from C is reducible for every g\geq 1. We also extend the number of examples where we can show reducibility significantly by establishing the existence of algebras with the required properties using methods developed by Fuchs, Runkel and Schweigert. As a result we show that the quantum representations are reducible in the SU(N) case, N>2, for all levels k\in \mathbb{N}. The SU(2) case was treated explicitly in [1], showing reducibility for even levels k\geq 4.