The aim was to provide an overview of the different statistical methods for validation of patient-reported outcome measures, ranging from simple statistical methods available in all software packages to advanced statistical models that require specialized software. A non-technical summary of classical test theory (CTT) and modern test theory (MTT) is provided. Specifically, confirmatory factor analysis, item response theory, and Rasch analysis is outlined. One CTT and three MTT methods were used to validate the two subscales (Symptoms and Quality of Life) from the Knee Injury and Osteoarthritis Outcome Score (KOOS). For each methodology, two analyses were considered: (i) a unidimensional analysis ignoring the pre-specified dimensionality, and (ii) a two-dimensional analysis using the pre-specified dimensionality. While CTT did not adequately address central issues regarding the validity of the KOOS subscales, the three MTT methods yielded very similar results. In conclusion, MTT methods offer analysis of all relevant properties related to the validity of patient-reported outcome measures, while this is not the case for CTT. Claims about sufficient validity based on CTT methods are inadequate and should not be trusted.