The convergence of the recently developed cluster perturbation (CP) expansions [Pawlowski et al., J. Chem. Phys. 150, 134108 (2019)] is analyzed with the double purpose of developing the mathematical tools and concepts needed to describe these expansions at general order and to identify the factors that define the rate of convergence of CP series. To this end, the CP energy, amplitude, and Lagrangian multiplier equations as a function of the perturbation strength are developed. By determining the critical points, defined as the perturbation strengths for which the Jacobian becomes singular, the rate of convergence and the intruder and critical states are determined for five small molecules: BH, CO, H2O, NH3, and HF. To describe the patterns of convergence for these expansions at orders lower than the high-order asymptotic limit, a model is developed where the perturbation corrections arise from two critical points. It is shown that this model allows for rationalization of the behavior of the perturbation corrections at much lower order than required for the onset of the asymptotic convergence. For the H2O, CO, and HF molecules, the pattern and rate of convergence are defined by critical states where the Fock-operator underestimates the excitation energies, whereas the pattern and rate of convergence for BH are defined by critical states where the Fock-operator overestimates the excitation energy. For the NH3 molecule, both forms of critical points are required to describe the convergence behavior up to at least order 25. Published under an exclusive license by AIP Publishing.