SWI/SNF and NuRD are protein complexes that antagonistically regulate DNA accessibility. However, repression of their activities often leads to unanticipated changes in target gene expression (paradoxical), highlighting our incomplete understanding of their activities. Here we show that SWI/SNF and NuRD are in a tug-of-war to regulate PRC2 occupancy at lowly expressed and bivalent genes in mouse embryonic stem cells (mESCs). In contrast, at promoters of average or highly expressed genes, SWI/SNF and NuRD antagonistically modulate RNA polymerase II (Pol II) release kinetics, arguably owing to accompanying alterations in H3.3 and H2A.Z levels at promoter-flanking nucleosomes, leading to paradoxical changes in gene expression. Owing to this mechanism, the relative activities of the two remodelers potentiate gene promoters toward Pol II–dependent open or PRC2-dependent closed chromatin states. Our results highlight RNA Pol II occupancy as the key parameter in determining the direction of gene expression changes in response to SWI/SNF and NuRD inactivation at gene promoters in mESCs.