Common CCR5-∆32 and HLA alleles only explain a minority of the HIV long-term non-progressor (LTNP) and elite controller (EC) phenotypes. To identify rare genetic variants contributing to the slow disease progression phenotypes, we performed whole exome sequencing (WES) on seven LTNPs and four ECs. HLA and CCR5 allele status, total HIV DNA reservoir size, as well as variant-related functional differences between the ECs, LTNPs, and eleven age- and gender-matched HIV-infected non-controllers on antiretroviral therapy (NCARTs) were investigated. Several rare variants were identified in genes involved in innate immune sensing, CD4-dependent infectivity, HIV trafficking, and HIV transcription mainly within the LTNP group. ECs and LTNPs had a significantly lower HIV reservoir compared to NCARTs. Furthermore, three LTNPs with variants affecting HIV nuclear import showed integrated HIV DNA levels below detection limit after in vitro infection. HIV slow progressors with variants in the TLR and NOD2 pathways showed reduced pro-inflammatory responses compared to matched controls. Low-range plasma levels of fibronectin was observed in a LTNP harboring two FN1 variants. Taken together, this study identified rare variants in LTNPs as well as in one EC, which may contribute to understanding of HIV pathogenesis and these slow progressor phenotypes, especially in individuals without protecting CCR5-∆32 and HLA alleles.