Previous research has shown that polygenic risk scores (PRSs) can be used to stratify women according to their risk of developing primary invasive breast cancer. This study aimed to evaluate the association between a recently validated PRS of 313 germline variants (PRS313) and contralateral breast cancer (CBC) risk. We included 56,068 women of European ancestry diagnosed with first invasive breast cancer from 1990 onward with follow-up from the Breast Cancer Association Consortium. Metachronous CBC risk (N = 1,027) according to the distribution of PRS313 was quantified using Cox regression analyses. We assessed PRS313 interaction with age at first diagnosis, family history, morphology, ER status, PR status, and HER2 status, and (neo)adjuvant therapy. In studies of Asian women, with limited follow-up, CBC risk associated with PRS313 was assessed using logistic regression for 340 women with CBC compared with 12,133 women with unilateral breast cancer. Higher PRS313 was associated with increased CBC risk: hazard ratio per standard deviation (SD) = 1.25 (95%CI = 1.18-1.33) for Europeans, and an OR per SD = 1.15 (95%CI = 1.02-1.29) for Asians. The absolute lifetime risks of CBC, accounting for death as competing risk, were 12.4% for European women at the 10th percentile and 20.5% at the 90th percentile of PRS313. We found no evidence of confounding by or interaction with individual characteristics, characteristics of the primary tumor, or treatment. The C-index for the PRS313 alone was 0.563 (95%CI = 0.547-0.586). In conclusion, PRS313 is an independent factor associated with CBC risk and can be incorporated into CBC risk prediction models to help improve stratification and optimize surveillance and treatment strategies.