Aim: Patients with rectal cancer often experience sexual dysfunction after treatment. The aim of this study was to evaluate sexual function in a prospective cohort of patients regardless of treatment and tumour stage and explore what factors might affect sexual activity 1 year after diagnosis. Method: The QoLiRECT study (Quality of Life in RECTal cancer) is a prospective study on the health-related quality of life in patients with rectal cancer in Denmark and Sweden. Questionnaires were completed at diagnosis and 1 year. Clinical data were retrieved from national quality registries. Results: Questionnaire data were available from 1085 patients at diagnosis and 920 patients at 1 year. Median age was 69 years (range 25–100). At diagnosis, 29% of the women and 41% of the men were sexually active, which was lower than an age-matched reference population. This was further reduced to 25% and 34% at 1 year. Risk factors for sexual inactivity were absence of sexual activity prior to the diagnosis and the presence of a stoma. Women experienced reduced lubrication and more dyspareunia at 1 year compared with the time of diagnosis. In men, erectile dysfunction increased from 46% to 55% at 1 year. Conclusion: Sexual activity in patients with rectal cancer is lower at diagnosis compared with the population norm and is further reduced at 1 year. The presence of a stoma contributed to reduced sexual activity after operation. Sexual dysfunction was difficult to evaluate due to low sexual activity in the cohort. In men, erectile dysfunction is common.