Social impairments are suggested as vulnerability markers for schizophrenia and bipolar disorder. Therefore, we investigated the development of social responsiveness and theory of mind (ToM) in children at familial high-risk of schizophrenia (FHR-SZ) or bipolar disorder (FHR-BP). This study is part of The Danish High Risk and Resilience Study, a longitudinal cohort study of children at FHR-SZ or FHR-BP and population-based controls (PBC). Social responsiveness was measured with the Social Responsiveness Scale (SRS-2), completed by teachers and primary caregivers. ToM was measured using The Animated Triangles Task (ATT). Both SRS-2 and ATT were applied at age 7 and 11. A total of 520 children participated (FHR-SZ, n = 201; FHR-BP, n = 119; PBC, n = 200). Results showed no significant time by group interactions. At follow-up, children at FHR-SZ exhibited impaired social responsiveness compared with PBC regardless of the informant. At both timepoints, a higher proportion of children at FHR-SZ were rated at a clinically significant level, implying inference in everyday social interactions. Compared with PBC, primary caregivers reported impairments in social responsiveness in children at FHR-BP at follow-up. The three groups did not differ in ToM at follow-up. Social responsiveness and ToM do not develop differently in children at FHR-SZ, FHR-BP and PBC from age 7 to 11, but impairments in social responsiveness remain stable and may constitute a vulnerability marker particularly in children at FHR-SZ, but also FHR-BP. ToM abilities seem to improve and remain intact, but ToM development and ToM task properties should be taken into consideration.