Objectives: (1) To investigate daily smartphone-based self-reported and automatically generated sleep measurements, respectively, against validated rating scales; (2) to investigate if daily smartphone-based self-reported sleep measurements reflected automatically generated sleep measurements and (3) to investigate the differences in smartphone-based sleep measurements between patients with bipolar disorder (BD), unaffected first-degree relatives (UR) and healthy control individuals (HC). Methods: We included 203 patients with BD, 54 UR and 109 HC in this study. To investigate whether smartphone-based sleep calculated from self-reported bedtime, wake-up time and screen on/off time reflected validated rating scales, we used the Pittsburgh Sleep Quality Index (PSQI) and sleep items on the Hamilton Depression Rating Scale 17-item (HAMD-17) and the Young Mania Rating Scale (YMRS). Findings: (1) Self-reported smartphone-based sleep was associated with the PSQI and sleep items of the HAMD and the YMRS. (2) Automatically generated smartphone-based sleep measurements were associated with daily self-reports of hours slept between 12:00 midnight and 06:00. (3) According to smartphone-based sleep, patients with BD slept less between 12:00 midnight and 06:00, with more interruption and daily variability compared with HC. However, differences in automatically generated smartphone-based sleep were not statistically significant. Conclusion: Smartphone-based data may represent measurements of sleep patterns that discriminate between patients with BD and HC and potentially between UR and HC. Clinical implication: Detecting sleep disturbances and daily variability in sleep duration using smartphones may be helpful for both patients and clinicians for monitoring illness activity. Trial registration number: clinicaltrials.gov (NCT02888262).