Autonomic imbalance reflected by higher resting heart rate and reduced parasympathetic tone may be driven by low-grade inflammation (LGI) and impaired glycemic control in type 2 diabetes mellitus (T2DM) and pre-diabetes. We examined the interaction of parasympathetic components of heart rate variability (HRV), variables of LGI, and glucose metabolism in people with T2DM, pre-diabetes, and normal glucose metabolism (NGM). We recorded HRV by Holter (48 h) in 633 community-dwelling people of whom T2DM n = 131, pre-diabetes n = 372, and NGM n = 130 and mean HbA1c of 7.2, 6.0 and 5.3%, respectively. Age was 55-75 years and all were without known cardiovascular disease except from hypertension. Fasting plasma glucose, fasting insulin, HOMA-IR, HbA1c and LGI (CRP, Interleukin-18 (IL-18), and white blood cells) were measured. Root-mean-square-of-normal-to-normal-beats (RMSSD), and proportion of normal-to-normal complexes differing by more than 50 ms (pNN50) are accepted measures of parasympathetic activity. In univariate analyses, RMSSD and pNN50 were significantly inversely correlated with level of HbA1c and CRP among people with T2DM and pre-diabetes, but not among NGM. RMSSD and pNN50 remained significantly inversely associated with level of HbA1c after adjusting for age, sex, smoking, and BMI among people with T2DM (β = - 0.22) and pre-diabetes (β = - 0.11); adjustment for LGI, HOMA-IR, and FPG did not attenuate these associations. In backward elimination models, age and level of HbA1c remained associated with RMSSD and pNN50. In people with well controlled diabetes and pre-diabetes, a lower parasympathetic activity was more related to age and HbA1c than to markers of LGI. Thus, this study shows that the driver of parasympathetic tonus may be more the level of glycemic control than inflammation in people with prediabetes and well controlled diabetes.