Despite the multitude of available methods, the characterization of ultrafast pulses remains a challenging endeavor, especially at the single-photon level. We introduce a pulse characterization scheme that maps the magnitude of its short-time Fourier transform. Contrary to many well-known solutions it does not require nonlinear effects and is therefore suitable for single-photon-level measurements. Our method is based on introducing a series of controlled time and frequency shifts, where the latter is performed via an electro-optic modulator allowing a fully-electronic experimental control. We characterized the full spectral and temporal width of a classical and single-photon-level pulse and successfully tested the applicability of the reconstruction algorithm of the spectral phase and amplitude. The method can be extended by implementing a phase-sensitive measurement and is naturally well-suited to partially-incoherent light.