Background: Due to an expected surge of COVID-19 patients in need of mechanical ventilation, the intensive care capacity was doubled at Rigshospitalet, Copenhagen, in March 2020. This resulted in an urgent need for doctors with competence in working with critically ill COVID-19 patients. A training course and a theoretical test for non-intensivist doctors were developed. The aims of this study were to gather validity evidence for the theoretical test and explore the effects of the course. Methods: The 1-day course was comprised of theoretical sessions and hands-on training in ventilator use, hemodynamic monitoring, vascular access, and use of personal protective equipment. Validity evidence was gathered for the test by comparing answers from novices and experts in intensive care. Doctors who participated in the course completed the test before (pretest), after (posttest), and again within 8 weeks following the course (retention test). Results: Fifty-four non-intensivist doctors from 15 different specialties with a wide range in clinical experience level completed the course. The test consisted of 23 questions and demonstrated a credible pass–fail standard at 16 points. Mean pretest score was 11.9 (SD 3.0), mean posttest score 20.6 (1.8), and mean retention test score 17.4 (2.2). All doctors passed the posttest. Conclusion: Non-intensivist doctors, irrespective of experience level, can acquire relevant knowledge for working in the ICU through a focused 1-day evidence-based course. This knowledge was largely retained as shown by a multiple-choice test supported by validity evidence. The test is available in appendix and online.