With increasing interest in peptide-based therapeutics also the application of computational approaches such as peptide docking has gained more and more attention. In order to assess the suitability of docking programs for peptide placement and to support the development of peptide-specific docking tools, an independently constructed benchmark data set is urgently needed. Here we present the LEADS-PEP benchmark data set for assessing peptide docking performance. Using a rational and unbiased workflow, 53 protein-peptide complexes with peptide lengths ranging from 3 to 12 residues were selected. The data set is publicly accessible at www.leads-x.org . In a second step we evaluated several small molecule docking programs for their potential to reproduce peptide conformations as present in LEADS-PEP. While most tested programs were capable to generate native-like binding modes of small peptides, only Surflex-Dock and AutoDock Vina performed reasonably well for peptides consisting of more than five residues. Rescoring of docking poses with scoring functions ChemPLP, ChemScore, and ASP further increased the number of top-ranked near-native conformations. Our results suggest that small molecule docking programs are a good and fast alternative to specialized peptide docking programs.