Developing a Test Collection for the Evaluation of Integrated Search

Research output: Chapter in Book/Report/Conference proceedingArticle in proceedingsResearchpeer-review


he poster discusses the characteristics needed in an information retrieval (IR) test collection to facilitate the evaluation of integrated search, i.e. search across a range of different sources but with one search box and one ranked result list, and describes and analyses a new test collection constructed for this purpose. The test collection consists of approx. 18,000 monographic records, 160,000 papers and journal articles in PDF and 275,000 abstracts with a varied set of metadata and vocabularies from the physics domain, 65 topics based on real work tasks and corresponding graded relevance assessments. The test collection may be used for systems- as well as user-oriented evaluation.
Original languageEnglish
Title of host publicationAdvances in Information Retrieval : 32nd European Conference on IR Research, ECIR 2010, Milton Keynes, UK, March 28-31, 2010, Proceedings
EditorsCathal Gurrin, Yulan He, Gabriella Kazai, Udo Kruschwitz, Suzanne Little, Thomas Roelleke, Stefan Rüger, Keith van Rijsbergen
Publication date2010
Publication statusPublished - 2010
EventEuropean Conference on IR Research - Milton Keynes, United Kingdom
Duration: 28 Mar 201031 May 2010
Conference number: 32


ConferenceEuropean Conference on IR Research
LandUnited Kingdom
ByMilton Keynes
SeriesLecture notes in computer science

    Research areas

  • Test collection design, Task-based IR, Integrated search

Number of downloads are based on statistics from Google Scholar and

No data available

ID: 47047675