Evaluación de sistemas de búsqueda de respuestas con restricción de tiempo

  1. Llopis Pascual, Fernando
  2. Noguera Robles, Elisa
  3. Ferrández Rodríguez, Antonio
  4. Escapa García, Alberto
Revista:
Procesamiento del lenguaje natural

ISSN: 1135-5948

Año de publicación: 2007

Número: 39

Páginas: 45-52

Tipo: Artículo

Otras publicaciones en: Procesamiento del lenguaje natural

Resumen

Previous works on evaluating the performance of Question Answering (QA) systems are focused in the evaluation of the precision. Nevertheless, the importance of the answer time never has been evaluated. In this paper, we developed a mathematic procedure in order to explore new evaluation measures in QA systems considering the answer time. Also, we carried out an exercise for the evaluation of QA systems within a time constraint in the CLEF-2006 campaign, using the proposed measures. The main conclusion is that the evaluation of QA systems in realtime can be a new scenario for the evaluation of QA systems.