Evaluation of Information Retrieval Systems

One of the challenges of modern information retrieval is to adequately evaluate Information Retrieval System (IRS) in order to estimate future performance in a specified application domain. Since there are many algorithms in literature the decision to select one for usage depends mostly on the evaluation of the systems' performance in the domain. This paper presents how visual and scalar evaluation methods complement one another to adequately evaluate information retrieval systems. The visual evaluation methods are capable of indicating whether one IRS performs better than another IRS fully or partially. An overall performance of IRS is revealed using scalar evaluation methods. The use of both types of evaluation methods will give a clear picture of the performance of the IRSs. The Receiver Operator Characteristic (ROC) curve and Precision-Recall (P-R) curve were used to illustrate the visual evaluation methods. Scalar methods notably precision, recall, Area Under Curve (AUC) and F measure were used.