Author Details

This paper seeks to analyze information search process in complex tasks1. Complex tasks are larger tasks, which lead people to engage in search tasks for finding information to advance those tasks. Search process consists of activities from query formulation to working with sources selected for task outcome. This paper approaches task performance from the cognitive point of view conceptualizing it as changes in knowledge structures. These structures consist of concepts and their relations representing some phenomenon. Changes in knowledge structures are associated to query formulation and search tactics, selecting contributing sources and working with sources for creating task outcome. As a result, hypotheses concerning associations between changes in knowledge structures and search behaviors are suggested. The paper also presents some ideas for success indicators at various stages of search processes.

Keywords

Cognitive Search, Information Seeking, Information Search Process, Information User, Search Outcome.

Full Text

References

Ahn, J.-W., Brusilovsky, P., He, D., Grady, J. and Li, Q. (2008). Personalized web exploration with task models. In Proc WWW 2008 (pp. 1-10). New York, NY: ACM. https://doi.org/10.1145/1367497.1367499

Bates M. J. (1979). Information search tactics. Journal of the American Society for Information Science, 30(4), 205-214. https://doi.org/10.1002/asi.4630300406

Belkin N.J. (1980). Anomalous states of knowledge as a basis for information retrieval. Canadian Journal of Information and Library Science, 5, 133-143.

Belkin, N. J., Cole, M., and Liu, J. (2009). A model for evaluating interactive information retrieval. In SIGIR Workshop on the Future of IR Evaluation, July 23, 2009, Boston.

Belkin, N. et al. (2017). 2nd Workshop on Supporting Complex Search Tasks. In Proceedings of CHIIR’17. https://doi.org/10.1145/3020165.3022163

Bron M, van Gorp J, Nack F, de Rijke M, Vishneuski A and de Leeuw S. (2012). A subjunctive exploratory search interface to support media studies researchers. In: Proceedings of the 35th international ACM SIGIR conference on Research and development in information retrieval (SIGIR ‘12) (pp. 425-434). New York, NY: ACM. https://doi.org/10.1145/2348283.2348342

Brookes, B. (1980). The foundation of information science. Part I: Philosophical aspects. Journal of Information Science, 2,125-133. https://doi.org/10.1177/016555158000200302

Budzik, J. and Hammond, K. (2000). User interactions with everyday applications as context for just-in-time information access. In Proc UIU’00 (pp. 44-51). New York, NY: ACM. https://doi.org/10.1145/325737.325776

Butcher KR, Davies S, Crockett A, Dewald A and Zheng R. (2011). Do graphical search interfaces support effective search for and evaluation of digital library resources? In: Proceedings of the 11th annual international ACM/IEEE joint conference on Digital libraries (JCDL ‘11) (pp. 315-324). New York, NY: ACM. https:// doi.org/10.1145/1998076.1998136 PMid:21816928 PMCid:PMC3150100

Campbell, D. J. (1988). Task complexity: a review and analysis. Academy of Management Review, 13, 40-52. https://doi.org/10.2307/258353

Cho, B.-Y., Woodward, L., Li, D. and Barlow, W. (2017). Examining adolescents’ strategic processing during online reading with a question-generating task. American Educational Research Journal, 54(4): 691-724. https://doi.org/10.3102/0002831217701694

Collins-Thompson, K., Rieh, S. Y., Haynes, C. C., Syed, R. (2016). Assessing learning outcomes in web search: A comparison of tasks and query strategies. Proc of the 1st CHIIR Conf (pp. 163-172). https://doiorg/10.1145/2854946.2854972

Fox,S., Karnawat, K., Mydland, M., Dumais, S., and White, T. (2005). Evaluating implicit measures to improve web search. ACM TOIS, 23(2), 147-168. https://doi. org/10.1145/1059981.1059982

Gwizdka, J. (2014). Characterizing relevance with eye-tracking measures. In Proceedings of the 5th Information Interaction in Context Symposium (pp. 58- 67). ACM. https://doi.org/10.1145/2637002.2637011

Hagen, M., Potthast, M., Völske, M., Gomoll, J. and Stein, B. (2016). How Writers Search: Analyzing the Search and Writing Logs of Non-fictional Essays. In Proc CHIIR’16, Diane Kelly, Rob Capra, Nick Belkin, Jaime Teevan, and Pertti Vakkari (Eds.) (pp. 193-202). ACM. https://doi. org/10.1145/2854946.2854969

He, D., Brusilovsky, P, Ahn, J., Grady, J., Farzan, R., Peng, Y., Yang, Y., and Rogati, M. (2008). An evaluation of adaptive filtering in the context of realistic task-based information exploration. IP&M, 44, 511-533. https:// doi.org/10.1016/j.ipm.2007.07.009

Hersh, W. (2003). Information retrieval: A health and biomedical perspective (2nd ed.). New York: Springer.

Hersh, W, Pentecost, J, and Hickam, D. (1996). A taskoriented approach to information retrieval. Journal of the American Society for Information Science, 47(1), 50-56. https://doi.org/10.1002/(SICI)1097- 4571(199601)47:1<50::AID-ASI5>3.0.CO;2-1

Ingwersen, P. (1992) Information Retrieval Interaction. London: Taylor Graham.

Jarvelin, K., Vakkari, P., Arvola, P., Baskaya, F., Jarvelin, A., Kekalainen, J., Keskustalo, H., Kumpulainen, S., Saastamoinen, M., Savolainen, R., and Sormunen, E. (2015). Task-based information interaction evaluation: The viewpoint of program theory. ACM Trans Inf Syst, 33(1), 3:1-3:30. https://doi.org/10.1145/2699660

Kelly, D. and Cool, C. (2002). The effects of topic familiarity on information search behavior. In Proc. JCDL’02. (pp. 74-75). New York, NY: ACM. https://doi. org/10.1145/544220.544232

Kong, W. and Allan, J. (2014). Extending faceted search to the general web. In Proc. CIKM’14 (pp. 839-848). New York, NY: ACM. https://doi.org/10.1145/2661829.2661964

Kuhlthau C. (1993). Seeking Meaning. Norwood, N.J.: Ablex. Lancaster, W., and Warner, A. (1993). Information retrieval today. Arlington, VA: Information Resources Press.

Liu, C., Gwizdka, J., and Liu. J. (2010). Helping identify when users find useful documents: Examination of query reformulation intervals. In Proceedings of IIiX’10 (pp. 215-224). https://doi.org/10.1145/1840784.1840816

Liu, C. Belkin, N. J., and Cole, M. J. (2012). Personalization of search results using interaction behaviors in search sessions. In Proc. SIGIR’12. (pp. 205-214). ACM. https:// doi.org/10.1145/2348283.2348314

Liu, J., and Belkin, N. J. (2010). Personalizing information retrieval for multi-session tasks: the roles of task stage and task type. In Proc. SIGIR’10 (pp. 26-33). ACM. https://doi.org/10.1145/1835449.1835457

Liu, J., and Belkin, N. J. (2012). Searching vs. writing: Factors affecting information use task performance. In Proceedings of the American Society for Information Science and Technology (pp. 1-10). https://doi.org/10.1002/meet.14504901127

Liu, J., Belkin, N.J. Zhang, X. and Yuan, X. (2013). Examining users’ knowledge change in the task completion process. IP&M, 49, 1058-1074. https://doi.org/10.1016/j.ipm.2012.08.006

Marchionini G. (1995). Information seeking in electronic environments. Cambridge University Press. https://doi.org/10.1017/CBO9780511626388

Pennanen M. and Vakkari P. (2003). Students’ conceptual structure, search process and outcome while preparing a research proposal. Journal of the American Society for Information Science and Technology, 54(8), 759-770. https://doi.org/10.1002/asi.10273

Qu, Y., and Furnas, G. (2008). Model-driven evaluation of exploratory search: A study under sensemaking framework. Information Processing and Management, 44(2), 534-555. https://doi.org/10.1016/j.ipm.2007.09.006

Raman, K., Bennett, P. and Collins-Thompson, K. (2013). Toward whole-session relevance: Exploring intrinsic diversity in web search. In Proc. SIGIR’13. (pp. 463-472). New York, NY: ACM. https://doi.org/10.1145/2484028.2484089

Robertson, S. (2001). Problem solving. Hove: Psychology Press. https://doi.org/10.4324/9780203457955_chapter_1

Russell, D., Stefik, M., Pirolli, P. and Card, S. (1993). The cost structure of sensemaking. In Proc. CHI ‘93 (pp. 269-276). New York, NY: ACM. https://doi.org/10.1145/169059.169209

Serola, S. and Vakkari, P. (2005). The anticipated and assessed contribution of information types in references retrieved for preparing a research proposal. Journal of the American Society for Information Science, 56(4), 373-381. https://doi.org/10.1002/asi.20113

Smucker, M. D., and Jethani, C. (2012). Time to judge relevance as an indicator of assessor error. In Proc. SIGIR’12. (pp. 1153-1154). ACM. https://doi.org/10.1145/2348283.2348515

Syed, R. and Collins-Thompson, K. (2017). Retrieval algorithms optimized for human learning. In Proc. SIGIR’17 (pp. 555-564). New York, NY: ACM. https://doi.org/10.1145/3077136.3080835

Vakkari, P. (1999). Task complexity, problem structure and information actions: integrating studies on information seeking and retrieval. Information Processing and Management, 35(6), 819-837. https://doi.org/10.1016/ S0306-4573(99)00028-X

Vakkari, P. (2003) Task-based information searching. ARIST 2003, (pp. 413-464). Medford, NJ: Information Today. https://doi.org/10.1002/aris.1440370110

Vakkari, P. (2010). Exploratory searching as conceptual exploration. In: Proceedings of 4th Workshop on Human-Computer Interaction and Information Retrieval (pp. 24-27). New Brunswick, NJ.

Vakkari, P. (2016). Searching as Learning. A Systematization based on Literature. Journal of Information Science, 42(1), 7-18. https://doi.org/10.1177/0165551515615833

Vakkari, P. (2000). Relevance and contributing information types of searched documents in task performance. In: Proceedings of the 23rd Annual International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR ‘00) (pp. 2-9). New York, NY: ACM. https://doi.org/10.1145/345508.345512

Vakkari, P. (2001). A theory of the task-based information retrieval process: A summary and generalization of a longitudinal study. Journal of Documentation, 57(1), 44-60. https://doi.org/10.1108/EUM0000000007075

Vakkari, P and Hakala, N. (2000). Changes in relevance criteria and problem stages in task performance. Journal of Documentation, 56, 540-562. https://doi.org/10.1108/ EUM0000000007127

Vakkari, P. and Huuskonen, S. (2012). Search effort degrades search output but improves task outcome. Journal of the American Society for Information Science and Technology, 63(4), 657-670. https://doi.org/10.1002/asi.21683

Vakkari, P., Jones, S., MacFarlane, A., and Sormunen, E. (2004). Query exhaustivity, relevance feedback and search success in automatic and interactive query expansion. Journal of Documentation, 60(2), 109-127. https://doi.org/10.1108/00220410410522016

Vakkari, P. and Kuokkanen, M. (1997).Theory growth in information science: Applications of the theory of science to a theory of information seeking. Journal of Documentation, 53(5), 497-519. https://doi.org/10.1108/EUM0000000007210

Vakkari, P., Pennanen, M. and Serola, S. (2003). Changes of search terms and tactics while writing a research proposal. Information Processing and Management, 39(3), 445-463. https://doi.org/10.1016/S0306-4573(02)00031-6

Vakkari, P., Volske, M., Potthast, M., Hagen, M. and Stein, B. (2019). Modelling the usefulness of search results as measured by information use. Information Processing and Management, 56(3), 879-894. https://doi.org/10.1016/j.ipm.2019.02.001

Vakkari, P., Volske, M., Potthast, M., Hagen, M. and Stein, B. (2021). Predicting essay quality from search and writing behavior. JASIST, 72(7), 839-852. https://doi.org/10.1002/asi.24451

Walhout, J., Oomen, P., Halszka, J. and Brand-Gruwel, S. (2017). Effects of task complexity on online search behavior of adolescents. Journal of the Association for Information Science and Technology, 68(6), 1449-1461. https://doi.org/10.1002/asi.23782

Wang P. (1997). User’s information needs at different stages of a research project: A cognitive view. In: Vakkari P, Savolainen R and Dervin B. (eds.), Information Seeking in Context (pp. 307-318). London and Los Angeles: Taylor Graham.

Wang, P. and Soergel, D. (1998). A cognitive model of document use during a research project: Study I: Document selection. Journal of the American Society for Information Science and Technology, 49(2), 115-133. https://doi.org/10.1002/(SICI)1097-4571(199802)49:2<115::AID-ASI3>3.0.CO;2-T

Wang, P., and White, M. D. (1999). A cognitive model of document use during a research project. Study II. Decisions at the reading and citing stages. Journal of the American Society for Information Science, 50, 98-114. https:// doi.org/10.1002/(SICI)1097-4571(1999)50:2<98::AIDASI2>3.0.CO;2-L

Wildemuth, B. M., de Bliek, R., Friedman, C. P., and File, D. D. (1995). Medical students’ personal knowledge, searching proficiency, and database use in problem solving. J Am Soc Inf Sci Technol, 46(8), 590-607. https://doi. org/10.1002/(SICI)1097-4571(199509)46:8<590::AID-ASI7>3.0.CO;2-#

Zhang, P. and Soergel, D. (2014). Towards a comprehensive model of the cognitive process and mechanisms of individual sensemaking. Journal of the Association for Information Science and Technology, 65(9), 1733-1758. https://doi.org/10.1002/asi.23125

Zhang, P. and Soergel, D. (2016) Process patterns and conceptual changes in knowledge representations during information seeking and sensemaking: A qualitative user study. Journal of Information Science, 42(1), 59-78. https://doi.org/10.1177/0165551515615834

Zhang, X., Liu, J., Cole, M. and Belkin, N. (2015). Predicting users’ domain knowledge in information retrieval using multiple regression analysis of search behaviors. Journal of the Association for Information Science and Technology, 66(5), 980-1000. https://doi.org/10.1002/asi.23218

Assessing Information Search by Task Outcome

Abstract Views :159 | PDF Views:7

Authors

Pertti Vakkari ¹

Affiliations
1 Tampere University, FIN-33014, FI

Source

Journal of Information and Knowledge (Formerly SRELS Journal of Information Management), Vol 60, No 1 (2023), Pagination: 31-37

Abstract

People do not search information as such but seek to get a job done or to manage a situation by the help of search results. Therefore, the ultimate goal of information search is to advance task performance. It should be evaluated accordingly, i.e. by its contribution to task outcome. This implies an extended notion of search process, which also covers the use of information in search results for task outcome. For measuring the effect of search to task outcome we propose both indirect and direct indicators which measure search success.

Keywords

Information Seeking, Information Search Model, Knowledge Structures, Measuring Search Outcome.

Full Text

References

Budzik, N. and Hammond, K. (2000). User interactions with everyday applications as context for just-in-time information access. In Proc. UIU’00 (pp. 44-51). New York, NY: ACM. https://doi.org/10.1145/325737.325776

Cooper, W. S. (1973). On selecting a measure of retrieval effectiveness. JASIST, 24( 2), 87-100. https://doi.org/10.1002/asi.4630240204

Harter, S. (1992). Psychological relevance and information science. JASIST, 43(9), 602-615. https://doi.org/10.1002/(SICI)1097-4571(199210)43:9<602::AID-ASI3>3.0.CO;2-Q

Hersh, W., Pentecost, J. and Hickam D. (1996). A taskoriented approch to information retrieval. JASIST, 47(1), 50-56. https://doi.org/10.1002/(SICI)1097-4571(199601)47:1<50::AID-ASI5>3.0.CO;2-1

Ja¨rvelin, K., Vakkari, P., Arvola, P, Baskaya, F., J¨arvelin, A., Kek¨ala¨inen, J., Keskustalo, H., Kumpulainen, S., Saastamoinen, M., Savolainen, R. and Sormunen, E. (2015). Task-based information interaction evaluation: The viewpoint of program theory. ACM Trans. Inf. Syst., 33(1), 3:1-3:30. https://doi.org/10.1145/2699660

Kekäläinen J. and Järvelin, K. (2002). Evaluating information retrieval systems under the challenges of interaction and multi-dimensional dynamic relevance. In Proc. CoLIS4 (pp. 253-270).

Kintsch, W. 1998. Comprehension. A paradigm for cognition. Cambridge UP, Cambridge. Lancaster, W. and Warner A. (1993). Information retrieval today. Arlington, VA: Information Resources Press.

Liu, M., Liu, Y., Mao, X., Luo, C., Zhang, M. and Ma, S. (2018), “Satisfaction with failure” or “unsatisfied success”: Investigating the relationship between search success and user satisfaction. In Proc. WWW2018 (1533-1542). New York, NY: ACM. https://doi.org/10.1145/3178876.3186065

Rees, A. (1967). Evaluation of information systems and services. ARIST, 2, 63-86.

Rossi, P., Lipsey, M. and Freeman, H. (2004). Evaluation. A systematic approach. 7th ed. Sage, Thousand Oaks.

Rubin R. J. (2006). Demonstrating results using outcome measurement in your library. Chicago: ALA.

Sperber, D. and Wilson, D. (1986). Relevance: Communication and cognition. Cambridge, MA: Harvard UP.

Syed, R. and Collins-Thompson, K. (2017). Retrieval algorithms optimized for human learning. In Proc. SIGIR’17 (pp. 555-564). New York, NY: ACM. https://doi. org/10.1145/3077136.3080835

Tague-Sutcliffe, J. (1995). Measuring information. Academic Press, San Diego. Vakkari, P. (1999). Task complexity, problem structure and information actions. IP&M, 35, 819-837. https://doi. org/10.1016/S0306-4573(99)00028-X

Vakkari, P. (2003). Task-based information searching. ARIST 2003, vol. 37. Medford, NJ: Information Today, p. 413-464. https://doi.org/10.1002/aris.1440370110

Vakkari, P. (2010). Exploratory searching as conceptual exploration. In: Proc. of 4th Workshop on Human-Computer Interaction and Information Retrieval (pp. 24-27). New Brunswick, NJ. Vakkari, P. (2020). The Usefulness of search results. A Systematization of types and predictors. In Proc. CHIIR2020. https://doi.org/10.1145/3343413.3377955

Vakkari, P., Völske, M., Potthast, M., Hagen, M. and Stein, B. (2019). Modeling the usefulness of search results as measured by information use. IPM, 56(3), 879-894. https://doi.org/10.1016/j.ipm.2019.02.001

Wildemuth, B.M., de Bliek, R., Friedman, C.P. and File, D.D. (1995). Medical students’ personal knowledge, searching proficiency, and database use in problem solving. JASIST, 46(8), 590-607.https://doi.org/10.1002/(SICI)1097-4571(199509)46:8<590::AID-ASI7>3.0.CO;2-#

Yu, R., Gadiraju, U., Holtz, P., Rokicki, M., Kemkes, P. and Dietze, S. (2018). Predicting user knowledge gain in informational search sessions. In Proc. SIGIR’18 (pp. 75-84). New York, NY: ACM. https://doi.org/10.1145/3209978.3210064 Conference Name:ACM Woodstock conference

Username
Password
Remember me