The Information Engineering Lab IELab @ UQ | The University of Queensland

2024

Search under Uncertainty: Cognitive Biases and Heuristics: A Tutorial on Testing, Mitigating and Accounting for Cognitive Biases in Search Experiments
Jiqun Liu, Leif Azzopardi
Proceedings of the 47th International ACM SIGIR Conference on Research and Development in Information Retrieval  ·  10 Jul 2024  ·  10.1145/3626772.3661382
TREC iKAT 2023: A Test Collection for Evaluating Conversational and Interactive Knowledge Assistants
Mohammad Aliannejadi, Zahra Abbasiantaeb, Shubham Chatterjee, Jeffrey Dalton, Leif Azzopardi
Proceedings of the 47th International ACM SIGIR Conference on Research and Development in Information Retrieval  ·  10 Jul 2024  ·  10.1145/3626772.3657860
Report on the Collab-a-Thon at ECIR 2024
Sean MacAvaney, Adam Roegiest, Aldo Lipani, Andrew Parry, Björn Engelmann Engelmann, …, Shubham Chatterjee, Thomas Jaenich, Udo Kruschwitz, Xi Wang, Zijun Long
ACM SIGIR Forum  ·  01 Jun 2024  ·  10.1145/3687273.3687287
Can We Use Large Language Models to Fill Relevance Judgment Holes?
Zahra Abbasiantaeb, Chuan Meng, Leif Azzopardi, Mohammad Aliannejadi
arXiv  ·  10 May 2024  ·  arXiv:2405.05600
A Conceptual Framework for Conversational Search and Recommendation: Conceptualizing Agent-Human Interactions During the Conversational Search Process
Leif Azzopardi, Mateusz Dubiel, Martin Halvey, Jeffery Dalton
arXiv  ·  15 Apr 2024  ·  arXiv:2404.08630
Measuring the retrievability of digital library content using analytics data
Hamed Jahani, Leif Azzopardi, Mark Sanderson
Journal of the Association for Information Science and Technology  ·  19 Mar 2024  ·  10.1002/asi.24886
Seeking Socially Responsible Consumers: Exploring the Intention-Search-Behaviour Gap
Leif Azzopardi, Frans Van Der Sluis
Proceedings of the 2024 ACM SIGIR Conference on Human Information Interaction and Retrieval  ·  10 Mar 2024  ·  10.1145/3627508.3638324
The Influence of Presentation and Performance on User Satisfaction
Kanaad Pathak, Leif Azzopardi, Martin Halvey
Proceedings of the 2024 ACM SIGIR Conference on Human Information Interaction and Retrieval  ·  10 Mar 2024  ·  10.1145/3627508.3638335
Search under Uncertainty: Cognitive Biases and Heuristics - Tutorial on Modeling Search Interaction using Behavioral Economics
Leif Azzopardi, Jiqun Liu
Proceedings of the 2024 ACM SIGIR Conference on Human Information Interaction and Retrieval  ·  10 Mar 2024  ·  10.1145/3627508.3638297
Uncharted Territory: Understanding Exploratory Search Behaviours in Literature Reviews
Ayah Soufan, Ian Ruthven, Leif Azzopardi
Proceedings of the 2024 ACM SIGIR Conference on Human Information Interaction and Retrieval  ·  10 Mar 2024  ·  10.1145/3627508.3638334
Measuring Bias in a Ranked List Using Term-Based Representations
Amin Abolghasemi, Leif Azzopardi, Arian Askari, Maarten de Rijke, Suzan Verberne
Lecture Notes in Computer Science  ·  01 Jan 2024  ·  10.1007/978-3-031-56069-9_1
The Search Futures Workshop
Leif Azzopardi, Charles L. A. Clarke, Paul B. Kantor, Bhaskar Mitra, Johanne R. Trippas, Zhaochun Ren
Lecture Notes in Computer Science  ·  01 Jan 2024  ·  10.1007/978-3-031-56069-9_57
Ranking Heterogeneous Search Result Pages Using the Interactive Probability Ranking Principle
Kanaad Pathak, Leif Azzopardi, Martin Halvey
Lecture Notes in Computer Science  ·  01 Jan 2024  ·  10.1007/978-3-031-56060-6_7

2023

Retrievability Bias Estimation Using Synthetically Generated Queries
Amin Abolghasemi, Suzan Verberne, Arian Askari, Leif Azzopardi
Proceedings of the 32nd ACM International Conference on Information and Knowledge Management  ·  21 Oct 2023  ·  10.1145/3583780.3615221
A Systematic Review of Cost, Effort, and Load Research in Information Search and Retrieval, 1972–2020
Molly McGregor, Leif Azzopardi, Martin Halvey
ACM Transactions on Information Systems  ·  18 Aug 2023  ·  10.1145/3583069
What the Dickens: Post-mortem privacy and intergenerational trust
Burkhard Schafer, Jo Briggs, Wendy Moncur, Emma Nicol, Leif Azzopardi
Computer Law & Security Review  ·  01 Jul 2023  ·  10.1016/j.clsr.2023.105800
Driven to Distraction: Examining the Influence of Distractors on Search Behaviours, Performance and Experience
Leif Azzopardi, David Maxwell, Martin Halvey, Claudia Hauff
Proceedings of the 2023 Conference on Human Information Interaction and Retrieval  ·  19 Mar 2023  ·  10.1145/3576840.3578298

2022

Revealing Cumulative Risks in Online Personal Information: A Data Narrative Study
Emma Nicol, Jo Briggs, Wendy Moncur, Amal Htait, Daniel Paul Carey, Leif Azzopardi, Burkhard Schafer
Proceedings of the ACM on Human-Computer Interaction  ·  07 Nov 2022  ·  10.1145/3555214
Comparing Levels and Types of Situational-Awareness based Agent Transparency in Human-Agent Collaboration
Sylvain Daronnat, Leif Azzopardi, Martin Halvey
Proceedings of the Human Factors and Ergonomics Society Annual Meeting  ·  01 Sep 2022  ·  10.1177/1071181322661498
Are Taylor's Posts Risky? Evaluating Cumulative Revelations in Online Personal Data
Leif Azzopardi, Jo Briggs, Melissa Duheric, Callum Nash, Emma Nicol, Wendy Moncur, Burkhard Schafer
Proceedings of the 45th International ACM SIGIR Conference on Research and Development in Information Retrieval  ·  06 Jul 2022  ·  10.1145/3477495.3531659
A Flexible Framework for Offline Effectiveness Metrics
Alistair Moffat, Joel Mackenzie, Paul Thomas, Leif Azzopardi
Proceedings of the 45th International ACM SIGIR Conference on Research and Development in Information Retrieval  ·  06 Jul 2022  ·  10.1145/3477495.3531924
Searching the Literature: An Analysis of an Exploratory Search Task
Ayah Soufan, Ian Ruthven, Leif Azzopardi
ACM SIGIR Conference on Human Information Interaction and Retrieval  ·  14 Mar 2022  ·  10.1145/3498366.3505818
An Initial Analysis of E-Procurement Search Behaviour
Stuart Mackie, Leif Azzopardi, Yashar Moshfeghi
Lecture Notes in Computer Science  ·  01 Jan 2022  ·  10.1007/978-3-030-96960-8_1
Early Risk Detection of Self-Harm Using BERT-Based Transformers
Rodrigo Martínez-Castaño, Amal Htait, Leif Azzopardi, Yashar Moshfeghi
Studies in Computational Intelligence  ·  01 Jan 2022  ·  10.1007/978-3-031-04431-1_8
Improving BERT-based Query-by-Document Retrieval with Multi-task Optimization
Amin Abolghasemi, Suzan Verberne, Leif Azzopardi
Lecture Notes in Computer Science  ·  01 Jan 2022  ·  10.1007/978-3-030-99739-7_1
Making sense of Trifles: Data Narratives and Cumulative Data Disclosure
Callum Nash, Dan Carey, Emma Nicol, Amal Htait, Burkhard Schafer, Jo Briggs, Wendy Moncur, Leif Azzopardi
Jusletter-IT  ·  01 Jan 2022  ·  10.38023/e77f5f7d-98c2-4759-a036-b1539bc4a8ae
Towards Building Economic Models of Conversational Search
Leif Azzopardi, Mohammad Aliannejadi, Evangelos Kanoulas
Lecture Notes in Computer Science  ·  01 Jan 2022  ·  10.1007/978-3-030-99739-7_4

2021

Analysing Mixed Initiatives and Search Strategies during Conversational Search
Mohammad Aliannejadi, Leif Azzopardi, Hamed Zamani, Evangelos Kanoulas, Paul Thomas, Nick Craswell
Proceedings of the 30th ACM International Conference on Information & Knowledge Management  ·  26 Oct 2021  ·  10.1145/3459637.3482231
Towards Identifying, Understanding and Controlling Cumulative Revelations in Social Media
Emma Nicol, Amal Htait, Leif Azzopardi, Wendy Moncur
Proceedings of the Association for Information Science and Technology  ·  01 Oct 2021  ·  10.1002/pra2.566
Investigating the Impact of Visual Environmental Uncertainty on Human-Agent Teaming
Sylvain Daronnat, Leif Azzopardi, Martin Halvey
Proceedings of the Human Factors and Ergonomics Society Annual Meeting  ·  01 Sep 2021  ·  10.1177/1071181321651209
Sentiment Intensity Prediction using Neural Word Embeddings
Amal Htait, Leif Azzopardi
Proceedings of the 2021 ACM SIGIR International Conference on Theory of Information Retrieval  ·  11 Jul 2021  ·  10.1145/3471158.3472254
ERR is not C/W/L: Exploring the Relationship Between Expected Reciprocal Rank and Other Metrics
Leif Azzopardi, Joel Mackenzie, Alistair Moffat
Proceedings of the 2021 ACM SIGIR International Conference on Theory of Information Retrieval  ·  11 Jul 2021  ·  10.1145/3471158.3472239
Untangling the Concept of Task in Information Seeking and Retrieval
Ayah Soufan, Ian Ruthven, Leif Azzopardi
Proceedings of the 2021 ACM SIGIR International Conference on Theory of Information Retrieval  ·  11 Jul 2021  ·  10.1145/3471158.3472259
Inferring Trust From Users’ Behaviours; Agents’ Predictability Positively Affects Trust, Task Performance and Cognitive Load in Human-Agent Real-Time Collaboration
Sylvain Daronnat, Leif Azzopardi, Martin Halvey, Mateusz Dubiel
Frontiers in Robotics and AI  ·  08 Jul 2021  ·  10.3389/frobt.2021.642201
Investigating the Influence of Ads on User Search Performance, Behaviour, and Experience during Information Seeking
Olivia Foulds, Leif Azzopardi, Martin Halvey
Proceedings of the 2021 Conference on Human Information Interaction and Retrieval  ·  14 Mar 2021  ·  10.1145/3406522.3446024
Untangling Cost, Effort, and Load in Information Seeking and Retrieval
Molly McGregor, Leif Azzopardi, Martin Halvey
Proceedings of the 2021 Conference on Human Information Interaction and Retrieval  ·  14 Mar 2021  ·  10.1145/3406522.3446026
User Models, Metrics and Measures of Search
Leif Azzopardi, Alistair Moffat, Paul Thomas, Guido Zuccon
Proceedings of the 2021 Conference on Human Information Interaction and Retrieval  ·  14 Mar 2021  ·  10.1145/3406522.3446049
Cognitive Biases in Search
Leif Azzopardi
Proceedings of the 2021 Conference on Human Information Interaction and Retrieval  ·  14 Mar 2021  ·  10.1145/3406522.3446023
AWESSOME: An Unsupervised Sentiment Intensity Scoring Framework Using Neural Word Embeddings
Amal Htait, Leif Azzopardi
Lecture Notes in Computer Science  ·  01 Jan 2021  ·  10.1007/978-3-030-72240-1_56
BERT-Based Transformers for Early Detection of Mental Health Illnesses
Rodrigo Martínez-Castaño, Amal Htait, Leif Azzopardi, Yashar Moshfeghi
Lecture Notes in Computer Science  ·  01 Jan 2021  ·  10.1007/978-3-030-85251-1_15

2020

Impact of Agents’ Errors on Performance, Reliance and Trust in Human-Agent Collaboration
Sylvain Daronnat, Leif Azzopardi, Martin Halvey
Proceedings of the Human Factors and Ergonomics Society Annual Meeting  ·  01 Dec 2020  ·  10.1177/1071181320641091
Impact of Agent Reliability and Predictability on Trust in Real Time Human-Agent Collaboration
Sylvain Daronnat, Leif Azzopardi, Martin Halvey, Mateusz Dubiel
Proceedings of the 8th International Conference on Human-Agent Interaction  ·  10 Nov 2020  ·  10.1145/3406499.3415063
Interactive Evaluation of Conversational Agents: Reflections on the Impact of Search Task Design
Mateusz Dubiel, Martin Halvey, Leif Azzopardi, Sylvain Daronnat
Proceedings of the 2020 ACM SIGIR on International Conference on Theory of Information Retrieval  ·  14 Sep 2020  ·  10.1145/3409256.3409814
Using Sentiment Analysis for Pseudo-Relevance Feedback in Social Book Search
Amal Htait, Sébastien Fournier, Patrice Bellot, Leif Azzopardi, Gabriella Pasi
Proceedings of the 2020 ACM SIGIR on International Conference on Theory of Information Retrieval  ·  14 Sep 2020  ·  10.1145/3409256.3409847
Analyzing the Influence of Bigrams on Retrieval Bias and Effectiveness
Abdulaziz AlQatan, Leif Azzopardi, Yashar Moshfeghi
Proceedings of the 2020 ACM SIGIR on International Conference on Theory of Information Retrieval  ·  14 Sep 2020  ·  10.1145/3409256.3409831
Predicting Perceptual Speed from Search Behaviour
Olivia Foulds, Alessandro Suglia, Leif Azzopardi, Martin Halvey
Proceedings of the 43rd International ACM SIGIR Conference on Research and Development in Information Retrieval  ·  25 Jul 2020  ·  10.1145/3397271.3401210
DataMirror: Reflecting on One's Data Self
Amal Htait, Leif Azzopardi, Emma Nicol, Wendy Moncur
Proceedings of the 43rd International ACM SIGIR Conference on Research and Development in Information Retrieval  ·  25 Jul 2020  ·  10.1145/3397271.3401398
Data-Driven Evaluation Metrics for Heterogeneous Search Engine Result Pages
Leif Azzopardi, Ryen W. White, Paul Thomas, Nick Craswell
Proceedings of the 2020 Conference on Human Information Interaction and Retrieval  ·  14 Mar 2020  ·  10.1145/3343413.3377959
Reflecting upon Perceptual Speed Tests in Information Retrieval
Olivia Foulds, Leif Azzopardi, Martin Halvey
Proceedings of the 2020 Conference on Human Information Interaction and Retrieval  ·  14 Mar 2020  ·  10.1145/3343413.3377982

2019

Building Economic Models and Measures of Search
Leif Azzopardi, Alistair Moffat, Paul Thomas, Guido Zuccon
Proceedings of the 42nd International ACM SIGIR Conference on Research and Development in Information Retrieval  ·  18 Jul 2019  ·  10.1145/3331184.3331379
Looking for Opportunities
Stuart Mackie, David Macdonald, Leif Azzopardi, Yashar Moshfeghi
Proceedings of the 42nd International ACM SIGIR Conference on Research and Development in Information Retrieval  ·  18 Jul 2019  ·  10.1145/3331184.3331428
cwl_eval
Leif Azzopardi, Paul Thomas, Alistair Moffat
Proceedings of the 42nd International ACM SIGIR Conference on Research and Development in Information Retrieval  ·  18 Jul 2019  ·  10.1145/3331184.3331398
The impact of result diversification on search behaviour and performance
David Maxwell, Leif Azzopardi, Yashar Moshfeghi
Information Retrieval Journal  ·  16 May 2019  ·  10.1007/s10791-019-09353-0
Building Economic Models of Human Computer Interaction
Leif Azzopardi, Guido Zuccon
Extended Abstracts of the 2019 CHI Conference on Human Factors in Computing Systems  ·  02 May 2019  ·  10.1145/3290607.3298809
Overview of the CLEF eHealth Evaluation Lab 2019
Liadh Kelly, Hanna Suominen, Lorraine Goeuriot, Mariana Neves, Evangelos Kanoulas, …, Leif Azzopardi, Rene Spijker, Guido Zuccon, Harrisen Scells, João Palotti
Lecture Notes in Computer Science  ·  01 Jan 2019  ·  10.1007/978-3-030-28577-7_26
VES Team at TREC Conversational Assistance Track (CAsT) 2019
Vasileios Stamatis, Leif Azzopardi, Alan Wilson
National Institute of Standards and Technology (NIST)  ·  01 Jan 2019  ·  10.6028/nist.sp.1250.cast-ves

2018

Information retrieval in the workplace: A comparison of professional search practices
Tony Russell-Rose, Jon Chamberlain, Leif Azzopardi
Information Processing & Management  ·  01 Nov 2018  ·  10.1016/j.ipm.2018.07.003
Measuring the Utility of Search Engine Result Pages
Leif Azzopardi, Paul Thomas, Nick Craswell
The 41st International ACM SIGIR Conference on Research & Development in Information Retrieval  ·  27 Jun 2018  ·  10.1145/3209978.3210027
Query Variation Performance Prediction for Systematic Reviews
Harrisen Scells, Leif Azzopardi, Guido Zuccon, Bevan Koopman
The 41st International ACM SIGIR Conference on Research & Development in Information Retrieval  ·  27 Jun 2018  ·  10.1145/3209978.3210078
Economic Models of Interaction
Leif Azzopardi, Guido Zuccon
Oxford Scholarship Online  ·  22 Mar 2018  ·  10.1093/oso/9780198799603.003.0012
Report on the Second International Workshop on the Evaluation on Collaborative Information Seeking and Retrieval (ECol?2017 @ CHIIR)
Leif Azzopardi, Jeremy Pickens, Chirag Shah, Laure Soulier, Lynda Tamine
ACM SIGIR Forum  ·  22 Feb 2018  ·  10.1145/3190580.3190599
Information Scent, Searching and Stopping
David Maxwell, Leif Azzopardi
Lecture Notes in Computer Science  ·  01 Jan 2018  ·  10.1007/978-3-319-76941-7_16
The impact of fielding on retrieval performance and bias
Colin Wilkie, Leif Azzopardi
Proceedings of the Association for Information Science and Technology  ·  01 Jan 2018  ·  10.1002/pra2.2018.14505501061
Overview of the CLEF Dynamic Search Evaluation Lab 2018
Evangelos Kanoulas, Leif Azzopardi, Grace Hui Yang
Lecture Notes in Computer Science  ·  01 Jan 2018  ·  10.1007/978-3-319-98932-7_31
Overview of the CLEF eHealth Evaluation Lab 2018
Hanna Suominen, Liadh Kelly, Lorraine Goeuriot, Aurélie Névéol, Lionel Ramadier, …, Leif Azzopardi, Dan Li, Jimmy, João Palotti, Guido Zuccon
Lecture Notes in Computer Science  ·  01 Jan 2018  ·  10.1007/978-3-319-98932-7_26

2017

An Empirical Analysis of Pruning Techniques
Ruey-Cheng Chen, Leif Azzopardi, Falk Scholer
Proceedings of the 2017 ACM on Conference on Information and Knowledge Management  ·  06 Nov 2017  ·  10.1145/3132847.3133151
Algorithmic Bias
Colin Wilkie, Leif Azzopardi
Proceedings of the 2017 ACM on Conference on Information and Knowledge Management  ·  06 Nov 2017  ·  10.1145/3132847.3133135
Integrating the Framing of Clinical Questions via PICO into the Retrieval of Medical Literature for Systematic Reviews
Harrisen Scells, Guido Zuccon, Bevan Koopman, Anthony Deacon, Leif Azzopardi, Shlomo Geva
Proceedings of the 2017 ACM on Conference on Information and Knowledge Management  ·  06 Nov 2017  ·  10.1145/3132847.3133080
An Initial Investigation of Query Expansion Bias
Colin Wilkie, Leif Azzopardi
Proceedings of the ACM SIGIR International Conference on Theory of Information Retrieval  ·  01 Oct 2017  ·  10.1145/3121050.3121097
The Lucene for Information Access and Retrieval Research (LIARR) Workshop at SIGIR 2017
Leif Azzopardi, Matt Crane, Hui Fang, Grant Ingersoll, Jimmy Lin, Yashar Moshfeghi, Harrisen Scells, Peilin Yang, Guido Zuccon
Proceedings of the 40th International ACM SIGIR Conference on Research and Development in Information Retrieval  ·  07 Aug 2017  ·  10.1145/3077136.3084374
A Test Collection for Evaluating Retrieval of Studies for Inclusion in Systematic Reviews
Harrisen Scells, Guido Zuccon, Bevan Koopman, Anthony Deacon, Leif Azzopardi, Shlomo Geva
Proceedings of the 40th International ACM SIGIR Conference on Research and Development in Information Retrieval  ·  07 Aug 2017  ·  10.1145/3077136.3080707

2015

Integrating and Evaluating Neural Word Embeddings in Information Retrieval
Guido Zuccon, Bevan Koopman, Peter Bruza, Leif Azzopardi
Proceedings of the 20th Australasian Document Computing Symposium  ·  08 Dec 2015  ·  10.1145/2838931.2838936

2012

Crowdsourcing interactions: using crowdsourcing for evaluating interactive information retrieval systems
Guido Zuccon, Teerapong Leelanupab, Stewart Whiting, Emine Yilmaz, Joemon M. Jose, Leif Azzopardi
Information Retrieval  ·  13 Jul 2012  ·  10.1007/s10791-012-9206-z

Search for Leif Azzopardi's papers on the Research page