Abstract
Given the very large numbers of documents involved in e-discovery investigations, lawyers face a considerable challenge of collaborative sensemaking. We report findings from three workplace studies which looked at different aspects of how this challenge was met. From a sociotechnical perspective, the studies aimed to understand how investigators collectively and individually worked with information to support sensemaking and decision making. Here, we focus on discovery-led refinement; specifically, how engaging with the materials of the investigations led to discoveries that supported refinement of the problems and new strategies for addressing them. These refinements were essential for tractability. We begin with observations which show how new lines of enquiry were recursively embedded. We then analyse the conceptual structure of a line of enquiry and consider how reflecting this in e-discovery support systems might support scalability and group collaboration. We then focus on the individual activity of manual document review where refinement corresponded with the inductive identification of classes of irrelevant and relevant documents within a collection. Our observations point to the effects of priming on dealing with these efficiently and to issues of cognitive ergonomics at the human–computer interface. We use these observations to introduce visualisations that might enable reviewers to deal with such refinements more efficiently.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
Explore related subjects
Discover the latest articles, news and stories from top researchers in related subjects.Notes
Whereas physical ergonomics is concerned with the design of tools and environments to fit human abilities and limitations, cognitive ergonomics is concerned with the fit between these things and human cognitive abilities and limitations involved in task performance.
Documents relating to client-attorney communication are ‘privileged’, meaning that they can legitimately be withheld from production during a litigation.
By ‘cognitive momentum’ we mean the facility that comes about through the activation of task-specific cognitive resources (e.g. memories, strategies etc.).
References
Attfield S, Blandford A (2008) E-disclosure viewed as ‘sensemaking’ with computers: The challenge of ‘frames’. Digital Evidence and Electronic Signature Law Review 5
Baron J, Braman R, Withers K, Allman T, Daley M, Paul G (2007) The Sedona Conference best practice commentary on the use of search and information retrieval methods in e-discovery. Sedona Conf J 8:189–223
Benedetti V, Castellani S, Grasso A, Martin D, O’Neill J (2008) Towards an Expanded Model of Litigation, DESI Workshop on Supporting Search and Sensemaking for Electronically Stored Information in Discovery Proceedings. http://www.cs.ucl.ac.uk/staff/S.Attfield/desi/DESI_II_agenda.html. Accessed 14 December 2009
Brassil D, Hogan C, Attfield S (2009) The centrality of user modelling to high recall with high precision search. In Proc. of the IEEE International Conference on Systems, Man and Cybernetics 2009
Conrad, JG (2007) E-discovery revisited: a broader perspective for IR Researchers’, DESI Workshop on Supporting Search and Sensemaking for Electronically Stored Information in Discovery Proceedings. http://www.umiacs.umd.edu/~oard/desi-ws/papers/conrad.pdf. Accessed 14 December 2009
Johnson JC (1990) Selecting ethnographic informants. Sage, CA
Kaplan A (2008) A conversation with corporate counsel: e-Discovery Trends and Perspectives, Industry Research Report. http://pdfserver.amlaw.com/ltn/conversation_corp_counsel.pdf. Accessed 14 December 2009
Klein HK, Myers MD (1999) A set of principles for conducting and evaluating interpretive field studies in information systems. MIS Q 23(1):67–94
Klein G, Moon B, Hoffman R (2006) Making sense of sensemaking 2: A macrocognitive model. IEEE Intell Syst 21(5):88–92
Klein G, Phillips JK, Rall EL, Peluso DA (2007) A data-frame theory of sensemaking. In: Hoffman R (ed) Expertise out of context: Proc. of the Sixth International Conf. on Naturalistic Decision Making (Pensacola Beach, Florida, May 15–17, 2003). Lawrence Erlbaum, US, pp 113–155
Luthans F, Davis TRV (1982) An idiographic approach to organizational behavior research: the use of single case experimental designs and direct measures. Acad Manage Rev 7(3):380–391
McNamara RP (2005) Semantic priming (Essays in Cognitive Psychology), Taylor & Francis, NY
McNee SM, Arnette B (2008) Productivity as a metric for visual analytics: reflection on e-discovery, Proc. of the 2008 Conference on Beyond Time and Errors: Novel Evaluation Methods for Information Visualization
Paul GL, Baron JR (2007) Information inflation: can the legal system adapt? Richmond J Law Technol, 13(3). http://law.richmond.edu/jolt/v13i3/article10.pdf. Accessed 14 December 2009
Pirolli P, Card S (2005) The sensemaking process and leverage points for analyst technology as identified through cognitive task analysis. In: Proc. International Conference on Intelligence Analysis (McLean, VA, May 2-6, 2005). https://analysis.mitre.org/proceedings/index.html. Accessed 14 December 2009
Russell DM, Stefik MJ, Pirolli P, Card SK (1993) The cost structure of sensemaking. In: Proc. of INTERACT ‘93 and CHI ‘93 Conf. on Hum. Factors in Comp. Sys. (Amsterdam, The Netherlands), ACM Press, New York, pp 269–276
Socha G, Gelbmann T (2009) Strange times: 2009 Socha Gelbmann report, Law Technology News, August 2009
Solomon RD, Baron JR (2009) Bake offs, demos and kicking the tires: a practical litigator’s brief guide to evaluating early case assessment software and search and review tools. http://www.kslaw.com/portal/server.pt. Accessed 14 December 2009
Strauss A, Corbin J (1998) Basics of qualitative research: techniques and procedures for developing grounded theory, 2nd edn. Sage Publications, London
Thomas JB, Clark SM, Gioia DA (1993) Strategic sensemaking and organisational performance: linkages among scanning, interpretation, action and outcomes. Acad Manage J 36:239–270
Tidwell J (2006) Designing interfaces: patterns for effective interaction design. O’Reilly, CA
Wildisen G (2009) March of the regulators, New Law J, 159(7356) http://www.newlawjournal.co.uk/nlj/content/march-regulators. Accessed 14 December 2009
Zhang X, Qu Y, Lee Giles C, Soong P (2008) CiteSense: Supporting Sensemaking of Research Literature. In: Proc. of the CHI ‘08 Conf. on Hum. Factors in Comp. Sys., (Florence, Italy), ACM Press, New York, pp 677–680
Acknowledgments
The work reported in this paper was funded under the Engineering and Physical Sciences Research Council funded project Making Sense of Information (EP/D056268). We are grateful to all participants in this study and to Freshfields Bruckhaus Deringer for hosting it.
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Attfield, S., Blandford, A. Discovery-led refinement in e-discovery investigations: sensemaking, cognitive ergonomics and system design. Artif Intell Law 18, 387–412 (2010). https://doi.org/10.1007/s10506-010-9091-y
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10506-010-9091-y