BY DR JAMES WALSH, DR ANDREW CUNNINGHAM & PROFESSOR BRUCE THOMAS (UNISA)

Criminal narratives seem to intrigue people across cultures and generations, with countless television dramas over the years chronicling the timelines of fictional crimes and their equally fictionally investigation by the on-screen “authorities”. Whilse these shows present a single, cohesive storyline, the reality is that this is far from the truth, with the content of “CSI: Miami” not just presenting fictional technology, but also a fictional representation of the investigatory processes.

The reality of the process is that it is more akin to managing an incoming stream of ad-hoc, randomly returned investigation enquiries and supporting evidence.

This not only poses a problem for current agents on the case - tracking what the whole case looks like, having to mentally piece together disparate facts from a number of sources - but also for agents coming in part-way through a case, or public prosecutors tasked with understanding a case as it has been assigned to them.

Given what Hollywood promised us in movies such as “Minority Report” and other media, there must be a better way of gaining an understanding of, and insight into, a case. This gap between the case investigation and the case comprehension is what the D2D CRC’s Narrative Visualisation project seeks to address.

Part of the Integrated Law Enforcement (ILE) program, the Narrative Visualisation project is looking to extend the functionality provided by the ILE case management platform being developed by the CRC by providing agents with a single, cohesive view of the case that leverages state-of-the-art information visualisation and narrative structure theory.

There are clear advantages to moving cases that contain an immense number of inter-related documents, photos, recordings, and other evidence into the digital realm.

However, the inherent downside is that an agent no longer has the ability to physically spread the case out across a desk to gain an understanding, instead having to open/search/close a countless sea of digital documents without any cohesive “cognitive glue” linking them together.

Furthermore, there is essential implicit knowledge that an agent formulates as they understand a case that is not easily transferred between agents. At best, an agent has the ability to re-trace the steps of the investigation to-date, re-tracing all the successes and dead-end enquiries of the case. While this can be valuable, it can also be prohibitively complex for someone wanting to gain an initial overview of case, with the eventual success of a case often dependant on that initial comprehension of a case by the prosecutor.

By leveraging Information Visualisation theory, we look to externalise this process of understanding, and aid in the communication of a case between users.

Information visualisation is the process of translating data into information by creating external visual representations of that data.

For an investigation, these external representations allow a user to offload some of the cognitive complexity of understanding a case; essentially providing a digital version of the cognitive glue that would have previously occurred with physical documents on an agent’s desk.

Forensics imagery, video recordings, and even three-dimensional scans of the crime scene can all be visualised as rich, interactive contextual representations that facilitate this understanding. More so, abstract information, such as locations, dates and times, relationships between persons of interest, can be visualised in a meaningful way that is not possible through textual representation alone.

To capture and externalise the implicit knowledge formulated by agents while investigating a case, we turn to Narrative Visualisation.

Narrative visualisation, a sub-discipline of information visualisation, is the process of imbuing a visualisation system with a cohesive narrative. Through our research, we have recognised that much of the implicit information within a case can be made explicit by providing users’ a “narrative voice” that links all the pieces of information together.

From a narrative visualisation perspective, law enforcement cases have some unique aspects that present novel opportunities in the research field.

Notably, we have identified two distinct but related narrative timelines­—the criminal timeline (CT) and investigative timeline (IT). The CT refers to the actual events directly relating to the crime, negating any influence from authorities, while the IT chronicles the involvement of law enforcement in the case and the investigative steps.

Both the CT and IT represent narratives in their own right and also form the greater narrative of the whole case. Understanding both these timelines and the relationship between the two is paramount to the successful prosecution of a case.

Existing chronologies presented to agents and prosecutors generally focus on the IT, ignoring the context of the case the CT provides. As such, we are developing novel, interactive representations of the CT and IT. This will allow agents to not only view the case overall from both the CT and IT view, but also to view the case as it relates to different charges, individuals, and other entities of interest, (e.g. locations or businesses of interest), with the narrative presented dynamically adjusted to reflect the content of interest.

Moving investigation case work from the traditional paper-based approach to a purely digital system seems like a natural progression, but how to do so in the best way is an open research question.

Our research goes someway to answering this question by applying information visualisation and narrative visualisation theory to represent complex information and externalise the implicit understanding of users.

By doing so, we reduce the workload of the user in comprehending and communicating a case and potentially bring reality one step closer to the future that Hollywood fiction promises.