6 July 2022
MDOH: The Forward-View with Almila Akdag Salah, Francisca Pessanha, and Myriam Fellous-Sigrist
The third in the Multimodal Digital Oral History seminar series welcomed Almila Akdag Salah and Francisca Pessanha from Utrecht University and Myriam Fellous-Sigrist from Kings College London.
The recording of the seminar can be accessed here: Multimodal Digital Oral History: The Forward-View Seminar - 3
Firstly, Almila and Francisca’s paper, titled What can breathing patterns tell us to analyze Oral History Archives?, was based on their overall research goal to make interviews with high emotional content more computationally accessible and analysable. Their research looks at the non-verbal cues in oral history interviews such as breaks, gaps, gestures, facial expressions, silences etc. that are poorly represented or lost in the transcription, and which allow us to study feelings, mood, and culture at scale: ‘Sometimes silences speak volumes’. They firstly discussed the algorithmic biases in and ethical challenges of using computational tools and approaches, such as automatic speech recognition (ASR) software, as well as the limitations of emotion detection in recorded materials through various non-verbal cues and techniques. Their particular focus is on trauma-related interview data and biological signals, specifically breathing, and this was partly inspired by ‘Records of Breath’, an installation by artist Evrim Kavcar based, which was on her own remembrance of traumatic events. Reflecting firstly on the challenges of working on breathing signals in oral history archives, such as the lack of objective measures or ‘ground truths’, they outlined their exploratory study of breathing features for the analysis of depression. They trained a model to predict breathing signals using the ‘Breath Speech Corpus’, which consists of spontaneous speech, and tested this on an unannotated dataset, the ‘Distress Analysis Corpus’ consisting of semi-clinical interviews about depression symptoms. The results were then tested for correlation with features in the psychopathology literature on depression. They finished by reflecting on the research opportunities that this type of research may open up around subjective memory and storytelling, the interview as a dialogue, and identity protection for interviewees in politicised or life-threatening situations.
Secondly, Myriam presented her paper: Between access and protection: applied ethics for curating digital oral history. This was based on her PhD research on digital oral history and the ethical problems that are amplified by greater discoverability, and focused on the curation of oral history interviews. Her research involved interviews with curators who shared their experiences of the complex issues in curating oral history interviews in the digital context with regards to the conflicting interests of open access and information privacy. Arising from her findings that there are inadequate workflows and resources for dealing with these complex issues, Myriam made a number of recommendations for anticipating and preparing for various privacy issues, improving mutual understanding of shared responsibilities between curators and interviewees, and bridging gaps in the provision of guidance.
Questions that followed focused on the ethical questions that are raised by the computational analysis of trauma-related interview data, the ethics of the re-use of interview data beyond their original purpose, and the extent to which ‘informed consent’ can be meaningfully achieved and maintained in the digital context.
- Hannah K. Smyth, 14 July 2022
Speakers:
Almila Akdag Salah & Francisca Pessanha, Utrecht University, Netherlands
Title
What can breathing patterns tell us to analyze Oral History Archives?
Abstract
Computational technologies have revolutionized the archival sciences field, prompting new approaches to process the extensive data in these collections. Automatic speech recognition (ASR) and natural language processing (NLP) create unique possibilities for analysis of oral history interviews, where otherwise the transcription and analysis of the full recording would be too time-consuming. However, many oral historians note the loss of aural information when converting the speech into text, pointing out the relevance of subjective cues for a full understanding of the interviewee narrative.
For the oral historians who emphasize the shortcomings of memory and storytelling as the strong points of oral history studies, the nonverbal cues of the interviewees, as well as the dialogue between the interviewee and the interviewer contain important information that needs to be included in the archive, and should be analyzed further to complete the research. The ability to look at human emotions, and somatic reactions while narrating important life events, both on an individual level, as well as on a collective level, gives scholars from many fields the means to focus on the feelings, mood, culture and subjective experiences on a mass scale. We argue in this talk that the advancements in affective computing, social signal processing, and automatic analysis of non-verbal communication is providing us with new tools that can capture the subjective and the emotional content of the OH-archives as well. To demonstrate our point, we will focus on one of the somatic cues of emotionality, namely breathing. Studies show a correlation between respiratory feedback and emotions such as joy, anger, fear and sadness. However, unlike speech and paralinguistic speech that is developed to a point where automatic tools perform relatively well, to capture and analyze breathing signals automatically in oral history archives is a challenge due to the lack of ground truth of breathing signals. We will highlight first the challenge of using cross dataset training to predict breathing signals automatically in a depression dataset, while also summarizing reports on affective states and expected/observed breathing patterns from the literature.
Bio
Dr. Almila Akdag Salah is an assist prof. at the Utrecht University, Dept. of Information and Computational Sciences. Her research interests combine qualitative and quantitative methods to study mainly humanities and social data. She is currently the Visual Media and Interactivity WG leader at DARIAH (the Digital Research Infrastructure for the Arts and Humanities.
Francisca Pessanha is a PhD candidate at the Information and Computer Sciences Department of Utrecht University. They obtained an integrated M.Sc degree in Bioengineering with a specialization in Biomedical Engineering at Porto University in 2020. During this period, Francisca was a visiting researcher at both Cambridge and Utrecht University, working in facial pain estimation assessment in animals. Their current research focuses on affective computing, particularly computer vision and paralinguistics for affective state analysis.
Myriam Fellous-Sigrist, King's College London, United Kingdom
Title
Between access and protection: applied ethics for curating digital oral history
Abstract
I will show how these concepts can assist us to better understand why digital (and especially online) dissemination still makes many oral history custodians uncomfortable across a large range of professional contexts, including universities, archives centres, libraries, museums and charities. The scalability and sustainability of solutions found so far are some of the remaining challenges which need to be addressed if practitioners want to live up to the democratic ambitions shared by oral history and digital humanities.
Bio
I am an oral historian and a researcher in Digital Humanities. I recently defended my PhD at King’s College London; my project focused on the ethical, legal and curatorial issues of digital oral history. I am currently working at the British Library as a rights officer for the Unlocking Our Sound Heritage project. I previously worked in the research data management team at University College London Library and as a project manager in an industrial heritage organisation in France. Since 2014 I have been a member of the executive and editorial boards of the French Association for Audiovisual and Oral Archives (AFAS).