For this publication, a Creative Commons Attribution 4.0 International license has been granted by the author(s), who retain full copyright.
born digital
The digitization of audiobooks has led to a significant increase in listening to literature. Despite their popularity, audiobooks are often considered to be only a superficial and less demanding alternative to reading printed books. However, there is very little empirical research on the listening experience of literature, making it essential to systematically and empirically examine the specific practices and experiences involved in listening to audiobooks and to compare them with the act of reading books. This article outlines the central research questions and the methodological approach of the FWF project
Die Digitalisierung von Hörbüchern hat zu einem erheblichen Anstieg des Hörens von Literatur geführt. Trotz ihrer Beliebtheit wird Hörbüchern oft zugeschrieben, dass sie nur eine oberflächliche und weniger anspruchsvolle Alternative zum Lesen gedruckter Bücher darstellen. Jedoch liegen sehr wenige empirische Studien zum Hören von Literatur vor und es scheint unerlässlich, die spezifischen Praktiken und Erfahrungen beim Hören von Hörbüchern systematisch und empirisch zu untersuchen sowie mit dem Lesen von Büchern in Beziehung zu setzen. Der Beitrag erläutert die zentralen Forschungsfragen und die methodische Vorgehensweise des FWF-Projekts
For a long time, poetry and narratives were transmitted primarily orally through
recitation or performance, and literary texts continued to be read aloud in
various contexts even after the widespread adoption of print (e.g., Schön 1987).
The first encounter with literature in a person's life takes place in childhood
when listening to stories and rhymes in storytelling and situations of reading
aloud. Research on the influence of reading aloud on reading socialization and
the widespread practice of reading aloud in primary school shows time and again
that auditory reception is the beginning of engagement with literature (e.g.,
Duursma et al. 2008; Müller 2004). From a media history
perspective, it was Edison’s invention of the phonograph in 1877 that made it
possible for the first time to record and reproduce sound, and thus to hear
spoken words independently of a person being present (Rodero 2018). From the
term ‘talking book’, emerging in the 1930s in the context of government programs
for blind readers, to the term ‘audiobook’ in the 1970s, with the use of audio
cassettes as an alternative to records, the use of audiobooks has changed
fundamentally due to technological developments: Whereas an unabridged recording of
Tolstoy’s
(Rubery 2011: 8). New
portable delivery technologies like the smartphone and streaming services have
not only made audiobooks more affordable to use but have also expanded the
places and situations in which audiobooks are received.
It is therefore not surprising that the digitization of audiobooks in particular has led to an increase in people listening to literature and that the status of this practice needs to be reexamined. In addition, the twenty-first century has seen a general resurgence in significant and dynamic engagement with literature through various auditory mediums: alongside audiobooks, there are author readings, poetry slams or podcasts. Certain scholars contend that this phenomenon signifies an ‘acoustic turn’ that influences modern culture, as discussed by Meyer (2008) and Steiner (2017).
Market research numbers provide a good reason for focusing on the reception of digital audiobooks. A representative survey about respondents’ reading and purchasing behaviors conducted in September 2020 in the US, Germany, the UK, France, Italy, Sweden, Switzerland, and the Netherlands has shown that while only one-third of consumers exclusively read printed media, 40 percent are so-called ‘multi-format users’ – i.e. they consume both print media and e-books, as well as audiobooks (Ott 2020). In Germany, where sales of e-book downloads increased by 16.2 percent from 2019 to 2020, audiobook downloads increased by 24.5 percent. For the same period, e-book subscriptions grew by 28.4 percent and audiobook subscriptions expanded by a remarkable 78.3 percent (Börsenverein des Deutschen Buchhandels 2021). In short, with more and more people consuming literature via digital audiobooks, it appears highly relevant to better understand how such a change in the mediality of literature consumption might affect the reader’s or listener’s literary experience.
Nevertheless, the consequences of the rise of aural digital media as a form of
engaging with literature has received little attention in academic research.
Academic publications remain remarkably rare and Rubery's conclusion still
appears to be valid: The
audiobook has struggled to gain acceptance among the humanities as a
legitimate aesthetic form despite its growing popularity. The absence of
critical discussion of a format that has been around for over a century is
one indication of its marginal status
(Rubery 2011: 10).
Clearly, skepticism toward the audiobook medium stems from the fact that it is
compared to the printed book as the leading medium for experiencing literature.
Because of concern that listening to books might lead to a poorer reception of
literature than reading a book, listening to audiobooks is often considered a
shallow alternative (see Rubery 2016).
However, the need for research on audiobooks is obvious, as recently stated by Baron in her book
regarding the mediality aspects of reading, just as it questions the primacy of the printed book(Have/Pedersen 2016: 27). Yet beyond such pleas, the study and especially the empirical investigation of reception differences between reading and listening to literature is still a desideratum in research in general, and in literary studies in particular. For a long time, literary studies have focused on aspects of writing, aesthetics, interpretation, and literary history, leaving the question of reading and reception largely to other disciplines. Only medieval studies have dealt with the orality of literature, but mostly from the perspective of text production, the role of the speaker/singer, and the consequences for the development of oral or written cultures (e.g., Haferland 2004; Wenzel 2002). Other disciplines like cultural studies, media studies, sound studies, and poetry performance studies, which have also been dealing with the phenomenon of listening to texts, agree that media play a crucial role in the formation of culture in general and artistic communication in particular. Above all, the history of reading shows that changes in reading media go hand in hand with changes in reading habits, reading processes, and the effects of reading (e.g., Chartier 1990; Debray 2003).
In short, what is needed is an evidence-based understanding of what the transition from reading to listening means for the literary experience. As Binczek (2020) recently stated, especially in literary studies, there is a lack of differentiated, proven descriptive criteria that take into account the specific auditory signification processes, including all the relevant paralinguistic features. In light of the growing prevalence of digital audiobooks, the key question within the field of literary studies that has once again become pressing is whether listening results in a distinct literary experience and textual comprehension of a literary work compared to reading.
Eighteenth-century aesthetics fought a competition between the senses and
Lessing, for example, recognized the acoustic channel as the medium that, due to
its affective charge, harbored the greatest aesthetic potential (Lessing
2001[1790]: 571). Apart from the preference of one sense over another
for the optimal aesthetic experience, the question of the merits and demerits of
silent reading or reading aloud has been raised again and again since antiquity,
and today silent reading especially is considered the primary technique for
sense making and correct comprehension when reading (Bickenbach 2020).
However, literary texts are never totally silent and are always related to the
world of sound: as Ong (2012: 8) pointed out: ‘Reading’ a text means converting it
to sound, aloud or in the imagination.
The ‘vocalization’ of the
printed pages by the reader is thus seen as a central aspect of the imaginative
apprehension of literary texts, which is why critics see the loss of the
reader's inner voice as a weak point of the audiobook (see Rubery 2011).
Nevertheless, it is unclear what differences arise according to whether one’s
own inner voice makes the text sound or whether the text is transferred into
speech by another voice, which actually is emitted acoustically and exists
physically. In contrast to the reading of a book, in which the reader is
visually engaged in the text, listening frees up the eyes to observe and imagine
(Bull 2007:
40). Therefore, Wittkower (2011) concludes that the
experience of an audiobook is very different from written text due to the
simultaneous experience of an arbitrarily related visual field. Reading is often
associated with the invitation temporarily to shut out the sounds and sights of
our present, and significant acoustic stimuli are regarded as disturbance. But
what happens when we listen to literature is that it is precisely the acoustic
stimuli which enable us to enter an imagined world while the sense of sight is
open for input beyond the text.
Furthermore, Binczek and Epping-Jäger (2014)
critically point out that the evaluation of the audiobook as a medium of
literature is determined by its mode of use, and that its reception is often not
understood as an exclusive activity, but reduced to an activity taking place
alongside driving, sport or household chores. According to the Immersive Media
& Books 2020 Consumer Survey, compared to the reading of print books, 70
percent of audiobook engagers are multitasking while listening to a novel or a
tale (Anderson
2021). It seems important, however, that the audiobook is not
condemned per se as an inferior medium for the reception of literary texts but
that the modes of reception are examined closely. Reading and listening to
literature depend on context to a certain degree; and because it is possible to
detach the eyes from the text, to be mobile and able to use one’s hands when
listening compared to reading books, the surroundings and possible activities
play a potentially important role which must be considered: The shift from experiencing the book
with the eyes to experiencing it with the ears has a great impact both on
the literary experience and on how we experience our surroundings while
reading
(Have/Pedersen 2016: 29). The question
that now arises is: if either the sense of sight or the sense of hearing can be
used in the reception of one and the same literary work, what exactly are the
differences in the literary experience and which perceptual characteristics are
determined by the mode of reception?
Our Austrian Science Fund (FWF)-funded project
This project encourages a holistic view of the literary experience of narrative fiction, trying to understand what happens to different dimensions when the literary text is heard and not read. Given the lack of a coherent theoretical framework of literary experience, for the purpose of our study we isolate four concrete dimensions: (1) text comprehension; (2) aesthetic emotions; (3) identification and engagement with narrative characters; and (4) immersion and mental representation of the narrated world. These dimensions are centrally related to a literary reading experience but also relate to listening to literary texts and lend themselves to systematic assessment through empirical measurement. In the following, these theoretical dimensions of literary experience will be elaborated in greater detail and the potential differences between listening and reading will be identified.
The comprehension of a literary text from the surface structure of the textbase to the situation model and to more complex aesthetic macrostructures (van Dijk/Kintsch 1983; Zwaan 1996) is the logical prerequisite for any literary experience and therefore important to consider when studying the effects of media modality. From a cognitive perspective, it is still unclear if the brain processes written and spoken literature similarly. Some studies show that specific semantic information is processed in the same brain regions regardless of modality (e.g., Deniz et al. 2019). However, other studies show differences in brain activation depending on modality and individual differences in working memory (e.g., Buchweitz et al. 2009). Subjects have shown similar competencies whether reading or listening to the test materials (Rogowsky et al. 2016). However, the majority of existing studies on reading and audiobooks are clearly related to reading and language acquisition (for struggling readers in particular); they deal with children's listening and reading, and are classroom-based studies with material created especially for the classroom (Moore/Cahill 2016). Regarding memory, as part of text comprehension, Baron attributes a primacy of print to physical differences between speech and writing such as the durability of writing, the control over pace, the ease of rereading, the easy ability to skim or skip passages, and the visual landmarks in the text (Baron 2021: 166); but this applies to longer texts, because with words and sentences, recall and comprehension are better with audio (Baron 2021: 166). Whether these assumptions are confirmed is unclear, as listening to literature outside the school context and as a leisure activity – arguably the most common use of digital audiobooks among adults – has hardly been studied empirically.
Beyond the mere comprehension of the text, reading narrative fiction can be a
profoundly emotional experience (Oatley 2002). In particular, aesthetic
emotions form the literary experience, being the emotions that can
arise when a person perceives and evaluates a stimulus for its aesthetic
appeal or virtues
(Schindler et al. 2017: 2).
Aesthetic emotions are based on the intrinsic appeal of their stimuli; they
are traditionally
limited to perceptual input from the distance senses
(i.e.
hearing, vision) and they are intertwined with aesthetic judgment
(Schindler
et al. 2017: 3).
When a written text is performed by an audiobook narrator, literature is given a facet that the written original lacks. In addition to the genuine aesthetic structure of the literary text, there are the specific features of spoken language such as rhythm, articulation, pace, and sound. The shift from print to sound is embodied above all in the performing voice, which can hardly be emotionless, even if the spoken content contains no emotions:
For the reader who comes upon the word on a page, the field and its unsaid significance is a dark obscurity […]. But if this word is spoken, there is already a certain potential field and presence of unsaid significance in the voice. If 'Adam' is said in an angry voice, imploringly, or in a quiet whisper, each sounded presence allows the 'bare word' to emerge from some of its obscurity in the sounding of its presence (Ihde 2007: 154).
It is necessary to explore whether the emotional involvement of listening to
a novel is different from that of reading a novel, as the intensity of
emotional involvement is by itself a prime factor of aesthetic enjoyment
and liking
(Menninghaus et al. 2019:
186). The central question is whether the performed interpretation of
the literary text via the voice opens up new or different aesthetic emotions
and a different aesthetic evaluation of the linguistic work of art.
Voice is an expression of individual personality traits, because apart from lexical messages, it also carries intimate information about a person (Schnickmann 2007). Therefore, in the case of the audiobook, the text is always interpreted by a physical and performing voice with specific characteristics. An unpublished study from 2017 confirmed a greater identification and connection with characters, greater emotional engagement measured by recording physiological activity, greater creation of mental images, and greater involvement when listening to a story compared to reading it (Rodero 2018).
Modern narratology takes for granted that first person (i.e., homodiegetic)
and third person (i.e., heterodiegetic) narrations lead to different
aesthetic experiences (Martínez/Scheffel 2019: 23).
Readers' reactions to a story are influenced by the narrative stance because
it can affect identification and the general assessment of a character (van Krieken et al. 2017). As Dixon
et al. (2020) reported, the intuitive assumption that, for
example, an autodiegetic narrator (i.e., first-person narration by the main
character) produces more identification and sympathy for the main character
than a heterodiegetic narrator has been empirically studied but has led to
different results. Nevertheless, the type of narrator influences the
reader’s reactions and an autodiegetic narration seems to reduce the tendency to experience
the circumstances of the character
(Dixon et al.
2020: 222). However, it is relevant to explore if those results
can be transferred to the reception of audiobooks. Empathy, identification,
and other parasocial engagement with a character might change if a text is
received as an audiobook with a speaker narrating a story referring to the
character as ‘I’ or ‘he/she/they’ and replacing the inner voice with an
actual real human being.
It is a distinctive element of fictional texts – and for many people the main
reason for reading literature – that readers move into the fictitious world.
Gerrig coined the term ‘transportation’ for this specific mechanism of
experiencing narratives, the impression of having left the real world
behind while visiting narrative worlds
(Gerrig 1993:
157), but the terms ‘immersion’ and ‘absorption’ are used
synonymously. Conjectures about the extent to which listening may have a
more immersive effect than reading, or vice versa, are yet to be empirically
investigated. In particular, beyond the basic text comprehension, it is
important to investigate whether there are differences in the emergence and
intensity of situation models, the mental representations of the people, objects,
locations, events, and actions described in a text, not of the words,
phrases, clauses, sentences, and paragraphs of a text
(Zwaan 1999:
15), when listening to or reading the same text. The quality of
the mental representation of the narrated world can be considered a basic
condition for successful reception of a literary work, and therefore must
receive special attention in the study of reading and listening to
literature.
Due to the lack of empirical studies on the reception of digital audiobooks, especially with a focus on literary texts and literary experiences, this project is based on a mixed-methods approach to explore the phenomenon of audiobook listening in as many ways as possible, drawing on our backgrounds in literary studies, communication science, and psychology, as well as our previous research experience on a similar topic (e.g., Schwabe et al. 2021). In concrete terms, to explore possible hypotheses and gain a feeling for the subjectively important aspects of the audiobook listening experience, we first conducted a series of qualitative focus group interviews. On the basis of the insights extracted through these interviews, we designed a series of quantitative experiments.
The goal of the focus group study was to explore the actual audiobook listening practices of regular audiobook users and their subjective evaluation of their listening experiences. Because the strength of focus group studies lies primarily in exploration – i.e., in generating rather than testing hypotheses – they are a productive complement to quantitative experiments. Rather than seeking to generalize the findings, focus group studies strive to uncover essential aspects of the data, such as behavioral, experiential, and attitudinal patterns (Bloor et al. 2001). The addition of qualitative methods to empirical reading research provides particular insights into habitual and actual media use which cannot be collected in any other way but are an essential prerequisite for multidimensional research. As the published literature exploring leisure audiobook reading is sparse, we needed a starting point for our research as a base we could build on. Thus, our goal with the focus group study was to gather as much subjective and conscious information as possible about the experiences and practices surrounding the topic of audiobook listening. Consequently, the data are delivered by self-description and collective discussion of the listening practices of persons who are familiar with using audiobooks. We conducted seven focus group interviews with four to six participants, accompanied by a short online questionnaire. In total, the sample consisted of 34 participants between 18 and 57 years old. The qualitative analysis of the transcripts came down to collecting various experiences, identifying common patterns, and classifying opinions on our focused topics. For coding, we followed a framework that we developed for describing the auditory reception of literature (Kosch et al. 2024). The framework builds on the fundamental assumption that listening to literature occurs in a complex interplay among the text, medium, listener, situational factors, and associated effects.
In brief, the analysis of the interviews showed that audiobook listening clearly differs from the reading of books as it entails distinct expectations and objectives, occurs within diverse situations, and involves different textual content. Furthermore, the act of consuming literature through auditory means is not clearly perceived as inferior or superior in terms of comprehension, aesthetic emotions, immersion, recall, visualization, etc., compared to the act of reading. Instead, a variety of distinct perceptual perspectives prevail. This finding underscores the imperative to examine the distinctions between the experiences of engaging with literature through reading or auditory reception, encompassing both favorable and adverse aspects, within a controlled laboratory environment and quantitative studies.
The goal of the quantitative experiments is to capture both conscious and more unconscious aspects of audiobook listening and compare those with reading a printed book. We will connect our initial theoretical assumptions – that the text comprehension, aesthetic emotions, identification and engagement with narrative characters, and the immersion and mental representation of the narrated world might be different between listening and reading a printed book – with the insights provided by our focus group study.
First of all, the main goal of the first lab experiment, which will be conducted from January to June 2024, is to explore the differences in text comprehension and aesthetic emotions between reading the same literary text in print and listening to the audiobook. Secondly, the focus study revealed that many audiobook listeners rarely sit or lie down with an audiobook and with the sole purpose of receiving the text. For many of our focus group participants, audiobooks are a tool to make chores and other less cognitively demanding tasks more interesting. Therefore, we will study the effect of letting participants do an extra manual task during the stimulus presentation. Thus, we will conduct an experiment where we will have one group sitting in a reading chair reading a literary text, one group sitting in a reading chair listening to an audiobook version of the same text, and one group sitting at a table coloring mandalas while listening to the audiobook recording. Afterwards the participants will fill out a reading comprehension test and a standardized questionnaire assessing their aesthetic emotions toward the text or the recording.
The second experiment will deal with differences in identification and emotional engagement with the narrative character. In our focus group study, the participants highlighted the importance of the audiobook narrator's voice and the effect it has on their emotions, such as empathy or sympathy. An audiobook narrator’s voice adds a layer to the text that could be beneficial for identification and emotional engagement with the narrative characters, but it could also be a factor disrupting a reader’s relationship with the literary characters. Additionally, the audiobook narrator’s voice might change the dynamic of the narrative perspective and the reader. A self-read “I” might affect the reader differently than an “I” read by an actual other human person. For the experiment, we will manipulate the short story
Lastly, we will conduct an experiment to focus on immersion and the mental representation of the narrated world. Many of the participants in the focus group study, along with participants in a focus group study and a survey study we carried out with the theme of digital reading (see Kosch et al. 2021; Schwabe et al. 2023), reported that they read or listen to audiobooks while in public spaces, like parks or on public transport. With a focus on immersion and mental representation of the narrated world, we plan to explore how these situations can affect the reception of a text. When reading a printed book, the eyes are occupied while the ears can still perceive sounds and noises. The opposite is the case during audiobook reading. While listening to an audiobook, the eyes observe the surrounding environment. In this experiment, we study the influence of different channels of perception of a literary text (ears and eyes) while processing the information from different surrounding elements in the environment. Two of the groups will be confronted with more distracting surroundings, the other two groups with less distracting surroundings. The participants will either read a text in a printed book while they hear distracting traffic noises or less distracting forest sounds, or they will listen to the audiobook of the same text while they wear a head-mounted virtual display seeing traffic or a forest virtual reality (VR) video. If there are problems due to the VR setup, we will change these conditions to a dark room with a projector screening the video on a wall. Afterwards, the participants will fill out a series of standardized questionnaires measuring their immersion and mental representation.
Due to the digitization of audiobooks, which are now published at the same time as the printed originals, their status has changed fundamentally and figures from recent years show that audiobooks have gained a considerable market share and established themselves as a popular medium for literature consumption. Nevertheless, to this day, silent reading in particular is considered the primary technique for sense-making and proper comprehension in reading (e.g., Bickenbach 2020), and cultural skeptics often see audiobooks as a secondary form of literature that not only encroaches upon traditional reading but also tempts readers to engage with literature in a superficial manner (e.g., Klotz 2022).
Despite the extensive and still-growing prevalence of digital audiobooks, research in this domain remains sparse. Empirical investigations into the differences between listening and reading texts are scarce outside an educational context and are rarely centered on literary materials. It is within this research gap that this project situates itself, aiming to ascertain potential distinctions between the literary experiences of reading books and listening to digital audiobooks through a range of empirical studies. The focus group study has already revealed that audiobooks serve different functions and objectives for users compared to traditional books. Contexts and situations of literary reception also differ significantly between the two mediums. Perhaps most strikingly, the audiobook has been attributed both advantages and disadvantages concerning various dimensions of the literary experience, including comprehension, aesthetic emotions, immersion, and engagement with characters. These subjective disparities in the perception of differences between visual and auditory reception of a literary text will be examined thoroughly through experimental studies. This endeavor aims to shed light on the actual impact on the literary experience of transitioning from reading to listening to literary texts, thereby making a significant contribution to the research on digital audiobooks.
This article is part of the