In the Literary Lab Project Suspense: Language, Narrative, Affect, our goal is to use computational methods to trace the experience of suspense as it is felt by the reader back to the formal features that create the conditions of possibility for this experience. As this project seeks to bridge the gap between the affective or emotional experience of reading and the structural elements of narrative, we combine methods taken from critical theory, quantitative analysis and social psychology, and the data that we mobilize for this project reflects this diversity. Drawing on a corpus of 200 suspenseful works (written between 1764 to 2013) and 100 unsuspenseful texts, this project combines methods native to the quantitative analysis of text, such as topic modeling or distinctive words, with reader-derived tags identifying suspenseful or unsuspenseful parts of novels in the corpus. Our innovative methodology allows us to trace the patterns of the various word groups that we derive through these models across the narratives of individual texts and test which sets of data accurately predict passages marked as suspenseful or unsuspenseful by the readers in our study.
The sheer diversity of this data, however, presents a challenge for our analysis. We are faced with combining not just distinctive word frequencies with model-based posterior probabilities, but also scored passages by readers. Although our early findings seemed to indicate a relationship between the pattern and volatility of highly suspenseful and unsuspenseful passages across a narrative in heightening the experience of suspense within a text, the complications inherent in combining this data within a single model presented a significant barrier to exploring this direction further. Our most promising finding thus far has been to identify potential topics whose rise or fall across the narrative seems to predict highly suspenseful passages: both those tagged and those missed by readers. Drawing on this success, this project has recently turned to developing a model that could accurately predict whether a given passage would be rated as suspenseful or unsuspenseful based on our feature set and leveraging these predictions to draw conclusions about the efficacy of our various features in creating the affect of suspense. In other words, by creating a virtual reader whose responses to texts imitated that of our readers, we could determine which features (which topics, or clusters of words) were most important in deciding whether or not a given passage was suspenseful.
Given the sheer quantity and variety of our data, however, standard models, such as discriminant function analysis (DFA) or logistic regression proved problematic. We therefore turned to a model capable of processing both a large number and variety of features within a classification-based model: a neural network.
Through an iterative selection process we trained a series of neural networks using our feature set and reader tagged passages and tested them for accuracy using data withheld from the training set. Our final model for this stage of the project is a network with three hidden levels (with 75, 45 and 10 nodes per level respectively).
The network uses a reduced set of 91 features to predict whether a passage (whose length is 2% of the total book length) should be classified as suspenseful or unsuspenseful. On the test data set, the accuracy of the model is an impressive 80.6% with 83% of suspenseful passages and 78% of unsuspenseful passages successfully identified.
The relative weights of the various input variables as modeled by the neural network are recoverable with Garson’s algorithm. This allowed us to not only gauge the accuracy of the model overall, but also to reveal which features were most important for determining the suspensefulness of a passage (either through their presence of absence).
Here, the most important feature for our model is the topic that we’ve titled ‘Drawing Room Conversation, ’ whose absence is highly predictive of suspense. Other key topics that our model has revealed include ‘Physical Pain,’ ‘Sentimental Romanticism’ and ‘Female Hardboiled’. Using this diagram, therefore, our current work involves investigating the narrative progression of individual texts to recover the ways in which these sets of topic words interact on a text by text basis.
In addition to ranking the inferential ability of our features, the model also allows us to predict the level of suspense in texts that have not been tagged by our readers, allowing us to visualize the pattern of suspenseful and unsuspenseful passages across a much larger sample of novels than could reasonably be tagged by our participants. The neural network returns not only a prediction for a passage, but also a confidence interval. By leveraging the classification and confidence per passage within an entire novel, we are able to tag all passages of a novel for their suspense rating. This allows us to visualize a much more comprehensive view of the pattern of suspenseful and unsuspenseful scenes across an entire narrative.
These three graphs all show sections within the narrative of Shelley’s Frankenstein tagged for suspense. The bluer the bar, the more unsuspenseful the passage, the redder the bar, the more suspenseful the passage is. The top two graphs represent the tags of two of our readers (sections without colors were not tagged by the readers). At the bottom is a graph tagged by the neural net. Not only does it reveal a more striking pattern of suspenseful and unsuspenseful passages across the text, but the tags of the neural net line up, to a surprising degree with the tags of the readers.
Although not among the initial goals of this project, the virtual reader that we have created has offered a new way to explore the phenomenon of suspense. In our current work, we continue to unpack these findings, refine our feature sets and leverage these new comprehensively tagged texts to investigate patterns in the text that are indicative of suspense at the level of narrative, rather than just the passage.