The American Historical Review

Abstract

Can statistics help historians to identify the events that are most distinctive of a particular era of time? This essay explores the use of a distinctiveness algorithm from library science for measuring the distinctiveness of manuscripts, tf-idf, recast as "tf-ipf" for the study of the terms most distinctive of historical periods. In a case study, tf-ipf is applied to the text of Hansard's Parliamentary Debates, varying the "period" from a 20-year horizon to a 6-month or one-day horizon. It is shown that the algorithm's assessments of what is most distinctive of 20-year and 10-year periods largely matches the consensus of British historians, while debates that held parliament's attention for six months or fewer have largely fallen beneath the threshold of scholarly attention. Attending to concerns that took up parliamentary debate for six months or fewer, the essay argues that tf-ipf thus presents a metric of parliamentary attention that mirrors the exclusions of class, allowing scholars to retrieve a timeline of when the concerns of the distillers, bleachers, dyers, Chartists, crofters, and miners reached a national debate. More generally, tf-ipf represents an important new tool for discovering the distinctive aspects of historical periods based on past experience, not historical bias.

A perennial challenge for scholars of history is refining the ability to reason critically about the relationship between long-term trends and short-term change. The skillful practitioner has much to accomplish: they must sort, accurately, through an enormous volume of evidence, all the while noticing the structural changes that might have shifted the standing of some group of actors. They must map the slow-moving influences of climate, migration, identity, and status against smaller ripples in the pulse of history. In short, they have a great deal of information to organize. Any tool that helps them to organize their information more robustly is an aid.

In the first decades of the nineteenth century, as British troops countered French ones on the continent, British elites were well aware that violence from below had recently decapitated a king in France and that revolutionary ideas posed a challenge to the existing political order at home. Aristocratic representatives in Parliament debated how best to avert the threat, invoking a phrase important in 1817-19 but less frequently invoked after 1820: “seditious meetings.” With this phrase, parliamentary representatives invoked radical political meetings as conspiracies, resulting in the censorship of the press and jailing of publishers.1

Within a decade of these events, however, the gag on the press had been removed. A new consensus was reached such that the previous political paranoia about sedition was abated, with the result that the language of “seditious meetings” was entirely outmoded by a new ideal of liberalism that privileged individual liberty and the freedom of the press as defining virtues of British identity (the trend has been verified, even though, as Michael Lobban found in 1990, the more general language of “unlawful assemblies” was intermittently invoked over much of the rest of the century to thwart newer popular movements).2

The steep falloff of the political concept of “sedition” presents one mark of an event distinctive of a moment in history, where the concept is strongly linked to one era but minimally present (if at all) in any respect in the periods that follow. After 1820, the phrase rapidly became a kind of political fossil, rarely again invoked with the same insistence, whatever the lasting consequences of the original moment. Even though the phrase “seditious meetings” exhibits low frequency in the period 1800–1820, the phrase is nevertheless mathematically distinctive of the era. Translating the statistical concept of “distinctiveness” or “surprise” into an idea about history, we might explain that “sedition” as a category was a defining political movement of the first decade of the nineteenth century.

This article explores a statistical approach to discovering events via tracking the language that is most distinctive of a particular period in time. It proposes the use of a statistical measure for scoring shifts of collective attention of the kind exemplified by the phrase “seditious meetings” in the first decade of the nineteenth century.

We are entering an age where a rigorous interrogation of what happened when might be best accomplished by critical thinking allied to the algorithm. The advantage of translating concern for temporal relationship into the language of code is that algorithms don’t experience fatigue when looking for repeated words over long series of pages. As a result, they can help scholars to carefully identify moments of change with contextual depth and accuracy. Faced with the knot of unconscious bias, algorithms can help to unsettle historicism, interrupting scholars’ tendency to read importance back into prior events based on the bias of the present rather than the historical reality of what happened in its unexpected diversity.

This article charts a new method for mapping the relationship between the pulses of long-term trends and short-term change, drawn out of a shift at the level of mathematics. Analysts who use text mining have frequently focused on frequency (as in word count) or regularity (as in topic models), typically using algorithms to discern a single linear trend over time. This paper proposes an alternative mathematical approach—the math of distinctiveness—as a tool for identifying the words most unique to each period of time.

This article introduces an algorithmic application that I call “term frequency–inverse period frequency,” or TF-IPF, alongside a technique for implementing the algorithm in the comparison of temporal divisions of increasing granularity, from a twenty-year period to a day. In contrast to counts that privilege frequency and average behavior, TF-IPF returns very different answers depending on the scale of aggregation—whether the week, month, or year is at stake. Looking at the same data through the perspective of a new algorithm, we can vary the scale of aggregation and so learn fundamentally new things about the past. This article examines how the algorithm delivers the material for a synthesis of long-term trends and short-term change that profiles the role of dissent in the British parliamentary debates of the nineteenth century. By the article’s end, it will present the evidence that such an approach can result in a subtly new perspective on the dynamics of elite liberalism and popular pressure in Parliament.

Working with the material of British Parliament—which has long been familiar to many historians—provides a useful index to the method’s veracity and power. In the case studies that follow, TF-IPF is applied to the British debates of the House of Commons and the House of Lords (commonly known as “Hansard”).3

As we shall see, using TF-IPF to guide a review of the most distinctive concerns of each period in Hansard reveals blind spots in the historical record that signify an enduring lack of attention in the historical profession to most episodes of historical experience in the nineteenth century that occupied fewer than six months of parliamentary attention. Applying TF-IPF allows historians to track and compare episodes that have escaped the notice of previous generations of historians and to bring them into dialogue with long-term forces. In the hands of a critical historian, algorithms can serve as a check on the biases of the historical profession. Instead of deferring to secondary sources, historians armed with statistics have the opportunity to recognize the meaningfulness of those episodes of past experience that might have seemed most surprising or unusual to a contemporary.

Introducing TF-IPF: Document Statistics Applied to a Temporal Context

This article introduces a new algorithm—TF-IPF—to serve the purpose of detecting the words that are most distinctive of certain time periods in the historical record. TF-IPF allows a scholar to return, for any given period of time that represents a subset of a longer archive, a list of terms that are ranked in terms of distinctiveness. As the user varies the period in question from decade to year to month to week, the algorithm returns a new list of terms that are distinctive of each, typically showing little, if any, overlap between the words most distinctive of two adjacent weeks, or between a week and the month that contains it. The potential contribution of TF-IPF derives from its power as an algorithm for comparing the components of historical change on different scales: it can allow the user to generalize about the slow course of change from decade to decade, against the faster clip of ideas that captured the imagination week in, week out.

TF-IPF represents the barest intervention in mathematical terms; it is cribbed from another algorithm, TF-IDF, long familiar to students of the digital humanities and library science, which was developed for the purpose of indexing documents for electronic library catalogs by identifying the terms that are most distinctive of any given paper, article, or book.4 Invented in the 1970s by mathematician Karen Spärck Jones to provide automated detection of subject headers for librarians assembling catalogs for journal articles, TF-IDF historically has been used for indexing words specific to one document but not to others for the purposes of suggesting keywords or subject-term indexes for journal articles or books.5

The algorithms in question are special because they use comparisons of word count in different subsections of the purpose to produce a measure not of averages but rather of distinctiveness. The key feature of TF-IDF is that it is designed to show a kind of “differential” calculus of terms found in one heading but not another.

When the scholar applies TF-IPF to Hansard, divided into periods of twenty years, the method produces a new overview of long-running political debates that punctuated the century, albeit one that is largely familiar to British historians (see fig. 1). It seems that around 1800, raw commodities, trade, and industrial and agricultural processes dominated parliamentary debate: the “corn trade,” “corn distillery,” “sugar distillation,” “ships registry,” “framework,” “distillation,” and “distillery”—basically the bureaucracy and infrastructure of industrialization. Alongside these commodities and their trade is concern with institutions and civic order, signaled by debates about “seditious meetings” and “blasphemous libel.” Next, 1820–40 appears as the period of church against the Enlightenment, featuring debates about the “Anatomy” Act, the observance of the “Sabbath,” and “liberty of the press.” The 1840s and 1850s are distinguished as the era of public works, with “drainage,” “museums,” “new houses of Parliament,” and “incumbered estates” predominating. The 1860s and 1870s appear as the biological era: “cattle plague,” “contagious diseases,” and “intoxicating liquors” all feature as headlines of debate, while, for the first time, empire appears on the top list, condensed into the “Eastern Question” of war with the Ottoman Empire. The 1880s and 1890s become the era of industrial socialism, with “teachers,” “labourers cottages,” the management of the “congested districts,” and the plight of “crofters” as specific social topics receiving attention, in comparison with earlier social issues, handled under the heading of “distress of the country” or the plight of the “destitute”—all paired with the ideal liberal aesthetic framework of “art.” The period 1900–1910 shows the intensification of focus on social issues, on “workmens compensation,” the “unemployed,” “evicted” tenants, “small holdings,” “housing,” and “meals,” along with “teachers,” continuing the theme of the 1880s.6

It’s helpful here to compare both TF-IPF and raw count (see fig. 2). The phrase “seditious meetings” does not appear frequently, and in the raw count visualization, it is scored quite low. However, because “seditious meetings” in 1800-20 is the most distinctive phrase-period relationship of any phrase-period relationship in the century, the phrase receives the highest TF-IPF score. How can a phrase be distinctive if it is not also prevalent? The significance of “distinctiveness” is partially explained if we consider the size of the corpus being measured within each period. The average count of each keyword in figure 8 goes up over the century from under one thousand to more than three thousand, a reflection of the increasing number of words spoken every year in Parliament that resulted from rising pressure on speakers from the culture of newspapers and the second reform act.7

Because there are fewer words overall in 1806–10, the score of TF-IPF, or the distinctiveness of the concern per period, is higher. A high-TF-IPF phrase like “seditious meetings” may be rare overall, but its appearances are concentrated within a single period; it is therefore a highly significant phrase for caricaturing parliamentary debates in 1806–10.

Given a basic understanding of TF-IPF as a mark of distinctiveness, the research project in question proceeded by inquiring into the consequence of narrowing the temporal period. TF-IPF was then applied on the scale of different temporal horizons, including day, week, month, year, five-year period, ten-year period, and twenty-year period. Each adjustment produced a different set of keywords that were differentially statistically correlated with one period but not with others, or, said otherwise, a list of keywords that were spoken about intensely within particular periods but not others. Each refraction represents a different possible overview of the century, with some characterized by the swiftly concluded debates of a day and others marked by the longue durée tides of twenty-year units. Through such a method as this, different timelines characterized by different scales can be measured and put in dialogue with one another.

The humanist who follows the tenets of the research approach that I have elsewhere named “critical search” does not take for granted an algorithm’s exact usages or best application but rather experiments with the algorithm by adjusting the inputs and carefully examining the results.8 As we shall see, the method of comparing different approaches to the same algorithm produced surprising results. At shorter horizons of time, concerns less familiar to the scholar trained in British history began to surface, suggesting that a “hidden” dimension of discourse had been revealed. The following questions arise: What is the significance of the apparent revelation of certain concerns at finer granularities of time? Why are certain subjects hidden or revealed by this method at different resolutions of time?

TF-IPF as a Method for Revealing Distinctiveness at Different Levels of Temporal Granularity

At the threshold of six-month time periods, many familiar terms are still visible, but the method also returns results less familiar to the historian of the nineteenth century (fig. 3). One set of terms from the model—including “wine” (1860) and “sugar” (1901)—suggests different aspects of know historical cases—for example, various commodities’ trades.9 Other terms—a debate around “bleaching” (1865), “refreshment houses” (1860), “insolvent debtors” (1813), “pillory” (1815), and “dyeing” (1860)—collect a series of lesser-known events involving labor and the economy, as if events that were debated over the course of six months but not a year had drifted below an invisible threshold of significance in the imagination of subsequent historians.

As I have suggested previously, the invisibility of stories that took up fewer than twelve months suggests that historians have often turned to Hansard informed by present-day historians’ concerns; they have had less room for identifying new events, even when the material was evidently available. Because historians of the past have had to depend on contemporary observers and secondary sources for much of their orientation toward the past, such an “invisible threshold of significance” may in fact structure many of the silences of the past: cholera is harder to ignore because sewers attracted twenty years’ worth of attention, but the dyeing and bleaching trades are lesser known to most historians of Britain. By condensing information about concerns that took up relatively large portions of national attention, TF-IPF can help historians to overcome our tendency toward historicism, or cherry-picking events worthy of attention based on the concerns of the present. TF-IPF offers an objective approach to uncovering the distinctiveness of certain periods, or differences in temporal experience that might have been felt and recognized by contemporaries. More generally, working with distinctiveness suggests how distant reading can help scholars to uncover pasts that have not otherwise made it to the level of “significant” events to be remembered.

A question of interpretation arises: What does the currency of parliamentary attention mean for the purposes of historians working today? Given a set of unfamiliar events that consumed parliamentary attention on the scale of six months or fewer, does less attention in the past imply that the events should necessarily matter less? Is there some cutoff for how little attention an event might receive and still matter? The question of significance is a deep one, of course, but quantitative measures can nevertheless illuminate some aspects of the meaning of parliamentary attention as a measure of contemporary politics.

It is unsurprising that when faced with the data about events that consumed parliamentary attention for a single day, we know little about most of the topics that received differential parliamentary attention at only this level—for example, the celebration of the queen’s “birthday” (ca. 1870), the “Alkali” Acts (ca. 1870), and debates about “paving” this or that street (ca. 1800–1820) (fig. 4). That the significance of none of these events is immediately and intuitively familiar to a historian of Britain is suggestive that the algorithm has guided us to a frontier of historical knowledge; it will remain for future historians to discover whether events that merited a single day of parliamentary discussion also deserve sustained examination in retrospect.

We should not be too hasty to discard out of hand events that merited little attention. Parliamentary attention is at best a partial proxy for significance. Sometimes the lack of attention merely reflects a hierarchy of power and money attached to a particular subject. For example, less-monetarily-valued industries, such as “alehouses” and “plumbers,” consumed debate for a single day, while more time was allotted to questions about regulating more-valuable industries. Attention on the scale of a week (not shown) was allotted to questions of taxing, regulating, and trading “salt,” “guns,” “silk,” “silver,” “newspapers,” “paper,” “iron,” “diamonds,” “coaches,” “woolens,” and “sugar.” Sustained attention at the month level (not shown) was given to another set of commodities and included debates about “guns” (ca. 1825), “copper” (ca. 1845), and “lead” (ca. 1860). A hitherto invisible hierarchy of commodities’ access to parliamentary attention begins to emerge. The shorter timescales assigned to “pavement” and the relatively longer ones given to “coaches” suggest how differently many political interests were represented in one debate or another, signaling the different quantities of parliamentary attention allotted to local interests (with paving) versus national interests (with coaches). Looking at the debates through different temporal scales, the historian can glimpse a rigorous list of which political alliances were able to dominate national debate and for how long.

Indeed, a hierarchy of power and time is even more visible when we turn to social issues. Most of the events of social history that are familiar to the nineteenth-century historian dominated parliamentary attention only for the period of a single month (not shown), including the discussion of “interments” (ca. 1850), “seditious meetings” (ca. 1815), and “popular petitions” (ca. 1810).10 Likewise, “crofters” dominated only for a week, as did the “revolution” in France of 1832; the striking workers at the “dockyards” in 1886; and the “yeomanry,” “Chartists,” and talk of the “constitution” in the 1840s.11 While historians are familiar with those social concerns that merited a month’s attention, they are less familiar with other concerns important to the popular imagination that held parliamentary attention for only a single day: “vagrants,” “mendicity,” “beerhouses,” and the registration of “plumbers” (see fig. 4).12 Some of these terms may reflect the regulation of commodities (e.g., checks on drink and homelessness), but others reflect the unresolved issue of the rights of laborers, whose political organizations were strong enough to present Parliament with a charter granting all men the right to vote but not strong enough to see the charter debated in any level of seriousness.

Beyond identifying events that were previously invisible, the algorithm also supports the analyst’s work of thinking about the themes and chronology of history. Reading the algorithm’s results with questions about labor in mind supports a new synthesis where the stories of vagrants, Chartists, dockworkers, and crofters together produce an irregular pattern of political eruption over the course of the century, each explosion of which was introduced by a changing cast of players voicing a changing set of concerns unified by the theme of political representation. Because their concerns were only ever treated to six months of debate, labor’s troubles were never fully investigated and its grievances never satisfied.

If this general pattern has long been understood by British social historians, the algorithm accurately tracks the pulse of social eruption, organizing diverse political participants into a narrative synthesis ordered by the level of parliamentary attention they were able to demand: a day, a week, a month, perhaps five years, but rarely more. Such an organization begs for comparison between the issues that commanded short-term attention and the interests that were able to struggle in a more concerted fashion, commanding parliamentary attention over decades at a time. We will return to such a comparison in the essay’s last section.

The experiment has already given us ample material for exploring the dynamics of short- and medium-term change, however. At minimum, the experiment demonstrates how looking at different subsections of time produces different stories—a familiar theme to theory that has been less explored through quantitative measures. At the five-year threshold, the abolition of slavery appears, but it disappears at the ten-year threshold, when the relative uniqueness of slavery abolition to that time period is dwarfed by the relative time-period uniqueness of Sabbath observance and dissent.13

The model demonstrates how careful digital scholars must be when counting the “top terms” of any single period: slight adjustments in the beginning and end dates could illuminate some aspects of the archive but obscure others. More broadly, however, the anecdote suggests that even an archive as well plowed as that of Britain’s parliamentary debates can hide issues of deep importance that historians have accidentally ignored. The history of British debates about slavery is not, mercifully, one of them today, thanks to scholars who have pursued the history of slavery’s abolition through a variety of sources, archives, and representations, both microhistorical and macrohistorical, traditional and digital.14 But if algorithmic overviews easily miss a subject as consequential as slavery, what other events might still lie undiscovered beneath the threshold of regular scholarly attention?

To be sure, the granularity of time occupied by any of these subjects can be measured in different ways besides TF-IPF. The TF-IPF algorithm ranks a set of keywords according to differential correlation to a particular time period—how tightly correlated with any twenty-year window a particular term is. Other approaches to the identification of top “concerns” per might include a raw count of days on which each word appears in the text of passage as a named subject of debate, or an approximation through a raw count of words of how long named discussions of the subject took. Visualizing the same terms in different ways can elucidate the relationships among different metrics of time.

The Political Algorithm

To entertain an extended metaphor, one might think of word count (and most other forms of measurement of time) as a grid. One can look at the grid from far away, and the grid will appear in miniature. One can squint at the grid up close, and the grid will appear huge. But the grid is always a grid. In contrast, TF-IPF is like a Persian rug: Look at it from far away, and one pattern appears. Look at any part of it up close, and a new pattern appears—typically one entirely distinct from the pattern seen from far away, and often distinct from the pattern visible in any other part of the rug.

There is also a political tilt to this power of tacking from large-scale generalizations to smaller intervals. In the context of parliamentary debates, TF-IPF provides a particularly powerful political antidote to quantitative measures that rely on generalization—it has the ability, that is, to dramatize political hierarchies implicit in an archive. The political interests who could claim Parliament’s attention for a single day at a time were entirely different from those political interests who could regularly demand Parliament’s attention for a week. Importantly, in the context of Parliament, “differential attention”—or the measure of how much time a particular concern commanded—can serve as a proxy for a measurement of power exerted by a particular lobby in Parliament. TF-IPF thus can work as an aid to scholars who would read into their synthesis of the past an account of the exertions of power and the hierarchies on political exchanges thereby imposed. By employing TF-IPF, scholars may fold into their analysis of long- and short-term forces a political critique that reveals (institutional) time as a reflection of hegemony.

Can such a method therefore support a synthesis of the longue durée? Would such a synthesis have the capacity to teach us anything new, or to make use of the analytics of power, which, as we have seen, is one of the peculiar merits of TF-IPF applied to an institution whose publishing resources were limited? An outline of British history in the nineteenth century that relied on fig. 4 would differ significantly from the outline of any extant table of contents in British history; its chapters would review “seditious meetings,” “Orange lodges,” “matrimonial clauses,” “merchant shipping,” and “[agricultural] holdings.” Such a synthesis begins with repression—the gag orders around fears of Jacobinism in the early decades of the century. But it also continues into sustained inquiries into imperial racism (with debates about the Orange lodges in the era of the Catholic Relief Act of 1829), into gender (with the Matrimonial Causes Act of 1857), and into class (with the Agricultural Holdings Acts of 1875 and 1883 and the Crofters Holdings Act of 1886, which provided the first protections for working people against eviction). A synthesis supported by fig. 3 would offer us a portrait of an age of liberalism, persistently working from era to era to gradually extend the pleasures of privilege to the many, reversing the exclusions of the past. It endorses the long-term view of progress suggested by fin de siècle historians like the Élie Halévy and the Webbs.

Does that mean that algorithm-supported history relentlessly endorses a metanarrative about progress? Absolutely not. The algorithmic technique presented in this essay also supports the reality of an alternative history of the century.15Figures 3 and 4 support an alternative synthesis, focused on the pulse of short-term causes that failed to command sustained attention in Parliament: the repeal of the Corn Laws, the banning of intoxicating liquors, cheap grain and sugar, the Chartists’ gambit for democracy, and the crofters’ plea for land redistribution in Scotland. Some of the short-lived concerns were radical—for instance, Chartism and the crofters. Many others were the causes of labor—the distillers, bleachers, dyers, Chartists, crofters, and miners. The algorithm offers an index of when labor politics reached a national debate.16

The real payoff is what happens when the portrait of long-term forces from figure 4 is married to the diagnostic of short-term episodes from figures 3 and 4. A narrator working with both sets of data might explain how the choke hold on parliamentary attention by British elites made room for only one or two reforms meaningful per generation—enough perhaps to reflect the self-consciousness of a liberal elite, not enough to undo the pattern of racial, sexual, and class exclusions with which modern historians understand the era of British liberalism.

In systematically organizing the themes of parliamentary debate into long-term trends and short-term change, TF-IPF opens up to view where contemporary time is always experienced as layered. In contrast to the temporal layers described by Braudel—who conceived of the past as having geological, sociocultural, and individual times, each of which was objective and external and had components laid down in the past that delimited contemporary possibility—the layers of time revealed by algorithm are composed of slow-moving forces and quickly moving trends, each of which is at work (and capable, perhaps, of being interrupted) at every moment in human experience.

Some readers will be disappointed because the results of the method do not radically upend our portrait of the nineteenth century.17 Yet the quantitative survey suggests several open questions about lesser-known events in the history of empire—the controversy over “hypothec” in the 1860s, for instance, a lesser-known byway in the vast matrix of British social history and property law. Hypothec governed the experience of farmers in Scotland and Ireland, commanded the attention of lawyers, and commanded attention equal to the governance of usury or cotton weavers earlier in the century, but no sustained inquiry into the politics of hypothec has yet been published. Similarly, there are tempting questions about Parliament’s relationship to teachers in 1870–1910, to “refreshment houses” in the 1860s, and to “monasteries” in the 1850s—material whose parliamentary significance could be relatively quickly ascertained through the application of a variety of algorithms trained on semantic cohorts alongside close readings of the passages where they appear.18 A new synthesis and characterization of time is one facet of the algorithm’s contribution to a new history; mapping missing bricks of knowledge to be filled in by later historians is another.

Of course, text mining isn’t for everyone. The historian who has identified a particular event, actor, or debate for inspection will have no need of TF-IPF or similar algorithms for the inspection of temporality. However, for scholars who wish to describe the rich context of experience against which a particular event took place, or scholars working with the longue durée of decades or centuries, the ability to identify the patterns surrounding a particular event allows a richer sense of time and place. The results of such a process imply, for scholars, the ability to step outside the examination of one particular set of events and actors and to ask what sorts of rhythms, crises, and competing events contextualized a particular moment in time.

In summary, algorithms—especially the algorithms of distinction—represent a new method for organizing and prioritizing information about the past in such a way as to help scholars to act with sustained rigor and critical attention as they engage with cases from long time spans. By offering an opportunity to peer into historical experience at different scales, the quantitative findings here also supply a critically aware historian with the materials for syntheses that acknowledge the diversity of experience and perspectives.

Jo Guldi is currently full professor of the History of Britain and its Empire at Southern Methodist University, where she teaches the history of capitalism and text mining as a historical method in R and Python. Her second archivally-researched monograph, The Long Land War: The Global Struggle for Occupancy Rights, was published by Yale University Press in May 2022. She also recently closed a $1 million NSF grant of which she was PI. Her forthcoming book on the philosophy of history applied to statistics and machine learning, The Dangerous Art of Text Mining, is in process with Cambridge University Press.