Humanities Data Visualization

Signal problems in humanities data visualization:

The specific is more interesting than the general
The data shows its irregularities, if you ask it nicely.

![The data shows it's irregularities, if you ask it nicely.](/slides/images/ghostmaps/701.png)

Deck 701: American Shipping, 1800-1850

Deck 892, US shipping 1980-1997

Decks of punchcards

0.0000001% of the ICOADS data set.

1848 6 1     3723 29038 02 4    10ISABE*_N   1   5                                                           
165 20779701 69 5 0 1                  FFFFFF77AAAAAAAAAAAA     99 0 790044118480601
3714N 6937W                                                                           NW     51 NW     57 NW   
51                                          201A.STEWART       NEW BEDFORD             WHALING V
OYAGE           2620 199

Digitized logbooks, c. 1930

Wallbrink and Koek

Logbook Digitization in the 1920s

Wallbrink, H. and F.B. Koek, Data Acquisition And Keypunching Codes For Marine Meteorological Observations At The Royal Netherlands Meteorological Institute, 1854–1968

Deck 720:German weather data

Deck 735:Soviet weather data

Soviet Closeup

Bookworm: Visualizing Libraries

Visualizing Libraries

Not visualizing texts.

Acknowledgements

Institutions

Northeastern University/Rice University Cultural Observatory

People

Erez Lieberman Aiden
Neva Cherniavsky, Martin Camacho, Matt Nicklay, Billy Janitsch, JB Michel, Piotr Organisciak.

Acknowledgements

Funders

Digital Public Library of America
Harvard Cultural Observatory
National Endowment for the Humanities
Mellon Foundation

Partners

Hathi Trust Research Center

github.com/Bookworm-project
bookworm.htrc.org
bookworm.culturomics.org

02138

Some Bookworm instances at Northeastern and Rice:

Science articles:
bookworm.culturomics.org/arxiv
Movies and television:
movies.benschmidt.org
Historical Newspapers: benschmidt.org/ChronAm

Yale University: full run of Vogue Magazine

http://bookworm.library.yale.edu/collections/vogue/

Medical Heritage Library (30,000 medical books)

US State Department

http://bookworm.htrc.illinois.edu/bookworm/

DSL for large text databases

A query language modeled on MongoDB
A visualization grammar modeled on ggplot

Grammar, part I: Creating a corpus

"search_limits": {
    "publish_country":["United States"],
   "year": {
      "$lte": 1920,
      "$gte": 1890
   },

Grammar, Part II: Defining texts and tokens

{
"plotType": "heatmap",
"search_limits": {
   "word": ["concentrate attention"],"year":{"$lte":1922,"$gte":1850}
},
"aesthetic":{"color":"WordsPerMillion","x":"decade","y":"BenSubject"},
"database":"presidio"
}

View

Grammar, Part II: Defining texts and tokens

Time interactions.

{
"database": "ChronAm",
"plotType": "heatmap",
"method": "return_json",
"search_limits": {
   "publish_year": {
      "$gte": 1860
  },
  "word": ["bicycle"]
},
"aesthetic": {
    "x": "publish_year",
    "y": "publish_day_year",
    "color": "WordsPerMillion"
}
}

View

Library composition

Language of the State of the Union

Comparing the Vocabulary of two TV shows

{
"database": "movies",
"plotType": "worddiv",
"search_limits": {
  "TV_show": ["Seinfeld"]
},
"compare_limits": {
  "TV_show": ["Cheers"]
},
"aesthetic": {
  "label": "unigram",
  "size": "Dunning"
}}

View

Comparing speeches by language

Reading texts in context

Incorporating

{"database": "ChronAm",
"plotType": "map",
"projection":"USA",
"method": "return_json",
"search_limits": {},
"aesthetic": {
  "point": "placeOfPublication_geo",
  "size": "TextCount"
}}

View

{"database": "ChronAm",
"plotType": "map",
"projection":"USA",
"method": "return_json",
"search_limits": {},
"aesthetic": {
  "point": "placeOfPublication_geo",
  "size": "TextCount",
  "time": "publish_year"}}

View

Newspaper flu coverage, 1917-1919

{"database": "ChronAm",
"plotType": "map",
"projection":"USA",
"method": "return_json",
"search_limits": {"word":["flu","influenza","pneumonia"],
           "publish_year":{"$lte":1920,"$gte":1918}},
"smoothingSpan":25,
"aesthetic": {
    "time":"publish_day",
  "point": "placeOfPublication_geo",
  "size": "TextPercent"}}

View

Mapping Places in the State of the Union

Data:

~14 million teaching evaluations from ~2003 to 2014

Scraped (slowly) from ratemyprofessors.com

Processed and tokenized using Bookworm

benschmidt.org/profGender

Same interaction easily adaptable to other collections

Biased Language, Biased Data

Usage

"Such a nasty woman"

Usage: "consciousness raising"

Testing

A-B Testing

Web users randomly assigned to one of two groups; stored in locally in browser cache, persistent through multiple visits.
Page renders differently based on group

Adding related words

Subtracting Transitions

Confirmation bias

Top search terms on benschmidt.org/profGender

Users are (slightly) more likely to exit after looking at a chart that doesn't show bias

Narratives

Census Data

link

Boats

Deck 720:German weather data

Deck 735:Soviet weather data

Soviet Closeup

Shipping

The expansion of whaling

An elaborate API call to the Maury browser.

    [
    {
            "DCK":[701],
            "start":365.25*1840,
            "end":365.25*1940,
            "non_standard_fields":["origin","destination"],
            "color":function(d) {
                    if (d.origin.match("BOSTON|NEW BEDFORD|SALEM") !== null) {return true}
                    if (d.destination.match("BOSTON|NEW BEDFORD|SALEM") !== null) { return true}
                    return false;
            },
            "continuation":true,
            horizon:25,
            "projection":"oceanicHomolosine"
    },
    [...]

Minard

Minard reproduction