SELECT
year,classification,
IFNULL(numerator.WordCount,0)*100000000/IFNULL(denominator.WordCount,0)/100 as WordsPerMillion
FROM
(
SELECT
lc1,year,
sum(main.count) as WordCount
FROM
fastcat
JOIN
master_bookcounts as main
ON (fastcat.bookid=main.bookid)
JOIN ( wordsheap as words1) ON (main.wordid = words1.wordid)
WHERE
(( ( (year<=1922) ) ) AND ( ( (year>=1850) ) ))
AND (( ( (words1.casesens = (SELECT casesens FROM wordsheap WHERE
casesens='library')) ) ) OR
( ( (words1.casesens = (SELECT casesens FROM wordsheap WHERE casesens='libraries')) ) ))
GROUP BY
lc1,year
) as numerator
RIGHT OUTER JOIN
(
SELECT
lc1,year,
sum(nwords) as WordCount
FROM
fastcat
WHERE
(( ( (year<=1922) ) ) AND ( ( (year>=1850) ) )) AND TRUE
GROUP BY
lc1,year
) as denominator
USING (lc1,year )
JOIN LCC USING (lc1)
GROUP BY lc1,year;
Rate of mentions of Topeka (each dot is one newspaper)
States and Regions are both as important as distance
Does race change imagined geographies?
Does race change imagined geographies?
Digital as play
"The Hermeneutics of Screwing Around" (Ramsay)
Play as Discovery
Digital reconfigurations for insight
Tools to democratize algorithmic exploration
Voyant tools
Google Ngrams
http://books.google.com/ngrams
Google Ngrams
http://books.google.com/ngrams
Distant Reading
Topic Modeling
Network Analysis
Geospatial visualization/Named Entity Recognition
Relationships among novels
Matthew Jockers and Elijah Meeks
Texts without Authors
Whatever vision of the digital humanities is proclaimed, it will have little place for the likes of me and for the kind of criticism I
practice:
a criticism that narrows meaning to the significances designed by an author, a criticism that generalizes from a text as small as half a line, a criticism that insists on the distinction between the true and the false, between what is relevant and what is noise, between what is serious and what is mere play.
Stanley Fish
What writers don't mean to say:
or, 'Tis 60 (± 10) Years Since
Wallbrink, H. and F.B. Koek, Data Acquisition And Keypunching
Codes For Marine Meteorological Observations At The Royal Netherlands
Meteorological Institute, 1854–1968
Logbooks as punchcards
Wallbrink, H. and F.B. Koek, Data Acquisition And Keypunching
Codes For Marine Meteorological Observations At The Royal Netherlands
Meteorological Institute, 1854–1968
Undigitized Elements
New Bedford Whaling Museum
Undigitized Elements
New Bedford Whaling Museum
Undigitized Elements
New Bedford Whaling Museum
Whaling Vessel Crews
New Bedford Whaling Museum
Physical Descriptions of Whaling Crewmembers
New Bedford Whaling Museum
German Merchant Marine Voyages
Whaling Voyages
Whaling Voyages
Ocean Navigation, 1750-1850
Ocean Navigation, 1750-1850
Climatogical Metadata is Historical Data
Shipping Routes
Textual Metadata: Newspaper Locations
Bill Lane Center for the American West: http://www.stanford.edu/group/ruralwest/cgi-bin/drupal/visualizations/us_newspapers
"We begin to feel, Monsieur L'Abbe," answered the vicar, with some
asperity, "that a Continental war entered into for the defence of an ally
who was unwilling to defend himself, and for the restoration of a royal
family, nobility, and priesthood who tamely abandoned their own rights,
is a burden too much even for the resources of this country."
Walter Scott, Waverley
Anachronisms Reflect What Change is Considered "Historical"
Mad Men with Computers.
What to do with millions of texts?
Nothing
What to do with millions of texts?
Keyword Search
What to do with millions of texts?
Science
Harvard Cultural Observatory
What to do with millions of texts?
Hire Programmers
What to do with millions of texts?
Focused Reading
Big Data as just another source
Digital sources contribute knowledge beyond individuals.
Metadata lets us look at social structures.
Once we know how to read it, we work around its biases like any other source.
3 Arenas for digital history
Working with digital/digitized sources.
Inventing and applying algorithmic methods.
Publishing and Preserving electronically.
Tools for working with text
Ngrams/Search engines
Voyant Tools (voyant-tools.org)
MALLET (topic modeling and machine learning)
Python, R, or maybe Java (real analysis)
Your computer's command line.
Historical Geography
ArcGIS.
QGIS.
Web-based geographic visualization.
Leaflet
D3.js
MapBox
Working with Statistics
Excel, databases.
The R Language (use the "Hadleyverse")
Publishing and Preserving electronically.
--Ted Underwood, University of Illinois
What to do with a new source
Source Criticism
What to do with a new source
Source Criticism
What to do with a new source
Source Criticism
02138
02138
Google Ngrams
http://books.google.com/ngrams
Google Ngrams
http://books.google.com/ngrams
Google Ngrams
http://books.google.com/ngrams
What to do with a new source
Source Criticism
What to do with a new source
Source Criticism
Hermeneutics
Argument
What to do with a new source
Source Criticism
Hermeneutics
Argument
What do we need to describe large data sets?
Basic Technical Skills
Understanding of biases (Source Criticism)
Way of reading (Hermeneutics)
Argument
"Big data" needs humanists
Reading Digital Sources
Technical Competence
Understanding of biases (Source Criticism)
Technique for reading (Hermeneutics)
Argument
Reading Digital Sources
Technical Competence
Understanding of biases (Source Criticism)
Reading Digital Sources
Technical Competence
Understanding of biases (Source Criticism)
Technique for reading (Hermeneutics)
Argument
Textual Metadata: Trial lengths
Data Mining with Criminal Intent/The Old Bailey Online
Textual Metadata: Correspondence Networks
Mapping the Republic of Letters
How should historians approach digitization
1. Improve our digital copies
2. Treat them as new sources altogether, and see what they're good for.
Grounding Words in Texts
Bookworm: Exploring Texts through Metadata
(http://bookworm.culturomics.org)
c. 1 million books, 80 billion words
Library metadata via Open Library
Digital Public Library of America funding
Team: Harvard Cultural Observatory, Rice Cultural Observatory,
Northeastern University Martin Camacho * Neva Cherniavsky * Erez Lieberman-Aiden * JB
Michel * Billy Janitsch
Comparing Custom Corpora
Bookworm: Exploring Texts with Metadata
Bookworm Arxiv
600,000 math and physics articles from the last 20 years
arxiv.culturomics.org
Bookworm JStor
600,000 math and physics articles from the last 20 years
Everyone knows what attention is. It is the taking possession by the mind, in clear and vivid form, of one out of what seem several simultaneously possible objects or trains of thought.
William James, Principles of Psychology
Focusing Attention: a psychological metaphor
Everyone knows what attention is. It is the taking possession by
the mind, in clear and vivid form, of one out of what seem several
simultaneously possible objects or trains of thought.
Focalization, concentration, of consciousness are of its essence.
William James, Principles of Psychology
Mind as Camera
Those who lay stress on the unity of mind regard it as almost evident a priori, that but one concept can occupy the focus of attention at a time... Attention, like the lens of the eye, is now [ie, first] accommodated to act as an instrument of near focus, high magnification, but limited aperture, and again [then] as one of distant focus, small magnifying-power, but wide range
Can the Mind attend to two things at once? Science: July 18, 1887
Epilogue: The Signification of the Frontier in American History
Shifts in language happen across different temporal dimensions at once
Cohort effects and temporal effects are evenly split
The 'first' Digital Humanists
In His mercy, around 1955, God led men to invent magnetic tapes.
Robert Busa in Perspective on the Digital Humanities: Schreibman et al, 2004.
Textual Research as Positivism
Mosteller and Wallace: Authorship attribution in the federalist papers
'Stylometrics' and 'Cliometrics'
Automated Classifications
Digital as play
"The Hermeneutics of Screwing Around" (Ramsay)
Play as Discovery
Digital reconfigurations for insight
Tools to democratize algorithmic exploration
Big Data
Funding
Media Attention
Culturomics and Google Ngrams
Distant Reading
Topic Modeling
Network Analysis
Geospatial visualization/Named Entity Recognition
Relationships among novels
Matthew Jockers and Elijah Meeks
Distant Reading
Topic Modeling
Network Analysis
Geospatial visualization/Named Entity Recognition
Relationships among novels
Matthew Jockers and Elijah Meeks
Texts without Authors
Whatever vision of the digital humanities is proclaimed, it will have little place for the likes of me and for the kind of criticism I
practice:
a criticism that narrows meaning to the significances designed by an author, a criticism that generalizes from a text as small as half a line, a criticism that insists on the distinction between the true and the false, between what is relevant and what is noise, between what is serious and what is mere play.
Stanley Fish
Working with Metatadata
Metadata shows hidden constraints: paths of ships.
Mad Men with Computers.
