It's not very hard to get individual texts in digital form. But working with grad students in the humanities looking for large sets of texts to do analysis across, I find that larger corpora are so hodgepodge as to be almost completely unusable. For humanists and ordinary people to work with large textual collections, they need to be distributed in ways that are actually accessible, not just open access. [Read complete post...]
Day of DH, 2022 Mar 27 2022
A Rose for Ruby Feb 27 2022
Ruby Logo There are programming languages that people use for money, and programming languages people use for love. There are Weekend at Bernie's/Jeremy Bentham corpses that you prop up for the cash, and there are "Rose for Emily" corpses you sleep with every night for decades because it's too painful to admit that the best version of your life you ever glimpsed is not going to happen. [Read complete post...]
I've been spending more time in the last year exploring modern web stacks, and have started evangelizing for Svelte-Kit, which is a new-ish entry into the often-mystifying world of web frameworks. As of today, I've migrated this, personal web site from Hugo, which I've been using the last couple years, to svelte-kit. Let me know if you encounter any broken links, unexpected behavior, accessibility issues, etc. I figured here I'd give a brief explanation of why svelte-kit, and how I did a Hugo-Svelte kit migration. [Read complete post...]
Scott Enderle is one of the rare people whose Twitter pages I frequently visit, apropos of nothing, just to read in reverse. A few months ago, I realized he had at some point changed his profile to include the two words “increasingly stealthy.” He had told me he had cancer months earlier, warning that he might occasionally drop out of communication on a project we were working on. I didn’t then parse out all the other details of the page—that he had replaced his Twitter mugshot with a photo of a tree reaching to the sky, that the last retweet was my friend Johanna introducing a journal issue about “interpretive difficulty”—the problems literary scholars, for all their struggles to make sense, simply can’t solve. I only knew—and immediately stuffed down the knowledge—that things must have gotten worse. [Read complete post...]
This article in the New Yorker about the end of genre prompts me to share a theory I've had for a year or so that models at Spotify, Netflix, etc, are most likely not just removing artificial silos that old media companies imposed on us, but actively destroying genre without much pushback. I'm curious what you think. [Read complete post...]
I've been yammering online about the distinctions between different entities in the landscape of digital publishing and access, especially for digital scholarship on text. So I've collected everything I've learned over the last 10 years into one, handy-to-use, chart on a 10-year-old meme. The big points here are: [Read complete post...]
I mentioned earlier that I've been doing some work on the old Bookworm project as I see that there's nothing else that occupies quite the same spot in the world of public- facing, nonconsumptive text tools. [Read complete post...]
Bookworm Caching Mar 07 2021
I used to blog everything that I did about a project like Bookworm, but have got out of the habit. There are some useful changes coming through through the pipeline, so I thought I'd try to keep track of them, partly to update on some of the more widely used installations and partly [Read complete post...]
I last looked at the H-Net job numbers about a month ago. [Read complete post...]
History Jobs Update Oct 01 2020
Out of a train-wreck curiosity about what's been happening to the historical profession, I've been watching the numbers on tenure-track hiring as posted on H-Net, one of the major venues for listing history jobs. [Read complete post...]
Circle Packing Sep 01 2020
I've been doing a lot of my data exploration lately on Observable Notebooks, which is--sort of--a Javascript version of Jupyter notebooks that automatically runs all the code inline. Married with Vega-Lite or D3, it provides a way to make data exploration editable and shareable in a way that R and python data code simply can't be; and since it's all HTML, you can do more interesting things. [Read complete post...]
Every year, I run the numbers to see how college degrees are changing. The Department of Education released this summer the figures for 2019; these and next year's are probably the least important that we'll ever see, since they capture the weird period as the 2008 recession's shakeout was wrapping up but before COVID-19 upended everything once again. But for completism, it's worth seeing how things changed. [Read complete post...]
As part of the Asylum Lab project at NYU, I've started poking around the database of the Executive office of immigration review (EOIR) published by the department of justice every month. It's a monster of a database; millions of rows detailing each person, charge, and motion in the enormous parallel federal court system that has been set up to track applicants for asylum, proposed deportees, and so on. [Read complete post...]
While I was choosing graduate programs back in 2005, I decided to come up with my own ranking system. I had been reading about the Google PageRank algorithm, which essentially imagines the web as a bunch of random browsing sessions that rank pages based on the likelihood that you--after clicking around at random for a few years--will end up on any given page. It occurred to me that you could model graduate school rankings the same way. It's essentially a four-step process: [Read complete post...]
As I often do, I'm going to pull away from various forms of Internet reading/engagement through Lent. This year, this brings to mind one of my favorite stray observations about digital libraries that I've never posted anywhere. [Read complete post...]
(This is a talk from a January 2019 panel at the annual meeting of the American Historical Association. You probably need to know, to read it, that the MLA conference was simultaneously taking place about 20 blocks north.) [Read complete post...]
Web Migration Jun 30 2019
Since 2010, I've done most of my web hosting the way that the Internet was built to facilitate: from a computer under the desk in my office. This worked extremely well for me, and made it possible to rapidly prototype a lot of of websites serving large amounts of data which could then stay up indefinitely; I have a curmudgeonly resistance to cloud servers, although I have used them a bit in the last few years (mostly for course websites where I wanted to keep student information separate from the big stew.) [Read complete post...]
Some news: in September, I'll be starting a new job as Director of Digital Humanities at NYU. There's a wide variety of exciting work going on across the Faculty of Arts and Sciences, which is where my work will be based; and the university as a whole has an amazing array of programs that might be called "Digital Humanities" at another university, as well as an exciting new center for Data Science. I'll be helping the humanities better use all the advantages offered in this landscape. I'll also be teaching as a clinical associate professor in the history department. [Read complete post...]
Critical Inquiry has posted an article by Nan Da offering a critique of some subset of digital humanities that she calls "Computational Literary Studies," or CLS. The premise of the article is to demonstrate the poverty of the field by showing that the new structure of CLS is easily dismantled by the master's own tools. It appears to have succeeded enough at gaining attention that it clearly does some kind of work far outsize to the merits of the article itself. [Read complete post...]
I wrote this year's report on history majors for the American Historical Association's magazine, Perspectives on History; it takes a medium term view of at the significant hit the history major has taken since the 2008 financial crisis. You can read it here. [Read complete post...]
As part of the Creating Data project, I've been doing a lot of work lately with interactive scatterplots. The most interesting of them is this one about the full Hathi collection. But I've posted a few more I want to link to from here: [Read complete post...]
I have a new article on dimensionality reduction on massive digital libraries this month. Because it's a technique with applications beyond the specific tasks outlined there, I want to link to a few things here. [Read complete post...]
New Site Oct 21 2018
I'm switching this site over from Wordpress to Hugo, which makes it easier for me to maintain. [Read complete post...]
I have a new article in the Atlantic about declining numbers for humanities majors. [Read complete post...]
I put up a new post at Sapping Attention about how bad the decline in humanities majors has been since 2013. In short, it's been bad enough to make me recant earlier statements of mine about the long-term health of the humanities discipline. [Read complete post...]
This is some real inside baseball; I think only two or three people will be interested in this post. But I’m hoping to get one of them to act out or criticize a quick idea. This started as a comment on Scott Enderle’s blog, but then I realized that Andrew Goldstone doesn’t have comments for the parts pertaining to him… Anyway. [Read complete post...]
I’ve gotten a couple e-mails this week from people asking advice about what sort of computers they should buy for digital humanities research. That makes me think there aren’t enough resources online for this, so I’m posting my general advice here. (For some solid other perspectives, see here). For keyword optimization I’m calling this post “digital humanities.” But, obviously, I really mean the subset that is humanities computing, what I tend to call humanities data analysis. [Edit: To be clear, ] Moreover, the guidelines here are specifically tailored for text analysis; if you are working with images, you’ll have somewhat different needs (in particular, you may need a better graphics card). If you do GIS, god help you. I don’t do any serious social network analysis, but I think the guidelines below should work relatively with Gephi. [Read complete post...]
Practically everyone in Digital Humanities has been posting increasingly epistemological reflections on Matt Jockers’ Syuzhet package since Annie Swafford posted a set of critiques of its assumptions. I’ve been drafting and redrafting one myself. One of the major reasons I haven’t is that the obligatory list of links keeps growing. Suffice it to say that this here is not a broad methodological disputation, but rather a single idea crystallized after reading Scott Enderle on “sine waves of sentiment.” I’ll say what this all means for the epistemology of the Digital Humanities in a different post, to the extent that that’s helpful. [Read complete post...]
Rate My Professor Feb 06 2015
Just some quick FAQs on my professor evaluations visualization: adding new ones to the front, so start with 1 if you want the important ones. [Read complete post...]
I promised Matt Jockers I’d put together a slightly longer explanation of the weird constraints I’ve imposed on myself for topic models in the Bookworm system, like those I used to look at the breakdown of typical TV show episode structures. So here they are. [Read complete post...]
I’ve been thinking a little more about how to work with the topic modeling extension I recently built for bookworm. (I’m curious if any of those running installations want to try it on their own corpus.) With the movie corpus, it is most interesting split across genre; but there are definite temporal dimensions as well. As I’ve said before, I have issues with the widespread practice of just plotting trends over time; and indeed, for the movie model I ran, nothing particularly interesting pops out. (I invite you, of course, to tell me how it is interesting.) [Read complete post...]
I’ve been seeing how deeply we could integrate topic models into the underlying Bookworm architecture a bit lately. [Read complete post...]
This is a post about several different things, but maybe it’s got something for everyone. It starts with 1) some thoughts on why we want comparisons between seasons of the Simpsons, hits on 2) some previews of some yet-more-interesting Bookworm browsers out there, then 3) digs into some meaty comparisons about what changes about the Simpsons over time, before finally 4) talking about the internal story structure of the Simpsons and what these tools can tell us about narrative formalism, and maybe why I’d care. [Read complete post...]
Like many technically inclined historians (for instance, Caleb McDaniel, Jason Heppler, and Lincoln Mullen) I find that I’ve increasingly been using the plain-text format Markdown for almost all of my writing. [Read complete post...]
I thought it would be worth documenting the difficulty (or lack of) in building a Bookworm on a small corpus: I’ve been reading too much lately about the Simpsons thanks to the FX marathon, so figured I’d spend a couple hours making it possible to check for changing language in the longest running TV show of all time. [Read complete post...]
Here’s a very technical, but kind of fun, problem: what’s the optimal order for a list of geographical elements, like the states of the USA? [Read complete post...]
String distance measurements are useful for cleaning up the sort of messy data from multiple sources. [Read complete post...]