Omnivark

For a few years, I helped teach nonfiction writing to first-year college students. Long after, I wondered whether a computer could be taught to recognize great nonfiction writing.

The result was Omnivark, a six-month personal project in 2012. It produced a daily edition of recommended articles and essays from around the web—“a highlight reel for the written word.”

An example:

Much of the work to identify the recommendations was done by the computer, not me. I wrote software to scan hundreds of new essays and articles per day from various sources across the web. Whereas normal text analysis looks for keywords to indicate what the content is about, Omnivark analyzed the text’s structural features for indicators of high-quality writing. This was well before ChatGPT or large language models, so there was no AI like we have today.

Over time, and a lot of experimentation, Omnivark became smart enough to reduce hundreds of candidate articles per day down to about ten that I would review. I’d then choose three for Omnivark’s daily edition. The software would try to recommend a quote for each article but usually missed the mark because the quote needed to not just be something well-said but also signal what the article was about. So I mostly chose the quotes.

Omnivark ran for six months, with more than one hundred editions. By the end, I felt like I had sufficiently answered the original question. Yes, circa 2012, a computer could be taught to recognize great writing to a meaningful extent. It did not “understand” the writing except as a set of quality scores, and it could not make the final editorial choices. But as a first reader, the software could evaluate a large volume of writing every day, turning the human work of creating daily editions into something pleasantly manageable.

I blogged about the project at the time:

• Why Omnivark? summarized my motivations and the technology.

• Following the Elites discussed the challenge of who/what to follow to find great writing.

• Stylistic Signals explained how Omnivark determined the great in great writing.

• Omnivark’s Division of Labor explored the roles of human and computer during the project.

The Omnivark website is long gone, but to those who visited back then, and to those who received the daily emails, thank you for reading. I hope you enjoyed it.

For me, Omnivark was a small and satisfying project, exploring how far software could get and where human judgment should have the final word.