Cracking a 1700s Code

Neat. [Link]

Now a team of Swedish and American linguists has applied statistics-based translation techniques to crack one of the most stubborn of codes: the Copiale Cipher, a hand-lettered 105-page manuscript that appears to date from the late 18th century. They described their work at a meeting of the Association for Computational Linguistics in Portland, Ore.

Discovered in an academic archive in the former East Germany, the elaborately bound volume of gold and green brocade paper holds 75,000 characters, a perplexing mix of mysterious symbols and Roman letters. The name comes from one of only two non-coded inscriptions in the document.

Kevin Knight, a computer scientist at the Information Sciences Institute at the University of Southern California, collaborated with Beata Megyesi and Christiane Schaefer of Uppsala University in Sweden to decipher the first 16 pages. They turn out to be a detailed description of a ritual from a secret society that apparently had a fascination with eye surgery and ophthalmology.

It began as a weekend project this year, Dr. Knight said in an interview, adding: “I don’t have much experience in cryptography. My background is primarily in computational linguistics and machine translation.”

Uncertain of the original language, the researchers went down several blind alleys before following their hunches. First, they assumed the Roman characters and not the abstract symbols contained all of the information.

But when that approach failed, they figured that the code was what cryptographers call a homophonic cipher — a substitution code that does not have a straightforward correspondence between the original and encoded information. And they decided the original language was probably German.

Eventually they concluded that the Roman letters were so-called nulls, meant to mislead the code breaker, and that the letters represented spaces between words made up of elaborate symbols. Another crucial discovery was that a colon indicated the doubling of the previous consonant.

The researchers used language-translation techniques like expected word frequency to guess what a symbol might equal in German.

“It turned out that we can apply a lot those techniques to code breaking,” Dr. Knight said.

The work is being praised by other experts. “Cracking the Copiale Cipher was a neat bit of work by Kevin Knight and his collaborators,” said Nick Pelling, a British software designer and a security specialist who maintains Cipher Mysteries, a cryptography news blog.

But while the cipher was a notable success, Dr. Knight and his colleagues have been frustrated by other, more impenetrable ciphers.

“There are these books and ancient languages of real historical value that contain historical information that we just can’t get out yet, and that’s of interest to a lot of people,” he said in a filmed interview describing the Copiale project.

The work has value to historians who are trying to understand the spread of political ideas. Secret societies were all the rage in the 18th century, Dr. Knight said, and they had an influence on both the American and French Revolutions. He recently shared the decoded Copiale text with Andreas Onnerfors, a historian at Lund University in Sweden and an expert on secret societies.

“When he saw the book and the decoded version, he was very excited about it,” Dr. Knight said. “He found a political commentary at the end that talked about the natural rights of man. That was pretty interesting and early.”