The epistemology of document examination

The debate over the authenticity of the Killian memos has been very disappointing.

A lot of bloggers have been crowing about their ability to create documents similar to the Killian memos in Word. As Brian Weatherson notes, they are fallaciously affirming the consequent. Of course word processors can imitate typewritten documents. That's what they were designed to do! This is especially true if you already know what the Killian memos look like and set about duplicating them. The question is whether there is anything about the documents that's rare or impossible in typed documents but ubiquitous in word processed documents. So far, I haven't seen any evidence of that. The discredited typography allegations were intended to show that the documents couldn't have been produced by a typewriter. If the critics had found traits of the memos that would have been unlikely or impossible from a a typewriter but nearly universal from a word processor, that would have suggested that a word processor was responsible. In fact, without the typography claims, all the critics are doing with their homemade Word demos is pointing out that a similar but not identical document can be produced in Word. So far, no one has been able to produce a document in Word that is truly identical at the level that would convince a questioned document examiner. All that has been shown so far is that Word can do many of the same things as typewriters.

Matt Yglesias agrees that the right wing blogosphere has been peddling some specious arguments which unaccountably convinced real journalists to investigate further. Matt thinks that the latest Washington Post story comes closer to debunking the memos.

I disagree. Notice how this bullshit comes in waves? First typography. Then sourcing. Now military jargon. Just as the typography allegations get refuted, people start harping on the fact that the memos came from private files. Now we're on to the "doesn't match military jargon" narrative. Luckily for the right wing few of us are in a position to evaluate the jargon of the TANG in the 1970s. (Though arguably almost any sentient being is better position to know how people talked in the TANG in the 1970s than W.) Independent experts on the TANG are fewer still. Most people in a position to know are either gruntled or disgruntled TANG members. If the memos are real, the implication is that the Guard was disgracefully if not criminally corrupt. Though hardly conclusive, we should keep this fact in mind when weighing the WaPo's claims to have cross checked the Killian memos with documents recently released by the Guard itself. If the Guard released these records after the appearance of the Killian memos became public knowledge, we have to wonder whether they released a random, representative sample of the documents available.

Unaccountably, the supposedly compelling Washington Post article leads with the weakest point of their story, namely, that handwriting expert Marcel Matley can't vouch for the authenticity of the documents. As Matley explains, CBS asked to him to examine documents that were many photocopies removed from the originals. Therefore, he was only able to authenticate Killian's signatures, not the memos themselves. The Post article disingenuously ignores the fact Matley was only one of three forensic document examiners who vetted the memos on behalf of CBS. CBS hasn't released the names of the other two examiners or explicitly relayed their conclusions.

Matley didn't say that would have been impossible for anyone to authenticate documents like these. He just said that it was beyond his expertise to do so. We don't know where the expertise of the other two examiners lies or how strongly they endorsed the memos. Handwriting and typography are two sub-specialties within questioned document examination. No doubt, there are also people who specialize in projecting what original documents would have looked like based on assumptions about photocopying technology. Still others focus on historical, semantic, and stylistic details. We shouldn't necessarily assume that any one expert ought to be in a position to conclusively authenticate a document, even if that document is authentic.

What nobody really makes clear is the overall logic of questioned document examination. Generally speaking, documents are authenticated by comparing them to known examples of their type. Killian's signatures were authenticated by comparing them to known samples of Killian's handwriting. Matley was able to say that they matched the prototypes enough and in the right ways. He was also able to affirm that they didn't match each other or the prototype too much--it would have been a dead giveaway to have multiple identical tokens of the same signature type. Part of the expertise of a questioned document examiner lies in knowing what tends to vary and what tends to stay the same.

A questioned document examiner may also able to recognize characteristic ways in which evidence degrades. The Washington Post uncritically repeats the proportional font canard. In fact, the documents have been copied so many times that the type looks kerned because of cumulative distortions.

The Post story also cites Adobe type experts who claim that the width of the Killian font is closer to modern word processor editions of the same font than to what an IBM typewriter would have produced. The Post doesn't explain whether the type experts are also qualified to factor in the distortion induced by photocopying. One presumes not.

The Post also claims that there are stylistic and content inconsistencies in the memos. These questions can only be answered by comparison to known examples of the relevant sort. The TANG released some documents from the era, but the Post doesn't elaborate on what kind of documents they were or who wrote them. The Post story also claims that the memos are inconsistent with the historical record. For example, one of the people mentioned by Killian had already quit the Guard by the time the memo alleges that he was pressuring Killian. The memos also mention an address for W. that wasn't accurate at the time the memos were allegedly written. These are the sort of thing that, if substantiated, would raise doubts about the authenticity of the memos. At this point we can't assume that they accurate, given the sad history of factual allegations in this case (e.g., "There was no such thing as Times New Roman in 1972.") On the other hand, it's quite possible that a former TANG leader was pressuring Killian and that the Guard was still sending mail to Bush's parents' address. If I had a nickel for every institution that still sends mail to my old addresses, I'd be able to buy a PR firm and my own staff of experts.

A questioned document examiner should also be able to pronounce on the expected variability in a particular type of sample. For example, signatures always vary slightly on each instance. Some people have more consistent signatures than others. In order to differentiate between incriminating inconsistencies and natural variability it is important to compare documents within the relevant class. The mere fact that Killian's memos differ from some norms for Guard documents doesn't necessarily mean that the memos are forged. If someone compared my fridge-door shopping list with a written invitation in my best handwriting, they would observe all kinds of inconsistencies between the two. The disputed memos are Killian's "notes to self"--it would be interesting to compare the Bush memos to known informal memos by Killian.

Authentication is not some magic epistemological seal of approval. It's a judgment based on a accumulated balance of probabilities. Sometimes an expert has to conclude that the evidence is ambiguous. This appears to have been the case with Matley. He concluded that it was impossible to tell whether these were real memos with Killian's signature or faked memos with genuine signatures tacked on. If the physical evidence is ambiguous, it becomes necessary to turn to other sources. If the questioned document examiners were able to affirm that the documents could be real, the evidential burden shifts to the reporters. They must ask questions about the quality of the source, the plausibility of the provenance, the consistency of the content with our other well-founded beliefs about Bush's record, and so on.

Some commentators have complained that it's implausible that the memos came from Killian's private files. Mark Kleiman argues that the CBS story was "at best weakly sourced." We know that the CBS story was anonymously sourced. It may also have been weakly sourced, but we can't know unless CBS reveals its sources. CBS's reticence is an epistemological obstacle. How can we decide how good their evidence is if they won't share it with us? On the other hand, CBS and 60 Minutes are generally reliable sources. I have more confidence in their journalistic competence and integrity than I do in the ravings of the right-wing blogosphere, or the verdicts of self-styled outside experts assembled to pronounce authoritatively on the basis of evidence too flimsy to allow the CBS expert to positively identify the memos. The outside experts are even worse off than the inside experts because they relied on .pdfs of photocopies which introduced even more degradation and distortion.

Whether the "private document" story is plausible depends on details we don't know. Our view on the matter should depend on who allegedly produced these documents and what access they would have had to Killian. If I claimed that I had private documents from the secret files of Bush's former squadron commander, I'd be mocked. But that's because there's no plausible reason to think that I'd have access to that kind of information. However, if CBS knows that documents came from a former Killian employee or a close friend his, it would be more plausible that s/he might have acquired these private documents.

Addendum: Thanks to SoCal Justice for this link to an ABC story: allegedly, CBS's two other document examiners are claiming that they didn't vouch for the memos either. One claims that she sent an email to CBS urging them not to go ahead with the story because she had doubts about the memos' authenticity. I'd like to see that email. I'd have to question the integrity of the CBS team behind the story if I had evidence that she and her colleague explicitly told CBS that the documents were fake or even that they weren't credible enough to base a story on.

I do have to wonder why these experts didn't come forward earlier, though. It's one thing to say that you had doubts all along, it's another to prove it, and to prove that you expressed them forcefully when it counted. Remember how Hodges changed his story?

Comments

[SoCalJustice]

Hi there, I saw a trackback from MY's site, and wanted to see what you had to say.

I think you should take a look at this very recent article from Brian Ross of ABC News:

"Casting Further Doubt; Document Analysts: CBS News Ignored Concerns About Disputed Bush Military Records"
http://abcnews.go.com/sections/WNT/Investigation/bush_guard_documents_040914-1.html

Do you still have "confidence in their journalistic competence and integrity"?

Obviously, only very few people at CBS are involved in this story. And there must be hundreds of diligent and competent CBS staffers who are pissed that a few people from 60 Minutes and Dan Rather have dragged everybody down with them.

[SoCalJustice]

I just want to quote a few key graphs from that article, for your readers who might not view the link (although everyone should read the whole story):

"Emily Will, a veteran document examiner from North Carolina, told ABC News she saw problems right away with the one document CBS hired her to check the weekend before the broadcast.

"I found five significant differences in the questioned handwriting, and I found problems with the printing itself as to whether it could have been produced by a typewriter," she said.

Will says she sent the CBS producer an e-mail message about her concerns and strongly urged the network the night before the broadcast not to use the documents.

"I told them that all the questions I was asking them on Tuesday night, they were going to be asked by hundreds of other document examiners on Thursday if they ran that story," Will said."

[The Commissar]

"If the critics had found traits of the memos that would have been unlikely or impossible from a typewriter but nearly universal from a word processor, that would have suggested that a word processor was responsible."

That is exactly what computer typography expert "no fan of George Bush" Joseph Newcomer says.

http://www.flounder.com/bush.htm

[john brothers]

I wrote up a rather lengthy response to your article here:(
http://www.undefined.com/ia/archives/2004/09/laugh_cry_howl.html)

Unfortunatley my trackback pinging isn't working well right now.

[John]

Just to note, re: the ABC story, that the implication from CBS is that there were more experts involved than just these two and Matley. At least, that was how I read it...

[Kriston]

One detail from the Post story that mystified me: How is Matley to have done any efficient handwriting analysis from a photocopy of the document? Surely even if Matley were privy to the identity of CBS's anonymous source he would acknowledge that the potential for tampering inherit to a photocopied signature negates the art of handwriting analysis.

[Brandon]

Hi, Lindsay,

I'm finding your posts on the epistemology issue to be great stuff! While I'm a bit skeptical about much of the furor, too, Weatherson's fallacy charge seems simply irrelevant; arguments in matters of forgery and fraud are never (as far as I can see) monotonic, but nonmonotonic - i.e., they aren't deductive but evidential, and therefore can't be accused of deductive fallacies. (On this point they are exactly parallel to claims of authenticity, which you rightly describe below.)

But as far as I can see, the critics of the memos are (still) claiming to have identified characteristics unlikely or impossible for a typewriter. It's worth pointing out that word processing software was not designed to mimic typewritten documents, but typeset documents, and the two are not the same (this is why, for instance, it's correct to doublespace after a period on a typewriter and not on a computer: computers space after periods like typesetters, not typewriters). The arguments appear (again, as far as I can see) to be that the memos would have had to have been done by someone with typesetting expertise and either professional typesetting equipment or the most expensive state-of-the-art IBM typewriter painstakingly measured, set, and reset to mimic typesetting. (This, in effect, is the Flounder.com argument linked to above, and it seems to be the standard argument at present.)

[Brandon]

Whoops! That should be: "they aren't deductive arguments but circumscribed searches". I forgot to change it in review....

[James Nightshade]

I got to agree with Brandon that "word processing software was not designed to mimic typewritten documents, but typeset documents."

Some high-end typewriters of the 1970s also tried to mimic typeset documents. But none of them were ever as capable as Microsoft Word and its ilk.

[Thad]

Brandon, James,

Actually, the big selling point of the IBM Executive series of typewriters was their ability to more closely mimic the look of typeset documents, hence the use of proportional width fonts. And the IBM Selectric Composer actually was capable of truly professional typesetting techniques, up to and including justification -- not used in the Killian memos, of course, but an indication of just how powerful and flexible the Selectric Composer was. It's not outrageously implausible that the TANG offices (or whatever office Killian had this typed at) would have had a Selectric Composer on hand, and yes, it could certainly handle light typing duties just as well as professional typesetting. Also, contrary to what has been reported elsewhere, it">http://www.ibmcomposer.org/SelComposer/images/ComposerBrochurePic.jpg&imgrefurl=http://www.ibmcomposer.org/SelComposer/brochure.htm&h=818&w=638&sz=284&tbnid=j-dJcX2UYsMJ:&tbnh=142&tbnw=111&start=1&prev=/images%3Fq%3Dselectric%2Bcomposer%26svnum%3D100%26hl%3Den%26lr%3D%26ie%3DUTF-8%26safe%3Doff%26sa%3DG">it was not much bigger than a standard typewriter.

Anyway, so far, the only person quoted in the media who actually worked for IBM in the 1970's says the memos could easily have been written on readily available 1970's IBM machines, and the "experts" who say otherwise (presumably including Newcomer, who talks a good game but doesn't seem to know much about these machines) are all "full of crap."

[Just an addendum -- the freeper charge that annoys me the most -- and it's really hard to pick -- is that the documents just happen to have 1.25 inch margins on each side, just like MS Word, and that this is somehow remarkable. Well, DUH -- of course they have 1.25 inch side margins, that's been the standard for business correspondence going back to at least the 1930's.

[Brandon]

Hi, Thad,

I agree entirely with your first paragraph; I didn't intend to imply that no typewriters could mimic any of the more sophisticated typesetting techniques, simply that the issue is more complicated than was suggested in the post. The typesetting argument is still being raised, though, with regard to the Selectric Composer, which is the typewriter mentioned in the dilemma in my comments, although on further consideration I can't remember if they've actually been saying that it was the most expensive. In any case, my points here are merely to try to clarify what the argument being made actually is; one of the problems respondents to the critics have faced is that their responses have sometimes lagged a great deal behind the critics, who are continually trying to strengthen their argument, and this lag makes the discussion difficult to follow.

[Brandon]

As I said, I'm mostly just trying to clarify what the argument is, and looking around I see that I garbled it completely in both my comments on it above.

For information on the Selectric Composer see here, which is the source for part of the information on which the argument is based (and which insists that the Composer is a typesetting machine requiring typesetting skills to mimic some of the elements of the memos; he suggests that the Executive is far more likely, although he admittedly doesn't have much acquaintance with Executives).

[abdul abulbul amir]

"It's not outrageously implausible that the TANG offices (or whatever office Killian had this typed at) would have had a Selectric Composer on hand..."

Well Ms Knox the woman who did the typing down there said they had a manual typewriter and later a Selectric. Further no living witness or subject supports the the notion that the memos are anything else but fake.

"Of course word processors can imitate typewritten documents. That's what they were designed to do!"

You must be joking! Have you ever compared word processed vs typed docs?

[Karl Baumgarten]

Are there any certified forensic document experts that have stated the documents were NOT forgeries? If so, I haven't heard of them, and I don't think CBS has, either.

[Walter]

This argument is already over. The documents are forgeries plain and simple. The attorney that is representing the forger is a hard core Liberal with heavy ties to the DNC.

[Walter]

http://www.weeklystandard.com/Content/Public/Articles/000/000/004/640pgolk.asp?pg=1 Pretty much tells the whole story.

[Josh Narins]

SoCalJustice is just a freaking moron.

Anyone who would quote someone who, early on, pointed out that it couldn't have been done on a typerwriter is simply parroting the Freep.

SoCalJustice is probably a right-wing flak who enjoys, as many undoubtedly do, the name MajikThise and the picture in the upper-left corner of the blog, and so regurgitates the required anti-meme here first.

[Josh Narins]

"Abdul Abulbul Amir" (a character from an old song my father sang) is also full of shite.

There distinctly ARE people who are alive today who support the memos.

Facts:
1. Knox says they the memos match the office talk.
2. Knox says Killian had a secret file (which he typed himself?)
3. Killian's CO says the memos are real.

Ergo, Amir and SoCalJustice are either moronic freep-alongs, or actually agent provacateurs.

Can you delete their trash comments? Please don't this time, because then my response comments look out of place ;)

[Jakemeister]

The documents are false. The bloggers uncovered the fact. Your whole blog is just a stew of cheesy rationalizations. Just a bunch of hot air!

[Jakemeister]

Hey Josh, the documents are false and all you ae doing by slamming SoCalJustice is proving that the left is impervious to facts and doesn't have the honor to admit an error. You are a true Leninist in that regard. The truth is merely a servant to your cause. Granted, there were top of the line typewriters in existence that could have printed it, the TANG didn't have them and a General wouldn't have taken the time and expense to use it for a personal file. Besides, teh all important jargon is incorrect. Maybe you should try to find some vets that didn't denounce the DoD and the troops when they returned to auhenticate that. SoCalJustice, you're a lizaroid!

[RightWingConspiracy]

Actually, the documents are proved fake at http://www.flounder.com/bush2.htm

Engaging in a more productive discussion over who faked them is a better use of your -- clearly finite -- intellectual capacity.

[leonide777]

comment6 сочинение по литературе на тему жизненные искания Андрея Болконского в романе Война и мир .Л.Н.Толстой можно ли беременным уходить на час раньше с работы скачать полную версию Англо-Русского словаря для Нокиа 6670 полный альбом группы стрелки бесплатно оптимальная модель политической модернизации