Criteria for Martial History Team Source Reviews

Inventing the Way of the Samurai, Dr Oleg Benesch, 2014, book cover

Introduction

Martial History Team’s mission is to promote martial arts history based on sound evidence and sourced research. This post will explain how the project evaluates books that will eventually be added to “recommended reading” lists. It will also describe criteria that are useful for anyone trying to weigh whether a book is a reliable historical reference.

1. Citation Inclusion

Inventing the Way of the Samurai, pages 1-2

The number one criteria for books that will make the MHT lists is extensive use of citations. The author makes a claim, and then supports it with a citation. For example, on the very first page of Dr. Oleg Benesch’s 2014 book Inventing the Way of the Samurai, he has already made three citations, shown at the end of the three highlighted sentences in the image above.

Books without citations could be opinions, or they could be well-informed. The problem for the reader is determining which situation is true. Without citations, the reader has to treat the text with suspicion at best, and distrust at worst.

2. Citation Correctness

Assuming citations are present, the next criterion is evaluating the correctness of the citations. If the citations are poor, then the reader will begin to lose faith in the work's argument.

This problem is rife on Wikipedia. For example, the entry for Mushin (mental state) originally contained this "quotation":

Mushin (mental state), old entry, https://en.wikipedia.org/wiki/Mushin_(mental_state)

The citation (number 7) once read as follows:

Mushin (mental state), old entry, https://en.wikipedia.org/wiki/Mushin_(mental_state)

The problem with citation 7 is that is totally false. In The Unfettered Mind, available online, there is no text at all matching that found in the old Wikipedia entry. 

In fact, the old Wikipedia text appeared in Joe Hyams' famous book Zen in the Martial Arts. The fixed Wikipedia entry appears below.

Fixed Wikipedia entry

Fixed Wikipedia citation

Therefore, one can have citations in a work, but the citations must be correct. Incorrect citations cause the reader to lose faith in the accuracy and work ethic of the writer.

3. Citation Referenceability

Inventing the Way of the Samurai, pp 15-16

Citation referenceability is another criterion used by MHT to evaluate books. Referenceability refers to the degree to which the reader can track down the citations. Let's use the figure above as an exercise to assess the citation quality for endnote 1. 

The next figure shows the associated endnote entry.

Inventing the Way of the Samurai, notes, p 41

Endnote 1 makes two references. A search online reveals excellent referenceability. 

The first reference, Death, Honor, and Loyalty: The Bushidó Ideal, is easily available online courtesy of the Electronic Journals of Martial Arts and Sciences. 

The second reference, Bushido or Bull? A Medieval Historian's Perspective on the Imperial Army and the Japanese Warrior Tradition, is also available online. The link to a PDF is ungated.

Obviously a document freely available online for anyone with an Internet connection is the pinnacle of referenceability. However, that is not always the case, and it should not automatically count against the citation if it is not freely available online.

Citations may point to books or articles, and sometimes those sources have sections available online via Google Books or Amazon previews. Sometimes there is no content available for the book or article, but there is a record of its existence in the Worldcat database or Google Scholar. 

Gaining access to the source may require a visit to the library or a request via an inter-library loan program. While not ideal, there are still mechanisms to obtain the source and then assess it using the criteria in this post. Old-school historians scoff at anyone who relies on online resources while ignoring libraries. Thanks to massive digitization projects, however, much material is appearing online, making referenceability ever easier.

Aside: An Example of Poor Referenceabilty

Dr. Yang, Jwing-Ming, in his preface to "Chinese Fast Wrestling - The Art of San Shou Kuai Jiao"

On the other hand, some authors may make claims that refer to books which are difficult or impossible to find, and for which there are no other references. 

One example of this weakness appears in the 1997 book Chinese Fast Wrestling for Fighting, a YMAA publication, by Liang Shou-Yu and Tai D Ngo.

In the foreword, reproduced here and shown in the figure above, Dr. Yang, Jwing-Ming writes:

"Not only that, it is recorded in Japanese documents, Collection of Ancestor's Conversations, Volume 2, Biography of Chen, Yuan-Yun that Chen, Yuan-Yun (587-1671 A.D. Ming dynasty) was the person who brought the "soft techniques" (i.e., wrestling) into Japan in 1659 that became today's Judo."

That is a pretty strong claim, repeated by several other Shuai Jiao practitioners. However, they all use the same language and it seems to originate from this statement by Dr Yang. Could there be a Chinese source out there waiting to be read? Certainly. However, in the order of preference we need to 1) be able to read it, 2) see some bibliographical entry for it, and 3) see other references to it that do not originate with Dr. Yang. None of that appears to exist, but should someone find it, MHT is interested!

4. Citation Reputation

The Origins Of Judo’s Kyu / Dan Grading System, Phil Morris, 2005, http://www.abergavennyjudo.co.uk/originsofjudosgradingsystem.pdf

A common problem with some citations, especially on Wikipedia, involves "sources" that show limited reputation. For example, entries and articles on the origin of the dan and kyu ranking system in judo sometimes claim that Japanese swimming inspired Prof Kano's invention of those methods for demonstrating martial arts progress.

Most or all of these discussions appear to trace back to the excerpt above, from a 2005 article by a 4th dan judoka named Phil Morris. While many of the references point to an archived question-and-answer post no longer online, there is a modern copy at the web site of a judo school in the UK.

The problem with this citing this document as a source is that it, itself, has no sources. One can see the author's name and the year he wrote it, but he provides no indication of where he got his material. Readers can tell Mr. Morris has written many short articles (2-4 pages generally) on judo history, but all of them lack citations. There doesn't appear to be anything online about Mr. Morris, and the site hosting his articles hasn't answered requests for information. 

In fact, based on Martial History Team research posted recently, swimming was not the origin (partly or otherwise) for dan or kyu grades in judo. It's far more likely that Kano brought dan grades to school swimming. 

On the other hand, citations pointing to works done by authors like Hurst or Friday, mentioned earlier, have great reputation. Those authors demonstrate scholarly rigor in their works and integrate the recommendations in this post in their writing. MHT also prefers primary sources over secondary sources whenever possible.

5. Readability

Covers of the Three Volume Series Choque by Roberto Pedreira

While all of the criteria in this post are subjective to a degree, the concept of readability is definitely in the eyes of the reader. Readability includes proper grammar, spelling, and composition. These are table stakes that one expects from any work done by a high school student.

Tone, organization, and narrative are a bit more complicated, but they are also key to readability. 

There are generally four sorts of sources that Martial History Team recommends:

1. History books and articles tell the story of a topic.

2. Biography books and articles tell the story of a person.

3. Encyclopedias (generally books) provide references for hundreds or thousands of topics, as an introduction to each.

4. Reference books, articles, or sites are generally collections of material addressing multiple related topics.

Sources that mix these types can be a bit more difficult to digest. 

As an example, Roberto Pedreira has done amazing work researching and writing about the history of Japanese and Brazilian grappling arts in his monumental multi-volume works Choque and Craze. However, large portions of each book are collections of fight records. While these are important, they make it difficult for readers to extract the key points from the overall work. (Might Mr. Pedreira produce a new volume that distills the key points of the multiple books into a single work, referencing the others as necessary?)

Books like Choque and Craze might appear in MHT lists, but probably with an asterisk warning the reader that mining for those golden nuggets is required to extract the most value from the work.

6. Message

Bushido Explained by Dr Alex Bennett, 2020

Once one reads a book or article based on solid sources and delivering its content in a readable manner, the reader can turn to the message the author is delivering. Does it make sense? Does it integrate and synthesize material? Does it build logically from the foundation, its sources? Does it deliver a message that would better inform MHT readers? These are the questions that MHT will evaluate when recommending a work.

7. Value and Obtainability

A very expensive book that might be a great resource?

The final considerations for inclusion in a Martial History Team recommended reading list are value and obtainability. Can readers purchase or otherwise acquire a copy of the work, and if yes, can they do so at a reasonable price?

For example, consider the book shown in the figure. Martial History Team would love to get a copy and evaluate it according to these criteria, but it is on sale only for hundreds of dollars on sites like Amazon and elsewhere. It has some availability via libraries, but inter-library loans are shut due to Covid19. The publisher and author have not responded to queries. Therefore, MHT cannot recommend this book in its present state of availability, unfortunately.

Conclusion

Ninja: Unmasking the Myth, by Stephen Turnbull

The seven criteria for evaluating a Martial History Team reviewed book are therefore the following:

1. Citation inclusion

2. Citation correctness

3. Citation referenceability

4. Citation reputation

5. Readability

6. Message

7. Value and obtainability

Did MHT miss any criteria? It's possible the team uses criteria without making them explicit here. If that is the case, they deserve to be mentioned. Readers can expect to see reviews that examine sources using these standards, with recommended reading lists to follow.

If you like this article, check out our Facebook page, Instagram account, and Twitter feed. Be devoted!