reading

In dental school, I was taught to “do dentistry” based on my many instructors' clinical anecdotes that appeared to be true and tested over time in their respective hands. However. this led to inconsistencies in recommendations from different instructors. creating confusion on how to best manage patient care. Although hours of instruction were given to basic science, little instruction was given on clinical research design and how to critically evaluate it and apply it to patient care. The 1990s brought the era of evidence-based medicine, which promoted translating clinical research outcomes and consideration of patient preferences as well as clinical anecdotes in guided clinical decision making. The first edition of Critical Thinking: Understanding and Evaluating Dental Research (1997) was the first textbook to fill this void in dental education and practice.

Critical Thinking became an award-winning textbook and is now available in this third edition. Its success lies in its unique writing style. Textbooks are historically written in the dry tone of the passive voice. consistent with the objectivity of science. In contrast. Dr Brunette adopted a narrative writing style that engaged the reader to view the complex concept of scientific investigation through personal experience. At the time, teaching through stories was rare and almost frowned upon as being "beneath" science, which is supposed to be free of any perceived subjective influence. Yet. Dr Brunette was ahead of his time, and this style of writing has been shown to be effective for teaching complex concepts. especially to students reading about them for the first time. This style has since become more common and is now even encouraged as a mode of teaching.

This book is fundamental for teaching the basic skills of evidence-based dentistry. These skills are key to managing the dentist-patient relationship, which has changed since I graduated in 1987. At that time, the relationship was predominately paternalistic, where I was the gatekeeper of dental knowledge and my patient's predominant access was through me. I had the power to discriminate which knowledge I felt the patient should have access to and the patient had to take me at face value with no opportunity to verify what I decided to share. However. as patients gained more access to information through the Internet. dental knowledge became mainstream. This allowed patients to assert autonomy in their dental care. and my relationship with my patients changed from one where I dictated treatment to one of shared decision making between us.Today, my patients expect me to help them critically evaluate dental knowledge so we can make informed decisions about their treatment together. This is where the skills taught in Critical Thinking become invaluable to a clinician.

Title: Critical thinking : understanding and evaluating dental research / Donald Maxwell Brunette.

Description: Third edition. | Batavia, IL : Quintessence Publishing Co, Inc, [2019] | Includes bibliographical references and index.

Identifiers: LCCN 2019017538 (print) | LCCN 2019018451 (ebook) | ISBN 9780867158014 (ebook) | ISBN 9780867158007 (softcover)

Classification: LCC RK80 (ebook) | LCC RK80 (print) | NLM WU 20.5 | DDC 617.6/027--dc23

© 2020 Quintessence Publishing Co, Inc
Quintessence Publishing Co Inc
411 N Raddant Road
Batavia, IL 60510
www.quintpub.com

All rights reserved. This book or any part thereof may not be reproduced, stored in a retrieval system, or transmitted in any form or by any means, electronic, mechanical, photocopying, or otherwise, without prior written permission of the publisher.

To the late Doug Waterfield, with whom I collaborated extensively once it became clear through the in vivo studies of Babak Chehroudi that macrophages played an important role in tissue response to the topography of implants. Doug was a tireless advocate for students in the Faculty of Dentistry and spared no effort to help them succeed.

The need for a third edition of Critical Thinking was driven by developments in dental and biomedical research conditions and practice, as well as reader comments. The third edition emphasizes the practical application of critical thinking in the production and evaluation of scientific publications. A number of issues are now more prominent.

Thanks to the Internet, patients of today have much more information (both accurate and inaccurate) than the patients of yesteryear. The consequence to dentists is that they must explain and be prepared to justify their clinical decisions with evidence. This requires that dentists be capable of unearthing relevant information and evaluating alternatives by accepted criteria so that patients can be confident in their dentist’s recommendations. In line with the virtual revolution entailed in dealing with the ever-expanding scientific literature, the chapter on searching the dental literature now emphasizes scholarly publishing issues at all stages of research in addition to covering more traditional topics such as types of resources and searching techniques. The preparation of systematic reviews is discussed in greater detail, as are the strengths and weaknesses of various services such as Web of Science and Google Scholar as well as approaches to identifying relevant evidence in grey literature and its use in reviews. The author of the revised chapter is Helen Brown, who has taken over from Kathy Hornby, for whose contribution I continue to be grateful. Helen has master’s degrees in English literature and library science, along with frontline experience resolving student problems in literature searches. She has made significant changes in the emphasis of this chapter. For example, the increasing trend toward open access publications and open science initiatives is addressed, providing important information for both prospective authors and readers with limited access to expensive subscription-based resources. The strengths and weaknesses of the h-index, now widely used by researchers to demonstrate the impact of their work, is also now considered, and there is updated information on impact factors of dental journals and other bibliographic tools.

A second new contributor is Ben Balevi, who takes over the chapter on diagnostic tests and measures from my former student Carol Oakley, whose contributions continue to influence the current chapter. Ben has a master’s degree in evidence-based dentistry and has been recognized for his contributions in this area by the American Dental Association.

There have been considerable changes in other aspects of the book as well. A new chapter discusses the function of each section of the scientific paper so that researchers can check that their papers are fulfilling the expectations of readers and referees.

The coverage on logic has expanded to include more material on informal logic, for example, the use of Walton’s critical questions for evaluating causation. The informal logic used in the refereeing process in which the burden of proof shifts from authors to reviewers and back has been added to complement the argumentation maps found in the chapter on judgment that features tactical details such as rebuttals, caveats, and supporting evidence. The intent is to give investigators a practical approach to anticipating and dealing with criticisms. The chapter on judgment now includes materials of both historical (Darwin’s decision to marry and Benjamin Franklin’s moral algebra) and novel decision-making heuristics, such as fast frugal heuristics.

Clinical decision making has been given its own chapter and expanded to include detailed, worked-out calculations and clinical scenarios. In addition to decision tree analysis, two elements of central importance to patients are presented with worked-out calculations: cost-effectiveness analysis and willingness to pay analysis.

The chapter on diagnostic tools and testing has been expanded to include new worked-out calculations and examples, as well as illustrations to clarify this topic, which many find difficult and confusing.

The chapter on experimentation now covers randomized field trials, a method employed in causally dense environments such as business problems, often combined with trial-and-error approaches. The technique is evolving but has been found useful by organizations such as Google that run thousands of randomized field trials.

The statistics chapter has expanded coverage of Bayesian statistics, as this approach is expected to become more widely used. The chapter on presenting results contains new illustrations and more extensive discussion of the integrity-based principles for displaying information emphasized by Tufte. The section on the behavior of scientists has been expanded with discussion on the reproducibility crisis now shaking some scientists’ faith in the reliability of reported results, even including those published in high-impact journals. All chapters have also been improved by the addition of updates, revisions, or examples.

I acknowledge the support of my colleagues who have never hesitated to provide advice in their respective domains of expertise. I also thank my former students, friends, and collaborators, whose insights inform numerous sections. Regular University of British Columbia (UBC) contributors in this way include Babak Chehroudi, Tim Gould, Hugh Kim, Chris Overall, Mandana Nematollahi, Eli Konorti, and Nick Jaeger (Electrical and Computer Engineering). Colleagues outside UBC, Christopher McCulloch (University of Toronto), Ken Yaegaki (Nippon Dental University), and Doug Hamilton (Western University), have also provided advice and encouragement. I also thank my collaborators at the Swiss Institute of Technology (Zurich), Professors Marcus Textor and Nicolas Spencer, and my former post-doc Dr Marco Wieland, for continuing my education in surface science. Of course, I would be an ingrate if I failed to acknowledge the continuing support of my wife Liz, sons Max and Regan (and their partners Heather and Malizza), who provided various examples used in the book including the exploits of my problem-solving grandson Calixte and developing gymnast granddaughter Genèvieve.

I also thank Quintessence for their ongoing support through three editions of this work. This edition has been facilitated through the efforts of Bill Hartman and Bryn Grisham. Zach Kocanda has been exemplary in his attention to detail, often under trying circumstances, in editing the manuscript.

“

It has happened more than once that I found it necessary to say of one or another eminent colleague, ‘He is a very busy man and half of what he publishes is true but I don’t know which half.’”

ERWIN CHARGAFF1

Critical thinking has been defined many ways, from the simple—“Critical thinking is deciding rationally what to or what not to believe”2—to the more detailed “Critical thinking is concerned with reason, intellectual honesty, and open-mindedness, as opposed to emotionalism, intellectual laziness, and closed-mindedness”3—to the nearly comprehensive:

Critical thinking involves following evidence where it leads; considering all possibilities; relying on reason rather than emotion; being precise; considering a variety of possible viewpoints and explanations; weighing the effects of motives and biases; being concerned more with finding the truth than with being right; not rejecting unpopular views out of hand; being aware of one’s own prejudices and biases; and not allowing them to sway one’s judgment.3

Self-described practitioners of critical thinking range from doctrinaire postmodernists who view the logic of science with its “grand narratives” as inherently subordinating4 to market-driven dentists contemplating the purchase of a digital impression scanner. In this book, critical thinking, and in particular the evaluation of scientific information, is conceived as “organized common sense” following Bronowski’s view of science in general.5 Of course, common sense can be quite uncommon. A secondary use of the term critical thinking implies that common sense involves a set of unexamined and erroneous assumptions. For example, prior to Galileo, everyone “knew” that heavy objects fell faster than lighter ones. Critical thinking as organized common sense takes the systematic approach of examining assumptions. The professional use of critical thinking is particularly complex for dental professionals because they live in two different worlds. On the one hand, they are health professionals treating patients who suffer from oral diseases. On the other hand, dentists typically also inhabit the business world, where decisions may be based on the principle of maximizing income from their investment. Dental practice is based only very loosely on responding to disease6; less than one-third of patient visits result in identifying a need for restorative care.7 Twenty percent of work is elective, such as most of orthodontics, tooth whitening, and veneers, and typically that work comprises the most lucrative aspects of practice. Thus, the information that must be evaluated in performing these disparate roles covers the spectrum from advertisements to financial reports to systematic meta-analysis of health research.

Dentists are health professionals, people with specialized training in the delivery of scientifically sound health services. The undergraduate dental curriculum is designed to give dental students the basic knowledge to practice dentistry scientifically, at least to the extent allowed by the current state of knowledge. But if any guarantee can be made to dental students, it is that dentistry will change, because the knowledge base of biomedical and biomaterial sciences grows continually. Most dentists today have had to learn techniques and principles that were not yet known when they were in dental school. In the future, as the pace of technologic innovation continues to increase and the pattern of dental diseases shifts, the need to keep up-to-date will be even more pressing. Means of staying current include interacting with colleagues, reading the dental literature, and attending continuing education courses—activities that require dentists to evaluate information. Yet, there is abundant historical evidence that dentists have not properly evaluated information. Perhaps the best documented example in dentistry of a widely accepted yet erroneous hypothesis is the focal infection theory. Proposed in 1904 and accepted by some clinicians until the Second World War, this untested theory resulted in the extraction of millions of sound teeth.8 But errors are not restricted to the past; controversial topics exist in dentistry today because new products or techniques are continually introduced and their usefulness debated. Ideally, dentists should become sophisticated consumers of research who can distinguish between good and bad research and know when to suspend judgment. This goal is different from proposing that dentists become research workers. One objective of this book is to provide the skills enabling a systematic method for the evaluation of scientific papers and presentations.

A marked addition to the challenges of dental practice in recent years is that patients have increased access through the Internet to information as well as misinformation. Dentists thus are more likely to be questioned by patients on proposed treatment plans and options. In responding to such questions, it is clearly advantageous for dentists to be able to present a rational basis for their choices. Chapter 23 covers an evidence-based approach to clinical decision making and appendix 9 provides a template for dental offices to use in documenting their decisions based on recent evidence.

A systematic approach to analyzing scientific papers has to be studied, because this activity requires more rigor than the reasoning used in everyday life. Faced with an overabundance of information and limited time, most of us adopt what is called a make-sense epistemology. The truth test of this epistemology or theory of knowledge is whether propositions make superficial sense.9 This approach minimizes the cognitive load and often works well for day-to-day short-term decision making. In 1949, Zipf of Harvard University published Human Behaviour and the Principle of Least Effort, in which he stated:

The Principle of Least Effort means, for example, that in solving his immediate problems he will view these against a background of his probable future problems, as estimated by himself. Moreover, he will strive to solve his problems in such a way as to minimize the total work that he must expend in solving both his immediate problems and his probable future problems.10

Zipf used data from diverse sources ranging from word frequencies to sensory sampling to support his thesis. Although the methods and style of psychologic research have changed, some more recent discoveries, such as the concept of cognitive miser in studies of persuasion,11 coincide with Zipf’s principle. Kahneman in Thinking, Fast and Slow has elevated the principle to a law noting that we “conduct our mental lives by the law of least effort.”12

In science, the objective is not to make easy short-term decisions but rather to explain the phenomena of the physical world. The goal is accuracy, not necessarily speed, and different, more sophisticated, more rigorous approaches are required. Perkins et al9 have characterized the ideal skilled reasoner as a critical epistemologist who can challenge and elaborate hypothetical models. Where the makes-sense epistemologist or naive reasoner asks only that a given explanation or model makes intuitive sense, the critical epistemologist moves beyond that stage and asks why a model may be inadequate. That is, when evaluating and explaining, the critical epistemologist asks both why and why not a postulated model may work. The critical epistemologist arrives at models of reality, using practical tactics and skills and drawing upon a large repertoire of logical and heuristic methods.9

Table 1-1 | Level of evidence guideline recommendations of the United States Agency for Healthcare Research and Quality

Psychologic studies have indicated that everyday cognition comprises two sets of mental processes, System 1 and System 2, which work in concert, but there is some debate whether they operate in a parallel or sequential manner. System 1 operates quickly and effortlessly, whereas System 2 is deliberate and requires attention and effort.12 System 2 is a rule-based system, and engaging System 2 is the surest route to fallacy-free reasoning.13 System 2 becomes engaged when it catches an error made by the intuitive System 1. The good news is that extensive work by Nisbett and colleagues (briefly reviewed by Risen and Gilovich13) showed that people can be trained to be better reasoners and that people with statistical backgrounds were less likely to commit logical fallacies. Nisbitt and colleagues further demonstrated that even very brief training was effective in substantially reducing logical and statistical errors. Thus this book has chapters on logic and statistics.

A second objective of the book is to inculcate the habits of thought of the critical epistemologist in readers concerned with dental science and clinical dentistry.

Level	Type of study	Grade of recommendation
1	Supportive evidence from well-conducted RCTs that include 100 patients or more	A
2	Supportive evidence from well-conducted RCTs that include fewer than 100 patients	A
3	Supportive evidence from well-conducted cohort studies	A
4	Supportive evidence from well-conducted case-control studies	B
5	Supportive evidence from poorly controlled or uncontrolled studies	B
6	Conflicting evidence with the weight of evidence supporting the recommendation	B
7	Expert opinion	C

In brief, the problems facing anyone wishing to keep up with developments in dentistry or other health professions are that (1) there is a huge amount of literature, (2) it is growing fast, (3) much of it is useless in terms of influencing future research (less than 25% of all papers will be cited 10 times in all eternity,14 and a large number are never cited at all), and (4) a good deal of the research on a clinical problem may be irrelevant to a particular patient’s complaint

The actual rate of growth of the scientific literature has been estimated to be 7% per year of the extant literature, which in 1976 comprised close to 7.5 million items.11 This rate of growth means that the biomedical literature doubles every 10 years. In dentistry, there are about 500 journals available today.15 Many dental articles are found in low-impact journals, but, ignoring these, there were still 2,401 articles published in 1980 in the 30 core journals.16 More recently, it has been estimated that about 43,000 dental-related articles are published per year.

However, the problem is not intractable. Relman,17 a former editor of the New England Journal of Medicine, believes that most of the important business of scientific communication in medicine is conducted in a very small sector of top-quality journals. The average practitioner needs to read only a few well-chosen periodicals.17 The key to dealing with the problem of the information explosion is in choosing what to read and learning to evaluate the information.

Dentists are exposed to diverse information sources, and the important issues vary depending on the source. For example, a dentist may wish to determine whether potassium nitrate toothpastes reduce dentin hypersensitivity. One approach would be to look up a systematic review on this topic in the Cochrane Library,18 which many regard as the highest level in the hierarchy of evidence (Table 1-1). The skills required to understand the review would include a basic knowledge of statistics and research design. The same dentist, facing the competitive pressures of his or her local market, might also want to determine whether a particular laser-bleaching process should be adopted for the practice. In that instance, there might not be a relevant Cochrane review, and there may not even be a relevant paper in a refereed journal to support a decision. Available evidence might consist of advertising brochures and anecdotes of colleagues. The dentist may have to employ a different set of skills, ranging from evaluating the lie factor in graphics (see chapter 14) to disentangling rhetoric from fact. Advertisements and salesmanship are persuasive exercises; the chapter on rhetoric (chapter 4) deals with means of persuasion.

Typically, dentists acquire information on innovative procedures through participation in networks in which their colleagues supply informal data on the effectiveness of the innovations. Nevertheless, dentists cite reading peer-reviewed dental literature and experimental studies as the gold standard for determining the quality of innovations.19 New technology is often introduced into their practices through trial and error; dentists take the pragmatic approach of directly determining what works in their hands in their practice.19 Doubtless, some of the personal and financial expenses typical of the trial-and-error approach could be reduced with more effective evaluation of information prior to selecting a material or technique for testing.

This book focuses on evaluating refereed scientific papers, but many of the issues of informational quality and questions that should be asked apply equally to other less formal channels of communication.

An acceptable primary scientific publication must be the first disclosure containing sufficient information to enable peers (1) to assess observations; (2) to repeat experiments; and (3) to evaluate intellectual processes; moreover, it must be sensible to sensory perception, essentially permanent, available to the scientific community without restriction, and available for regular screening by one or more of the major recognized secondary services.20

Similar ideas were stated more succinctly by DeBakey,21 who noted that the contents of an article should be new, true, important, and comprehensible. A good deal of the literature circulated to dentists does not meet these requirements. But even excluding the throwaway or controlled-circulation magazines that are little more than vehicles for advertisements, the amount of information published annually appears formidable.

One approach to dealing with a large number of papers is to disregard original papers and receive information secondhand. Dental and medical journals present reviews of current research in specific clinical or scientific fields; some journals, such as Dental Clinics of North America and the Journal of Evidence Based Dentistry, are exclusively devoted to this approach. Although this tactic reduces the volume of literature to be covered, it does not solve the problem of evaluating the information contained in the reviews. To perform this task effectively, a researcher must be able to assess the soundness of the reviewer’s conclusions. In deciding to accept information secondhand, the researcher is also deciding whether the author of the review is a reliable, objective authority. Thus, the problem of evaluation has been changed, but not eliminated.

This book focuses on the primary literature, where it is hoped that new, true, important, and comprehensible information is published. The systematic review, a relatively new review form, attempts to deal with some of the more glaring problems of traditional reviews and is covered briefly in chapter 5. Although useful for some purposes, the systematic review has its own shortcomings, and the researcher must judge how these affect the conclusions. Journals vary in quality; chapter 5 also discusses bibliometric approaches of ranking journals. In the following section, I present a brief review of how articles get published that may help explain some of this variation.

The author’s goal is to make a significant contribution to the scientific literature: a published paper. To accomplish that goal, the author will have to produce a submission for publication whose contents are new, true, important, and comprehensible. Moreover, the author wants to publish the paper in a journal whose readers will likely find the paper of interest and hopefully be influenced by it. As journals vary in the rigor they demand and the length of papers they accept, the author needs to identify the best journal for his or her purposes.

The first hurdle faced by an article submitted for publication is an editor’s decision on the article’s suitability for the journal. Different journals have different audiences, and the editors are the arbiters of topic selection for their journal. Editors can reject papers immediately if they think the material is unsuited to their particular journal.

In some journals, acceptance or rejection hinges solely on the opinion of the editor. However, this method is problematic because informed decisions on some papers can only be made by experts in a particular field. Therefore, as a general rule, the most highly regarded journals ask the opinion of such specialists, called referees or editorial consultants. Referees attempt to ensure that a submitted paper does not demonstrably deviate from scientific method and the standards of the journal. Whether a journal is refereed can be determined by consulting Ulrich’s Periodicals Directory (ulrichsweb.com). Editors usually provide referees with an outline of the type of information that they desire from the referee. The criteria for acceptance will necessarily include both objective (eg, obvious errors of fact or logic) and subjective (eg, priority ratings) components. Unfortunately, the task of refereeing is difficult and typically unpaid. Refereeing is often squeezed in among other academic activities, so it should not be surprising that it sometimes is not done well and that referees often disagree.

Studies of the reliability of peer-review ratings are disappointing for readers wanting to keep faith in the peer-review system. Reliability quotients, which can range from 0 (no reliability) to 1 (perfect reliability), for various attributes of papers submitted to a psychology journal22 follow:

Despite such issues, there is evidence that the review process frequently raises important issues that, when resolved, improve the manuscript substantially.23

After consulting with referees, the editor decides whether the paper should be (1) published as is—a comparatively rare event; (2) published after suitable revision; or (3) rejected. Journals reject papers in proportions varying from 0% to greater than 90%. The literature available to dental health professionals ranges the spectrum of refereed to nonrefereed, from low (or no) rejection rates to high rejection rates. The Journal of Dental Research, for example, used to have a 50% rejection rate (Dawes, personal communication, 1990), but that has risen so that 25 years later some 90% of submissions are rejected.24 Even among high-impact journals, however, there is no guarantee that the referees did a good job. In fact, these considerations only serve to reinforce the view “caveat lector”—let the reader beware.

The editor of the journal and the referees are the “gatekeepers” who decide whether a manuscript is accepted. In science the basic rule appears to be something akin to “if it doesn’t get published, it doesn’t exist.” Thus the rewards in science go to those who publish first, not the first scientist to observe a phenomenon. Obviously, pleasing these gatekeepers is essential to a scientific career.

The editor and the referees are the representatives of the readers of the journal. They protect the readers from wasting their time on obviously erroneous or uninteresting or unsuitable or unoriginal or opaque or trivial submissions. The papers in the journal must be relevant to the readership.

An important part of the editor’s job is to protect authors from unjust criticism that can arise from such things as personal animosity between an author and a referee or an attempt by a referee to block publication of a competitor’s work. Unfortunately, the scientists who are best able to evaluate a submission may be individuals who can suffer most from its publication, as for example occurs when the referee’s own work is “scooped” (ie, published earlier by a competitor).

To justify readers’ expenditure of time the paper should address a significant problem or concern and provide a significant amount of information. The length of journal articles varies; some journals publish “letters” rather than full-length papers for interesting but only briefly developed findings. Editors are interested in publishing papers that are likely to be cited in the future or, expressed another way, are building blocks for future research or clinical application. Tacker25 notes that journals differ in the sophistication of their readership. A general medical journal (eg, JAMA) is written at the comprehension level of a third-year medical student, whereas a specialty journal is written for a first- or second-year resident. A scientific journal should be understandable to third- or fourth-year PhD candidates or above in the general field.

The editor decides ultimately whether to accept or reject a submission. As a general rule the editor is an unpaid (or lowly paid) volunteer of distinguished standing in the field covered by the journal. The editor defines the scope of the journal (ie, what subjects are published in it), and if a manuscript falls outside the journal’s mandate, it will probably be returned promptly to the author. Similarly, an editor may reject a paper on the grounds that a submission does not advance the field sufficiently or has a low potential for future impact. Such judgments are subjective but nevertheless may need to be made. I call this the “de gustibus” standard after the Latin adage, De gustibus non est disputandum: “In matters of taste, there can be no disputes.” As the adage indicates, if a decision is made on this basis it will be difficult to persuade the editor to reverse it.

Editors are often responsible for diverse other tasks such as recruiting referees and persuading them to submit their reviews in a timely manner. Some journals have associate editors who oversee submissions in their area of expertise, and the editor must coordinate their activities as well as consult with editorial boards and deal with the various business matters. Despite the importance of their job, editors are not always appreciated by their colleagues, who may resent some decisions. Chernin playfully suggests, “Editors are also the people who separate the wheat from the chaff and frequently publish the chaff.”26

After the manuscript is accepted by the editor, it may be passed on to a managing editor to take the manuscript though the production and publication process. Day27 states that editors and managing editors have jobs that are made impossible by the attitudes of authors who submit to their journals. For example, authors might ignore the rules and conventions specified by the journal (eg, the format for the references). Or authors and referees may have irreconcilable views, and the editor may be caught in the middle. Given that the editor’s decision could affect the author’s career, it is clearly wise not to irritate editors or referees, but rather to make their job in dealing with the submission as easy as possible. That is, authors want the editor to like them, and as has been extensively studied in the psychology literature,28 liking can be a key factor in persuasion, in this case persuading the editor that the submission should be published.

An indicator of what editors want is provided by the instructions given to referees of journals, often in the form of a checklist or a score sheet that incorporates specific questions for reviews completed online. As an example, I compiled an indicator of some of editors’ concerns by simply looking at the instructions sent to me by ten journals. The following characteristics were emphasized:

Overall the instructions emphasize economy of expression, ignoring the folk wisdom that “Sermons on brevity and chastity are about equally effective.”26 Nevertheless it is useful for prospective authors to obtain a specific checklist for the journal to which they are submitting so that they can attempt to meet the journal’s expectations.

The referees are unpaid volunteers; nevertheless they do receive some nonmonetary rewards. They get first access to new information in a field that interests them, and their decisions can influence the direction of that field. On occasion that information may be useful—for example, a submission could contain a reference of which the referee was unaware or a new technique that might be beneficial to the referee’s own research, or reading the article might prompt an idea for the referee’s future research. Finally, in doing a favor to the editor in refereeing a manuscript, the referee might acquire a store of goodwill that might help when his or her own manuscript is submitted to the journal. (Another well-accepted persuasive factor—reciprocation).28 Nevertheless, refereeing papers is a low-yield task—the referees’ efforts help the editor and those whose papers are published, but the referee typically gets no tangible benefit save the good feeling that comes from doing the right thing. Spending their own time on work for which others will benefit is bound to lead to resentment if those potentially benefitted make the task more difficult than it need be. The applicable golden rule then is to do unto the referees as you would have them do unto you. Make it easy for the referees in the hope they will make it easy for you. In this spirit then authors should attempt to meet the expectations of referees, in particular not wasting their time. In general, referees expect a scientific writing style characterized by the following qualities:

• Objectivity. Data obtained from scientific observation should be capable of being repeated by any competent observer, and the interpretations should be similarly identical among investigators. Expressed another way, in the “Storybook” version of scientific method, there is no room in science for subjective personal data collection and interpretation. Sometimes writers attempt to emphasize their objectivity, and this desire to appear objective can lead to overuse of the passive voice. Of course investigators do have an axe to grind, as they want to be published so that they can reap the rewards of publication—recognition and employment being the chief among these. So a tradition has arisen whereby authors attempt to appear to be objective while being strong advocates for their position. Thus, authors make “Verbal choices . . . that capitalize on a convenient myth . . . reason has subjugated the passions.”29 In any case, readers have come to expect that scientific writers will present at least a veneer of objectivity (practitioners of qualitative methods might disagree), but readers have other expectations of authors as well.

• Logic. Logic not only in organization but in sentence structure and meeting reader expectations.30

• Modesty. Related to the scientific norm of humility (extravagant claims will attract close and probably critical attention).

• Clarity. Scientific writers should follow the common advice to all writers such as avoiding misplaced modifiers, dangling participles, nonparallel constructions, stacked modifiers, etc. (There are numerous books on writing style, such as Strunk and White’s The Elements of Style31 or Zinnser’s On Writing Well.32)

• Precision. Use of precise terminology to avoid confusion and the fallacy of equivocation.

• Justified reasoning. Making the reason for statements clear by referring to data in the paper (eg, “see Figure 1”) or references to the literature.

• Use of signposts (eg, “our first objective…,” “our second objective…”), linkage, etc.

Referees typically submit their reports by filling out forms online often accompanied by explanatory remarks in an uploaded text file.

Typically the form starts with what might be called high-level assessments—questions like accept or reject, priority, overall length of the paper. More detailed points are given in the comments to authors or the editor or editor’s assistant. The confidential comments to the editor give the referee the possibility to offer frank criticism that might be construed by some authors as being insulting. For example, a referee might comment that the paper is poorly written and needs revision by a native English speaker, and such a comment might be insulting to an author who was in fact a native English speaker.

Typically referees make critical comments on the papers they are reviewing ranging from the easily correctable, such as typographic errors or formatting, through more difficult problems to correct, such as lack of clarity in organization, deficiencies such as inappropriate methodology, or erroneous logic, that lead to unsupported conclusions. Typically the referees will number their comments, and the editor will require the author to address each of them. So in effect the authors and each of the referees enter into a debate presided over by the editor, who might also provide some comments, that might be classed according to the conventions of informal logic or pragmatics as a “persuasion dialogue.”33 The participants are obligated to give helpful and honest replies to their opponents’ questions. In theory each participant in the dialogue is supposed to use arguments exclusively composed of premises that are commitments of the other participant. But in argumentation, as in life, commitments are notoriously difficult to extract from an opponent, and pretty much the best one can hope for is plausible commitment to an opinion based on reasoned evidence. In conducting the argument the participants are also obligated with a burden of proof, which shifts from one to the other during the dialog. For example, in submitting the paper the author, as proponent, assumes the burden of proof for the conclusions of the paper, and the components of the paper (ie, methods, data, figures, tables, and logic), constitute the means of bearing that burden. Similarly the referee, in making a criticism, assumes the burden of proof of justifying the criticism. This may be done by various means such as citing deficiencies in the evidence in the paper, external scientific evidence (such as previously published papers), or expected standards in the field of study. The editor forwards the referees’ criticisms along with a preliminary decision to the authors who, if they want the submission to proceed to publication, are expected to bear the burden of proof in responding to the criticisms. This dialogue can be carried over several cycles. Often in my experience, it seems that referees seldom accept or commit to the author’s arguments; rather they merely concede by terminating discussion. In science, as in life, it is difficult to say “Sorry, I was wrong.” In some instances agreement between the referees and the authors is never achieved, but the issues are clarified to an extent that the editor can make a decision.

The question arises of the logic used by editors in making their decisions. First it should be noted that different types of reasoning employ different standards of proof, and this is not unusual in human affairs. In law for example the standard of proof in criminal cases is “beyond reasonable doubt” whereas civil cases are decided on the “balance of probabilities.” Scientific arguments can be complex and may entail various forms of logic ranging from the certainty of deductive logic employed in mathematics to inductive logic, which can deal with calculated probabilities, to informal logic that balances many factors but does not necessarily proceed by strict numeric calculation so that the conclusions are classed qualitatively in terms of their relative plausibility.

Perhaps the reasoning process most employed by editors, who have to make a practical decision, would be the pragmatic model devised by the philosopher Stephen Toulmin34 (see also chapter 22 for more on argumentation maps), which specifies a system for scientific explanation that includes Claims (such as conclusions in the paper) justified by Evidence and Warrants. A Warrant is the means that connects the Claim to the Evidence; it may be, for example, a scientific principle or a connection established by previous work. An important aspect of Toulmin’s approach is that it is field dependent so that appropriate standards are employed for differing types of scientific endeavor. One can see this aspect in action in the scientific literature by observing the content of papers where the rigor of the methods, the quantity of data, or the articulation of the findings differ among different fields of science or among the journals within one field of science. It is the editor who determines the standards of his/her journal, and differences between editors in what they consider important findings or flaws can result in a paper rejected by an editor of one journal being accepted by another one. There are other elements in the Toulmin model, including the Rebuttal arguments that restrict or counter the claim and the Qualifier, which indicates the degree of certainty that the proponent assigns to the Claim (eg unlikely, possibly, highly probable, or beyond any reasonable doubt) and this feature can hold the key to resolution of conflicts. Authors can back off or limit their claims to account for the views of the referees, and the editor can in good conscience publish the article.

The end users of the published paper, the readers, have been defined as anyone who reads the text “with an intentional search for meaning.”35