Once upon a time a single cat, incidentally named Pebbles, came to refute a whole theory of cats. The general theory was “All cats are black”. Not a very good theory, since Pebbles is a grey cat. The power of a single counterexample is enormous. In mathematics a general question can be settled by a proof or a counterexample. Fermat’s last theorem was conjectured in 1637, and a proof was provided by Andrew Wiles in 1944. Euler’s sum of power conjecture was phrased in 1769, and a counterexample was provided by Leon and Parkin in 1966. OK, it took a bit of time, but the point is that both questions were settled. All mathematicians are comfortable with counterexamples. A calculus student knows that f(x)=|x| is a counterexample to the claim that every continuous function is differentiable.
In contrast, counterexamples are fairly rare in mainstream biology. In Rational design of antibiotic treatment plans: A Treatment Strategy for Managing Evolution and Reversing Resistance (joint work with Portia Mira, Devin Greene, Juan Meza, Bernd Sturmfels and Miriam Barlow) we provide a counterexample. Briefly, we identified optimal antibiotic treatment plans intended for hospital use in our article. A treatment plan consists of a sequence of drugs in a specified order. Optimal here refers to maximal probability to “reverse antibiotic resistance” and return to the wild type (see the article for details). By increasing the number of drugs allowed in a treatment plan we initially got better results, meaning higher probability to return to the wild-type. Allowing for at most 3 drugs, gave better results than allowing at most 2 drugs. However, the maximum probabilities seemed to stagnate after a few steps, allowing for at most 6 drugs gave no better results than allowing at most 4 drugs. This observation provoked a theoretical question: Will the maximum probabilities always stagnate? In general that is not the case, which we showed by providing a counterexample. Specifically, we constructed two substitution matrices [representing two antibiotics] on a 3-locus system where the maximum probabilities can be increased indefinitely.
A slightly confused biologist asked me if our argument really works in general. That’s exactly the point. The implication of a counterexample is general, even if the counterexample may be special. The most common misconception of counterexamples is exactly that they are alternative models/theories/methods. I have encountered biologist reviewers who, translated to the example with Pebbles, believe that “Pebbles is a grey cat” is a general model assuming that “All cats are grey and have the name Pebbles”. One can hardly blame reviewers for finding the model “All cats are grey and have the name Pebbles” no more convincing than the model “All cats are black”.
The problems with counterexamples was beautifully illustrated in a recent debate on occupancy modelling. Welsh, Lindenmayer and Donnelly provided a counterexample in Fitting and interpreting occupancy models. Their work was criticized (Ignoring imperfect detection in biological surveys is dangerous: a response to ‘Fitting and Interpreting Occupancy Models’), and the authors responded in Adjusting for One Issue while Ignoring Others Can Make Things Worse. From the article:
“Guillera-Arroita et al have presented in [2] a counter counter example to show that occupancy modelling sometimes works and, on this basis, argue that it is dangerous not to adjust for non-detection. Counter counter examples have no standing in logic and do not refute counter examples.”
Indeed, a black cat is in no way a counterexample to the grey cat Pebbles. Black and grey cats can happily co-exist, and so can examples. American University students are known for their political interest. Counterexamples may not be revolutionary but they do come in handy in critical thinking.