Originally posted September 30, 2013

First of all, I wanted to offer a big thank you to everyone who participated. My co-conspirators (CJ, See Arr Oh, and Stu) and I are truly grateful to everyone who took time out of their day to answer our survey questions. We’ve had 401 responses (as of Monday evening at 8pm EST), which, in all honesty, is so much more than we had ever expected. We need to send a special word of thanks to the ever-generous, and awesome, Derek Lowe who, I am sure, was the reason that many of those respondents found the survey in the first place.

I wanted to give a brief run-down of the responses that we received from our first 19 guinea pigs. Not including the four of us, the remaining 15 included: Christopher Cramer, a crew from Bruce Turnbull, Dr. Rubidium, Grant Hill, Jessica Breen, Marcel Swart, Katherine Haxton, Aaron Finke, Gavin Armstrong, Ash Jogalekar, Stuart Conway, Vikki Cantrill, Paul Bracher, and Kathy Franz.

At the outset, we all (CJ, See Arr OH, Stu, and I) agreed that importance and impact and citations are not equivalent and are also difficult to define. What we were/are after is an understanding of the following.
1) Can chemists agree on what papers are “important”?
2) Can chemists predict what papers are cited the most?
3) Can we identify and agree on papers we want to talk about with chemists (or with non-chemists)
4) Is there any overlap between what we find important and what gets cited?
And, finally
5) There’s lots of really cool chemistry out there; what do I (we) miss that other people see?

Our respondents gave some initial perceptions on how their thought process. In general, there was an opinion that papers from a persons own field would dominate question one (perceived importance) along with titles that included words like “first”. Several respondents thought the second question (most citations) would be dominated by “people worship”. That is, papers from “big name” PI’s would be cited the most. Another person thought the best cited papers would include: “All synthesising things using methods that will undoubtedly have been picked up on by others and further developed.”

So, how did those predictions play out? And … what were the answers to our four questions? (1-Which three papers in the issue do YOU think are the most ‘significant’? 2-Which three papers do you think will have been cited the most to-date? 3-Which three papers would you most want to point out to other chemists? 4-Which three papers would you want to shout about from the rooftops?)

This pdf shows several things. First, it lists the articles from our issue of JACS in order of times cited. Second, it shows the number of times each paper was used as a response to one of our questions. Third, there is a comparison between total citations and total responses. There are several results here that I’d like to note:
1) Of the 63 papers, there were only 16 not mentioned by any of our respondents.
2) The fewest number of citations by any paper on this list is 11. (This seems remarkable. Should it be?)

Let’s have a look at some of the individual papers and their use in the responses (Disclaimer — I am writing this so I’m going to discuss the papers/responses that I think are most interesting):

“Factors that Determine the Protein Resistance of Oligoether Self-Assembled Monolayers − Internal Hydrophilicity, Terminal Hydrophilicity, and Lateral Packing Density”
Total Citations: 325; Total Responses: 4.
This paper addressed issues that many were having at the time and set a precedence for how researchers should construct their self-assembled monolayers. Obviously, there are a lot of chemists who would find this useful to their own research. I think the useful issue is important for determining citation number. I am heartened to see that “useful” seems to be key in deciding the top three cited papers from this issue.

“TNA Synthesis by DNA Polymerases”
Total Citations: 59; Total Responses: 19.
Many of our respondents found this paper interesting due to it’s origin of life (OoL) implications. Seeing as how OoL has come up often among the “grand challenges” for chemistry, it is somewhat surprising that this study hasn’t been cited more. Are chemists (as a whole) just not that interested in OoL studies? Is this OoL a more recent phenomenon? That is, have most of its citations come in just the last year or two? (This is a question we will be looking to answer in our publication of the larger study.)

Other Useful
“Easily Processable Phenylene−Thiophene-Based Organic Field-Effect Transistors and Solution-Fabricated Nonvolatile Transistor Memory Elements”
Total Citations: 275; Total Responses: 10 (This one *could* also be helped by “author-worship” – The PI is Tobin Marks).
“Transition Metal-Catalyzed Formation of Boron−Nitrogen Bonds: Catalytic Dehydrocoupling of Amine-Borane Adducts to Form Aminoboranes and Borazines”
Total Citations: 258; Total Responses: 2

Buzzword Bingo
“Single-Step in Situ Synthesis of Polymer-Grafted Single-Wall Nanotube Composites”
Total Citations: 251; Total Responses: 8
“An Ionic Liquid-Supported Ruthenium Carbene Complex: A Robust and Recyclable Catalyst for Ring-Closing Olefin Metathesis in Ionic Liquids”
Total Citations: 251; Total Responses: 14
“A Heterocyclic Peptide Nanotube”
Total Citations: 171; Total Responses: 15
“Solventless Polymerization: Spatial Migration of a Catalyst To Form Polymeric Thin Films in Microchannels”
Total Citations: 18; Total Responses: 10

Firsts
“First Examples of Organophosphorus-Containing Materials for Light-Emitting Diodes”
Total Citations: 103; Total Responses: 15
“First Pseudorotaxane-Like [3]Complexes Based on Cryptands and Paraquat: Self-Assembly and Crystal Structures”
Total Citations: 88; Total Responses: 3.
“ The First Triple Thiol-thiolate Hydrogen Bond versus Triple Diselenide Bond That Bridges Two Metal Centers”
Total Citations: 11; Total Responses: 1

Things that might show up in a textbook, or in a classroom, or in a conversation between a bunch of dorky chemists
“On the Nonpolar Hydration Free Energy of Proteins: Surface Area and Continuum Solvent Models for the Solute−Solvent Interaction Energy”
Total Citations: 129; Total Responses: 3
“Mechanisms of C−C and C−H Alkane Reductive Eliminations from Octahedral Pt(IV): Reaction via Five-Coordinate Intermediates or Direct Elimination?”
Total Citations: 94; Total Responses: 10
“Tin-Centered Radical and Cation: Stable and Free”
Total Citations: 38; Total Responses: 1
“Quantitative Measure for the “Nakedness” of Fluoride Ion Sources”
Total Citations: 32; Total Responses: 4
“A Cationic Guest in a 24⁺ Cationic Host”
Total Citations: 31; Total Responses: 5

Journalism Fodder
“Diamond Formation by Reduction of Carbon Dioxide at Low Temperatures”
Total Citations: 50; Total Responses: 7 (All of which were responses to the “tell non-chemists” question)

One more curious case
“Partitioning the Loss in Vancomycin Binding Affinity for d-Ala-d-Lac into Lost H-Bond and Repulsive Lone Pair Contributions”
Total Citations: 57; Total Responses: 11
A lot of us really “liked” this paper. This is another one of those papers that might not have been “timely” enough. MRSAand other antibiotic resistance bacteria have burst onto the scene much more in recent years. To many of us, this seemed like a great study for chemistry (structure, function, mutation, new function) study as well as a paper that might interest reporters covering antibiotic resistance.

The Horse Race
So … who did the best picking citations?
None of our initial respondents correctly guessed the top 3 cited papers. Our clear winner, however, was Gavin Armstrong. Kathy Franz also put in a very respectable showing! They were the only two to top 600 total citations (between the three articles they chose). In the analysis file, I show how respondents picks for each of the questions fared in terms of citations. It is also quite clear that, when asked to pick the most cited articles, chemists are capable of distinguishing papers that will be cited.

If your still reading, thanks! This is all I’m going to analyze for this post. We do expect to have a more detailed and in depth analysis for the data from the larger survey. These were just things that I found interesting. I’m sure that if I’ve missed anything important, CJ, CRO, and Stu will mention it in the comments. Also, if you see anything interesting, let me know. We had a lot of fun with this survey! And, if you’d like to see more, here are all of our respondents answers(with some comments on why they liked the papers they did). Enjoy!

Thanks
-CJ, See Arr Oh, Stu, and Matt