You mean the aether theory has no credibility any longer? Damn ...

So, you're suggesting a kind of ongoing within-science poll of what's plausible and what's not, science-theory and science-field-wise? That sounds like a good idea. It is funny to what extent interested outsiders have no idea how seriously to take findings or even debates within fields. I'm assuming you guys are tipping me off to what's hot and solid and where things are going in your fields. But you could be misleading me entirely. How would I know?

"a laughing stock"
or
"thinker of gigantic Aristotelian proportions"?

Pffft. How are these two options opposed? If Aristotle were doing philosophy today, he would be derided as a sloppy new-age dogmatist. I've never read George Lakoff, but if "Aristotelian" is the adjective that naturally describes his theory, then I automatically dismiss it as unwieldy and arbitrary.

If Aristotle were doing philosophy today

well, so? great minds are generally wrong about science, as are most humans. i'm basically referring to THE NEW YORK TIMES MAG profile where he talks about something he discovered which had been misunderstood since aristotle's time.

Why not, as per usual, measure the degree of influence by counting the number of citations a paper gets in the relevant citation index?

Bora has made the Kuhnian claim that multi-level selection's day will come when the older scientists die off, but how do we know that his perception is correct?

How many older scientists would have to be killed to get an operational answer to this question? (I suppose I should be asking Bora.)

Great minds are wrong mostly about things that no one else knows either in their time. They discover some things, but not everything, and become wrong when another great mind (possibly a contemporary) comes along with a new discovery. In other wrods, no one discovers everything. Newton, Galileo, etc. were wrong about some things (Galileo was resisted to the idea of gravity, for example, as a form of action-at-a-distance).

Aristotelianism was effectively anti-scientific during the long early-Christian period (until 1500 or so), but Aristotle himself was an active experimental biologist who dissected molluscs.

Now going to your major point.

One problem with polling is that the energy of science and the most interesting stuff tend to be concentrated in the areas of uncertainty, so that the things you would hope most to know are in the areas where it's least likely to get answers.

I think that sports is a better analogy than elections. Polls are just dry-run mock pre-elections before the real elections; polls are very similiar to elections, sample elections a little before the big counting one.

Scientific disputes are like games in progress, where you know a lot about the relative strength of the teams, but don't know who will win the game. Pauling and Watson-Crick were evenly matched, for example, but (as I remember) they scored some points when they realized DNA wasn't a protein, and won the game with the double helix.

One big question: how often does a genuine minority thinker revolutionize a field with an unexpected and contrarian discovery? The revolutionary myths of science say "often, and that's what's great about science", most recent science studies says "not as often as people think."

I've had lots of debates with economists, and that field seems pretty messed. There seems to be a dominant orthodox marginalist consensus, plus lots of mainstream revisors, and also large numbers of dissidents. After quite a lot of reading and discussion I don't know what the state of the field is at all. My suspicion is that the marginalists are maintaining power by institutional dominance rather than by winning arguments, but I'm biased and I really don't know -- the time I've spent trying to find out has made the subject more puzzling, not less.

Resolving disputes is also a lot harder when there's no possibility of resorting to an experiment or observation to decide between theories. In cryptography, you can see some of this with the style of papers that are popular. There's a big push for security proofs in papers, for reasons that have much less to do with certainty (it's pretty common for the proof and the attack to appear within a year of each other) than with a sort of community standard of what's important in a paper. By contrast, proposing actual schemes has a kind of experiment--people break them, or implement them for some application and find that they don't work well because the author of the paper screwed up his analysis of the work required to do his scheme.

I tend to think that the really bitter scientific disputes are in areas where there's no way to subject the question to an experimental or observational test. Or where the test is whether this approach to researching this problem, followed for a whole career, will yield good results. For psychological reasons, once you've committed yourself to that approach, it's natural to fight like hell to assert that it is not only a good approach, but indeed, the One True Correct Approach. The alternative is saying "I've probably wasted the last decade or two of my life, and I'll probably never recover."

Seems to me John Emerson makes an excellent point when he says that polls model elections (though with much more detail than the generally binary choice of the later), whereas determining who’s on the right track in science is more analogous to ways of determining at say the midpoint of a season which sports teams are likely to make it to the playoffs, or win them. (I guess I’ve taken is point a bit further.)

That thought brought to my mind a science podcast, I think from the longer series that Scientific American puts out once a week. In this case it was about some recent mathematical modeling work.

They mathematician interviewed (whose name I can’t remember or never really got but who had a s.A. Indian accent) talked about the degree of randomness or unpredictability in sports outcomes, and of predictability. This touched on the different levels of success the four leading US professional sports plus soccer have had in equalizing teams. (The NFL has had the most success at increasing this from earlier periods, through such mechanisms as salary caps.) Now for the bit that’s possibly relevant here. The best way of determing the best team or narrow group of best teams quickly (with fewer games), is to quickly match winners with winners and the reverse. That is a flexible schedule. Sort of like the playoffs from day one, or anyway early in the season. This isn’t what anyone does but he modeled it.

Isn’t there some similarity between this and what peer reviewed journal you can get to publish your scientific paper? As your work becomes more recognized professionally you get more prestigious exposure and the critique and head butting against it that goes with that as well, no? There may be some problems with this method in politically sensitive or passionate areas where the predilections of the top editors may be crucial (at like to hear about that), but in other areas doesn’t it work fairly well?

The trouble is, how often was Jensen able to write in Scientific American or Science after he became controversial – even though he was a widely recognized top scientist in psychometrics before that? Was it ever conceivable that Rushton could, no matter how much he tightened his range of evidence? And so on.

Anyway polling just doesn’t seem to me to cut it in evaluating scientific work. Copernicus with right even though the great majority of fellow astronomers at the time didn’t think so for quite a while. Lamarck was wrong even though his theory was widely popular for a while. Science is ultimately determined by undemocratic processes. As well who do you poll? In climate research the opinion of people working generally in the area or related areas seems to me nearly irrelevant when compared to that of someone expert in the same or closely related methodologies in the micro area (say of studying water vapor and cloud effects). And so on.

Just to make the last point about "who do you poll" crystal at the risk of needlessly belaboring it:

Isn't peer review a mechanism of "polling" by way of acceptance for publication or not and before that maybe suggestions for tightening up, by a good selection of those most expert in the micro area?

That's how I've long thought it works anyhow. Am I wrong?

Of course then the problem is how are those most expert chosen, and are papers on sufficiently controversial topics of with sufficiently controversial conclusions rejected out of hand before presentation to peers for review by many top journals? Jensen for example? (I'm asking because I don't know, not to make an assertion.)

the pear review process suffers from a good measure of good ole boyism!

just to be clear, i am not interested in the "right answer", i'm interested in the consensus. they overlap, but not very well :-)

Peer review is different from polling, because referees will accept a paper as long as the data look good, regardless of whether they sympathize with the larger motivating theoretical program. In my own field, I can think of one investigator whose results are published in top journals, yet a poll of experts would reveal that almost everyone dismisses this person's ideas.

Also, Razib, I'm puzzled by your implied claim (and forgive me if this is not what you intended to imply) that polling would lead to a better-informed public. Those who know, know; those who don't know, can not be trusted to interpret the results of the poll anyway. Even among scientists, there are clearly a few who are in the know, and others who suffer Rumsfeldian meta-igorance: they don't even know what they don't know (and ought to quit playing the game).

I was critical above, but let me switch. By polling we could at least differentiate the controversial, in-process areas from the consensus areas. A 51-49 vote wouldn't mean much, but a 90-10 vote would mean a lot, and my bet is that you'd get a lot of 99-1 votes, perhaps including some unexpected ones.

Those who know, know; those who don't know, can not be trusted to interpret the results of the poll anyway.

they are trusted with decisions anyone. e.g., funding decisions. polls can at least be used as a talking point to dampen down the misinformation, e.g., creationists do sincerely believe that scientists have rejected evolution because that's what their leaders tell them. consider the utility of the fact that only 40% of ph.d. scientists are theists, and only 10% of national academy of sciences members are theists. we know this from the larson & witham surveys. we should have more.

By polling we could at least differentiate the controversial, in-process areas from the consensus areas. A 51-49 vote wouldn't mean much, but a 90-10 vote would mean a lot, and my bet is that you'd get a lot of 99-1 votes, perhaps including some unexpected ones.

this is exactly what i'm talking about.

John Emerson--

but a 90-10 vote would mean a lot

The trouble is you would have gotten something like that against Copernicus for quite a while.

In general the more politically sensitive the area (and the lower the average IQ in the field e.g. sociology), the less the poll numbers would mean. But the public would be unlike to comprehend such things.

Oh, ok, you've got creationism in mind.

Yeah polling would work well there, and very legitimately so. The fraction supporting creationism would approach zero out to several decimal points.

Isn't that a special case though?

How well do you think polling would work on Cochran and Harpendig's work, or Bruce Lahn's controversial stuff? Or my examples of Jensen and Rushton and Lynn?

How legitimate would the results be, as a measure of likelihood of ending up notably more right than wrong?

Isn't the same true of all sorts of HBD research, genetic or otherwise, at least once other scientists see it being disparaged in the media, popular and scientific?

The trouble is you would have gotten something like that against Copernicus for quite a while.

so? science isn't about being right on all times all the time in all ways. we'e wrong on a lot of things right now that we think we're right on. so what? the key is to beat random expectation. over the long haul science is self-correcting, but my point isn't about being correct, it is to translate the general feel of the scientific consensus to the intelligent public with some level of quantitative precision.

the intelligent public is key, a lot of times you don't know what's going on in affinal disciplines and propogandists can quickly distort perceptions.

Isn't that a special case though?

people compare global warming to evolution all the time re: denial. well, compare the % of atmospheric physicists who are skeptical of the consenus vs. biologists.

Isn't the same true of all sorts of HBD research, genetic or otherwise, at least once other scientists see it being disparaged in the media, popular and scientific?

i am, believe it or not, interested in things aside from HBD. this isn't jesus, come to cleanse away all sins and offer everlasting truth.

in any case, if the poll was anonymous i'd suspect people would be surprised by the amt of interest/sympathy for lahn's work in particular. most evolutionary biologists might not be in the jensen camp, but they deal in variation as their stock and trade.

but seriously, you're misconstruing my intent. the poll isn't to validate whatever positions we hold as right, true and tenable, it is to gauge the nature of scientific opinion. if they're wrong, so what? that's just the nature of the data. i don't suggest that we take a poll of humans to ascertain whether god exists, but, i do suggest we take a poll if we are to argue whether a belief in the existence of god is modal in all cultures. it is confusion of the latter form i'm trying to address.

p.s. sam harris regularly states that the majority of swedes don't believe in god (i.e., an example of a culture where theism is not modal). this a false. so the analogy i'm using doesn't use a moot example. people regular make false assertions about the state of general opinion, whether that opinion is right or wrong.

Haha, razib just used the word "affinal" in a non-anthropological, non-mathematical sentence. That's why I like coming to gnxp.

I see how this poll can be useful for busting the creation-science bubble. But laypeople don't make specific funding decisions (with a few politicized exceptions like human embryonic stem cells). Congress doesn't poll scientists to figure out what science is worth doing; Congress just appropriates chunks of money according to the lobbyists' demands, trusting the scientists to sort it out amongst themselves.

surveys like this are already done when it becomes of interest to an individual researcher. expanding and regularizing the practice seems like a good idea.

Congress just appropriates chunks of money according to the lobbyists' demands, trusting the scientists to sort it out amongst themselves.

one thing lobbyists offer is specialized knowledge. but, we all know that lobbyists often used their 'expertise' to just lie about what the state of a given field is. to use an explicit example, assume that there is a tightly framed question re: anthropogenic global warming addressed to those with a ph.d. in the atmospheric scientists. what if the result was 90% yay, 10% nay. we know that 95% of republicans seem to reject this position, and a large number will continue to reject this position even if such a survey is delivered to them, but i think at least it could crystalize the nature of the debate and the root of the beliefs.

p.s. this is not to open up a debate on global warming, i'm offering a hypothetical as to the utility of the program that i'm suggesting.

When Copernicus first proposed his theory, it was quite reasonable to reserve judgement and ask skeptical questions. That was the "in process' period.

As far as HBD goes, I think that a poll of individuals in the field would be less monolithic than the official consensu. I don't say this as a HBD advocate either.

John Emerson-

As far as HBD goes, I think that a poll of individuals in the field would be less monolithic than the official consensu. I don't say this as a HBD advocate either.

Maybe so.

You might be somewhat changing my mind. I’m “in process” as you put it above. Which could go either way, let me add, if not I’d say right now I’ve changed it. The fulcrum for me is estimating how the intelligent public would evaluate the science expert poll info. As well, how would this effect actual science going forward?

I’m an HBD “advocate” only because it seems to me that there’s compelling evidence that that’s what’s true, speaking at a broad level of generality, obviously. True but broadly rejected among elites for religious or religion-like reasons. It’s not that I relish that it be true, as a buttress for my white maleness (and the Jewish and Japanese variety of whitish males would take more reflective pride for it’s confirmation, for example). It’s that iconoclastic truth seeking has always been a primary intellectual motive for me, or perhaps THE primary one.

I’ve always been attracted to debunking what the INTELLECTUAL establishment takes as dogma. Or debunking around the edges, not pulling down the whole edifice. As for the economic or governmental establishment at the moment – I don’t care so much. I probably would if I felt significantly threatened by them, but that would be a somewhat different sort of motive. More like my motive for opposing Islamist extremism – which as it happens meshes perfectly easily with wanting to debunk the dominant intellectual establishment views – but which I’d feel impelled to strongly oppose regardless.