The paper, https://openreview.net/pdf?id=gCPJFcHskT, an approach to the study of AI following the research procedures established in the study of animal cognition, starts with the story of Clever Hans, a case of mistaken attribution of intelligence to a horse.
See my take on the relevance of CleverHans to AI, http://drbean.sdf.org/CleverHansMind.html
Comparative psychologists studying animal cognition appear to have become wary of endorsing dramatic claims for the ability of animals, even of parrots1 , and philosophical speculation about animal sentience in the case of insects and other organisms has been made fun of, eg by Justin Leiber. See http://drbean.sdf.org/AnimalSentience.html
The paper extracts 5 principles from animal cognition research:
Behaviorists were unconvinced perhaps by the Harvard Law2 that their program was impossible. When they did something to animals in experiments seeking answers to questions they had, they did not think it was unnatural to cast the animals in ‘passive’ roles and require them to respond to their interventions.
Recognizing that animals have some free will according to the Harvard Law, they might have included a control group to check whether their answers had any relationship to the questions asked, but they didn’t.
Shown up by ethology and shamed by the cognitive revolution in psychology, behaviorism, focused on animals, morphed into comparative psychology.
Recognizing more active roles for animals, researchers studying what they know realized the cunning of animals required they themselves also to be more tricky asking their questions.
An adversarial attitude was required designing control conditions, but who is the adversary here?
The paper says it is alternative explanations to their theories, probably simpler mechanisms, that are the adversary, but perhaps the adversary is also the ideas of anthropomorphizing animal supporters and the belief that human thinking processes apply in animals also.
But perhaps the adversary is the animal under investigation using cunning to achieve the result it wants. In this case, the investigator needs to interrogate the animal, an uncooperative suspect, using questions crafted to reveal the truth. Adversariality and interrogation should not suggest antagonism, disagreement or opposition. Rather the investigator needs tactical empathy.3
The paper says researchers recognize interrogation is an ongoing process, not really captured by the idea of an experimental intervention with a control group. They recognize they need to continue to generate alternative hypotheses and a series of questions for the animal on the basis of these hypotheses to find out the truth.
Rigorous experimentation requires the careful selection of experimental stimuli on the basis of which experimental and control groups are distinguished and explicit research result claims can confidently be made. But the expansive claims researchers wish to make often extrapolate way beyond the data they have, and are only analogically related to the findings of the experiments they ran with those carefully distinguished experimental stimuli.
No, that’s true perhaps, but not the point the paper is making. It’s saying because cognition can be conceived as different phenomena occurring over a range from the concrete to the abstract, results found with any one set of stimuli need to be duplicated with different sets to establish animals have domain-general concepts like numerosity, search strategies and world models.
So if the response of LLM to variations in stimuli is drastically different, it could indicate over-reliance on stimuli seen before. If there is true domain-generality across many stimulus types, it could indicate an LLM has an effective world model.
Surprising failures to replicate or extend findings are more informative than expected success to do so. And the high moral and financial cost of running experiments on animals means studies must aim to use all the information captured by their experiments.
Analogously, instead of asking,
Does this LLM exhibit logical reasoning?the question should be,
In which scenarios does this LLM exhibit logical reasoning and what are the features of those scenarios? How do those features also characterize other scenarios?But making these judgments, we must avoid the Clever Hans trap and not rush to judgment, something we are able to do, should be able to do, or are advised to do evaluating people’s statements. Unfortunately, along with the Steve Jobs Reality Distortion Field4 and Trump Derangement Syndrome5, we have the AI Reality Distortion Field Derangement Syndrome, which prevents us doing this.
Anim Cogn. 2022 Dec 22;26(1):199–228. doi: 10.1007/s10071-022-01733-2 https://pmc.ncbi.nlm.nih.gov/articles/PMC9877086/↩︎
Under controlled experimental conditions of temperature, time, lighting, feeding, and training, the organism will behave as it damn well pleases–Joel Garreau↩︎