The True Nature of Knowledge Acquisition

Written by Jamie Friday, 30 December 2011 08:20

The first proper attempt to formalise knowledge acquisition was called deduction. Deduction is a demonstration of a fact arrived at by logical reasoning. For example:

All Greeks are mortal
Socrates is a Greek
Therefore, Socrates is a mortal.

Deductive logic relies on one statement that is absolutely true. From that statement, using skilled reasoning, we can learn specific things about the natural world. For example, from the general observation of human mortality, I can, even though I’ve never been there, conclude that ‘South Americans are mortal’.

The problem with deduction is that it is based on general truths. The search for general truths is never ending. Or, for some people, the search for truth did end in something they called ‘God’. Why was Socrates mortal? Because God made him like that. 

Any system of knowledge acquisition that leads to God is clearly broken. Therefore, deduction was superseded by another logical system: induction. Induction starts with specific observations and then works towards a general fact. Here is the classic example: 

The first swan ever to be seen in England was white.
The second swan ever to be seen was white.
The ten-thousandth swan ever to be seen in England was white.
Therefore, on the balance of probabilities, it can be said that all swans are white.

Inductive reasoning frees us from metaphysics and her ugly cousin, theological certainty. This means we can get on with our lives, gathering new knowledge as we go, without been married to ideas that are very likely to very quickly go out of date. Deductive reasoning traps us, inductive reasoning frees us. Bacon summarises this well in his opening lines of ‘Novum Organum’: 

Those who have taken upon thems to lay down the law of nature as a thing already searched out and understood, whether they have spoken in simple assurance or professional affectation, have therein done philosophy and the sciences great injury. For as they have been successful in inducing belief, so they have been effective in quenching and stopping inquiry; and have done more harm by spoiling and putting an end to other men’s efforts than good by their own.

This can be paraphrased for my purposes to say that: 

those who have taken it upon them to lay down the laws of software development have done the evolution of software development a great injury. For as they have been successful in inducing belief in five steps plans, mechanical TDD and Deming’s wheel, so they have been effective in quenching and stopping inquiry; and have done more harm by spoiling and putting an end to other men’s efforts than good by their own.

When we believe general truths we cobble together dysfunctional systems of belief (the brutal 20^th showed us this, with genocide been the natural conclusion of so many certainties). The same holds for software development. We are awash with methods that are composed of constituent 'truths' that we affectionately call best practices. Best practices, of course, being the result of other people’s thinking. Induction was the antidote to such laziness. 

However, induction was also logical, and at one point a lot of people were coming to the same conclusion: humans don’t acquire knowledge logically. Hypothetico-Deductivism – the method of hypothesis – considers a relation where none exists. A good, although dreadful example of this was the Dutch hunger winter of 1944. Children born to the babies who were born during the hunger winter, some twenty or thirty years later, were unusually small. It was as they ‘knew’ they were being born into a food shortage. The variables were food and stage of gestation (first, second or third trimester). Only after the experiment had completed were scientists able to look back and make the links between the variables.

It is fair to call hypothetico-deductivism the leap before you look method of knowledge acquisition. Similarly, software agility is not about best practices (general truths), such as automated testing or visual workspaces, rather it’s about experimentation. It’s about leaping and then looking back. Before each unit of work we hypothesize what we may be able to achieve, we take a leap forward, then in hindsight we study what we have done. When put like this, there is no wonder that many people are terrified of software agility. The tension between those who believe in logic and those who believe in experimentation is nothing new. Consider Gimbel’s take on this:

Where opponents of the method of hypothesis, like John Stuart Mill, argued that allowing blind guesses into the scientific game would introduce unfathomable degrees of error, error that could be avoided if only we stuck fast to the our logical machinery, [William] Whewell retorted that we learn valuable lessons from our mistakes, that without the mistakes, great advances might not be possible.

The tension between the logicians and the experimenters, I think, is based on the flawed belief that software development is about developing software. It is actually about knowledge acquisition.

Further Reading
Gimbel, S. (2011) Exploring the Scientific Method: Cases and Questions, University of Chicago Press.