COGNITION AS COMPRESSION

STOP PRESS: Key publications with brief notes.

From as long ago as the 19th century it has been recognised that information processing in brains and nervous systems (perception, learning, thinking, neuro-muscular control etc) may be governed by principles of economy or compression. Similar principles appear in the design of some of the current generation of artificial neural nets.

Although some researchers continued to develop these ideas, they went out of fashion for many years and have only recently begun to attract renewed attention. Part of the reason for preparing this page is to encourage more interest in 'cognitive economy' and principles of information compression which have been recognised by neurophysiologists for some time, especially the importance of inhibition in nervous tissue as a means of extracting redundancy from information. G Von Békésy's book on Sensory Inhibition (Princeton, 1967) deserves a wider audience. Concepts of economical coding in the nervous system have been a theme in Horace Barlow's work for many years. Also good value is Zipf's Human Behaviour and the Principle of Least Effort (1949), despite the passage of years and despite the fact that it was written before seminal concepts of 'information theory', introduced by Hartley and Shannon, became widely known.

Some relevant work is reviewed in Computing, cognition and information compression and Computing as compression: an overview of the SP theory and system.

Nice reviews of some of the issues and many pointers to relevant sources can be found in two articles by Nick Chater: Reconciling simplicity and likelihood principles in perceptual organization (Psychological Review 103 (3), pp. 566-581, 1996) and The search for simplicity: a fundamental cognitive principle? (The Quarterly Journal of Experimental Psychology A 52 (2), 273-302, 1999).

Principles of Minimum Length Encoding, and their possible role in explaining how a child may learn his or her first language, have been examined in a program of research on Language Learning as Compression. There is solid evidence that first language learning by children may, to a large extent, be understood as information compression (see the publications that may be obtained from that page, especially Learning syntax and meanings through optimization and distributional analysis).

'Cognition as compression' has provided the inspiration for a programme of research on Computing as Compression, developing the conjecture that all kinds of computing and formal reasoning may usefully be understood as information compression by pattern matching, unification and search - and developing a 'new generation' computing system based on the theory. Much of this research is relevant to understanding human cognition as described in two 'overview' articles (Multiple alignment, cognition and computing (short) and Information compression by multiple alignment, unification and search as a unifying principle in computing and cognition (long)) and in several other publications.

'Cognition and computing as compression' is related to work on Algorithmic Information Theory (AIT) and Minimum Length Encoding (MLE) (references in the sources cited above) but there are important differences.

"Fascinating idea! All that mental work I've done over the years, and what have I got to show for it? A goddamned zipfile! Well, why not, after all?"