THE SP THEORY OF INTELLIGENCE AND ITS REALISATION IN THE SP COMPUTER MODEL

COMPUTING AS COMPRESSION

Too blue for logic

My axioms were so clean-hewn,
The joins of 'thus' and 'therefore' neat
But, I admit
Life would not fit
Between straight lines
And all the cornflowers said was 'blue,'
All summer long, so blue.
So when the sea came in and with one wave
Threatened to wash my edifice away -
I let it.

Marianne Jones

SP GROUP: If you would like to receive occasional news about progress in the SP programme of research, please contact me, Gerry Wolff, via email or phone. There is information about the SP group here: SP GROUP.

NOTE: It may seem odd that, amongst the publications detailed below, there is a relatively big gap between the publication of the book Unifying Computing and Cognition (tinyurl.com/mw26xtdh) in 2006 and later publications beginning with The SP Theory of Intelligence: an overview (PDF, Information, 4 (3), 283-341, 2013, bit.ly/1NOMJ6l) in 2013. This is because, from early 2006 to late 2012, I was working full time on campaigns related to climate change. Because of the large carbon footprint from flying, I'm trying to avoid it as much as possible. Online conferences also have a carbon footprint but that is easier to fix than the carbon footprint from flying.

Overview:

• Key publications, with notes.
• There is an overview of the SP programme of research in a four-page leaflet (PDF) prepared by Research Outreach.

Notes:

• Documents, listed below, which are peer-reviewed and published, are marked 'PRP'.
• All the peer-reviewed, published papers listed on this page are 'open access' and can be read without charge or other impediment.
• There is a download link for every document except the book Unifying Computing and Cognition (tinyurl.com/mw26xtdh).

The SP system

• SP early version: PRP "Simplicity and power—some unifying ideas in computing" (PDF, The Computer Journal, 33(6), 1990. tinyurl.com/2p8zuy93). See also Early work.
• SP introduction: "Biochemistry provides inspiration for a new kind of AI" (PDF, tinyurl.com/473dawp8).
• Extended overview: PRP "The SP Theory of Intelligence: an overview " (PDF, Information, 4 (3), 283-341, 2013, bit.ly/1NOMJ6l).
• Strengths and potential of the SP System:
  • PRP "The SP Theory of Intelligence, and its realisation in the SP Computer Model, as a foundation for the development of artificial general intelligence", Analytics, 2023, 2(1), 163-197, tinyurl.com/yts34bvh . The theme of this paper is that the SP Theory of Intelligence (SPTI), and its realisation in the SP Computer Model (SPCM), is a promising Foundation for the Development of Artificial General Intelligence (FDAGI). Evidence is presented showing that the SPTI has substantial advantages as an FDAGI compared with six alternatives. This is a major advance, challenging the pervasive assumption that deep neural networks are the way forward for the development of AGI.
    
  • PRP "Twenty significant problems in AI research, with potential solutions via the SP Theory of Intelligence and its realisation in the SP Computer Model", Foundations, 2022, 2(4), 1045-1079, doi.org/10.3390/foundations2040070. This paper describes the clear potential of the SP System to help solve 20 problems in AI research, most of them described by influential experts in AI, in interviews with science writer Martin Ford, reported by him in his book Architects of Intelligence.
    
• Book: Unifying Computing and Cognition: the SP Theory and Its Applications (tinyurl.com/mw26xtdh).
• Tutorial:
  The SP-Multiple-Alignment concept
  • Article: "The potential of the SP-Multiple-Alignment concept as the 'double helix' of intelligence". This paper was presented as "Spotlight Research" at the SuperAGI, AGI-Leap Summit virtual conference on 2024-02-24 (tinyurl.com/mu7yj5z6).
  • Slide show: " href="http://www.cognitionresearch.org/papers/double-helix/sp-double-helix-slides.pdf"> The SP-Multiple-Alignment concept within the SP Theory of Intelligence" (tinyurl.com/4bnx5w4t).
  The SP Theory of Intelligence
  • Book: "Intelligence Via Compression of Information" This draft of a book is in broadly the right shape but not yet final. (tinyurl.com/2sp6uz6j).
  • Article: "Intelligence Via Compression of Information". This is a shortened version of the book. (tinyurl.com/3p42uxhw)
  • Papers: "Intelligence Via Compression of Information (2023)", with download links, tinyurl.com/2a5br5rf.
  • Foundations:
    • PRP "Information compression as a unifying principle in human learning, perception, and cognition," Complexity, vol. 2019, Article ID 1879746, 38 pages, 2019, DOI: doi.org/10.1155/2019/1879746 (PDF: bit.ly/2GdIqnY).
    • PRP "The SP-multiple-alignment concept as a generalization of six other variants of information compression via the matching and unification of patterns," Journal of Pure and Applied Mathematics, Volume 7, Number 4, Pages, 246-260, 2023. (PDF: tinyurl.com/2p37pcp2).
    • "Information compression, intelligence, computing, and mathematics (slides)" (PDF, slides).

• PRP Distinctive features and advantages: "The SP Theory of Intelligence: distinctive features and advantages" (PDF, IEEE Access, 4, 216-246, 2016, bit.ly/2qgq5QF). See also Awards for IEEE Access, and IEEE Access received an impact factor of 3.244 in the 2016 JCR release.
• Advantages of the SP system compared with deep learning: "Solutions to problems with deep learning" (PDF, bit.ly/2AJzu4j).
• Generalisation of the SP-multiple-alignment concept: "Information Compression via the Matching and Unification of Patterns (ICMUP) as a foundation for AI" (PDF, tinyurl.com/2p9asr47).
• SP-Neural:
  • PRP "Information compression, multiple alignment, and the representation and processing of knowledge in the brain" (PDF, Frontiers in Psychology, 7, 1584, 2016, bit.ly/2esmYyt).
  • Keynote presentation via webinar at the "7th International Conference on Neuroscience and Neurological Disorders" held during June 08-09, 2020:
    • Slides: streaming (via Google docs and browser) and download (PDF). These are largely self-explanatory but there is more information in the notes.
    • Notes to accompany slides.

Potential benefits and applications

• Overview of potential benefits and applications: PRP "The SP Theory of Intelligence: benefits and applications" (PDF, Information, 5 (1), 1-27, 2014, bit.ly/1FRYwew).
• Autonomous robots: PRP "Autonomous robots and the SP Theory of Intelligence" (PDF, IEEE Access, 2, 1629-1651, 2014, bit.ly/18DxU5K). See also Awards for IEEE Access, and IEEE Access received an impact factor of 3.244 in the 2016 JCR release.
• Big data:
  • PRP "Big data and the SP Theory of Intelligence" (PDF, IEEE Access, 2, 301-315, 2014, bit.ly/2qfSR3G). See also Awards for IEEE Access, and IEEE Access received an impact factor of 3.244 in the 2016 JCR release.
    • PRP This paper, with minor revisions, is reproduced in "Big Data: Storage, Sharing, and Security" (Fei Hu (Ed.), CRC Press, 2016).
    • Slides: "Big data and the SP Theory of Intelligence" (PDF).
  • "The curse of variety in computing, and what can be done about it" (PDF, bit.ly/2jsXaEG).
  • Oiling the wheels under big data. How big gains in efficiency may be achieved in transmitting big data from one place to another.
    • "Oiling the wheels under big data" (PDF, for non specialists).
    • "Oiling the wheels under big data" (PDF, more technical).
• Commonsense reasoning: "Commonsense reasoning, commonsense knowledge, and the SP Theory of Intelligence" (PDF, bit.ly/2T0UnRz, submitted for publication).
• Crime prevention and detection: "The SP system as an aid to crime prevention and detection" (CPD), slide show):
  • PPTX.
  • PDF.
• Defence: "The SP system and its potential applications in defence" (slide show):
  • PPTX.
  • PDF
• Intelligent databases: PRP "Towards an intelligent database system founded on the SP theory of computing and cognition" (PDF, Data & Knowledge Engineering, 60, 596-624, 2007, bit.ly/1CUldR6).
• Mathematics: PRP "Mathematics as information compression via the matching and unification of patterns," Complexity, vol. 2019, Article ID 6427493, 25 pages, 2019, DOI: doi.org/10.1155/2019/6427493 (PDF, bit.ly/2LqUHIr).
• Medical diagnosis: PRP "Medical diagnosis as pattern recognition in a framework of information compression by multiple alignment, unification and search" (PDF, Decision Support Systems, 42, 608-625, 2006, bit.ly/1F366o7).
• Natural language processing: "Interpreting Winograd Schemas via the SP Theory of Intelligence and its realisation in the SP Computer Model" (PDF, bit.ly/2ME8DOA, submitted for publication).
• Science: "How the SP Theory of Intelligence and its realisation in the SP Computer Model may be applied with advantage in science" (PDF, tinyurl.com/42j2vczu).
• Self-driving vehicles: "A proposed solution to problems in the development of the knowledge needed by self-driving vehicles" (PDF, tinyurl.com/n6vpxcuf, submitted for publication).
• Software engineering: "Software engineering and the SP Theory of Intelligence" (PDF, bit.ly/2w99Wzq, submitted for publication).
• Sustainability: PRP "How the SP system may promote sustainability in energy consumption in IT systems" (PDF, Sustainability 2021, 13 (8), article number 4565, 2021. doi.org/10.3390/su13084565, tinyurl.com/2ccs5z3b.).
• Transmission of information: "In the transmission of information, the great potential of model-based coding with the SP theory of artificial intelligence"(PDF, bit.ly/2n5o7OB).
• Transparency: PRP "Transparency and granularity in the SP Theory of Intelligence and its realisation in the SP Computer Model" (PDF, tinyurl.com/2f5hca8d, Published in the book Interpretable Artificial Intelligence: A Perspective of Granular Computing, Witold Pedrycz and Shyi-Ming Chen (editors), Springer: Heidelberg, 2021, ISBN 978-3-030-64948-7, DOI: 10.1007/978-3-030-64949-4).
• Vision, both artificial and natural. The SPS opens up a new approach to the development of computer vision and its integration with other aspects of intelligence, and it throws light on several aspects of natural vision:
• PRP "Application of the SP Theory of Intelligence to the understanding of natural vision and the development of computer vision," SpringerPlus, 3(1), 552-570, 2014.
• PRP "The potential of the SP system in machine learning and data analysis for image processing," (PDF, Big Data and Cognitive Computing, 5(1), 7, 2021, DOI: 10.3390/bdcc5010007).

 Development of the SP Machine

• Programme of development: PRP "A roadmap for the development of the 'SP Machine' for artificial intelligence" (PDF, V. Palade and J G Wolff, The Computer Journal, Volume 62, Issue 11, November 2019, Pages 1584-1604, DOI: doi.org/10.1093/comjnl/bxy126, PDF: bit.ly/2Vu0M9Q).
• ICMUP and SP-multiple-alignment: 'Information Compression via the Matching and Unification of Patterns' (ICMUP) as a foundation for AI (PDF).
• Research proposal:
• Short version: Short proposal for the development of the SP machine (PDF).
• Long version: Proposal for the creation of a research facility for the development of the SP machine (PDF)

Conference papers

• International Conference on Neuroscience and Neurological Disorders 2020, Keynote Speaker
  • "The potential of the SP-Multiple-Alignment concept as the "double helix" of intelligence" (PDF)
  • Information Compression and the Representation and Processing of Knowledge in the Brain, Abstract (PDF)
• IEEE IntelliSys 2017
  • Paper: PRP "Information compression, multiple alignment, and intelligence" (PDF, presented at IEEE IntelliSys 2017, 7-8 September, London).
  • Slides: PPX, PDF.

Media:

• Articles for people with a science background.
• Articles for non-specialists.
• Tweet references: short HTML pages expanding on tweets.
• Press releases.
• Reports about the SP system.
• Comments on published articles: Nature 538, 2016-10-06.

Software:

• SP71-console.zip.

INTRODUCTION

The SP Theory of Intelligence is a unique attempt to integrate and simplify observations and concepts across artificial intelligence, mainstream computing, mathematics, and human perception and cognition, with information compression as a unifying theme.

The name "SP" is short for simplicity and power, because compression of any given body of information, I, may be seen as a process of reducing informational “redundancy” in I and thus increasing its “simplicity”, whilst retaining as much as possible of its non-redundant expressive “power”.

Central in the theory is the powerful concept of multiple alignment, with versatility in the representation of knowledge and in AI functions.

The SP theory is realised in the form of a computer model, SP71. This model may be seen as a first version of the SP machine, an expression of the SP theory and a means for it to be applied. Source code for the model, with associated files, may be downloaded via links under the heading 'SOURCE CODE', below.

The SP theory combines conceptual simplicity with descriptive and explanatory power in several areas, including concepts of 'computing', the representation of knowledge, natural language processing, pattern recognition, several kinds of reasoning, the storage and retrieval of information, planning and problem solving, unsupervised learning, information compression, and human perception and cognition.

An introduction to the ideas is in the slide show The SP Theory of Intelligence (PDF)—see below. A much fuller account is in the book Unifying Computing and Cognition (tinyurl.com/mw26xtdh).

The background and motivation for the research is described in Computing, cognition and information compression (PDF).

PRESENTATIONS

• Information compression, intelligence, computing, and mathematics (slides) (PDF). This is about background observations and ideas that provide the foundations of the SP theory.
• The SP Theory of Intelligence (slides) (PDF). This talk describes the SP theory, the multiple alignment concept, and the SP machine, and potential benefits and applications. As a supplement to this talk, there are additional examples of multiple alignments (PDF), with notes.
• Big data and the SP Theory of Intelligence (slides) (PDF). These slides are based on Big data and the SP Theory of Intelligence (PDF, J G Wolff, IEEE Access, 2, 301-315, 2014).

Unless it is self-explanatory, each slide has one or more notes, each one shown with a 'speech bubble' icon, normally in the top left-hand corner. To see a note, position the cursor over the icon. If the whole note is not visible, right-click on the icon and choose "Open pop-up note".

To view a presentation, it is probably best to download the file and open it in the Adobe Reader. Other systems may not show the notes properly. Controls: Full screen: CTRL-L; Escape from full screen: ESC; Zoom in: CTRL-plus; Zoom out: CTRL-minus; Next slide: left-click; Previous slide: right-click; Scrolling left or right, up or down: use the 'hand'.

An article, giving an overview of the SP system and its expected benefits and applications is here: The SP Theory of Intelligence: benefits and applications (PDF).

PUBLICATIONS

Most of the publications from this section may be obtained as a PDF, Postscript or MS Word file. The articles that give the best overall view of the research programme and the SP framework are marked with . For readers who are not previously acquainted with this field, these are the articles to start with. For convenience, they are listed here in a suggested order for reading:

• Overview.
• Background thinking.
• Articles about syntax, parsing and production of natural language.
• Probabilistic reasoning in the SP framework.
• 'Computing' as information compression.
• Unsupervised learning.

Publications topics:

• Overview
• Natural language processing
• Probabilistic reasoning
• Pattern recognition (including medical diagnosis viewed as pattern recognition)
• Interpreting computing, mathematics and logic within the SP framework
• Medical applications
• Applications in the Semantic Web
• Unsupervised learning
• Realisation of the SP framework with neural mechanisms
• Software engineering
• Databases and information retrieval
• Background
• Conference papers about the SP framework
• Miscellaneous articles relating to the SP framework
• Early work

BOOK

Ebook cover	Print edition cover

Description of the book

Ordering the book

PROPOSAL FOR THE CREATION OF A HIGH-PARALLEL IMPLEMENTATION OF THE SP MACHINE

A good step forward in the development of these ideas would be the creation of a new version of the SP machine. It would be:

• Based on the SP71 computer model.
• Built as a software virtual machine, using high-parallel search mechanisms, and with a good user interface. The high-parallel search mechanisms in an existing search engine would provide a good foundation for this version of the SP machine.
• An open-source model, available, via the web, to the research community everywhere.
• A vehicle for experimentation and research with two main aims:
  • To explore the range of applications of the SP machine.
  • To refine the SP theory itself.

SOURCE CODE

If you wish to see the exact workings of the SP71 computer model or experiment with it, the source code and a Windows executable is available in the file SP71.ZIP, and via "Ancillary files" under www.arxiv.org/abs/1306.3888. A slightly earlier version, SP70, referenced in published papers and in Unifying Computing and Cognition (tinyurl.com/mw26xtdh), may be downloaded via SP70.ZIP.

Before looking at the detailed workings of SP71, it is probably best to understand the thinking behind it, described in Unifying Computing and Cognition (tinyurl.com/mw26xtdh) and The SP Theory of Intelligence: an overview (PDF, J G Wolff, Information, 4 (3), 283-341, 2013, bit.ly/1NOMJ6l). Also, there is a relatively high level description of the workings of the program in Sections 3.9, 3.10, and 9.2 in Unifying Computing and Cognition (tinyurl.com/mw26xtdh).

It is probably best to start by looking at SP62, a subset of SP71 that lacks any ability for learning. The source code and a Windows executable is available in the file SP62.ZIP.

Please note: these programs are for research, they are not shrink-wrapped software packages that you might buy from a shop. That said, there are detailed comments in the source code which should help to clarify how the programs works. I have released them mainly for people who want to dig around inside them and see what is going on. We aim to develop a high-parallel version of the "SP machine" with a user-friendly interface. This should make things easier for anyone who wants to try out the system or improve it.

This software is released to the public domain without any restrictions, but please read the notice on the source files.

ARCHIVING

Most of the papers from this programme of research, both those which have not yet been published and those that have been published, are archived in https://arxiv.org/. To find these papers, search for author: Wolff_J_G in that archive.

The moderators of arxiv.org would not accept some papers, saying that they did not fit any of the arxiv categories, or that there was too much overlap with my other papers. These papers are:

• On the "mysterious" effectiveness of mathematics in science (PDF, bit.ly/2otrHD0).
• Towards a new mathematics for science (PDF, bit.ly/2o1pr8p).
• Information compression as a unifying principle in human learning, perception, and cognition (PDF, bit.ly/2ruLnrV).
• Strengths and potential of the SP Theory of Intelligence in general, human-like artificial intelligence (PDF, bit.ly/2z3rM8b).
• And later papers.

These papers are now archived with http://vixra.org/ (search under "Full Site" for "J Gerard Wolff") and https://hal.archives-ouvertes.fr/ (Search for "J Gerard Wolff").

From January 2019, http://vixra.org/ and https://hal.archives-ouvertes.fr/ are the preferred archives because of the simplicity of archiving PDF files instead of TEX source files (with PDF figures) that have to be processed.

The SP71 computer model is archived here:

• As "ancillary files" with "The SP Theory of Intelligence: an overview" in https://arxiv.org/abs/1306.3888. Some but not all of the notes associated with this project are archived here.

A much fuller record of the research, including all versions of the software, all the associated notes, and all the publications, has been sent for digital archiving, with agreement of relevant people in each organisation, to:

• The National Museum of Computing at Bletchley Park, near Milton Keynes, UK. With the agreement of the archivist, Brian Aldous, a memory stick with a covering letter was sent to him on 24th of May 2016.
• Bangor University Library, Bangor University, North Wales, UK:
  • Files were deposited with the librarian about 2005.