One of my first jobs was helping build an expert system for a, even today, complex computational linguistics problem. The company had a rich corporate library full of academic books on expert systems, decision trees, first gen (pre-winter) AI, and some early books on early ML approaches. I remember seeing this book in particular and its evocative title caused me to look deeper into the library than I would have normally.
Our core system was built of thousands upon thousands of hand-crafted rules informed by careful statistical analysis of hundreds of millions of entries in a bulk data system.
Part of my job was to build the system that analyzed the bulk data and produced the stats, and the other part was carefully testing and fixing the rulesets for certain languages. It was mind-numbing work, and looking back we were freakishly close to all the bit and pieces needed for then bleeding-edge ML had we chosen to go that way.
However, we chose expert systems because it gave us tremendous insight into what was happening, and the opportunity to debug and test things at an incredibly granular scale. It was fully possible to say "the system has this behavior because of xyz" and it was fully possible to tune the system at individual character levels of finesse.
Had we wanted to dive into ML, we could have used this foundation as a bootstrap into building a massive training set. But the founders biased towards expert systems and I think, at the time, it was the right choice.
The technology was acquired, and I wonder if the current custodians use it for those obvious next-step purposes.
Back when expert systems were relevant, we had nowhere near the amount of compute that's basically required for modern day ML. Even if the data somehow was in place for it.
If a book or movie is ever made about the history of AI, the script would include this period of AI history and would probably go something like this…
(Some dramatic license here, sure. But not much more than your average "based on true events" script.)
In 1957, Frank Rosenblatt built a physical neural network machine called the Perceptron. It used variable resistors and reconfigurable wiring to simulate brain-like learning. Each resistor had a motor to adjust weights, allowing the system to "learn" from input data. Hook it up to a fridge-sized video camera (20x20 resolution), train it overnight, and it could recognize objects. Pretty wild for the time.
Rosenblatt was a showman—loud, charismatic, and convinced intelligent machines were just around the corner.
Marvin Minsky, a jealous academic peer of Frank, was in favor of a different approach to AI: Expert Systems. He published a book (Perceptrons, 1969) which all but killed research into neural nets. Marvin pointed out that no neural net with a depth of one layer could solve the "XOR" problem.
While the book's findings and mathematical proof were correct, they were based on incorrect assumptions (that the Perceptron only used one layer and that algorithms like backpropagation did not exist).
As a result, a lot of academic AI funding was directed towards Expert Systems.
The flagship of this was the MYCIN project. Essentially, it was a system to find the correct antibiotic based on the exact bacteria a patient was infected with. The system thus had knowledge about thousands and thousands of different diseases with their associated symptoms. At the time, many different antibiotics existed, and using the wrong one for a given disease could be fatal to the patient.
When the system was finally ready for use... after six years (!), the pharmaceutical industry had developed “broad-spectrum antibiotics,” which did not require any of the detailed analysis MYCIN was developed for.
The period of suppressing Neural Net research is now referred to as (one of) the winter(s) of AI.
--------
As said, that is the fictional treatment. In reality, the facts, motivations, and behavior of the characters are a lot more nuanced.
"AI" is too much of a broad umbrella term of competing ideas, from symbolic logic (FOL, expert systems) to statistical operations (NNs). It's clear today that the latter has won the race, but ignoring this history doesn't seem to be a very smart move.
I'm in no way an expert but I feel that today's LLMs lack some concepts well known in the research of logical reasoning. Something like: semantic.
MYCIN was a high profile project, but rule engines in this vein come in handy quite often. With some basic Prolog knowledge and a decent implementation pretty much any set of data files can quickly be turned into an ad hoc 'expert system' lite.
The simplest example and the one I usually bring up is log files, where the primary delimiter is \n and the secondary is likely some whitespace, which can easily be replaced with Prolog delimiters and a bit of decoration. This turns the data into Prolog code which can be consulted as is and complemented with rules abstracting complex queries.
6 comments
[ 3.3 ms ] story [ 26.3 ms ] threadOur core system was built of thousands upon thousands of hand-crafted rules informed by careful statistical analysis of hundreds of millions of entries in a bulk data system.
Part of my job was to build the system that analyzed the bulk data and produced the stats, and the other part was carefully testing and fixing the rulesets for certain languages. It was mind-numbing work, and looking back we were freakishly close to all the bit and pieces needed for then bleeding-edge ML had we chosen to go that way.
However, we chose expert systems because it gave us tremendous insight into what was happening, and the opportunity to debug and test things at an incredibly granular scale. It was fully possible to say "the system has this behavior because of xyz" and it was fully possible to tune the system at individual character levels of finesse.
Had we wanted to dive into ML, we could have used this foundation as a bootstrap into building a massive training set. But the founders biased towards expert systems and I think, at the time, it was the right choice.
The technology was acquired, and I wonder if the current custodians use it for those obvious next-step purposes.
If a book or movie is ever made about the history of AI, the script would include this period of AI history and would probably go something like this…
(Some dramatic license here, sure. But not much more than your average "based on true events" script.)
In 1957, Frank Rosenblatt built a physical neural network machine called the Perceptron. It used variable resistors and reconfigurable wiring to simulate brain-like learning. Each resistor had a motor to adjust weights, allowing the system to "learn" from input data. Hook it up to a fridge-sized video camera (20x20 resolution), train it overnight, and it could recognize objects. Pretty wild for the time.
Rosenblatt was a showman—loud, charismatic, and convinced intelligent machines were just around the corner.
Marvin Minsky, a jealous academic peer of Frank, was in favor of a different approach to AI: Expert Systems. He published a book (Perceptrons, 1969) which all but killed research into neural nets. Marvin pointed out that no neural net with a depth of one layer could solve the "XOR" problem.
While the book's findings and mathematical proof were correct, they were based on incorrect assumptions (that the Perceptron only used one layer and that algorithms like backpropagation did not exist).
As a result, a lot of academic AI funding was directed towards Expert Systems. The flagship of this was the MYCIN project. Essentially, it was a system to find the correct antibiotic based on the exact bacteria a patient was infected with. The system thus had knowledge about thousands and thousands of different diseases with their associated symptoms. At the time, many different antibiotics existed, and using the wrong one for a given disease could be fatal to the patient.
When the system was finally ready for use... after six years (!), the pharmaceutical industry had developed “broad-spectrum antibiotics,” which did not require any of the detailed analysis MYCIN was developed for.
The period of suppressing Neural Net research is now referred to as (one of) the winter(s) of AI.
--------
As said, that is the fictional treatment. In reality, the facts, motivations, and behavior of the characters are a lot more nuanced.
I'm in no way an expert but I feel that today's LLMs lack some concepts well known in the research of logical reasoning. Something like: semantic.
Code:
https://github.com/norvig/paip-lisp
Book
https://archive.org/details/github.com-norvig-paip-lisp_-_20...
As for the interpreter, SBCL works fine everywhere; if not, pick ECL and CCL.
The simplest example and the one I usually bring up is log files, where the primary delimiter is \n and the secondary is likely some whitespace, which can easily be replaced with Prolog delimiters and a bit of decoration. This turns the data into Prolog code which can be consulted as is and complemented with rules abstracting complex queries.
Something similar can be done with JSON files.
https://www.scryer.pl/
https://www.metalevel.at/prolog/dcg
https://www.metalevel.at/prolog/expertsystems