A short history of AI


The Dartmouth meeting did not mark the beginning of scientific inquiry into machines which could think like people. Alan Turing, for whom the Turing prize is named, wondered about it; so did John von Neumann, an inspiration to McCarthy. By 1956 there were already a number of approaches to the issue; historians think one of the reasons McCarthy coined the term artificial intelligence, later AI, for his project was that it was broad enough to encompass them all, keeping open the question of which might be best. Some researchers favoured systems based on combining facts about the world with axioms like those of geometry and symbolic logic so as to infer appropriate responses; others preferred building systems in which the probability of one thing depended on the constantly updated probabilities of many others.

 


View Full Image

A-short-history-of-AI

The following decades saw much intellectual ferment and argument on the topic, but by the 1980s there was wide agreement on the way forward: “expert systems” which used symbolic logic to capture and apply the best of human know-how. The Japanese government, in particular, threw its weight behind the idea of such systems and the hardware they might need. But for the most part such systems proved too inflexible to cope with the messiness of the real world. By the late 1980s AI had fallen into disrepute, a byword for overpromising and underdelivering. Those researchers still in the field started to shun the term.

It was from one of those pockets of perseverance that today’s boom was born. As the rudiments of the way in which brain cells—a type of neuron—work were pieced together in the 1940s, computer scientists began to wonder if machines could be wired up the same way. In a biological brain there are connections between neurons which allow activity in one to trigger or suppress activity in another; what one neuron does depends on what the other neurons connected to it are doing. A first attempt to model this in the lab (by Marvin Minsky, a Dartmouth attendee) used hardware to model networks of neurons. Since then, layers of interconnected neurons have been simulated in software.

These artificial neural networks are not programmed using explicit rules; instead, they “learn” by being exposed to lots of examples. During this training the strength of the connections between the neurons (known as “weights”) are repeatedly adjusted so that, eventually, a given input produces an appropriate output. Minsky himself abandoned the idea, but others took it forward. By the early 1990s neural networks had been trained to do things like help sort the post by recognising handwritten numbers. Researchers thought adding more layers of neurons might allow more sophisticated achievements. But it also made the systems run much more slowly.

A new sort of computer hardware provided a way around the problem. Its potential was dramatically demonstrated in 2009, when researchers at Stanford University increased the speed at which a neural net could run 70-fold, using a gaming PC in their dorm room. This was possible because, as well as the “central processing unit” (cpu) found in all pcs, this one also had a “graphics processing unit” (gpu) to create game worlds on screen. And the gpu was designed in a way suited to running the neural-network code.

Coupling that hardware speed-up with more efficient training algorithms meant that networks with millions of connections could be trained in a reasonable time; neural networks could handle bigger inputs and, crucially, be given more layers. These “deeper” networks turned out to be far more capable.

The power of this new approach, which had come to be known as “deep learning”, became apparent in the ImageNet Challenge of 2012. Image-recognition systems competing in the challenge were provided with a database of more than a million labelled image files. For any given word, such as “dog” or “cat”, the database contained several hundred photos. Image-recognition systems would be trained, using these examples, to “map” input, in the form of images, onto output in the form of one-word descriptions. The systems were then challenged to produce such descriptions when fed previously unseen test images. In 2012 a team led by Geoff Hinton, then at the University of Toronto, used deep learning to achieve an accuracy of 85%. It was instantly recognised as a breakthrough.

By 2015 almost everyone in the image-recognition field was using deep learning, and the winning accuracy at the ImageNet Challenge had reached 96%—better than the average human score. Deep learning was also being applied to a host of other “problems…reserved for humans” which could be reduced to the mapping of one type of thing onto another: speech recognition (mapping sound to text), face-recognition (mapping faces to names) and translation.

In all these applications the huge amounts of data that could be accessed through the internet were vital to success; what was more, the number of people using the internet spoke to the possibility of large markets. And the bigger (ie, deeper) the networks were made, and the more training data they were given, the more their performance improved.

Deep learning was soon being deployed in all kinds of new products and services. Voice-driven devices such as Amazon’s Alexa appeared. Online transcription services became useful. Web browsers offered automatic translations. Saying such things were enabled by AI started to sound cool, rather than embarrassing, though it was also a bit redundant; nearly every technology referred to as AI then and now actually relies on deep learning under the bonnet.

In 2017 a qualitative change was added to the quantitative benefits being provided by more computing power and more data: a new way of arranging connections between neurons called the transformer. Transformers enable neural networks to keep track of patterns in their input, even if the elements of the pattern are far apart, in a way that allows them to bestow “attention” on particular features in the data.

Transformers gave networks a better grasp of context, which suited them to a technique called “self-supervised learning”. In essence, some words are randomly blanked out during training, and the model teaches itself to fill in the most likely candidate. Because the training data do not have to be labelled in advance, such models can be trained using billions of words of raw text taken from the internet.

Mind your language model

Transformer-based large language models (LLMs) began attracting wider attention in 2019, when a model called GPT-2 was released by OpenAI, a startup (GPT stands for generative pre-trained transformer). Such LLMs turned out to be capable of “emergent” behaviour for which they had not been explicitly trained. Soaking up huge amounts of language did not just make them surprisingly adept at linguistic tasks like summarisation or translation, but also at things—like simple arithmetic and the writing of software—which were implicit in the training data. Less happily it also meant they reproduced biases in the data fed to them, which meant many of the prevailing prejudices of human society emerged in their output.

In November 2022 a larger OpenAI model, GPT-3.5, was presented to the public in the form of a chatbot. Anyone with a web browser could enter a prompt and get a response. No consumer product has ever taken off quicker. Within weeks ChatGPT was generating everything from college essays to computer code. AI had made another great leap forward.

Where the first cohort of AI-powered products was based on recognition, this second one is based on generation. Deep-learning models such as Stable Diffusion and DALL-E, which also made their debuts around that time, used a technique called diffusion to turn text prompts into images. Other models can produce surprisingly realistic video, speech or music.

The leap is not just technological. Making things makes a difference. ChatGPT and rivals such as Gemini (from Google) and Claude (from Anthropic, founded by researchers previously at OpenAI) produce outputs from calculations just as other deep-learning systems do. But the fact that they respond to requests with novelties makes them feel very unlike software which recognises faces, takes dictation or translates menus. They really do seem to “use language” and “form abstractions”, just as McCarthy had hoped.

This series of briefs will look at how these models work, how much further their powers can grow, what new uses they will be put to, as well as what they will not, or should not, be used for.

© 2024, The Economist Newspaper Limited. All rights reserved. From The Economist, published under licence. The original content can be found on www.economist.com



Source link

Share

Latest Updates

Frequently Asked Questions

Related Articles

Running out of storage? This tiny flash drive is your phone’s new BFF

TL;DR: Keep every photo, video, and file without stressing over storage limits —...

elon musk: Elon Musk says xAI’s Grok 3 chatbot to be unveiled on Monday

Billionaire Elon Musk said on Saturday that his xAI's artificial intelligence chatbot and...

Government Fires Specialists Without Realizing They Were in Charge of Nuclear Bombs, Then Panics and Tries to Rehire Them

Paging Dr. StrangeloveWe've all made mistakes before. Maybe we did something in the...

A look under the hood of transfomers, the engine driving AI model evolution

Join our daily and weekly newsletters for the latest updates and exclusive content...
SULTAN88
SULTANSLOT
RAJA328
JOIN88
GFC88
HOKIBET
RUSIASLOT88
TAHU69
BONANZA99
PRAGMABET
MEGA55
LUXURY777
LUXURY333
BORJU89
QQGAMING
KEDAI168
MEGA777
NAGASLOT777
TAKSU787
KKSLOT777
MAS77TOTO
bandar55
BOS303
HOKI99
NUSA365
YUHUSLOT
KTP168
GALAXY138
NEXIA138
PETIR33
BOOM138
MEGA888
CABE888
FOSIL777
turbospin138
KAPAKBET
SUPERJP
sultankoin99
dragon88
raffi888
kenzobet
aladin666
rgo365
ubm4d
GERCEP88
VIVA99
CR777
VOXY88
delman567
intan69
CABE888
RNR303
LOGO303
PEMBURUGACOR
mpo383
cermin4d
bm88
ANGKA79
WOWHOKI
ROKET303
MPOXL
GURITA168
SUPRASLOT
SGCWIN
DESA88
ARWANA388
DAUNEMAS
ALADDIN666
BIOWIN69
SKY77
DOTA88
NAGA138
API5000
y200m
PLAYBOOK88
LUXURY12
A200M
MPO700
KENANGAN4D
cakrabola
PANDAGENDUT
MARVEL77
UG300
HOKI178
MONTE77
JASABOLA
UNTAR4D
LIDO88
MAFIABOLA77
GASPOL189
mpo999
untung138
TW88
JAGUAR33
MPOBOS
SHIO88
VIVO4D
MPOXL
JARISAKTI
BBO303
AONCASH
ANGKER4D
LEVIS4D
JAGO88
REPUBLIK365
BOSDEAL88
BOLA168
akunjp
WARTEGBET
EZEBET
88PULSA
KITAB4D
BOSDEAL88
STUDIOBET
MESINKOIN
BIMA88
PPNUSA
ABGBET88
TOP77
BAYAR77
YES77
BBTN4D
BBCA4D
VSLOTS88
MPO800
PAHALA4D
KPI4D
JURAGAN77
QQ188
BOLAPELANGI
C200M
QQ998
GWKTOGEL
MEGABANDAR
COLOWIN
VIP579
SEVEN4D
MPO188
DEWATA88
SURAT4D
SINAR123
LAMBO77
GUDANG4D
AWAN4D
PLANETLIGA
GT88
ROYALSPIN88
MAMAJITU
MITO99
PEDIA4D
WIBU69JP
333HOKI
SIDARMA88
NAGAEMAS99
HOLA88
CAKAR76
KINGTOTO
RATUGAMING
SSI168
PILAR168
ACTOTO
EYANGTOGEL
KAISAR328
SLOT628
KAISAR88
DOTA88
MAXWIN369
ALIBABA99
MM168
SQUAD777
NAGABET88
JAYABOLA
SEMPATIGAME
PANDAJAGO
PIKAT4D
SINGA77
YUYU33
MASTERPLAY99
VICTORY39
NASA4D
PERMATA55
SAKAUSLOT
CK303
MPOTOWER
CIPUTRABET
WINJUDI
DEWI5000
IYA777
MAHIRTOTO
GOSLOT88
TIPTOP4D
RAJA787
JBO680
JOKER188
EPICPLAY88
TRIVABET
KAISAR189
JOKER81
JPSPIN88
MAYORA4D
DJARUMPLAY
OVO88
BAKTI78
WINGSLOT77
ICAFE4D
PDTOTO
JETPLAY88
PORN VIDEO
https://link.space/@Hikaribet
https://bio.site/Hikaribet
https://heylink.me/Hikaribet39
CMBET88
CMBET88
didascaliasdelteatrocaminito.com
glenellynrent.com
gypsumboardequipment.com
realseller.org
https://harrysphone.com/upin
gyergyoalfalu.ro/tokek
vipokno.by/gokil
winjospg.com
winjos801.com/
www.logansquarerent.com
internationalfintech.com/bamsz
condowizard.ca
jawatoto889.com
hikaribet3.live
hikaribet1.com
heylink.me/hikaribet
www.nomadsumc.org
condowizard.ca/aromatoto
euro2024gol.com
www.imaracorp.com
daftarsekaibos.com
stuffyoucanuse.org/juragan
Toto Macau 4d
Aromatoto
Lippototo
Mbahtoto
Winjos
152.42.229.23
bandarlotre126.com
heylink.me/sekaipro
www.get-coachoutletsonline.com
wholesalejerseyslord.com
Lippototo
Zientoto
Lippototo
Situs Togel Resmi
Fajartoto
Situs Togel
Toto Macau
Winjos
Winlotre
Aromatoto
design-develop-test.com
winlotre.online
winlotre.xyz
winlotre.us
winlotrebandung.com
winlotrepalu.com
winlotresurabaya.shop
winlotrejakarta.com
winlotresemarang.shop
winlotrebali.shop
winlotreaceh.shop
winlotremakmur.com
Dadu Online
Taruhantoto
a Bandarlotre
bursaliga
lakitoto
aromatoto
Rebahin
untungslot.pages.dev
slotpoupler.pages.dev
rtpliveslot88a.pages.dev
tipsgameslot.pages.dev
pilihslot88.pages.dev
fortuertiger.pages.dev
linkp4d.pages.dev
linkslot88a.pages.dev
slotpgs8.pages.dev
markasjudi.pages.dev
saldo69.pages.dev
slotbenua.pages.dev
saingtoto.pages.dev
markastoto77.pages.dev
jowototo88.pages.dev
sungli78.pages.dev
volatilitas78.pages.dev
bonusbuy12.pages.dev
slotoffiline.pages.dev
dihindari77.pages.dev
rtpdislot1.pages.dev
agtslot77.pages.dev
congtoto15.pages.dev
hongkongtoto7.pages.dev
sinarmas177.pages.dev
hours771.pages.dev
sarana771.pages.dev
kananslot7.pages.dev
balitoto17.pages.dev
jowototo17.pages.dev
aromatotoding.com
unyagh.org
fairparkcounseling.com/gap/
impress-newtex.com/ajax/
SULTAN88
SULTANSLOT
RAJA328
JOIN88+
HOKIBET
GFC88
RusiaSlot88
Tahu69
BONANZA99
Pragmabet
mega55
luxury777
luxury333
borju89
qqgaming
KEDAI168
mega777
nagaslot777
TAKSU787
kkslot777
MAS77TOTO
BANDAR55+
BOS303
Login-HOKI99/
NUSA365
YUHUSLOT
ktp168
GALAXY138