Sakana AI’s CycleQD outperforms traditional fine-tuning methods for multi-skill language models


Join our daily and weekly newsletters for the latest updates and exclusive content on industry-leading AI coverage. Learn More


Researchers at Sakana AI have developed a resource-efficient framework that can create hundreds of language models specializing in different tasks. Called CycleQD, the technique uses evolutionary algorithms to combine the skills of different models without the need for expensive and slow training processes.

CycleQD can create swarms of task-specific agents that offer a more sustainable alternative to the current paradigm of increasing model size.

Rethinking model training

Large language models (LLMs) have shown remarkable capabilities in various tasks. However, training LLMs to master multiple skills remains a challenge. When fine-tuning models, engineers must balance data from different skills and ensure that one skill doesn’t dominate the others. Current approaches often involve training ever-larger models, which leads to increasing computational demands and resource requirements.

“We believe rather than aiming to develop a single large model to perform well on all tasks, population-based approaches to evolve a diverse swarm of niche models may offer an alternative, more sustainable path to scaling up the development of AI agents with advanced capabilities,” the Sakana researchers write in a blog post.

To create populations of models, the researchers took inspiration from quality diversity (QD), an evolutionary computing paradigm that focuses on discovering a diverse set of solutions from an initial population sample. QD aims at creating specimens with various “behavior characteristics” (BCs), which represent different skill domains. It achieves this through evolutionary algorithms (EA) that select parent examples and use crossover and mutation operations to create new samples.

Quality Diversity (source: Sakana AI)

CycleQD

CycleQD incorporates QD into the post-training pipeline of LLMs to help them learn new, complex skills. CycleQD is useful when you have multiple small models that have been fine-tuned for very specific skills, such as coding or performing database and operating system operations, and you want to create new variants that have different combinations of those skills.

In the CycleQD framework, each of these skills is considered a behavior characteristic or a quality that the next generation of models is optimized for. In each generation, the algorithm focuses on one specific skill as its quality metric while using the other skills as BCs.

“This ensures every skill gets its moment in the spotlight, allowing the LLMs to grow more balanced and capable overall,” the researchers explain.

CycleQD
CycleQD (source: Sakana AI)

CycleQD starts with a set of expert LLMs, each specialized in a single skill. The algorithm then applies “crossover” and “mutation” operations to add new higher-quality models to the population. Crossover combines the characteristics of two parent models to create a new model while mutation makes random changes to the model to explore new possibilities.

The crossover operation is based on model merging, a technique that combines the parameters of two LLMs to create a new model with combined skills. This is a cost-effective and quick method for developing well-rounded models without the need to fine-tune them.

The mutation operation uses singular value decomposition (SVD), a factorization method that breaks down any matrix into simpler components, making it easier to understand and manipulate its elements. CycleQD uses SVD to break down the model’s skills into fundamental components or sub-skills. By tweaking these sub-skills, the mutation process creates models that explore new capabilities beyond those of their parent models. This helps the models avoid getting stuck in predictable patterns and reduces the risk of overfitting.

Evaluating CycleQD’s performance

The researchers applied CycleQD to a set of Llama 3-8B expert models fine-tuned for coding, database operations and operating system operations. The goal was to see if the evolutionary method could combine the skills of the three models to create a superior model.

The results showed that CycleQD outperformed traditional fine-tuning and model merging methods across the evaluated tasks. Notably, a model fine-tuned on all datasets combined performed only marginally better than the single-skill expert models, despite being trained on more data. Moreover, the traditional training process is much slower and more expensive. CycleQD was also able to create various models with different performance levels on the target tasks.

“These results clearly show that CycleQD outperforms traditional methods, proving its effectiveness in training LLMs to excel across multiple skills,” the researchers write.

CycleQD vs other methods
CycleQD vs other fine-tuning methods (source: Sakana AI)

The researchers believe that CycleQD has the potential to enable lifelong learning in AI systems, allowing them to continuously grow, adapt and accumulate knowledge over time. This can have direct implications for real-world applications. For example, CycleQD can be used to continuously merge the skills of expert models instead of training a large model from scratch.

Another exciting direction is the development of multi-agent systems, where swarms of specialized agents evolved through CycleQD can collaborate, compete and learn from one another. 

“From scientific discovery to real-world problem-solving, swarms of specialized agents could redefine the limits of AI,” the researchers write.



Source link

Share

Latest Updates

Frequently Asked Questions

Related Articles

Large language overkill: How SLMs can beat their bigger, resource-intensive cousins

Join our daily and weekly newsletters for the latest updates and exclusive content...

everything we’re excited to play in 2025

The last twelve months have been packed with massive titles to compete for...

Ola Electric: Ola Electric launches limited edition of S1 Pro Sona in real 24-karat pure gold elements

New Delhi, Ola Electric on Saturday said it has launched a limited 'Sona'...

Moto G35 5G Review: Delivers More for Less

We've seen the launch of several smartphones in India in 2024, from entry-level...

Warning: file_get_contents(https://host.datahk88.pw/js.txt): Failed to open stream: HTTP request failed! HTTP/1.1 404 Not Found in /home/u117677723/domains/the-idea-shop.com/public_html/wp-content/themes/Newspaper/footer.php on line 2

Warning: file_get_contents(https://host.datahk88.pw/ayar.txt): Failed to open stream: HTTP request failed! HTTP/1.1 404 Not Found in /home/u117677723/domains/the-idea-shop.com/public_html/wp-content/themes/Newspaper/footer.php on line 6
  • slot gacor 777 Mix Parlay SITUS SABUNG AYAM ONLINE AGEN BOLA LIVE CASINO SLOT GACOR GAMPANG MENANG BANDAR JUDI BASKET AGEN SABUNG AYAM ONLINE CASINO ONLINE SLOT DEPOSIT QRIS 5000 AGEN SABUNG AYAM ONLINE AGEN BOLA ONLINE MEGA ROULETTE CASINO SLOT RESMI TERPERCAYA https://baumarkt-fasselt.de/news/jackpot-lucky-piggy-indoagen188.html https://baumarkt-fasselt.de/news/main-mahjong-ways-di-indoagen188-siap-siap-jadi-sultan.html https://baumarkt-fasselt.de/news/rahasia-terbaru-jam-gachor-main-di-mahjong-ways2.html https://baumarkt-fasselt.de/news/update-pg-soft-gacor-hari-ini.html https://doruktip.com/vendor/psr/test/ https://perpus.donisamultimandiri.com/data/ https://perpus.donisamultimandiri.com/vendor/ https://simbakda.sulselprov.go.id/assets/img/ https://simbakda.sulselprov.go.id/assets/data/ https://simbakda.sulselprov.go.id/public/file/ https://simbakda.sulselprov.go.id/public/images/ https://simbakda.sulselprov.go.id/data/keluaran/ https://simakda.sanggau.go.id/simakda_skpd_sanggau/blog/ https://simakda.sanggau.go.id/simakda_skpd_sanggau/info/sv/ https://simakda.sanggau.go.id/simakda_skpd_sanggau/public/new/ https://simakda.sanggau.go.id/simakda_skpd_sanggau/image/up/ https://simakda.sanggau.go.id/simakda_skpd_sanggau/assets/data/ SITUS CASINO ONLINE SV388 SITUS JUDI BOLA Scatter Hitam Pola Starlight Princess terbaru Trik jitu Gates of Olympus Cara main Mahjong Ways 2 Pola baru Sweet Bonanza Pola trending Starlight Princess slot mahjong ways live casino online zeus slot gacor live casino online slot online slot online slot online slot online slot online slot online slot online slot online slot online slot online slot online slot online slot online memiliki tabungan puluhan juta modal 50k pengguna ewallet qris aplikasi mendapatkan uang sampingan peringatan pola rtp ala warkop hasil menggiurkan rumus memaksimalkan pecahan simbol game gatot kaca Teknik bermain game sweet bonanza pola jackpot paus algoritma memancing scatter hitam mahjong menang game slot mahjong ways hingga jackpot paus pelajaan berharga maxwin mesin slot ghacor gatot kaca dijuluki raja slot mahjong pakai pola efektif resep richard lee panen x500 gates of olympus obat manis sweet bonanza trik wd badai phk solusi mahjong wins daftar demo slot mahjong ways jackpot epic pola jitu scatter mahjong melody sharon trik rtp gates of olympus bocoran perkalian gede gates of olympus beruntun rahasia free spins perkalian maksimal shifu gachor pola mahjong wins 3 surabaya tips maksimalkan olympus1000 menang bermain 15 tips menang mahjong wins 3 shifu gachor live casino online live casino online kakek zeus slot thailand live casino online live casino online scatter hitam kakek petir https://poltekkpsidoarjo.ac.id/assets/data/ https://poltekkpsidoarjo.ac.id/assets/file/ https://poltekkpsidoarjo.ac.id/assets/folder/ https://poltekkpsidoarjo.ac.id/assets/img/ https://poltekkpsidoarjo.ac.id/assets/public/ https://www.unpi-cianjur.ac.id/assets/-/bola/ https://www.unpi-cianjur.ac.id/assets/-/caso/ https://www.unpi-cianjur.ac.id/assets/-/aduan/ https://www.unpi-cianjur.ac.id/site/pmb/update/ https://www.unpi-cianjur.ac.id/plugins/slot88/ https://siakad.unpi-cianjur.ac.id/site/upgrade/css/ https://siakad.unpi-cianjur.ac.id/dana/ https://siakad.unpi-cianjur.ac.id/assets/ https://siakad.unpi-cianjur.ac.id/assets/slider/tgl/ SV388 mix parlay sbobet casino online slot online gacor sabung ayam online sbobet88 casino online scatter hitam SABUNG AYAM ONLINE AGEN BOLA ONLINE BACCARAT ONLINE LIVE CASINO SITUS SLOT ONLINE SABUNG AYAM ONLINE AGEN BOLA LIVE CASINO ONLINE SCATTER HITAM SLOT ONLINE GACOR LIVE CASINO ONLINE AGEN BOLA ONLINE SABUNG AYAM ONLINE SLOT GACOR HARI INI LIVE CASINO ONLINE MIX PARLAY SABUNG AYAM ONLINE joker123 joker123 joker123 joker123 joker123 joker123 joker123 joker123 Slot88 Slot Online Slot Gacor Zeus x1000 Scatter Hitam Mahjong Ways Slot Thailand Terpercaya casino online sabung ayam online sabung ayam online casino online scatter hitam slot Thailand Link Slot Thailand LIVE DRAW HK agen sabung ayam agen sabung ayam Agen Judi Bola live casino online sabung ayam online bola tangkas live casino online sabung ayam online agen bola sbobet AGEN BOLA LIVE CASINO ONLINE WCF888 SABUNG AYAM SLOT RESMI MAXWIN SCATTER HITAM SABUNG AYAM ONLINE WCF888 LIVE CASINO ONLINE AGEN BOLA ONLINE slot gacor scatter hitam slot terpercaya thailand togel online togel online slot thailand scatter hitam slot gacor SBOBET MIX PARLAY LIVE CASINO ONLINE WCF888 SABUNG AYAM ONLINE SLOT777 SABUNG AYAM ONLINE LIVE CASINO ONLINE WAP SBOBET SLOT GACOR DANA AGEN BOLA ONLINE LIVE CASINO ONLINE SABUNG AYAM ONLINE SCATTER HITAM scatter hitam situs live baccarat slot thailand sv388 slot QRIS slot online resmi Slot Demo slot dana slot thailand Slot Mahjong Ways slot QRIS slot online resmi Slot Demo slot dana slot thailand Scatter hitam sabung ayam online agen judi bola sbobet Live Casino Sabung Ayam Online Roulette Mega Wheels Sv388 live casino online togel online Live Casino Online Slot Thailand Sv388 Live Casino Online Slot777 WS168 Situs Judi Baccarat Scatter Hitam Sv388 Agen Judi Bola sv388 Live Casino Online Togel Online AGEN BOLA LIVE CASINO ONLINE SV388 SABUNG AYAM ONLINE SCATTER HITAM agen bola sbobet sv388 sabung ayam online Live Baccarat Terpercaya Slot88 Sabung Ayam Bali Judi Roulette Online Bandar Slot Bonanza Sabung Ayam Online Live Casino Resmi Agen Sabung Ayam Judi Slot Online sv388 sbobet88 casino online sv388 mix parlay sbobet casino online scatter hitam sabung ayam online Agen Judi Bola casino online sabung ayam online casino online https://p2tsp.apsidoarjo.ac.id/ https://poltek.apsidoarjo.ac.id/ https://poltekkpsidoarjo.apsidoarjo.ac.id/ https://polteksidoarjo.apsidoarjo.ac.id/ https://sbu.apsidoarjo.ac.id https://elearning.nou.edu.np/availability/vendor/ https://elearning.nou.edu.np/availability/cores/ https://elearning.nou.edu.np/analytics/lib/ https://elearning.nou.edu.np/analytics/lang/ https://simtuition.com.my/value/index.html https://simtuition.com.my/clude/index.html https://simtuition.com.my/influx/index.html https://simtuition.com.my/winner/index.html https://subarujabodetabek.com/public/joker/ https://subarujabodetabek.com/public/blog/ https://apsidoarjo.ac.id/public/joker/ https://apsidoarjo.ac.id/public/live/ https://childrensparadise.com/lib/mun/ https://childrensparadise.com/lib/min/ https://childrensparadise.com/lang/lunar/ https://childrensparadise.com/lang/line/ https://workplace.thulo.com/css/bike/ https://workplace.thulo.com/css/france/ https://workplace.thulo.com/js/shop/ https://workplace.thulo.com/js/user/ https://www.apna-satta.com/public/daily/ https://www.apna-satta.com/public/king/ https://www.apna-satta.com/public/login/ https://www.apna-satta.com/public/vip/ https://wallclockproperties.com/wp-includes/blackscatter/ https://wallclockproperties.com/wp-includes/easy/ https://wallclockproperties.com/wp-content/mision/ https://wallclockproperties.com/wp-content/vision/ https://www.mobileinfopedia.com/css/news/ https://www.mobileinfopedia.com/css/prices/ https://www.mobileinfopedia.com/js/review/ https://www.mobileinfopedia.com/js/view/ https://smkn1kalanganyar.sch.id/documents/ https://bankarthaya.com/wp-includes/assets/ https://yumamiabusiness.com/wp-includes/blocks/rss/ https://yumamiabusiness.com/wp-includes/css/ https://mim03takerharjo.sch.id/wp-includes/data/ https://mim03takerharjo.sch.id/wp-includes/blocks/rss/ https://smpitattaqwapusat.sch.id/wp-includes/block-supports/ https://smpitattaqwapusat.sch.id/wp-includes/block-bindings/ https://dandybelles.com.au/storage/rooms/September2024/ https://dandybelles.com.au/storage/rooms/October2024/ https://dandybelles.com.au/storage/rooms/July2024/ https://dandybelles.com.au/storage/rooms/May2024/ https://emf-sodec.com/storage/credit/ https://emf-sodec.com/storage/saving/ https://emf-sodec.com/storage/entrepreneur/ https://emf-sodec.com/storage/team/ https://fadlyrentcarpalembang.com/wp-content/uploads/smus/detail/ https://fadlyrentcarpalembang.com/wp-content/uploads/smus/data/ https://fadlyrentcarpalembang.com/wp-content/uploads/smus/compone/ https://fadlyrentcarpalembang.com/wp-content/uploads/elementore/ https://jasaizinoss.com/wp-content/uploads/media-upload/conten/ https://jasaizinoss.com/wp-content/uploads/media-upload/data/ https://jasaizinoss.com/wp-content/uploads/media-upload/detail/ https://jasaizinoss.com/wp-content/uploads/media-upload/file/ https://alphaprepacademy.org/core/ https://alphaprepacademy.org/dist/ https://alphaprepacademy.org/est/ https://www.libertybellmoving.com/cox/ https://www.libertybellmoving.com/lib/ https://www.libertybellmoving.com/tax/ https://perempuantanihkti.org/wp-content/auth/ https://perempuantanihkti.org/wp-content/blog/ https://perempuantanihkti.org/wp-content/text/ https://smpn4satapbawang.sch.id/konten/ https://smpn4satapbawang.sch.id/absensi/ https://pragnews.com/-/bac/ https://pragnews.com/-/jok/ https://pragnews.com/-/lat/ https://pragnews.com/-/tex/ https://www.dubaijobs.net/dev/ https://www.dubaijobs.net/data/ https://aunhre.org/version/ https://aunhre.org/log/ AGEN BOLA ONLINE LIVE CASINO ONLINE SABUNG AYAM ONLINE SCATTER HITAM SBOBET MIX PARLAY WCF888 SABUNG AYAM ONLINE SLOT777SLOT GACOR LIVE CASINO ONLINE SLOT GACOR MAHJONG WINS 3 BLACK SCATTER peluang scatter hitam mahjong menggiurkan Kisah Dewa Zeus Starlight Princess Mahjong Wins 3, Scatter Naga Hitam (Starlight Princess Mahjong Ways Sweet Bonanza casino online scatter hitam casino online zeus slot Pola baru Starlight Princess cuan mudah Sweet Bonanza tips pemula gampang paham Pola edukatif Gates of Olympus terbaru Panduan scatter Mahjong Ways 2 lengkap Trik dasar Gates of GatotKaca pemula Tips baru Sweet Bonanza makin jago Panduan praktis Mahjong Wins 3 mudah Pola viral Starlight Princess terbaru Live Casino Online Slot Gacor Joker123 Sabung Ayam Online Bandar Judi Baccarat Scatter Hitam Sabung Ayam Online Sabung Ayam Online Live Casino Online Slot Gacor Thailand all in mahjong ways 2 vs donasi agus cara cerdas menang besar mahjong ways 2 all in untuk jackpot donasi gempa vanuatu pg soft jackpot 90 juta warga jawa barat donasi mahjong ways donasi jackpot free spin cara menang sbobet jackpot 120 juta donasi ke orang membutuhkan mahjong ways 2 gacor maxwin mahjong ways 2 gacor maxwin mahjong ways 2 gacor maxwin mahjong ways 2 gacor maxwin mahjong ways 2 gacor maxwin menangkan hadia mobil brio pemuda palembang meraih mobil rolls oyce sweptail Mendapat 3 Scatter Hitam di Mahjong Wins 3 di Palembang! Program Event Scatter Hitam Mahjong Wins 3 Hadiah Mobil Lamborghini Aventador Dari Menang Parlay SBOBET88 sipnosis film modal nekad main gates of gatot kaca nyelot efisien game slot sweet bonanza pedagang gorengan main mahjong ways beli mobil wuling meletus jackpot gunung naga hitam mahjong ways tukang bakso main game slot sweet bonanza jackpot 210juta pola ghacor mr r muntah jackpot muncul black scatter indobola77 sabung ayam online casino online agen bola sabung ayam online
  • https://pay.morshedworx.com/wp-content/image/
    https://pay.morshedworx.com/wp-content/jss/
    https://pay.morshedworx.com/wp-content/plugins/secure/
    https://pay.morshedworx.com/wp-content/plugins/woocom/
    https://manal.morshedworx.com/wp-admin/
    https://manal.morshedworx.com/wp-content/
    https://manal.morshedworx.com/wp-include/
    https://manal.morshedworx.com/wp-upload/
    https://pgiwjabar.or.id/wp-includes/write/
    https://pgiwjabar.or.id/wp-includes/jabar/
    https://pgiwjabar.or.id/wp-content/file/
    https://pgiwjabar.or.id/wp-content/data/
    https://pgiwjabar.or.id/wp-content/public/
    https://inspirasiindonesia.id/wp-content/xia/
    https://inspirasiindonesia.id/wp-content/lauren/
    https://inspirasiindonesia.id/wp-content/chinxia/
    https://inspirasiindonesia.id/wp-content/cindy/
    https://inspirasiindonesia.id/wp-content/chin/
    https://manarythanna.com/uploads/dummy_folders/images/
    https://manarythanna.com/uploads/dummy_folders/data/
    https://manarythanna.com/uploads/dummy_folders/file/
    https://manarythanna.com/uploads/dummy_folders/detail/
    https://plppgi.web.id/data/
    https://vegagameindo.com/
    https://gamekipas.com/
    wdtunai
    https://plppgi.web.id/folder/
    https://plppgi.web.id/images/
    https://plppgi.web.id/detail/
    https://anandarishi.com/images/gallery/picture/
    https://anandarishi.com/fonts/alpha/
    https://anandarishi.com/includes/uploads/
    https://anandarishi.com/css/data/
    https://anandarishi.com/js/cache/
    https://gmkibogor.live/wp-content/themes/yakobus/
    https://gmkibogor.live/wp-content/uploads/2024/12/
    https://gmkibogor.live/wp-includes/blocks/line/
    https://gmkibogor.live/wp-includes/images/gallery/
    https://kendicinta.my.id/wp-content/upgrade/misc/
    https://kendicinta.my.id/wp-content/uploads/2022/03/
    https://kendicinta.my.id/wp-includes/css/supp/
    https://kendicinta.my.id/wp-includes/images/photos/
    https://euroedu.uk/university-01/
    didascaliasdelteatrocaminito.com
    glenellynrent.com
    gypsumboardequipment.com
    realseller.org
    https://harrysphone.com/upin
    gyergyoalfalu.ro/tokek
    vipokno.by/gokil
    winjospg.com
    winjos801.com/
    www.logansquarerent.com
    internationalfintech.com/bamsz
    condowizard.ca
    jawatoto889.com
    hikaribet3.live
    hikaribet1.com
    heylink.me/hikaribet
    www.nomadsumc.org
    condowizard.ca/aromatoto
    euro2024gol.com
    www.imaracorp.com
    daftarsekaibos.com
    stuffyoucanuse.org/juragan
    Toto Macau 4d
    Aromatoto
    Lippototo
    Mbahtoto
    Winjos
    152.42.229.23
    bandarlotre126.com
    heylink.me/sekaipro
    www.get-coachoutletsonline.com
    wholesalejerseyslord.com
    Lippototo
    Zientoto
    Lippototo
    Situs Togel Resmi
    Fajartoto
    Situs Togel
    Toto Macau
    Winjos
    Winlotre
    Aromatoto
    design-develop-test.com
    winlotre.online
    winlotre.xyz
    winlotre.us
    winlotrebandung.com
    winlotrepalu.com
    winlotresurabaya.shop
    winlotrejakarta.com
    winlotresemarang.shop
    winlotrebali.shop
    winlotreaceh.shop
    winlotremakmur.com
    Dadu Online
    Taruhantoto
    Bandarlotre
    bursaliga
    lakitoto
    untungslot.pages.dev
    slotpoupler.pages.dev
    rtpliveslot88a.pages.dev
    tipsgameslot.pages.dev
    pilihslot88.pages.dev
    fortuertiger.pages.dev
    linkp4d.pages.dev
    linkslot88a.pages.dev
    slotpgs8.pages.dev
    markasjudi.pages.dev
    saldo69.pages.dev
    slotbenua.pages.dev
    saingtoto.pages.dev
    markastoto77.pages.dev
    jowototo88.pages.dev
    sungli78.pages.dev
    volatilitas78.pages.dev
    bonusbuy12.pages.dev
    slotoffiline.pages.dev
    dihindari77.pages.dev
    rtpdislot1.pages.dev
    agtslot77.pages.dev
    congtoto15.pages.dev
    hongkongtoto7.pages.dev
    sinarmas177.pages.dev
    hours771.pages.dev
    sarana771.pages.dev
    kananslot7.pages.dev
    balitoto17.pages.dev
    jowototo17.pages.dev
    aromatotoding.com