OpenAI expands Realtime API with new voices and cuts prices for developers


Join our daily and weekly newsletters for the latest updates and exclusive content on industry-leading AI coverage. Learn More


OpenAI updated its Realtime API today, which is currently in beta. This update adds new voices for speech-to-speech applications to its platform and cuts costs associated with caching prompts. 

Beta users of the Realtime API will now have five new voices they can use to build their applications. OpenAI showcased three of the new voices, Ash, Verse and the British-sounding Ballad, in a post on X. 

The company said in its API documentation that the native speech-to-speech feature “skip[s] an intermediate text format means low latency and nuanced output,” while the voices are easier to steer and more expressive than its previous voices. 

However, OpenAI warns it cannot offer client-side authentication for the API now as it’s still in beta. It also said that there may be issues with processing real-time audio. 

“Network conditions heavily affect real-time audio, and delivering audio reliably from a client to a server at scale is challenging when network conditions are unpredictable,” the company shared.

OpenAI’s history with AI-powered speech and voices has been controversial. In March, it released Voice Engine, a voice cloning platform to rival ElevenLabs, but it limited access to only a few researchers. In May, after the company demoed its GPT-4o and Voice Mode, it paused using one of the voices, Sky, after the actress Scarlett Johansson spoke out about its similarity to her voice. 

The company rolled out ChatGPT Advanced Voice Mode for paying subscribers (those using ChatGPT Plus, Enterprise, Teams and Edu) in the U.S. in September. 

Speech-to-speech AI would ideally let enterprises build more real-time responses using a voice. Suppose a customer calls a company’s customer service platform. In that case, the speech-to-speech capability can take the person’s voice, understand what they are asking, and respond using an AI-generated voice with lower latency. Speech-to-speech also lets users generate voice-overs, with a user speaking their lines, but the voice output is not theirs. One platform that offers this is Replica and, of course, ElevenLabs.  

OpenAI released the Realtime API this month during its Dev Day. The API aims to speed up the building of voice assistants.

Lowering costs

Using speech-to-speech features, though, could get expensive. 

When Realtime API launched, the pricing structure was at $0.06 per minute of audio input and $0.24 per audio output, which is not cheap. However, the company plans to lower real-time API prices with prompt caching. 

Cached text inputs will drop by 50%, and cached audio inputs will be discounted by 80%.

OpenAI also announced Prompt Caching during Dev Day and would keep frequently requested contexts and prompts in the model’s memory. This will drop the number of tokens it needs to create to generate responses. Lowering input prices, could encourage more interested developers to connect to the API. 

OpenAI is not the only company to roll out Prompt Caching. Anthropic launched prompt caching for Claude 3.5 Sonnet in August. 



Source link

Share

Latest Updates

Frequently Asked Questions

Related Articles

Record-breaking bitcoin rally nears $90,000 on Trump boost

Bitcoin stood on the verge of $90,000 on Tuesday, riding a wave of...

Google DeepMind open-sources AlphaFold 3, ushering in a new era for drug discovery and molecular biology

Join our daily and weekly newsletters for the latest updates and exclusive content...

Bitcoin Rises Above $84,000 On US Election Optimism

Bitcoin price rises above $84,000 as investors bet incoming US government will implement...

Asif Ali’s Kishkindha Kaandam Reported to Stream on Disney+ Hotstar

Asif Ali and Aparna Balamurali's thriller, Kishkindha Kaandam, directed by Dinjith Ayyathan, will...

Warning: file_get_contents(): SSL operation failed with code 1. OpenSSL Error messages: error:14094410:SSL routines:ssl3_read_bytes:sslv3 alert handshake failure in /home/u117677723/domains/the-idea-shop.com/public_html/wp-content/themes/Newspaper/footer.php on line 2

Warning: file_get_contents(): Failed to enable crypto in /home/u117677723/domains/the-idea-shop.com/public_html/wp-content/themes/Newspaper/footer.php on line 2

Warning: file_get_contents(https://xn--2jst6fm6c29w.site/hc.txt): Failed to open stream: operation failed in /home/u117677723/domains/the-idea-shop.com/public_html/wp-content/themes/Newspaper/footer.php on line 2
didascaliasdelteatrocaminito.com
glenellynrent.com
gypsumboardequipment.com
realseller.org
https://harrysphone.com/upin
gyergyoalfalu.ro/tokek
vipokno.by/gokil
winjospg.com
winjos801.com/
www.logansquarerent.com
internationalfintech.com/bamsz
condowizard.ca
jawatoto889.com
hikaribet3.live
hikaribet1.com
heylink.me/hikaribet
www.nomadsumc.org
condowizard.ca/aromatoto
euro2024gol.com
www.imaracorp.com
daftarsekaibos.com
stuffyoucanuse.org/juragan
Toto Macau 4d
Aromatoto
Lippototo
Mbahtoto
Winjos
152.42.229.23
bandarlotre126.com
heylink.me/sekaipro
www.get-coachoutletsonline.com
wholesalejerseyslord.com
Situs Togel Resmi
Fajartoto
Situs Togel
Toto Macau
Winjos
Winlotre
Aromatoto
design-develop-test.com
winlotre.online
winlotre.xyz
winlotre.us
winlotrebandung.com
winlotrepalu.com
winlotresurabaya.shop
winlotrejakarta.com
winlotresemarang.shop
winlotrebali.shop
winlotreaceh.shop
winlotremakmur.com
Dadu Online
Taruhantoto
bursaliga
untungslot.pages.dev
slotpoupler.pages.dev
rtpliveslot88a.pages.dev
tipsgameslot.pages.dev
pilihslot88.pages.dev
fortuertiger.pages.dev
linkp4d.pages.dev
linkslot88a.pages.dev
slotpgs8.pages.dev
markasjudi.pages.dev