Amazon launched a brand new synthetic intelligence (AI) mannequin in its flagship Nova household of fashions on Tuesday. Dubbed Amazon Nova Sonic, it’s a voice technology mannequin able to producing human-like speech. Nevertheless, it isn’t a text-to-speech (TTS) software; as a substitute, it will probably course of voice enter in actual time and reply to it. The Seattle-based tech big says builders can use the mannequin to construct conversational AI chatbots and comparable instruments. Notably, the Amazon Nova Sonic AI mannequin additionally helps useful calling and gear use, making it appropriate with agentic software developments as nicely.
Amazon Nova Sonic Is Accessible As an API
In a weblog submit, the tech big introduced the discharge of the Amazon Nova Sonic. The corporate mentioned conventional approaches to voice-enabled functions use a posh with a number of fashions comparable to textual content recognition, speech-to-text conversion, knowledge processing, and TTS fashions. This usually results in a rise in latency, and failure in preserving linguistic context, the submit added.
Amazon mentioned its strategy with the Nova Sonic mannequin was to unify speech understanding and speech technology elements. The AI mannequin is alleged to have the ability to course of knowledge and generate speech in actual time, giving it a conversation-like expertise. This unified system additionally permits the mannequin to higher perceive the tempo and timbre of enter speech to contextualise the intent of the consumer.
Moreover, the AI mannequin can perceive completely different talking types in addition to separate masculine and feminine-sounding voices in numerous accents. It will probably additionally perceive when a consumer misspeaks, mumbles, or pauses whereas talking. Amazon says the mannequin can choose up speech even in a loud setting.
In response technology, the corporate claims the mannequin could be extra expressive and human-like, and might modify its response fashion to match the context of the dialog. Presently, the AI mannequin solely helps the English language. Amazon mentioned help for extra languages will probably be added quickly. The mannequin helps a context window of 32,000 tokens for audio, with an extra window to deal with longer conversations. It has a default session restrict of eight minutes.
To make use of the Nova Sonic mannequin, builders can head to Amazon Bedrock and discover it underneath the mannequin entry choice. It will also be accessed by way of a bidirectional streaming software programming interface (API) that may each course of audio enter and generate output.