\n<\/p>\n
AI company\u00a0Sesame<\/a>\u00a0has released the base model that powers Maya, the\u00a0impressively realistic voice assistant<\/a>.<\/p>\n The model, which is 1 billion parameters in size (\u201cparameters\u201d referring to individual components of the model), is under an Apache 2.0 license, meaning it can be used commercially with few restrictions. Called CSM-1B, the model generates \u201cRVQ audio codes\u201d from text and audio inputs, according to Sesame\u2019s description on the AI dev platform Hugging Face<\/a>.<\/p>\n RVQ refers to \u201cresidual vector quantization,\u201d a technique for encoding audio into discrete tokens called codes. RVQ is used in a number of recent AI audio technologies<\/a>, including Google\u2019s SoundStream and Meta\u2019s Encodec.<\/p>\n CSM-1B uses a model from Meta\u2019s Llama family<\/a> as its backbone paired with an audio \u201cdecoder\u201d component. A fine-tuned variant of CSM powers Maya, Sesame says.<\/p>\n \u201cThe model open-sourced here is a base generation model,\u201d Sesame writes in CSM-1B\u2019s Hugging Face<\/a> and GitHub<\/a> repositories. \u201cIt is capable of producing a variety of voices, but it has not been fine-tuned on any specific voice [\u2026] The model has some capacity for non-English languages due to data contamination in the training data, but it likely won\u2019t do well.\u201d<\/p>\n It\u2019s unclear what data Sesame used to train CSM-1B. The company didn\u2019t say. <\/p>\n It\u2019s worth noting the model has no real safeguards to speak of. Sesame has an honor system and merely urges developers and users not to use the model to mimic a person\u2019s voice without their consent, create misleading content like fake news, or engage in \u201charmful\u201d or \u201cmalicious\u201d activities.<\/p>\n I tried the demo<\/a> on Hugging Face, and cloning my voice took less than a minute. From there, it was easy to generate speech to my heart\u2019s desire, including on controversial topics like the election and Russian propaganda.<\/p>\n Consumer Reports recently warned that many popular AI-powered voice cloning tools on the market don\u2019t have \u201cmeaningful\u201d safeguards<\/a> to prevent fraud or abuse.<\/p>\n Sesame, co-founded by Oculus co-creator Brendan Iribe, went viral in late February for its assistant tech, which comes close to clearing uncanny valley territory. Maya and Sesame\u2019s other assistant, Miles, take breaths and speak with disfluencies, and can be interrupted while speaking, much like OpenAI\u2019s Voice Mode<\/a>. <\/p>\n Sesame has raised an undisclosed amount of capital from Andreessen Horowitz, Spark Capital, and Matrix Partners. In addition to building voice assistant tech, the company says it\u2019s prototyping AI glasses \u201cdesigned to be worn all day\u201d that\u2019ll be equipped with its custom models. <\/p>\n<\/div>\n