+
+ + +
+
+

CosyVoice 3.6: Towards In-the-wild Multilingual Speech Generation

+


FunAudioLLM Team

+

SpeechLab@Tongyi, Alibaba Group

+
+

Abstract: + CosyVoice 3.6 further scales language coverage, acoustic robustness and controllability over the 3.x series. This page compares the latest CosyVoice 3.6-0.8B and CosyVoice 3.6-4B models, with CosyVoice 3.0 demos retained as historical references where available. +

+ +

Models shown: +
• CosyVoice 3.0-0.5B — historical demo reference +
• CosyVoice 3.0-1.5B — historical demo reference +
• CosyVoice 3.6-0.8B +
• CosyVoice 3.6-4B +

+ +

+ Contents +

+

+
+ + + +
+

Zero-shot In-context Generation

+
+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
LanguagePromptTextCosyVoice 3.0-0.5BCosyVoice 3.0-1.5BCosyVoice 3.6-0.8BCosyVoice 3.6-4B
ZH这个记者俱乐部将成为该部门发放资讯的唯一渠道。
HARD-ZH八百标兵奔北坡，北坡炮兵并排跑，炮兵怕把标兵碰，标兵怕碰炮兵炮。
ENButler tranquilizes Holly with a hypodermic dart gun.
HARD-ENFuzzy Wuzzy was a bear. Fuzzy Wuzzy had no hair. Fuzzy Wuzzy wasn't very fuzzy, was he?
+
+
+ + + +
+

Multilingual / Minority Languages

+

Zero-shot cloning across 14 languages: ja ko de es fr it ru + ar id pt th vi ms tl (7 newly added in 3.6)

+
+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
LanguageTextCosyVoice 3.0-0.5BCosyVoice 3.0-1.5BCosyVoice 3.6-0.8BCosyVoice 3.6-4B
JA結合することができないから矛盾するというのである。
KO동혁은 환자의 머리맡을 떠나지 않았다.
DEDie Gemeinde liegt im Westerwald im Einzugsbereich der Westerwälder Seenplatte.
ESEn el horizonte, un sol radiante, símbolo del nacimiento de la nación.
FRDe manière informelle, la capacité d'un modèle de classification correspond à sa complexité.
ITIl suo primo utilizzo è stato in Vietnam da parte dell'esercito statunitense.
RUТаким образом, мы услышим от них более сфокусированные и полезные выступления.
— 以下为 3.6 新增语种（3.0 未支持）—
ARالنهرُ الذي يجري بهدوءٍ يعلّمنا دروسًا في الصبرِ والمرونةِ حين يلتفُّ حولَ الصخورِ بدلَ أن يصطدمَ بها بعنفٍ غيرِ مجدٍ.——
IDSenyum tulus mampu menyampaikan kebaikan lebih dari ribuan kata indah.——
PTA arte contemporânea desafia frequentemente as convenções estéticas tradicionais, convidando o espectador a participar ativamente na construção de significado.——
THการมีเพื่อนที่กล้าพูดความจริงกับเรา แม้จะขัดหูบ้าง ก็เป็นของขวัญที่มีค่า เพราะช่วยให้เราเห็นจุดบอดของตัวเอง——
VISức khỏe thể chất và tinh thần luôn song hành với nhau nên ta phải chăm sóc cả hai một cách cân bằng và điều độ.——
MSOrang Sepanyol telah memulakan tempoh penjajahan yang berlangsung selama tiga abad.——
TLLibo-libong taon nang gumagawa at gumagamit ang mga tao ng mga lenteng pampalaki.——
+
+
+ + + +
+

Cross-lingual In-context Generation

+

Same prompt voice reading different target languages (0-shot) — listen whether the timbre stays consistent across languages.

+
+ + + + + + + + + + + + + + + + + + + + + + +
PromptTargetTextCosyVoice 3.6-0.8BCosyVoice 3.6-4B
ZH prompt

加菲迪曾经代表巴西国家足球队。		→ ENMany Christians are well-educated, but cannot rise to positions of responsibility.
→ JA歴史的世界においては、過去は単に過ぎ去ったものではない、プラトンのいう如く非有が有である。
→ KO가을철 들면서부터 덕호는 읍의 출입이 잦아졌다.
EN prompt

The log file reveals that nobody ever read the end user license agreement during installation.		→ ZH瓦尔韦尔迪是葡萄牙布拉干萨区的一个堂区。
→ JA病床の母親が、だれかに手紙の代筆を頼む。
→ KO선비는 씨아틀도 만지지 않으면 앞이 허전한 것 같아서 그냥 붙들고 있었다.
+
+
+ + + +
+

Emotional Voice Generation

+
+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
EmotionPromptTextCosyVoice 3.0-0.5BCosyVoice 3.0-1.5BCosyVoice 3.6-0.8BCosyVoice 3.6-4B
Happy · ENI actually managed to grab tickets to Eason Chan’s concert! It’s amazing! I can’t wait to hear his voice live!
Happy · ZH等到七月底项目结束,我就可以申请休年假了,好期待哦!
Sad · ENThe research institute won’t renew my contract, so I have to go back to my country. I don’t even have a job anymore.
Sad · ZH原来父皇什么都知道了,我真笨。
Fearful · ENI’m too scared to be here alone, can you stay with me?
Fearful · ZH求你了，别再说那些吓人的话，我真的快撑不住了。
Angry · ENI told you, stop nagging, alright? Haven’t you nagged enough in this lifetime?
Angry · ZH老喜欢出阴招,使手段,太不厚道了!
Surprised · ENWhat did you just say? Is this really true? I can’t believe it at all!
Surprised · ZH我怎么听着像在做梦一样，怎么可能会发生这样的事？
+
+
+ + + +
+

Chinese Dialect Voice Generation

+

Zero-shot voice cloning across 17 Chinese dialects (dialect timbre replication)

+
+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
Task / DialectPromptTextCosyVoice 3.6-0.8BCosyVoice 3.6-4B
粤语第二条问题咧，就系边一类嘅商户嘅租金下调最多咧？—
东北话你别总把自己逼到极限才休息，人不是机器，累了就歇一歇，硬撑只会让你更容易崩。—
甘肃话对于人来讲，智慧不取决于他里面的“有”，而取决于他里面的“空”。—
贵州话你要想学习有效，就每天固定一点时间噻，哪怕二十分钟，坚持下来比突然猛学一晚上更有用。—
河南话今儿个我晚上散步走到桥上，风吹得头发乱飞，河面黑亮黑亮的，我站那儿发了会儿呆，回家睡得挺香。—
湖北话你莫光看别个朋友圈晒得光鲜，人家背后吃的苦你看不到，把自个日子过稳当才是正经。—
湖南话你要是觉得心头乱，就把房间收拾一哈咯，衣服叠好，桌面擦干净，环境清爽人也清爽。—
江西话二零一七年二月二十一日，蒋某某、童某某与江西聚诚公司签订了三份商品房买卖合同（预售）。—
闽南语记者看到，伊得床得枕头套弄啊面皆红去，建议伊夺就枕头套拆落来，呼呼吸一下，啉一下水。—
宁夏话想专心干活你先把手机扔一边嘛，叮咚一下你就分神，分神多了，一上午啥也整不成。—
山西话今儿个坐公交，有人外放短视频，吵得人脑仁疼，旁边人都皱眉，我戴耳机都压不住。—
陕西话万一你连一万都没有，你还“万一”呐！算了，不指望你了。儿子，以后呀你要努力赚钱，你看看你归叔开这么好的车，你也可以做到地，别像你爸一样啊。—
山东话“永恒刹那，刹那永恒，等你在时间之外，在时间之内，在刹那，在永恒。”—
上海话文件版本侬若是不用“v1v2最终版最终最终版”这种瞎命名，直接用日期加里程碑，后头回溯一眼就懂，省得翻半天。—
四川话我在办公室做表格，电脑突然死机，半天没保存的东西全没了，我当场想哭，只能深呼吸重新做，顺便骂自己不长记性。—
天津话介小土布呃，小土布走到这个是从这个青县马场从那边传传过来的，到远到高阳织这个小土布的这机器，到了天津以后呢。—
云南话下午太阳一出来热得要死，我在办公室吹风扇还觉得闷，同事说你把窗开点，我开了又灰大，只能叹气说昆明天气真是怪。—
+
+
+ + + +
+

Acoustic Robustness

+

Robust synthesis under noisy / reverberant / unclear (telephone-band, high-freq cut) prompts

+
+ + + + + + + + + + + + + + + + + + +
ConditionPromptTextCosyVoice 3.6-0.8BCosyVoice 3.6-4B
noisy · zh青铜器在中国商周时期达到了极高的艺术水平，鼎和尊是典型器物。这些器物既是礼器，也是权力和地位的象征。
reverb · zh你昨天看的那部电影怎么样，剧情精彩吗？
unclear · zh昨天我去超市买水果，看到货架上摆满了新鲜的草莓和樱桃。
noisy · enCoffee beans are seeds inside bright red berries that grow on tropical bushes. They must be dried, roasted, and ground before they can be brewed into a drink.
reverb · enHave you ever wondered why we dream at night and what those images mean?
unclear · enI walked through the park this morning and saw a beautiful garden full of roses.
+
+
+ + + +
+

Long-text Generation

+

Continuous long-passage reading (zh/en, ~1–2.5 min each)

+
+ + + + + + + + + + + + + + +
LanguagePromptText (excerpt)CosyVoice 3.6-0.8BCosyVoice 3.6-4B
ZH记忆里的夏天总是从蝉鸣开始的。那时候的天蓝得透亮，太阳一出来就把院墙晒得发烫。我总爱趴在凉席上，看窗棂外的光斑一点点挪动。风是不怎么有的，连树叶都懒得晃悠。只有老槐树底下，知了扯着嗓子叫个没完，像是在跟谁较劲。祖母摇着蒲扇，扇出的风带着花露水和皂角的气味，凉丝丝地贴在皮肤上。我盯着天上飘过的云，觉得日子长得没有尽头。水蜜桃的汁水顺着指尖往下淌，黏糊糊的却舍不得擦。井水里镇着的瓜被刀切开时，那一瞬间的清甜好像能把整个午后都填满。影子被拉得很长，长到可以一直延伸到巷口的大石磨旁。那时候什么都不用赶，时间就像墙头的藤蔓，慢吞吞地往上爬。我常赤着脚踩在青砖地上，脚底能感觉到白日里积下来的余温。街坊邻居的闲聊声隔着院墙飘过来，带着柴火灶台上的烟火气。我把汗津津的手在裤腿上胡乱抹一把，抱着空玻璃瓶去供销社换汽水。瓶盖撬开时扑腾出的白沫，沾在手背上又粘又甜。夕阳西下的时候，天边会染上一层橘红色的霞光，把一切都镀上柔和的边缘。暮色四合的时候，蚊子开始在耳畔嗡嗡作响。我不急着回屋，只愿再多坐一会儿。晚风终于吹散了白日的燥热，带来泥土和水草混合的气息。远处的田野里传来蛙鼓声，一声接着一声，把黑夜推得更深了些。萤火虫提着小小的灯笼在草丛间游荡，明灭不定，像极了那些抓不住的念头。我把蒲扇丢在一边，闭上眼睛任由思绪漫无边际地飘荡。童年之所以漫长，或许是因为那时的心没有装太多的事情。只要眼前的瓜够甜，水够凉，日子就足够丰盈。如今人长大了，夏天还是那个夏天，可心境早已换了模样。我们学会了在喧嚣里找安宁，却再也拼凑不出当初那份毫无保留的从容。时光就这样默默往前走，带走了许多具体的物件，却把某些安静的瞬间酿成了酒。每次回味，都能尝出一点淡淡的怅惘，又夹杂着一丝妥帖的暖意。
ENAgain again. A voice from beyond the world was calling. Hello stephanos here comes the dedalus. He recognized their speech collectively before he distinguished their faces. Their diving stone poised on its rude supports and rocking under their plunges and the rough hewn stones of the sloping breakwater over which they scrambled in their horseplay gleamed with cold wet lustre. The towels with which they smacked their bodies were heavy with cold seawater and drenched with cold brine was their matted hair. He stood still in deference to their calls and parried their banter with easy words. How characterless they looked shuley without his deep unbuttoned collar ennis without his scarlet belt with the snaky clasp and connolly without his norfolk coat with the flapless side pockets. It was a pain to see them and a sword like pain to see the signs of adolescence that made repellent their pitiable nakedness. Perhaps they had taken refuge in number and noise from the secret dread in their souls. But he apart from them and in silence remembered in what dread he stood of the mystery of his own body. Stephanos dedalos. Now as never before his strange name seemed to him a prophecy. So timeless seemed the grey warm air so fluid and impersonal his own mood that all ages were as one to him. A moment before the ghost of the ancient kingdom of the danes had looked forth through the vesture of the hazewrapped city.
+
+
+ + + +
+

Text Normalization (TN)

+

Correct reading of numbers / abbreviations / symbols / named entities

+
+ + + + + + + + + + + + + + + + + + +
LanguagePromptRaw textCosyVoice 3.6-0.8BCosyVoice 3.6-4B
ZH档案馆B区三楼缩微胶片柜编号为B3-089、B3-097，调阅时请逐位确认编号。
ZH轧钢车间三号机组良品率98.7%，废品率1.3‰，返工率0.85%。
ZH车险精算师说：预期赔付率公式是\frac{E[L]}{P}，风险因子用\beta和\gamma表示。
ZH这款便携式快递秤最大承重15kg，精度±2g，体积仅20cm×15cm×8cm。
ZH主控板温度阈值设定为65℃—85℃，风扇启停区间为40℃～60℃。
EN
I can't believe it— the lions just broke out of their enclosure and are walking around freely!		Dr. Smith from NASA will arrive at 9:45 AM on Dec. 1st; the flight covers 1,200 km at 850 km/h, approx. 6 hrs.N/AN/A—
+
+
+ + + +
+

Instructed Voice Generation

+

Natural-language instructions controlling emotion / style / character / dialect

+
+ + + + + + + + + + + + + + + + + + + + + + + + + + +
InstructionPromptTextCosyVoice 3.6-0.8BCosyVoice 3.6-4B
中文 instruction
请用愤怒的情感朗读下面这句话。今天的会议改到下午三点。
请用很快的语速朗读下面这句话。她把茶杯放在窗台上，然后转身回到了书桌前。
请以机器人的口吻朗读下面这段话。桌子上的水杯里还剩下半杯水，已经冷了。
请用恐惧的情感、轻柔的音量朗读下面这段话。出租车在路口慢慢减速，等待红灯变绿。
语气慢条斯理、不急不躁，带着久经临床的从容感，每个用药细节都咬字清楚 —— 模仿一位四十多岁的中医大夫在安静的诊室里向中年女病人嘱咐用药。这次给你换一个新的方子，前两天那个先停掉。每天早晚各一次，记得用温水。睡前一个小时不要再喝茶了，影响吸收。
角色：四十岁出头的退役军人，性格沉稳，话不多。场景：在自家阳台抽着烟向多年未见的老战友讲述这些年的经历。风格：克制、沙哑，每句话之间留出停顿。语速：偏慢。请按以上设定朗读下面这段台词。退伍那年我以为这辈子就这样了。后来去了西北做工程，跑了五年。前年回的家，孩子已经认不出我了。
English instruction
Read the following sentence in an angry tone.The meeting was moved to three in the afternoon.
Read the following sentence quickly.She placed the teacup on the windowsill and turned back to her desk.
Read the following in the voice of a robot.Half a cup of water was left on the desk, and it had already gone cold.
Say the following cheerfully, and fast.The taxi slowed down at the intersection, waiting for the light to turn green.
Calm clinical tone, measured pacing, reassuring warmth in the closing line, key values lightly emphasized — scene: a senior male physician speaking with a middle-aged patient in his office about a recent test result.Your bloodwork came back, and there's nothing alarming in any of it. The cholesterol is just slightly above where I'd like it to be. We'll talk about diet, and we'll recheck in three months.
Character: a forty-something former soldier, quiet and reserved. Scene: smoking on his apartment balcony, telling an old comrade about the years since they last met. Style: restrained, slightly husky, with pauses between sentences. Pace: slow. Please read the following lines under this direction.The year I came home, I thought that was the end of the story. Then I spent five years on a project out west. Got back two years ago. The kid didn't recognize me anymore.
+
+
+ + + +
+

Fine-grained Tags

+

Inline tags controlling pause / breath / stress and other fine-grained articulation

+
+ + + + + + + + + + + + + + + + + + + + + + + + + + + + +
TagPromptTagged textCosyVoice 3.6-0.8BCosyVoice 3.6-4B
中文 Fine-grained Tags
[gasp][trembling] [panicked] 救命啊！电梯突然停电了，而且一直在往下坠！ [gasp] 有没有人能听到我说话？快帮帮我！
[sighing][sad, very slowly] 爷爷走的那天，天空正下着蒙蒙细雨。 [sighing] 他拉着我的手，一句话也没有说出来。
[clears throat][serious] 针对本次食品安全事件，我市已成立联合调查组。 [clears throat] 一旦发现违法违规行为，将依法依规严肃处理。
[giggles][mischievously] [giggles] 嘿嘿，趁他现在闭着眼睛睡觉，我们偷偷把他的鞋带跟桌子腿绑在一起，看他醒了怎么办！
[laughing][excited] 我们的新歌终于冲上音乐榜单第一名了！ [laughing] 感谢所有粉丝的支持，我们做到了！
[cough][tired] 刚跑完一万米，我现在连一根手指头都不想动。 [cough] 嗓子眼里全是一股铁锈的味道。
[snorts][like dracula] [whispers] 黑暗已经降临，属于我们的时间到了。 [snorts] 不要发出声音，跟我来。
English Fine-grained Tags
[gasp][amazed] The glass sculpture looked ordinary at first, [gasp] but under the morning light it seemed to glow from within.
[sighing][empathetic] I know this has been a difficult week for you, [sighing] and no one expects you to have every answer right now.
[clears throat][serious] The evacuation route must remain clear at all times, [clears throat] especially during inspections and emergency drills.
[giggles][mischievously] [giggles] I may have replaced every office pen with one that writes in glitter, but I refuse to call it a crime.
[laughing][excited] [laughing] We finally received the approval letter, and the whole team can start building the prototype tomorrow morning.
[cough][bored] The lecture continued for another hour, [cough] and even the clock seemed too tired to move.
[snorts][sarcastic] Wonderful, the printer jammed again right before the presentation, [snorts] because apparently timing is its greatest talent.
+
+
+ + +
+

Disclaimer

+

The content provided above is for academic purposes only and is intended to demonstrate technical capabilities. Some examples are sourced from the internet. If any content infringes on your rights, please contact us to request its removal.

+
+ +
+