Paper
arXiv
Yuki Saito, Shinnosuke Takamichi, Eiji Iimori, Kentaro Tachibana, and Hiroshi Saruwatari
Sample 1
| Speaker | Text (jp) | Text (en) | Intention | Emotion | Speaking style | Reliability score | GT Emotion |
| MStudent | 先生、今回のテストの点数すごくよかったよ！ | Hi, teacher. I got really good scores on this test! | 報告 (report) | 喜び (joy) | 爽やか (clear) | 5 | Happy |
| Teacher | おめでとう、頑張ったね！ | Congrats, you did great! | 祝福 (blessing) | 喜び (joy) | 落ち着いた (quiet) | 3 | Happy |
| MStudent | 先生のおかげだよ！ | Thank you for your help! | 報告 (report) | 信頼 (trust) | 穏やか (gentle) | 4 | Happy |
| Teacher | いやいや、あなたがすごく頑張ったからだよ。 | No, no. It's because you worked so hard. | 否定 (negation) | 期待 (anticipation) | 落ち着いた (quiet) | 5 | Happy |
Sample 2
| Speaker | Text (jp) | Text (en) | Intention | Emotion | Speaking style | Reliability score | GT Emotion |
| FStudent | 先週の地方大会で、すごくいい成績が残せたんです。 | I did really well at a regional tournament for my club last week. | 報告 (report) | 喜び (joy) | 生き生きした (vibrant) | 5 | Happy |
| Teacher | 聞いたよ、快挙だね！一生懸命練習した甲斐があったね！ | I hear you, that's an accomplishment! All that hard practice was worth it! | 祝福 (blessing) | 喜び (joy) | 爽やか (clear) | 5 | Happy |
| FStudent | ありがとうございます！努力が実を結ぶって、こんな感じなんだな、って思いました。 | Thank you! I think this is what it feels like when hard work pays off. | 感謝 (gratitude) | 喜び (joy) | 穏やか (gentle) | 5 | Happy |
| Teacher | 真面目にやってた人には、必ずいいことが起こるんだって。 | Good things always happen to people who were serious. | 賛辞 (praise) | 期待 (anticipation) | 知的 (intellectual) | 5 | Happy |
Sample 3
| Speaker | Text (jp) | Text (en) | Intention | Emotion | Speaking style | Reliability score | GT Emotion |
| MStudent | 今日学校で、些細なことで先生に注意されて、むかついたわ。 | I was so disgusted when a male teacher warned me about something so trivial at school today. | 注意されている (warned) | 怒り (anger) | 生き生きした (vibrant) | 3 | Angry |
| Teacher | なんで注意されちゃったの？ | Why did you get warned? | 問いかけ (question) | 信頼 (trust) | 落ち着いた (quiet) | 5 | Neutral |
| MStudent | 授業中にYouTubeみてたら、急にどなられたわ。 | I was watching YouTube in class and he suddenly yelled at me. | 報告 (report) | 怒り (anger) | 生き生きした (vibrant) | 3 | Angry |
| Teacher | 授業中にみてたら怒られるわよ。せめて休み時間にしたらいいのに。 | That's right. Why not at least do it during recess? | 疑念 (suspicion) | 期待 (anticipation) | クール (cool) | 4 | Neutral |
Sample 4
| Speaker | Text (jp) | Text (en) | Intention | Emotion | Speaking style | Reliability score | GT Emotion |
| FStudent | 今日は部活で「連帯責任」って言って全員お説教されちゃったよ。 | Today, everyone in the club got a sermon saying that "We are co-responsible for each other's performance." | 報告 (report) | 怒り (anger) | 生き生きした (vibrant) | 5 | Angry |
| Teacher | 連帯責任、いかにも学校らしいなぁ。 | "Co-responsible" sounds like how very school-like. | 疑念 (suspicion) | 落ち着いた (quiet) | 知的 (intellectual) | 2 | Neutral |
| FStudent | 先生が高校生のころもあった？ | Did you have the same experience when you were in high school? | 問いかけ (question) | 信頼 (trust) | 可愛い (cute) | 5 | Neutral |
| Teacher | あったあった。そのへんは今も昔も同じだから、怒る気持ちも分かるよ。 | Yes, yes. That part is the same now as it was then, so I can understand why you are angry. | 共感 (empathy) | 穏やか (gentle) | 誠実 (honest) | 5 | Neutral |
Sample 5
| Speaker | Text (jp) | Text (en) | Intention | Emotion | Speaking style | Reliability score | GT Emotion |
| MStudent | あそこまで行ったのに悔しい…。 | It's frustrating because we were so close to winning... | 悔しさ (regret) | 悲しみ (sadness) | 渋い (gravelly) | 4 | Sad |
| Teacher | じゅうぶん頑張ったじゃん。 | You've worked hard enough. | 鼓舞 (inspiration) | 喜び (joy) | 穏やか (gentle) | 3 | Neutral |
| MStudent | 自信持っていいのかな？ | Can I be confident? | 自信不足 (unconfident) | 悲しみ (sadness) | 落ち着いた (quiet) | 4 | Neutral |
| Teacher | あったり前じゃん！凄いことだよ！ | That's a thing! That's great! | 肯定 (affirmation) | 喜び (joy) | 生き生きした (vibrant) | 5 | Neutral |
Sample 6
| Speaker | Text (jp) | Text (en) | Intention | Emotion | Speaking style | Reliability score | GT Emotion |
| FStudent | こんなに点数が悪かったのははじめてで、すごくショックです。何が悪かったんでしょうか。 | This is the first time I have had such a bad score and I am very shocked. What did I do wrong? | 悲しみ (sadness) | ショック (shocked) | 落ち着いた (quiet) | 4 | Sad |
| Teacher | そういうこともあるよ。苦手なところからたくさん出題されたのかな。 | That's one of those things. I guess you got a lot of questions from areas you were not good at. | 同情 (sympathy) | 平静 (neutral) | 落ち着いた (quiet) | 4 | Neutral |
| FStudent | 単純ミスが多くて、自分でも、どうして間違ったのか分からないのばかりです。 | There are so many simple mistakes, and even I don't know how I got them wrong. | 悲しみ (sadness) | 悩み (worry) | 落ち着いた (quiet) | 5 | Sad |
| Teacher | 普段だったらできる問題ならだいじょうぶだよ。これを教訓にして、次回からは確実に点数を取っていくように頑張ろうね！ | If it's a problem you can normally do, you'll be fine. Let this be a lesson to you, so that next time you can get a good score for sure! | 励まし (encouragement) | 期待 (anticipation) | 穏やか (gentle) | 4 | Neutral |
Synthetic speech samples
| Method | Sample 1 | Sample 2 | Sample 3 | Sample 4 |
| Natural | ||||
| FS2+Emo | ||||
| FS2&CCE | ||||
| FS2&IES | ||||
| FS2&Emo&IES | ||||
| FS2&Emo&CCE | ||||
| FS2&CCE&IES |
(Bonus) Control of synthetic speech by changing IES words
Sample 1: text = "そうなんだね。" (en: I see.)| Speech | Intention | Emotion | Speaking style |
| 問いかけ (question) | 悲しみ (sadness) | 落ち着いた (quiet) | |
| 賛辞 (praise) | 喜び (joy) | 生き生きとした (vibrant) | |
| 共感 (empathy) | 怒り (anger) | 生き生きとした (vibrant) |
| 疑念 (suspicion) | 悲しみ (sadness) | クール (cool) | |
| 期待 (anticipation) | 喜び (joy) | 生き生きとした (vibrant) | |
| 問いかけ (question) | 平静 (neutral) | 丁寧 (polite) |
| 励まし (encouragement) | 喜び (joy) | 生き生きとした (vibrant) | |
| 共感 (empathy) | 悲しみ (sadness) | 落ち着いた (quiet) | |
| 助言 (advice) | 平静 (neutral) | 知的 (intellectual) |