Cross-lingual emotion disentanglement and transfer in speech synthesis

This paper addresses the issue of poor emotional expression in speech synthesis under low-resource conditions by exploring emotion disentanglement and transfer methods in a cross-lingual emotional text-to-speech. By utilizing speech data from languages with rich emotional datasets as references, the emotional expression capabilities of speakers in low-resource languages are expected to be enhanced. To tackle the entanglement of speech emotion and content information, an emotional disentanglement method is proposed that combines general speech features with supra-segmental emotional prosody features, distinguishing between articulatory-related and content-related emotional characteristics. Furthermore, to address the challenge of re-encoding information caused by differences in emotional prosody patterns across languages, a decorrelation projection transformation method is introduced, thereby improving the quality and emotion presentation of synthesized speech. Subjective and objective evaluation experiments demonstrate that the proposed method outperforms existing baselines and state-of-the-art (SOTA) methods in cross-lingual emotional transfer synthesis. The mean opinion score (MOS) for emotional expressiveness is increased by approximately 0.2 and the probability of emotional category confusion is reduced by over 10%. In the meantime, there have also been slight improvements in speech quality, content accuracy, and speaker similarity.