Intercultural expression and perception of Japanese, English & French social affects

A. Rilliard with T. Shochi & V. Aubergé

Introduction

Affects in speech are expressed according to different cognitive processing levels, from involuntarily controlled expressions of emotion to the intentionally, voluntarily controlled expressions of attitudes (or social affects) of the speaker. Attitudes and Emotions (as “labels” or “notions”) are sometimes confused since both use specialized prosodic encoding in the acoustic signal. Since the prosodic emotional expressions are carried in parallel with the prosodic linguistic structures, then we can view expressions of attitudes as an integral part of communication. Some affects such as surprise, are classified either as attitudes or emotions. Our position is to assume that surprise can be an emotion if it is involuntarily processed, and can be an attitude when it is voluntarily processed and needs to be learned. Studying affects in spoken interaction is crucial for expressive speech modelling, and we need to describe precisely what the attitudes of a speaker in a language and a culture can be and how they are linguistically encoded. This cross-cultural study is a helpful method to examine this problem. Some attitudes can be expected to be universal in part, both in terms of what is expressed and how it is expressed. But because attitudes are socially constructed for and by the language, they can exist in one language but not another; moreover the prosodic realization of one specific attitude in a specific language may not be recognized (or may be ambiguous) in another language.

Selection of attitudinal expressions in Japanese, English and French

Several studies on French (Auberge et al., 1997, Morlec et al., 2001), English (Diaferia, 2002) and Japanese (Shochi et al., 2006) draw up lists of attitudes or these three languages (some sound examples are also given – details on the corpuses are given in these publications):

This works mainly emphasis on the cross-cultural perception of socials affects, and our first results concern the perception of Japanese attitudes by either Japanese and foreign listeners. The set of 12 Japanese attitudes were selected for their representativeness according to the literature and Japanese language teaching methods. Some of these attitudes are specific or specifically important for the Japanese culture, especially those linked to the politeness strategy: simple-politeness, sincerity-politeness and kyoshuku vs. arrogance. The sincerity-politeness attitude appears when a speaker considered inferior is talking to someone superior in the Japanese society: the speaker expresses a serious and sincere intention by using this prosodic attitude. The kyoshuku attitude (there is no lexical entry to translate this in English) is a typically Japanese cultural attitude. Even if such situations occur in all cultures, the Japanese language has chosen to encode this situation as an attitude (“attitudineme”). A speaker uses kyoshuku when he wants to express a conflicting opinion to an interlocutor considered socially as superior – aiming at not disturbing him but to help him, or when the speaker is desiring to get a favour from his superior. It is described by Sadanobu (2004) as “a mixture of suffering ashamedness and embarrassment, (which) comes from the speaker’s consciousness of the fact that his/her utterance of request imposes a burden to the hearer” (p.34). The corpus consists of seven sentences of 1, 2, 5 or 8 moras, with a systematic variation of the lexical stress position for 8-maro length sentences. The sentences were constructed in order to have no particular affective connotations in any region of Japan, and a male Japanese native language teacher produces each sentence with all the 12 attitudes.

Table 1: Corpus of Japanese attitudes – the lexical stress is marked with a star.

Experiments & results

1- PARADIGM

A computer interface was developed and used amongst the different groups of listeners. It proposes a translation of each attitudinal label in the listener’s native language, who were introduced by a long oral explanation, giving many examples of relevant situations. Each subject listened to each stimulus one time only. For each stimulus, they were asked to indicate the perceived attitude amongst the twelve. Listeners from three languages were tested during the experiment: Japanese, English and French native speakers. Japanese listeners were used in effect to validate the quality of the recorded attitudes; French and Americans languages were used to represent cultures and languages distant from Japanese.

2- VALIDATION WITH JAPANESE LISTENERS

Native Japanese speakers evalute the corpus of attitudes in order to rate the quality of the performance. Results show:

• No effect of stress

• Restricted effect of length

• All attitudes are recognized

  - most without confusions: Arrogance, Declaration, Doubt, Irritation, Politeness, Question, Surprise
  - some with confusions with others attitudes:

Kyoshuku is mixed with Sincerity-Politeness SincerityPoliteness is mixed with Politeness and Kyoshuku Admiration is mixed with Politeness Authority is mixed with Evidence Evidence is mixed with Arrogance

Such results show that attitudinal expression are recognized by listeners, as all confusions remain inside more general categories.

Plot of the results of correspondence analysis made on the perception tests’ results (4 dimension explain 2/3 of the variance):

3- PERCEPTION BY NAIVE FOREIGN LISTENERS

AMERICAN ENGLISH

Results for American English listeners, naive in Japanese, show:

• No effect of stress

• Restricted effect of length

• Recognition of attitudes:

  - Without confusions: IR, PO, AU
  - With confusions: Declaration & Question mixed with Politeness; Surprise with Doubt + question
  - Not recognized: 
          Arrogance (mixed with: declaration, evidence, authority)
          Evidence (mixed with: authority, declaration)
          Admiration (mixed with: sincerity-politeness, surprise)
          Kyoshuku (mixed with: irritation): typical “false friend” behaviour
          Sincerity (no confusion tendency): misunderstood attitudinal expression
  - The Politeness attitudes function as an attractor.

Plot of the results of correspondence analysis made on the perception tests’ results (4 dimension explain 90% of the variance):

FRENCH

Results for French listeners, naive in Japanese, show:

• No effect of stress

• Restricted effect of length

• Recognition of attitudes

- without confusion: Admiration, Authority, Irritation
- with confusions: Sincerity-Politeness mixed with Politeness; Doubt with Surprise; Evidence with Declaration
- Low recognition: Question, Arrogance
- Not recognized: Kyoshuku (mixed with: irritation, arrogance, authority): typical “false friend” behaviour

Plot of the results of correspondence analysis made on the perception tests’ results (4 dimension explain 85% of the variance):

4- EFFECT OF JAPANESE LANGUAGE LEARNING ON FRENCH LEARNERS

LEVEL 1 (beginners)

At this level of Japanese (beginners), the subjects have learned to identify the sincerity and the doubt. They have changed some confusions and misinterpret sincerity-serious with declaration and arrogance with evidence. They ares some mutual confusions between doubt and surprise. They have learned to learned discriminate arrogance vs. authority, politeness vs. sincerity-serious, declaration -> evidence. But however, are still confused Kyoshuku with (irritation, arrogance-impoliteness, authority), arrogance with declaration and mainly interrogation with declaration, that can be a strong communicative handicap for such subjects starting to interact in Japanese. Graphical representation of the results obtained by level 1 French learners of Japanese:

LEVEL 2

At this level, where the French subjects start to be fluent in Japanese, they did not learn to identify any other attitudes. Arrogance and Kyoshuku are still not recognized. They however learned at least to discriminate question and declaration, but Kyoshuku is still confused with (irritation, arrogance-impoliteness, authority), and arrogance with (declaration,evidence). They have changed confusion of declaration with sincerity-serious. Doubt and surprise are still mutually confused (which is not the case for Japanese listeners, and not the case for French in French doubt and surprise)

Graphical representation of the results obtained by level 2 French learners of Japanese:

Perspectives

Some others analysis are currently proceeded, namely: - a detailed acoustical analysis of stimuli; - a perception test based on a gating paradigm, aimed at localising the relevant information in the sentences; - an Audio Visual perception test of both Japanese and French attitudes; - complete this cross cultural analysis by testing the same groups of listeners on French and English attitudes.

Other languages will also be studied (Vietnamese, Hungarian), and other groups of listeners (Chinese, Korean) in order to have a more detailed sketch of the cross cultural production and perception of social affects.

Acknowledgments

We are deeply grateful to T. Sadanobu and M. Nakagawa (Kobe University) for their financial support and their helpful advices on Japanese attitudes. We also thank D. Erickson and students at Black Hills State University, Spearfish, S.D. for the American perception tests. This work was performed as collaboration between Graduate School of Cultural Studies and Humane Science, Kobe university and GIPSA-lab in the frame of the “College Doctoral Franco-Japonais”, directed by French and Japanese foreign offices, and also as a part of Crest-JST “Expressive Speech Processing” Project, directed by Nick Campbell, ATR, Japan.

Refernces

Aubergé, V., Grépillat, T., Rilliard, A. 1997. Can we perceive attitudes before the end of sentences? The gating paradigm for prosodic contours. 5th Eurospeech, 871-874.

Diaferia, M.L. 2002. Les attitudes de l’anglais : premiers indices prosodiques. Mémoire de DEA de Sciences Cognitives, INPG, Grenoble.

Morlec, Y., Bailly, G., Aubergé, V. 2001. Generating prosodic attitudes in French: data, model and evaluation. Speech Communication 33 (4), 357-371.

Sadanobu, T. 2004. A natural history of Japanese pressed voice. J. of the Phonetic Society of Japan 8 (1), 29-44.

Shochi, T., Aubergé, V., Rilliard, A., 2006. How prosodic attitudes can be false friends: Japanese vs. French social affects. Speech Prosody, Dresden, 692-696.

Links

PS