Keywords

1 Introduction

When working in a company as a project member, the communication skill that connects the team is important. In recent years, there has been a growing number of companies that adopted group discussion in the recruitment of employers. In group discussion task, job applicants have to collaborate with each other on an assigned topic, where their communication skill and personality can be observed by the investigators of the companies. Therefore, giving the impression of a high communication skills to the recruiters, may increase one’s chance of success while job hunting.

In our ongoing project, we aim to develop a system that can provide advices to its users in improving the impression of their communication skill can be estimated from the verbal and nonverbal signals of the participants [1]. However, the estimation is based on the data of the whole period of the experiment session. There is only one result, that is, a judgment (high/low or a score) of the communication skill of each participant at the end of the discussion session. It could be difficult to utilize the results for developing a support system for the participants: the participants cannot know when and how to improve their behaviors.

In order to develop a support system, a finer unit at appropriate size of the behaviors of the participant is required. It is known from social science that people’s roles in group interaction structure nonverbal behavior in important ways [2]. In this paper, we propose the use of functional roles of the participants as the unit to trace the dynamics of the interaction among the participants. We considered that the functional roles can be treated as a template or a style of actual behaviors. By exploring the temporal transitions of the roles of an individual participant and the situation (the roles of other participants) were the participant was in, we expect that it is possible to derive the relationship between functional roles and impression of communication skill. The results are then supposed to be able be used in conducting the strategy in improving the impression of communication skill.

The analysis of the functional roles is based on a data corpus collected in actual group discussion experiments. It is composed of video and audio data of 10 groups by three sessions of group discussion conversation as well as sensor data like head motion and eye tracking. The participants’ communication skill on the whole corpus was evaluated by 21 experts who had the experience in personnel management or recruitment. After the definition of functional roles, firstly we analyzed the relationship between functional roles and communication skill impression to confirm whether the distribution of roles played by an individual participant has influence on the impression of his/her communication skill. Second, we analyzed the characteristics of the group evaluated as high communication skill comparing to the group evaluated as low in the sense of how they responded to the situation where they are in.

2 Related Works

Researchers in organizational psychology have studied the communication skill or the individual personality in group meetings for decades, uncovering statistical relationships between nonverbal behaviors, personality, hire ability, and professional performance. In the context of group meeting, based on the nonverbal features, including features like speaking turn, voice prosody, visual activity, and visual focus of attention feature and so on. Aran and Gatica-Perez [3] presented an analysis on the participants’ personality prediction in small group. Similarity, Okada et al. [1] developed a regression model to infer the score for communication skill using multi modal features including linguistic and non-verbal features: voice prosody, speaking turn, and head activity. Similar to our goal, Schiavo et al. [4] presented a system that monitors the group members’ non-verbal behaviors and acts for an automatic facilitator. It supports the follow of communication in a group conversation activity. Furthermore, job interviews also have been studied in the research field of multi modal interaction, too. Raducanu et al. [5] made use of “The Apprentice” reality TV shows, which features a competition for a real, highly paid corporate jobs. The study was carried out using non-verbal audio cues to predict the person with highest status and to predict the candidates going to be fired. Muralidhar et al. [6] implemented a behavioral training framework for students with the goal of improving the impressions their hospitality perceived by others. They also evaluated the relationship between automatically extracted nonverbal cues and various social signals in a correlation analysis.

Taken together, these studies show that high-prediction models could be achieved by using multi modal nonverbal information (speaking turn, voice prosody, visual activity, and visual focus of attention). Which can be used for predicting communication skills or personalities.

In the present study, we aim to develop a system that can provide advices to its users in improving the impression of their communication skill during group discussion. Our work therefore selected functional roles of participants by incorporating multi modal non-verbal and verbal features, to analyzed the relationship with the impression of communication skill. In order to be able to timely feedback on the performance of the participants, we also analyzed the relationships with the impression of communication skill and conversational situations in shorter interval compared with the former studies. And found the difference of performance between participants with high-score and low-score communication skill.

3 Group Discussion Corpus

In this section, in order to analyze the relationships between the functional roles and the impression of communication skill, it is necessary to achieve the communication skill models by using the group discussion corpus as training dataset. And the assessment of communication skills and the roles’ annotation of participants were also executed in this corpus.

Fig. 1.
figure 1

Setup of the experiment environment.

Full size image

3.1 Data Collection Environment

We therefore used a multi-modal corpus in which groups of four people discussed three different topics [7]. The experiment was to collect the verbal information and non-verbal information of the participants in the group discussion. Then, recruited 40 Japanese university students who did not know each other before to participant in the experiment, and divided them into 10 groups of four people.

Each group had three discussion sessions in Japanese. In order to simulate the actual job interviews, they reviewed the group discussion tasks which were used frequently in recruiting in Japanese companies. And three kinds of discussion themes were designed as following: Celebrity guest selection, Booth planning for school festival, and Travel planning for foreigner friends. And the state of the discussion has been recorded and measured using a variety of sensors (Fig. 1).

3.2 Assessment of Communication Skills

The corpus also includes the communication skills of each participant, which was assessed by 21 external observers with the experience of human resource management. In this paper, we presented a brief overview of the assessment of communication skills [1]. The communication skills were defined according to the paper [8, 9], including four categories [smooth interaction, aggregation of opinions, communicating one’s own claim, logical and clear presentation] and total communication skill to assess the communication skills of the participants. The scores of four traits were evaluated between one and five, and the total communication skill was evaluated between one and ten. The observers assessed the skills by watching the video of the group discussion. The procedure applied to the assessment was as follows. The video of a session was segmented into three portions (five to seven minutes). Of the 21 observers, seven assessed the skill score by watching one of three partitions in a session. This means that the session is assessed by all of the observers (seven observers * three partitions). The agreement between the observers for each skill index was confirmed in terms of Cornbach’s alpha (\(\alpha \)), and all of \(\alpha \) the values of are greater than 0.85.

3.3 Annotations of Roles

In terms of roles, the paper [10, 11] indicated that the participant plays one of three type of roles, including group-task role, maintenance role, and individual role in group discussion. According to the previous works and the observation on our data corpus, we also added non-verbal description into the definition of roles. And finally defined seven types of roles, including follower, gatekeeper, information giver, objector, opinion provider, passive participant, and summarizer (Table 1).

In the part of roles annotation, three annotators watched the videos and annotated the roles of participants according to the definition above. Each annotator was assigned with the data of four groups. One group was randomly selected for the measurement of inter-annotator reliability. The pair-wise Kappa Coefficients were 0.41, 0.51, and 0.63.

Table 1. The overview of roles’ definition
Full size table

4 Communication Skill Models

4.1 Guidelines for Creating Models

In order to identify whether there has relationships between roles and communication skills, we used the duration ratio of each type of role as feature at both regression and classification problems. Due to a part of audio files cannot be used for roles’ annotation, the 110 data points are used for the experiments. Evaluation is done with a cross validation testing, we used data samples observed from participants in one group for test data and remaining samples from groups for training.

4.2 Regression Model and Results

In the regression experiments, we assessed the models on predicting the actual communication skill impression scores for five traits. We used \(R^2\) (coefficient determination) as a criteria of performance of the regression model, and the results are shown in Fig. 2 From this figure, the \(R^2\) of the regression model of total communication ability is 0.55, while aggregation of opinions is 0.56.

Fig. 2.
figure 2

Result of support vector machine regression model

Full size image

4.3 Classification Model and Results

For the classification experiments, we converted the problem to a classification problem by defining two classes for each trait as high and low, based on the scores. More specifically, to calculate the mean m and the standard deviation \(\sigma \) of all evaluation value in five traits in first. Then, classify the evaluation values which were over m + \(\beta \sigma \) into the label of high, while below m − \(\beta \sigma \) were classified into the label of low. Considering the balance of the size of two classes, we decided to set \(\beta \) as 0.5. We chose to use logistic in the classification analysis, and estimated the model with the values of precision, recall and F-Measure. And Table 2 showed the estimation results of five traits.

Table 2. Results of logistic classification model
Full size table

From Table 2 we can indicate that the precisions and F-Measure of the classification model of aggregation of opinions is 0.903, while total communication skill is 0.892. That is to say that we can confirm the communication skill impression of participant is high or low from the duration ratios of roles in a high precision by using our classification model.

Table 3 showed the significance probability of each role that influence the classification between the label of high and low in independent T-tests.

Table 3. Significance test result of each role. +: p < .1, *: p < .05, **: p < .01, n.s.: not significant
Full size table

According to the result that illustrated by Table 3, we confirmed that the roles of gatekeeper and opinion provider are strongly correlated with classification between two classes, and those who are whether trying to maintain or improve the social relationships within group would have great difference in the communication skill impression. While follower and objector are weakly compared with other roles.

5 Analysis of Conversational Situations

We can confirm there is a certain relationship between roles and communication skill impression of the participants according to the result in last section. Moreover, these analyses were overlooked by a long time which about 15 min to 20 min in one session, and lead to cannot provide current advice for users in time. Therefore, in order to be suitable for real-time processing, we considered that if extracting the roles of participants in more details, and analyzing the conversational situations from participants’ individual point of view that would get some useful information that can be added into our supporting system. For example, which kind of situation often appear, and high-communication-skill people would to play which type of roles in this situation. On the other hand, we also tried to find what the difference between high-skill-score and low-skill-score’ performance in same situation, and analyzed which kind of mode impacts the communication skill impression.

5.1 Analysis Method

Although there were four participants in each group, the layout was symmetric from the view point view of each participant. In this section, we call the participant who is being analyzed as the center participant. The conversational situation is then modeled as the combination of the role of the center participant and a set of roles of the other participants. Each participant group conducted group discussion in three topics and therefore 30 experiment sessions were recorded. Due to partial data failures, 26 of them were annotated. Each session was divided to three segments with length from five to seven minutes. Since there are four participants in one group, this leads to 312 data. According to the Nyquist-Shannon sampling theorem, perfect reconstruction is guaranteed possible for a band limit B < fs/2. Considering that the minimum interval of role annotation in whole dataset is about 300 ms, in order to explore the temporal dynamics of group discussion, all 312 data points were sliced into 100 ms intervals.

5.2 Overall Distribution of Situations

There were 1,084,206 intervals in total. The occurrence frequency of the conversational situations is shown in Fig. 2. From the whole data corpus, 63 kinds of situations were found in total. The 36 situations which had occurrences less than 1% were combined in category, “Others.” Overall, there were no dominating situations, the distribution of situations was relatively averaged. However, if we look into the role distribution of more frequent situations, it could be found that the activity of the participants was not high while they were often playing passive or follower roles. With relatively fewer occurrences, one of the participants provided objective information or tried to facilitate the discussion, but few of them tried to express their opinions.

Fig. 3.
figure 3

Occurrence frequency of conversational situations. The letter outside of “{}” is role of center participant, and the roles of the other three participants were shown in “{}” ’s. Here, the abbreviations of functional roles are shown as the follows: F: Follower; G: Gatekeeper; I: Information giver; B: Objector; O: Opinion provider; P: Passive participant; S: Summarizer

Full size image

5.3 Analysis of Situations Difference

Next, we analyzed how different the participants with high scores responded to the situations than the participants with low scores. We selected the participants evaluated with scores at least 0.4 times of standard deviation \(\sigma \) higher than the mean m of all participants as high group and the ones with scores at least 0.4\(\sigma \) lower than m as low group.

Figure 3 shows the results of the comparison between high and low groups. Then, we conducted an independent t test to verify which situation differed between high-score and low-score. From which we understand the followings (Fig. 4):

Fig. 4.
figure 4

Comparison between the participants with high scores and low scores. The results of two-tailed t test: +: p < .1, *: p < .05, **: p < .01, n.s.: not significant

Full size image

  • The conversational situations that have a significantly high frequency of high-score communication skill at the time of F{G,P,P}, P{F,F,F}, O{P,P,P}, F{P,P,P}, F{O,P,P}, G{F,F,F}, O{F,F,P}, G{F,F,P}, and O{F,P,P}. That is to say, the participants with high-score communication skill preferred to state their own opinions when the others had lower activity. The high group participants also played the gatekeeper role more frequently in similar situations. Apart from P{F,F,F}, they almost to show a positive attitude even the other participants had no feedback to the problem being discussed.

  • Oppositely, the participants evaluated with low scores had significantly higher frequency in the situations: P{F,F,P}, P{F,F,G}, F{F,F,P}, P{F,F,O}, P{F,P,P}, P{O,P,P}, F{F,F,G}, F{F,P,P}, and P{F,O,P}. Compared with high-score participants, low-score participants tended to be “Follower” or “Passive participant” no matter whether anyone was expressing their opinion or not. To summarize, the high-score participants more actively joined the discussion, express their opinions more, and lead the discussion. On the other hand, the low-score participants were passive to the discussion and showed negative attitude.

6 Conclusion and Future Direction

In this study, seven functional roles of the participants were defined. We firstly proposed regression and classification model for analyzing the relationship between functional roles and communication skill of participants in group discussion, which were based on a previous collected database. The performance of the proposed classification model was 0.838 in its F-Measure. We also calculated the significance probability of the duration ration of each role, and confirmed four types of roles influenced the communication skill impression of a participant most. On the basis of these results, we analyzed the relationship between conversational situations and the total communication skill score. The conversational situation was modeled as the combination of the roles of the participants. From the analysis results, it was found that the participant with high scored of communication skill did show clearly different behaviors (playing the roles) than the ones evaluated as low. They were more active, tended to express their own opinions, and facilitate the discussion.

In terms of future direction, we note that only use the duration ratio of roles as feature is not sufficient, so we will pay attention to the characteristic of runtime, to add some more detailed features in the next step. For example, the verbal features, the interaction with other participants, the role change patterns of the participants. Finally, to establish the supporting system of communication skill that teach the users to play what kind of role will have high communication skill impression, based on the types of roles which participants play.