Hostname: page-component-8448b6f56d-c4f8m Total loading time: 0 Render date: 2024-04-25T05:44:50.978Z Has data issue: false hasContentIssue false
  • English
  • Français

Multimodal Emotion Integration in Bipolar Disorder: An Investigation of Involuntary Cross-Modal Influences between Facial and Prosodic Channels

Published online by Cambridge University Press:  11 April 2014

Tamsyn E. Van Rheenen  and
Susan L. Rossell
Tamsyn E. Van Rheenen*
Affiliation:
Brain and Psychological Sciences Research Centre, Swinburne University Monash Alfred Psychiatry Research Centre, Central Clinical School, Monash University and the Alfred Hospital
Susan L. Rossell
Affiliation:
Brain and Psychological Sciences Research Centre, Swinburne University Monash Alfred Psychiatry Research Centre, Central Clinical School, Monash University and the Alfred Hospital
*
Correspondence and reprint requests to: Tamsyn Van Rheenen, Monash Alfred Psychiatry research centre (MAPrc), Level 4, 607 St Kilda Road, Melbourne, VIC 3004, Australia. E-mail: tvanrheenen@swin.edu.au
Get access Rights & Permissions [Opens in a new window]

Abstract

The ability to integrate information from different sensory channels is a vital process that serves to facilitate perceptual decoding in times of unimodal ambiguity. Despite its relevance to psychosocial functioning, multimodal integration of emotional information across facial and prosodic modes has not been addressed in bipolar disorder (BD). In light of this paucity of research we investigated multimodal processing in a BD cohort using a focused attention paradigm. Fifty BD patients and 52 healthy controls completed a task assessing the cross-modal influence of emotional prosody on facial emotion recognition across congruent and incongruent facial and prosodic conditions, where attention was directed to the facial channel. There were no differences in multi-modal integration between groups at the level of accuracy, but differences were evident at the level of response time; emotional prosody biased facial recognition latencies in the control group only, where a fourfold increase in response times was evident between congruent and incongruent conditions relative to patients. The results of this study indicate that the automatic process of integrating multimodal information from facial and prosodic sensory channels is delayed in BD. Given that interpersonal communication usually occurs in real time, these results have implications for social functioning in the disorder. (JINS, 2014, 20, 1–9)

Keywords

CognitionAffectMood disordersMultisensory emotion processingUnimodal processingAudio-visual perceptionAffective prosodyFacial emotionNeuropsychology
Type
Research Articles
Information
Journal of the International Neuropsychological Society , Volume 20 , Issue 5 , May 2014 , pp. 525 - 533
Copyright
Copyright © The International Neuropsychological Society 2014 

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

References

REFERENCES

Alais, D., & Burr, D. (2004). The ventriloquist effect results from near-optimal bimodal integration. Current Biology, 14, 257–262. doi:http://dx.doi.org/10.1016/j.cub.2004.01.029Google Scholar
Australian Bureau of Statistics (2007). National survey of mental health and wellbeing: Summary of results. Canberra: Australian Bureau of Statistics.Google Scholar
Blairy, S., Linotte, S., Souery, D., Papadimitriou, G.N., Dikeos, D., Lerer, B., … Mendlewicz, J. (2004). Social adjustment and self-esteem of bipolar patients: A multicentric study. Journal of Affective Disorders, 79, 97–103.Google Scholar
Borod, J.C., Pick, L.H., Hall, S., Sliwinski, M., Madigan, N., Obler, L.K., … Tabert, M. (2000). Relationships among facial, prosodic, and lexical channels of emotional perceptual processing. Cognition & Emotion, 14, 193–211.Google Scholar
Brekke, J., Kay, D.D., Lee, K.S., & Green, M.F. (2005). Biosocial pathways to functional outcome in schizophrenia. Schizophrenia Research, 80, 213–225.Google Scholar
Coryell, W., Scheftner, W., Keller, M., Endicott, J., Maser, J., & Klerman, G.L. (1993). The enduring psychosocial consequences of mania and depression. The American Journal of Psychiatry, 150, 720–727.Google Scholar
Craddock, N., O’Donovan, M.C., & Owen, M.J. (2006). Genes for schizophrenia and bipolar disorder? Implications for psychiatric nosology. Schizophrenia Bulletin, 32, 9–16. doi:10.1093/schbul/sbj033Google Scholar
Craddock, N., O’Donovan, M.C., & Owen, M.J. (2005). The genetics of schizophrenia and bipolar disorder: Dissecting psychosis. Journal of Medical Genetics, 42, 193–204. doi:10.1136/jmg.2005.030718Google Scholar
Cramer, V., Torgersen, S., & Kringlen, E. (2010). Mood disorders and quality of life. A community study. Nordic Journal of Psychiatry, 64, 58–62. doi:10.3109/08039480903287565Google Scholar
De Gelder, B., & Bertelson, P. (2003). Multisensory integration, perception and ecological validity. Trends in Cognitive Sciences, 7, 460–467. doi:http://dx.doi.org/10.1016/j.tics.2003.08.014Google Scholar
de Gelder, B., Böcker, K.B.E., Tuomainen, J., Hensen, M., & Vroomen, J. (1999). The combined perception of emotion from voice and face: Early interaction revealed by human electric brain responses. Neuroscience Letters, 260, 133–136. doi:http://dx.doi.org/10.1016/S0304-3940(98)00963-XGoogle Scholar
de Gelder, B., & Jean, V. (2000). The perception of emotions by ear and by eye. [Article]. Cognition & Emotion, 14, 289–311. doi:10.1080/026999300378824Google Scholar
de Gelder, B., Meeren, H.K.M., Righart, R., Stock, J. v. d., van de Riet, W.A.C., & Tamietto, M. (2006). Beyond the face: Exploring rapid influences of context on face processing. In S. Martinez-Conde, S.L. Macknik, L.M. Martinez, J.M. Alonso & P.U. Tse (Eds.), Progress in brain research (Vol. 155, Part 2, pp. 37–48). Amsterdam: Elsevier.Google Scholar
de Gelder, B., Pourtois, G., Vroomen, J., & Bachoud-Lévi, A.-C. (2000). Covert Processing of faces in prosopagnosia is restricted to facial expressions: Evidence from cross-modal bias. Brain and Cognition, 44, 425–444. doi:http://dx.doi.org/10.1006/brcg.1999.1203Google Scholar
de Gelder, B., Vroomen, J., de Jong, S.J., Masthoff, E.D., Trompenaars, F.J., & Hodiamont, P. (2005). Multisensory integration of emotional faces and voices in schizophrenics. [3]. Schizophrenia Research, 72, 195–203.Google Scholar
de Jong, J.J., Hodiamont, P.P.G., Van den Stock, J., & de Gelder, B. (2009). Audiovisual emotion recognition in schizophrenia: Reduced integration of facial and vocal affect. Schizophrenia Research, 107, 286–293.Google Scholar
Dolan, R.J., Morris, J.S., & de Gelder, B. (2001). Crossmodal binding of fear in voice and face. Proceedings of the National Academy of Sciences of the United States of America, 98, 10006–10010. doi:10.1073/pnas.171288598Google Scholar
Ekman, P., & Friesen, W.V. (1976). Pictures of facial affect. Palo Alto, CA: Consulting Psychologists Press.Google Scholar
Freeman, A.J., Youngstrom, E.A., Michalak, E., Siegel, R., Meyers, O.I., & Findling, R.L. (2009). Quality of life in pediatric bipolar disorder. Pediatrics, 123, e446–e452. doi:10.1542/peds.2008-0841Google Scholar
Goldstein, T.R., Miklowitz, D.J., & Mullen, K.L. (2006). Social skills knowledge and performance among adolescents with bipolar disorder. Bipolar Disorders, 8, 350–361. doi:10.1111/j.1399-5618.2006.00321.xGoogle Scholar
Goodwin, F.K., & Jamison, K.R. (1990). Manic depressive illness. New York: Oxford University Press.Google Scholar
Hoertnagl, C.M., Muehlbacher, M., Biedermann, F., Yalcin, N., Baumgartner, S., Schwitzer, G., … Hofer, A. (2011). Facial emotion recognition and its relationship to subjective and functional outcomes in remitted patients with bipolar I disorder. Bipolar Disorders, 13, 537–544. doi:10.1111/j.1399-5618.2011.00947.xGoogle Scholar
Kim, R.S., Seitz, A.R., & Shams, L. (2008). Benefits of stimulus congruency for multisensory facilitation of visual learning. PLoS One, 3, e1532. doi:10.1371/journal.pone.0001532Google Scholar
Kohler, C.G., Walker, J.B., Martin, E.A., Healey, K.M., & Moberg, P.J. (2010). Facial emotion perception in schizophrenia: A meta-analytic review. Schizophrenia Bulletin, 36, 1009–1019. doi:10.1093/schbul/sbn192Google Scholar
Laurienti, P.J., Wallace, M.T., Maldjian, J.A., Susi, C.M., Stein, B.E., & Burdette, J.H. (2003). Cross-modal sensory processing in the anterior cingulate and medial prefrontal cortices. Human Brain Mapping, 19, 213–223. doi:10.1002/hbm.10112Google Scholar
MacQueen, G.M., Young, L.T., & Joffe, R.T. (2001). A review of psychosocial outcome in patients with bipolar disorder. Acta Psychiatrica Scandinavica, 103, 163–170.Google Scholar
Martino, D.J., Strejilevich, S.A., Fassi, G., Marengo, E., & Igoa, A. (2011). Theory of mind and facial emotion recognition in euthymic bipolar I and bipolar II disorders. Psychiatry Research, 189, 379–384. doi:10.1016/j.psychres.2011.04.033Google Scholar
McGurk, H., & MacDonald, J. (1976). Hearing lips and seeing voices. Nature, 264, 746–748.Google Scholar
McKirdy, J., Sussmann, J.E.D., Hall, J., Lawrie, S.M., Johnstone, E.C., & McIntosh, A.M. (2009). Set shifting and reversal learning in patients with bipolar disorder or schizophrenia. Psychological Medicine, 39, 1289–1293. doi:doi:10.1017/S0033291708004935Google Scholar
Melcher, T., Wolter, S., Falck, S., Wild, E., Wild, F., Gruber, E., … Gruber, O. (2013). Common and disease-specific dysfunctions of brain systems underlying attentional and executive control in schizophrenia and bipolar disorder. European Archives of Psychiatry and Clinical Neuroscience, [Epub ahead of print]. doi:10.1007/s00406-013-0445-9Google Scholar
Montgomery, S.A., & Asberg, M. (1979). A new depression scale designed to be sensitive to change. British Journal of Psychiatry, 134, 382–389.Google Scholar
Morriss, R., Scott, J., Paykel, E., Bentall, R., Hayhurst, H., & Johnson, T. (2007). Social adjustment based on reported behaviour in bipolar affective disorder. Bipolar Disorders, 9, 53–62. doi:10.1111/j.1399-5618.2007.00343.xGoogle Scholar
Neurobehavioral Systems Inc. (2012). Presentation,Version 14.8.Google Scholar
Nuechterlein, K., & Green, M.F. (2006). MATRICS Consensus Cognitive Battery manual. Los Angeles: MATRICS Assessment Inc.Google Scholar
Paulmann, S., & Pell, M.D. (2011). Is there an advantage for recognizing multi-modal emotional stimuli? Motivation and Emotion, 35, 192–201.Google Scholar
Paulmann, S., Titone, D., & Pell, M.D. (2012). How emotional prosody guides your way: Evidence from eye movements. Speech Communication, 54, 92–107. doi:10.1016/j.specom.2011.07.004Google Scholar
Pell, M. (2005). Nonverbal emotion priming: Evidence from the ‘facial affect decision task’. Journal of Nonverbal Behavior, 29, 45–73. doi:10.1007/s10919-004-0889-8Google Scholar
Phillips, M.L., Drevets, W.C., Rauch, S.L., & Lane, R. (2003). Neurobiology of emotion perception II: Implications for major psychiatric disorders. Biological Psychiatry, 54, 515–528.Google Scholar
Phillips, M.L., Ladouceur, C.D., & Drevets, W.C. (2008). A neural model of voluntary and automatic emotion regulation: Implications for understanding the pathophysiology and neurodevelopment of bipolar disorder. Molecular Psychiatry, 13, 833–857.Google Scholar
Pourtois, G., de Gelder, B., Bol, A., & Crommelinck, M. (2005). Perception of facial expressions and voices and of their combination in the human brain. Cortex, 41, 49–59. doi:http://dx.doi.org/10.1016/S0010-9452(08)70177-1Google Scholar
Rossell, S.L., Van Rheenen, T.E., Groot, C., Gogos, A., & Joshua, N.R. (2013). Investigating affective prosody in psychosis: A study using the Comprehensive Affective Testing System. Psychiatry Research, 210, 896–900. doi:10.1016/j.psychres.2013.07.037Google Scholar
Ryan, K.A., Vederman, A.C., Kamali, M., Marshall, D., Weldon, A.L., McInnis, M.G., & Langenecker, S.A. (2013). Emotion perception and executive functioning predict work status in euthymic bipolar disorder. Psychiatry Research, 210, 472–478. doi:http://dx.doi.org/10.1016/j.psychres.2013.06.031Google Scholar
Saarni, S.I., Viertiö, S., Perälä, J., Koskinen, S., Lönnqvist, J., & Suvisaari, J. (2010). Quality of life of people with schizophrenia, bipolar disorder and other psychotic disorders. The British Journal of Psychiatry, 197, 386–394. doi:10.1192/bjp.bp.109.076489Google Scholar
Samamé, C., Martino, D.J., & Strejilevich, S.A. (2012). Social cognition in euthymic bipolar disorder: Systematic review and meta-analytic approach. Acta Psychiatrica Scandinavica, 125, 266–280. doi:10.1111/j.1600-0447.2011.01808.xGoogle Scholar
Shams, L., Kamitani, Y., & Shimojo, S. (2004). Modulations of visual perception by sound. In G.A. Calvert, C. Spence & B.E. Stein (Eds.), The handbook of multisensory processes. Cambridge, MA: MIT Press.Google Scholar
Sheehan, D.V., Lecrubier, Y., Harnett Sheehan, K., Amorim, P., Janavs, J., Weiller, E., … Dunbar, G.C. (1998). The Mini-International Neuropsychiatric Interview (MINI): The development and validation of a structured diagnostic psychiatric interview for DSM-IV and ICD-10. The Journal of Clinical Psychiatry, 59, 22–33.Google Scholar
Tsai, S.-Y., Lee, J.-C., & Chen, C.-C. (1999). Characteristics and psychosocial problems of patients with bipolar disorder at high risk for suicide attempt. Journal of Affective Disorders, 52, 145–152. doi:10.1016/s0165-0327(98)00066-4Google Scholar
Van Rheenen, T.E., & Rossell, S.L. (2013a). Auditory-prosodic processing in bipolar disorder; from sensory perception to emotion. Journal of Affective Disorders, 151, 1102–1107.Google Scholar
Van Rheenen, T.E., & Rossell, S.L. (2013b). Is the non-verbal behavioural emotion-processing profile of bipolar disorder impaired? A critical review. Acta Psychiatrica Scandinavica, 128, 163–178. doi:10.1111/acps.12125Google Scholar
Van Rheenen, T.E., & Rossell, S.L. (2014). Let’s face it: Facial emotion processing is impaired in bipolar disorder. Journal of the International Neuropsychological Society, 20, 200–208. doi:10.1017/S1355617713001367Google Scholar
Vederman, A.C., Weisenbach, S.L., Rapport, L.J., Leon, H.M., Haase, B.D., Franti, L.M., … McInnis, M.G. (2012). Modality-specific alterations in the perception of emotional stimuli in bipolar disorder compared to healthy controls and major depressive disorder. Cortex, 48, 1027–1034.Google Scholar
Vroomen, J., & De Gelder, B. (2000). Sound enhances visual perception: Cross-modal effects of auditory organization on vision. Journal of Experimental Psychology: Human Perception and Performance, 26, 1583–1590.Google Scholar
Vroomen, J., Driver, J., & Gelder, B. (2001). Is cross-modal integration of emotional expressions independent of attentional resources? Cognitive, Affective, & Behavioral Neuroscience, 1, 382–387. doi:10.3758/cabn.1.4.382Google Scholar
Young, R., Biggs, J., Ziegler, V., & Meyer, D. (1978). A rating scale for mania: Reliability, validity and sensitivity. The British Journal of Psychiatry, 133, 429–435. doi:10.1192/bjp.133.5.429Google Scholar