Hostname: page-component-7c8c6479df-p566r Total loading time: 0 Render date: 2024-03-28T23:10:47.529Z Has data issue: false hasContentIssue false

REACTIVITY AND TYPE OF VERBAL REPORT IN SLA RESEARCH METHODOLOGY: Expanding the Scope of Investigation

Published online by Cambridge University Press:  22 July 2005

Melissa A. Bowles
Affiliation:
University of Illinois at Urbana-Champaign
Ronald P. Leow
Affiliation:
Georgetown University

Abstract

The present study addresses the reactivity of two types of verbal protocols in SLA research. It expands on the work of Leow and Morgan-Short (2004), who found nonmetalinguistic verbalization during a second-language reading task to be nonreactive for beginning learners' text comprehension, intake, and production of the targeted morphological form. The present study investigated the reactivity of both metalinguistic and nonmetalinguistic protocols, using a syntactic structure and advanced language learners of Spanish. Results indicated that neither type of verbalization significantly affected text comprehension or written production of old or new exemplars of the targeted structure when compared to a control group, although metalinguistic verbalization appeared to cause a significant decrease in text comprehension over nonmetalinguistic verbalization. Furthermore, both types of verbalization significantly increased the amount of time on task.

Type
Research Article
Copyright
© 2005 Cambridge University Press

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

References

REFERENCES

Ahlum-Heath, M. E., & di Vesta, F. J. (1986). The effect of conscious controlled verbalization of a cognitive strategy on transfer in problem solving. Memory and Cognition, 14, 281285.CrossRefGoogle Scholar
Alanen, R. (1995). Input enhancement and rule presentation in second language acquisition. In R. Schmidt (Ed.), Attention and awareness in foreign language learning (pp. 259302). Honolulu: University of Hawai‘i Press.
Allwood, C. M. (1990). On the relation between justification of solution method and correctness of solution in statistical problem solving. Scandinavian Journal of Psychology, 31, 181190.CrossRefGoogle Scholar
Anderson, M. (1985). Some evidence on the effect of verbalization on process: A methodological note. Journal of Accounting Research, 23, 843852.CrossRefGoogle Scholar
Berardi-Coletta, B., Buyer, L. S., Dominowski, R. L., & Rellinger, E. R. (1995). Metacognition and problem solving: A process-oriented approach. Journal of Experimental Psychology: Learning, Memory, and Cognition, 21, 205223.CrossRefGoogle Scholar
Berry, D. C. (1983). Metacognitive experience and transfer of logical reasoning. Quarterly Journal of Experimental Psychology, 35A, 3949.CrossRefGoogle Scholar
Berry, D. C., & Broadbent, D. E. (1984). On the relationship between task performance and associated verbalizable knowledge. Quarterly Journal of Experimental Psychology, 36A, 209231.CrossRefGoogle Scholar
Biehal, G., & Chakravarti, D. (1989). The effects of concurrent verbalization on choice processing. Journal of Marketing Research, 26, 8496.CrossRefGoogle Scholar
Bower, A. C., & King, W. L. (1967). The effect of number of irrelevant stimulus dimensions, verbalization, and sex on learning bi-conditional classification rules. Psychonomic Science, 8, 453454.CrossRefGoogle Scholar
Brehmer, B. (1974). Hypotheses about relations between scaled variables in the learning of probabilistic inference tasks. Organizational Behavior and Human Performance, 11, 127.Google Scholar
Brunk, L., Collister, G., Swift, C., & Stayton, S. (1958). A correlation study of two reasoning problems. Journal of Experimental Psychology, 55, 236241.CrossRefGoogle Scholar
Carpenter, P. A., Just, M. A., & Schell, P. (1990). What one intelligence measures: A theoretical account of the processing in the Ravens Progressive Matrices Test. Psychological Review, 97, 404431.CrossRefGoogle Scholar
Cavalcanti, M. C., & Cohen, A. D. (1990). Comentarios em composições: Uma comparação dos pontos de vista do professor e do aluno [Comments in compositions: A comparison of the teacher's and student's viewpoints]. Trabalhos em Lingüística Aplicada, 15, 723.Google Scholar
Chamot, A. U., & El Dinary, P. B. (1999). Children's learning strategies in language immersion classrooms. Modern Language Journal, 83, 319338.CrossRefGoogle Scholar
Cohen, A. D. (1986). Mentalistic measures in reading strategy research: Some recent findings. English for Specific Purposes, 5, 131145.CrossRefGoogle Scholar
Cohen, A. D. (1987). Recent uses of mentalistic data in reading strategy research. Revista de Documentação de Estudos em Lingüística Teorica e Aplicada, 3, 5784.Google Scholar
Cohen, A. D. (2000). Exploring strategies in test taking: Fine-tuning verbal reports from respondents. In G. Ekbatani & H. Pierson (Eds.), Learner-directed assessment in ESL (pp. 127150). Mahwah, NJ: Erlbaum.
Cohen, A. D., & Cavalcanti, M. C. (1987). Viewing feedback on compositions from the teacher's and the student's perspective. ESPecialist, 16, 1328.Google Scholar
Cohen, A. D., & Hosenfeld, C. (1981). Some uses of mentalistic data in second language research. Language Learning, 31, 285313.CrossRefGoogle Scholar
Corder, S. P. (1973). The elicitation of interlanguage. In J. Svartik (Ed.), Errata: Papers in error analysis (pp. 3648). Lund, Sweden: CKW Geerup.
Davis, J., & Bistodeau, L. (1993). How do L1 and L2 reading differ? Evidence from think aloud protocols. Modern Language Journal, 77, 459472.CrossRefGoogle Scholar
Davis, J. H., Carey, M. H., Foxman, P. N., & Tarr, D. B. (1968). Verbalization, experimenter presence, and problem solving. Journal of Personality and Social Psychology, 8, 299302.CrossRefGoogle Scholar
Dickson, J., McLennan, J., & Omodei, M. M. (2000). Effects of concurrent verbalization on a time-critical, dynamic decision-making task. Journal of General Psychology, 127, 217228.CrossRefGoogle Scholar
Ellis, R. (2001). Investigating form-focused instruction. Language Learning, 51, 146.Google Scholar
Enkvist, I. (1995). Intellectual and linguistic processes in foreign language students: Students' development during their first year of Spanish at a Swedish university. In Studies of higher education and research. Washington, DC: ERIC Clearinghouse. (ERIC Document Reproduction Service No. ED390253)
Ericsson, K. A. (2002). Towards a procedure for eliciting verbal expression of non-verbal experience without reactivity: Interpreting the verbal overshadowing effect within the theoretical framework for protocol analysis. Applied Cognitive Psychology, 16, 981987.CrossRefGoogle Scholar
Ericsson, K. A., & Simon, H. A. (1984). Protocol analysis: Verbal reports as data. Cambridge, MA: MIT Press.
Ericsson, K. A., & Simon, H. A. (1993). Protocol analysis: Verbal reports as data (rev. ed.). Cambridge, MA: MIT Press.
Færch, C., & Kasper, G. (1986). One learner—two languages: Investigating types of interlanguage knowledge. In J. House & S. Blum-Kulka (Eds.), Interlingual and intercultural communication (pp. 211227). Tübingen, Germany: Gunter Narr.
Fidler, E. J. (1983). The reliability and validity of concurrent, retrospective, and interpretive verbal reports: An experimental study. In P. Humphreys, O. Svenson, & A. Vari (Eds.), Analyzing and aiding decision processes (pp. 429440). Amsterdam: Elsevier.
Gagné, R. H., & Smith, E. C. (1962). A study of the effects of verbalization on problem solving. Journal of Experimental Psychology, 63, 1218.CrossRefGoogle Scholar
Gass, S. M., & Mackey, A. (2000). Stimulated recall methodology in second language research. Mahwah, NJ: Erlbaum.
Hosenfeld, C. (1976). Learning about learning: Discovering our students' strategies. Foreign Language Annals, 9, 117129.CrossRefGoogle Scholar
Hosenfeld, C. (1977). A preliminary investigation of the reading strategies of successful and nonsuccessful second language learners. System, 5, 110123.CrossRefGoogle Scholar
Hosenfeld, C. (1979). Cindy: A learner in today's foreign language classroom. In W. Borne (Ed.), The foreign language learner in today's classroom environment (pp. 5375). Montpelier, VT: Northwest Conference on the Teaching of Foreign Languages.
Hosenfeld, C. (1984). Case studies of ninth grade readers. In J. C. Alderson & A. H. Urquhart (Eds.), Reading in a foreign language (pp. 231249). London: Longman.
Hughes, J., & Parkes, S. (2003). Trends in the use of verbal protocol analysis in software engineering research. Behaviour & Information Technology, 22, 127141.CrossRefGoogle Scholar
Jaaskelainen, R. (2000). Focus on methodology in think-aloud studies on translating. In S. Tirkkonen Condit & R. Jaaskelainen (Eds.), Tapping and mapping the processes of translation and interpreting: Outlooks on empirical research (pp. 7182). Amsterdam: Benjamins.
Jourdenais, R. (2001). Cognition, instruction, and protocol analysis. In P. Robinson (Ed.), Cognition and second language instruction (pp. 354375). New York: Cambridge University Press.
Kasper, G., & Blum-Kulka, S. (1993). Interlanguage pragmatics. Oxford: Oxford University Press.
Kern, R. G. (1994). The role of mental translation in second language reading. Studies in Second Language Acquisition, 16, 441461.CrossRefGoogle Scholar
Lass, U., Klettke, W., Lüer, G., & Ruhlender, P. (1991). Does thinking aloud influence the structure of cognitive processes? In R. Schmid & D. Zambarbieri (Eds.), Oculomotor control and cognitive processes (pp. 385396). Amsterdam: Elsevier.
Leow, R. P. (1997). Attention, awareness, and foreign language behavior. Language Learning, 47, 467505.CrossRefGoogle Scholar
Leow, R. P. (1998). Toward operationalizing the process of attention in SLA: Evidence for Tomlin and Villa's (1994) fine-grained analysis of attention. Applied Psycholinguistics, 19, 133159.CrossRefGoogle Scholar
Leow, R. P. (2000). A study of the role of awareness in foreign language behavior: Aware versus unaware learners. Studies in Second Language Acquisition, 22, 557584.CrossRefGoogle Scholar
Leow, R. P. (2001a). Attention, awareness, and foreign language behavior. Language Learning, 51 (Suppl. 1), 113155.Google Scholar
Leow, R. P. (2001b). Do learners notice enhanced forms while interacting with the L2? An online and offline study of the role of written input enhancement in L2 reading. Hispania, 84, 496509.Google Scholar
Leow, R. P., & Morgan-Short, K. (2004). To think aloud or not to think aloud: The issue of reactivity in SLA research methodology. Studies in Second Language Acquisition, 26, 3557.CrossRefGoogle Scholar
Mackey, A., Gass, S., & McDonough, K. (2000). How do learners perceive interactional feedback? Studies in Second Language Acquisition, 22, 471497.Google Scholar
Mathews, R. C., Buss, R. R., Stanley, W. B., Blanchard-Fields, F., Cho, J. R., & Druhan, B. (1989). Role of implicit and explicit processes in learning from examples: A synergistic effect. Journal of Experimental Psychology: Learning, Memory, and Cognition, 15, 10831100.Google Scholar
McGeorge, P., & Burton, A. M. (1989). The effects of concurrent verbalization on performance in a dynamic systems task. British Journal of Psychology, 80, 455465.CrossRefGoogle Scholar
Nabei, T., & Swain, M. (2002). Learner awareness of recasts in classroom interaction: A case study of an adult EFL student's second language learning. Language Awareness, 11, 4363.CrossRefGoogle Scholar
Nevo, N. (1989). Test-taking strategies on a multiple-choice test of reading comprehension. Language Testing, 6, 199215.CrossRefGoogle Scholar
Norris, S. P. (1990). Effect of eliciting verbal reports of thinking on critical thinking test performance. Journal of Educational Measurement, 27, 4158.CrossRefGoogle Scholar
Norris, S. P. (1992). A demonstration of the use of verbal reports of thinking in multiple-choice critical thinking test design. Alberta Journal of Educational Research, 38, 155176.Google Scholar
O'Malley, J. M., Chamot, A. U., Stewner-Manzanares, G., Kupper, L., & Russo, R. P. (1985). Learning strategies used by beginning and intermediate ESL students. Language Learning, 35, 2146.CrossRefGoogle Scholar
Philp, J. (2003). Nonnative speakers' noticing of recasts in NS-NNS interaction. Studies in Second Language Acquisition, 25, 99126.Google Scholar
Rhenius, D., & Deffner, G. (1990). Evaluation of concurrent thinking aloud using eye-tracking data. Proceedings of the Human Factors Society 34th Annual Meeting: Orlando '90—Countdown to the 21st Century (pp. 12651269). Santa Monica, CA: Human Factors Society.
Robinson, K. M. (2001). The validity of verbal reports in children's subtraction. Journal of Educational Psychology, 93, 211222.CrossRefGoogle Scholar
Robinson, P. (1995). Aptitude, awareness, and the fundamental similarity of implicit and explicit second language learning. In R. Schmidt (Ed.), Attention and awareness in foreign language learning (pp. 303357). Honolulu: University of Hawai‘i Press.
Robinson, P. (1997a). Generalizability and automaticity of second language learning under implicit, incidental, enhanced, and instructed conditions. Studies in Second Language Acquisition, 19, 223247.Google Scholar
Robinson, P. (1997b). Individual differences and the fundamental similarity of implicit and explicit adult second language learning. Language Learning, 47, 4599.Google Scholar
Robinson, P. (2001). Individual differences, cognitive abilities, aptitude complexes, and learning conditions in second language acquisition. Second Language Research, 17, 368392.CrossRefGoogle Scholar
Rosa, E., & Leow, R. P. (2004a). Awareness, different learning conditions, and L2 development. Applied Psycholinguistics, 25, 269292.Google Scholar
Rosa, E., & Leow, R. P. (2004b). Computerized task-based exposure, explicitness, type of feedback, and Spanish L2 development. Modern Language Journal, 88, 193217.Google Scholar
Rosa, E., & O'Neill, M. D. (1999). Explicitness, intake, and the issue of awareness. Studies in Second Language Acquisition, 21, 511556.CrossRefGoogle Scholar
Russo, J. E., Johnson, E. J., & Stephens, D. L. (1989). The validity of verbal protocols. Memory & Cognition, 17, 759769.CrossRefGoogle Scholar
Selinker, L. (1972). Interlanguage. International Review of Applied Linguistics, 10, 209231.CrossRefGoogle Scholar
Short, E. J., Schatschneider, C., Cuddy, C. L., Evans, S. W., Dellick, D. M., & Basili, L. A. (1991). The effect of thinking aloud on the problem solving performance of bright, average, learning disabled, and developmentally handicapped students. Contemporary Educational Psychology, 16, 139153.CrossRefGoogle Scholar
Stanley, W. B., Mathews, R. C., Buss, R. R., & Kotler-Cope, S. (1989). Insight without awareness: On the interaction of verbalization, instruction, and practice in a simulated process control task. Quarterly Journal of Experimental Psychology, 41A, 553577.CrossRefGoogle Scholar
Stratman, J. F., & Hamp-Lyons, L. (1994). Reactivity in concurrent think-aloud protocols. In P. Smagorinsky (Ed.), Speaking about writing: Reflections on research methodology (pp. 89112). London: Sage.
Warren, J. (1996). How students pick the right answer: A ‘think aloud’ study of the French CAT. Occasional Papers Applied Linguistics Association of Australia, 15, 7994.Google Scholar
Wilder, L., & Harvey, D. J. (1971). Overt and covert verbalization and problem solving. Speech Monographs, 38, 171176.CrossRefGoogle Scholar
Wilson, T. D., & Schooler, J. W. (1991). Thinking too much: Introspection can reduce the quality of preferences and decisions. Journal of Personality and Social Psychology, 60, 181192.CrossRefGoogle Scholar
Yamashita, J. (2002). Reading strategies in L1 and L2: Comparison of four groups of readers with different reading ability in L1 and L2. ITL, Review of Applied Linguistics, 135–136, 135.CrossRefGoogle Scholar