Ir al menú de navegación principal Ir al contenido principal Ir al pie de página del sitio


Número 0

Nuevas voces

Building ethograms for interpreting research —A technical report

Publicado 20.10.2026

  • Du Zhiqiang

Du Zhiqiang
Università di Bologna

Resumen

By establishing precise temporal relationships across modalities, researchers can connect behavioral indicators with cognitive processes, revealing otherwise hidden patterns in interpreter performance. This paper presents a hands-on approach for aligning multimodal data in interpreting studies. It introduces four strategies for synchronizing audio, video, and keystroke data along a unified temporal framework: visual observation, auditory identification, signal sound recognition, and words and chunk levels. This human-centered approach envisions interpreters as cognitive agents interacting with physical and digital environments. Though time-intensive, these alignment protocols provide essential foundations for capturing interpreting’s multimodal nature and support more rigorous, ecologically valid research in cognitive translation and interpreting studies.

Palabras clave

Data cleaning, multimodal data alignment, interpreting research

PDF (English)

Citas

  1. Ahrens, Barbara, and Przemysław Janikowski. 2025. “Speech Recording”. In Research Methods in Cognitive Translation and Interpreting Studies, edited by Ana María Rojo López and Ricardo Muñoz Martín, 183–212. Amsterdam: John Benjamins. https://doi.org/10.1075/rmal.10.08ahr.
  2. Angelone, Erik. 2021. “Broadening the Scope of Error Categories in Translation Assessment through Screen Recording”. Across Languages and Cultures 22 (2): 143–57. https://doi.org/10.1556/084.2021.00021.
  3. Aston, Guy. 2018. “Acquiring the Language of Interpreters: A Corpus-Based Approach”. In Making Way in Corpus-Based Interpreting Studies, edited by Mariachiara Russo, Claudio Bendazzoli, and Bart Defrancq, 83–96. Singapore: Springer Singapore. https://doi.org/10.1007/978-981-10-6199-8_5.
  4. Borg, Claudine, Carmen Heine, and Hanna Risku. 2025. “Observations and Diaries”. In Research Methods in Cognitive Translation and Interpreting Studies, edited by Ana María Rojo López and Ricardo Muñoz Martín, 108–32. Amsterdam: John Benjamins. https://doi.org/10.1075/rmal.10.05bor.
  5. Chmiel, Agnieszka, Przemysław Janikowski, Danijel Koržinek, Agnieszka Lijewska, Marta Kajzer-Wietrzny, Dariusz Jakubowski, and Koen Plevoets. 2023. “Lexical Frequency Modulates Current Cognitive Load, but Triggers No Spillover Effect in Interpreting”. Perspectives, (June): 1–19. https://doi.org/10.1080/0907676X.2023.2218553.
  6. Chmiel, Agnieszka, Przemysław Janikowski, and Agnieszka Lijewska. 2020. “Multimodal Processing in Simultaneous Interpreting with Text: Interpreters Focus More on the Visual than the Auditory Modality”. Target 32 (1): 37–58. https://doi.org/10.1075/target.18157.chm.
  7. Chmiel, Agnieszka, Marta Kajzer-Wietrzny, Danijel Koržinek, Dariusz Jakubowski, and Przemysław Janikowski. 2024. “Syntax, Stress and Cognitive Load, or on Syntactic Processing in Simultaneous Interpreting”. Translation, Cognition & Behavior 7 (1): 22–47. https://doi.org/10.1075/tcb.00091.chm.
  8. Christodoulides, George. 2016. “Effects of Cognitive Load on Speech Production and Perception”. PhD dissertation, UCL - Université Catholique de Louvain. https://dial.uclouvain.be/pr/boreal/object/boreal:177060.
  9. Cowan, Nelson. 2005. Working Memory Capacity. Essays in Cognitive Psychology. New York: Psychology Press.
  10. Doherty, Stephen. 2021. “Multisensory Integration in Audiovisual Translation”. In Multilingual Mediated Communication and Cognition, edited by Ricardo Muñoz Martín and Sandra Halverson, 155–70. London: Routledge. https://doi.org/10.4324/9780429323867-7.
  11. Du, Zhiqiang. 2024. “Bridging the Gap: Exploring the Cognitive Impact of InterpretBank on Chinese Interpreting Trainees”. PhD dissertation, Alma Mater Studiorum - Università di Bologna. https://doi.org/10.48676/unibo/amsdottorato/11584.
  12. Enríquez Raído, Vanessa, Erik Angelone, and Christian Olalla Soler. 2025. “Screen Recording”. In Research Methods in Cognitive Translation and Interpreting Studies, edited by Ana María Rojo López and Ricardo Muñoz Martín, 133–56. Amsterdam: John Benjamins. https://doi.org/10.1075/rmal.10.06rai.
  13. Frittella, Francesca Maria. 2022. “CAI Tool-Supported SI of Numbers: A Theoretical and Methodological Contribution”. International Journal of Interpreter Education 14 (1): 32–56. https://doi.org/10.34068/ijie.14.01.05.
  14. Halverson, Sandra L., and Ricardo Muñoz Martín. 2020. “The Times, They Are a-Changin”: Multilingual Mediated Communication and Cognition”. In Multilingual Mediated Communication and Cognition, edited by Ricardo Muñoz Martín and Sandra L. Halverson, 1–17. London: Routledge. https://doi.org/10.4324/9780429323867-1
  15. Hooge, Ignace T. C., Antje Nuthmann, Marcus Nyström, Diederick C. Niehorster, Gijs A. Holleman, Richard Andersson, and Roy S. Hessels. 2025. “The Fundamentals of Eye Tracking Part 2: From Research Question to Operationalization”. Behavior Research Methods 57 (2): 73. https://doi.org/10.3758/s13428-024-02590-2.
  16. Huang, Dan Feng, Fang Li, and Hang Guo. 2023. “Chunking in Simultaneous Interpreting: The Impact of Task Complexity and Translation Directionality on Lexical Bundles”. Frontiers in Psychology 14 (August): 1252238. https://doi.org/10.3389/fpsyg.2023.1252238.
  17. Huxley, Julian S. 1914. “The Courtship - Habits of the Great Crested Grrebe (Podiceps Cristatus); with an Addition to the Theory of Sexual Selection.” Proceedings of the Zoological Society of London 84 (3): 491–562. https://doi.org/10.1111/j.1469-7998.1914.tb07052.x.
  18. Katerina, Tzafilkou, and Protogeros Nicolaos. 2018. “Mouse Behavioral Patterns and Keystroke Dynamics in End-User Development: What Can They Tell Us about Users” Behavioral Attributes?” Computers in Human Behavior 83 (June): 288–305. https://doi.org/10.1016/j.chb.2018.02.012.
  19. Kokkola, Sari, and Anne Ketola. 2015. “Thinking Outside the ‘Methods Box’: New Avenues for Research in Multimodal Translation”. In Rajojen Dynamiikkaa, Gränsernas Dynamik, Borders under Negotiation, Grenzen Und Ihre Dynamik, edited by Danel Rellstab and Nestori Siponkoski, 4: 219–228. Vaasa, Finland: Vakki Publications. https://doi.org/10.70484/vakki.149182.
  20. Korpal, Paweł, and Katarzyna Stachowiak-Szymczak. 2020. “Combined Problem Triggers in Simultaneous Interpreting: Exploring the Effect of Delivery Rate on Processing and Rendering Numbers”. Perspectives 28 (1): 126–43. https://doi.org/10.1080/0907676X.2019.1628285.
  21. Leijten, Mariëlle, and Luuk Van Waes. 2013. “Keystroke Logging in Writing Research: Using Inputlog to Analyze and Visualize Writing Processes”. Written Communication 30 (3): 358–92. https://doi.org/10.1177/0741088313491692.
  22. Lindgren, Eva, and Kirk P. H. Sullivan, eds. 2019. Observing Writing: Insights from Keystroke Logging and Handwriting. Studies in Writing, volume 38. Leiden; Boston: Brill. https://doi.org/10.1163/9789004392526
  23. Mankauskienė, Dalia. 2016. “Problem Trigger Classification and Its Applications for Empirical Research”. Procedia-Social and Behavioral Sciences 231 (October): 143–48. https://doi.org/10.1016/j.sbspro.2016.09.083.
  24. Messerman, Arik, Tarik Mustafić, Seyit Ahmet Camtepe, and Sahin Albayrak. 2011. “Continuous and Non-Intrusive Identity Verification in Real-Time Environments Based on Free-Text Keystroke Dynamics”. In 2011 International Joint Conference on Biometrics (IJCB), 1–8. https://doi.org/10.1109/IJCB.2011.6117552.
  25. Mizuno, Akira. 2017. “Simultaneous Interpreting and Cognitive Constraints”. In Journal of College of Literature, 58:1–28. Aoyama Gakuin University.
  26. Muñoz Martín, Ricardo, and Matthias Apfelthaler. 2022. “A Task Segment Framework to Study Keylogged Translation Processes”. The International Journal of Translation and Interpreting Research 14 (2): 8–31. https://doi.org/10.12807/ti.114202.2022.a02.
  27. Muñoz Martín, Ricardo, and José Ma. Cardona Guerra. 2018. “Translating in Fits and Starts: Pause Thresholds and Roles in the Research of Translation Processes”. Perspectives 27 (4): 525–51. https://doi.org/10.1080/0907676X.2018.1531897.
  28. Muñoz Martín, Ricardo, and Ana María Rojo López. 2025. “Introduction”. In Research Methods in Cognitive Translation and Interpreting Studies, edited by Ana María Rojo López and Ricardo Muñoz Martín, 1–20. Amsterdam: John Benjamins. https://doi.org/10.1075/rmal.10.intro.
  29. Muñoz Martín, Ricardo, Sanjun Sun, Zhiqiang Du, and Sara Puerini. 2025. “Keylogging”. In Research Methods in Cognitive Translation and Interpreting Studies, edited by Ana María Rojo López and Ricardo Muñoz Martín: 157–82. Amsterdam: John Benjamins. https://doi.org/10.1075/rmal.10.07mar.
  30. Niehorster, Diederick C., Marcus Nyström, Roy S. Hessels, Richard Andersson, Jeroen S. Benjamins, Dan Witzner Hansen, and Ignace T. C. Hooge. 2025. “The Fundamentals of Eye Tracking Part 4: Tools for Conducting an Eye Tracking Study”. Behavior Research Methods 57 (1): 46. https://doi.org/10.3758/s13428-024-02529-7.
  31. Plevoets, Koen, and Bart Defrancq. 2016. “The Effect of Informational Load on Disfluencies in Interpreting: A Corpus-Based Regression Analysis”. Translation and Interpreting Studies. The Journal of the American Translation and Interpreting Studies Association 11 (2): 202–24. https://doi.org/10.1075/tis.11.2.04ple.
  32. Prandi, Bianca. 2023. Computer-Assisted Simultaneous Interpreting: A Cognitive-Experimental Study on Terminology. Berlin: Language Science Press. https://doi.org/10.5281/zenodo.7143055.
  33. Robbins, Philip, and Murat Aydede, eds. 2009. The Cambridge Handbook of Situated Cognition. Cambridge: Cambridge University Press.
  34. Seeber, Kilian G. 2017. “Multimodal Processing in Simultaneous Interpreting”. In The Handbook of Translation and Cognition, edited by John W. Schwieter and Aline Ferreira, 1st ed., 461–75. Hoboken, NJ: John Wiley & Sons. https://doi.org/10.1002/9781119241485.ch25.
  35. —. 2011. “Cognitive Load in Simultaneous Interpreting: Existing Theories — New Models”. Interpreting. International Journal of Research and Practice in Interpreting 13 (2): 176–204. https://doi.org/10.1075/intp.13.2.02see.
  36. Shao, Zhangminzi, and Bart Defrancq. 2024. “Fundamental Frequency as an Acoustic Mirror of Interpreters” Cognitive States”. Interpreting 27 (1): 1–27. https://doi.org/10.1075/intp.00107.sha.
  37. Van Waes, Luuk, Mariëlle Leijten, Jens Roeser, Thierry Olive, and Joachim Grabowski. 2021. “Measuring and Assessing Typing Skills in Writing Research”. Journal of Writing Research 13 (1): 107–53. https://doi.org/10.17239/jowr-2021.13.01.04.
  38. Walker, Callum, Kristian Tangsgaard Hvelplund, and Victoria Lei. 2025. “Screen Eyetracking”. In Research Methods in Cognitive Translation and Interpreting Studies, edited by Ana María Rojo López and Ricardo Muñoz Martín, 213–34. Amsterdam: John Benjamins. https://doi.org/10.1075/rmal.10.09wal.
  39. Wang, Fuxiang. 2022. “Impact of Translation Difficulty and Working Memory Capacity on Processing of Translation Units: Evidence from Chinese-to-English Translation”. Perspectives 30 (2): 306–22. https://doi.org/10.1080/0907676X.2021.1920989.
  40. Wehrmeyer, Ella. 2019. “A Corpus for Signed Language Interpreting Research”. Interpreting 21 (1): 62–90. https://doi.org/10.1075/intp.00020.weh.
  41. Zhang, Xiaojun. 2016. “Semi-Automatic Simultaneous Interpreting Quality Evaluation”. International Journal on Natural Language Computing 5 (5): 1–12. https://doi.org/10.5121/ijnlc.2016.5501.