April 4-6, 2024 • Hyatt Regency • Lexington, KY
Innovations in Health Communication
Innovations in Health Communication
Abstract: Leveraging Virtual Reality (VR) With Eye-Tracking and Generative Artificial Intelligence (AI) for Message Effects Research
Background and Current Study. We are constantly exposed to persuasive messaging in our day-to-day lives, yet it is difficult for researchers to precisely measure how much attention people pay to these messages. First, it is difficult to study how messages attract attention in a natural-real-world setting with a stationary eye-tracking method. Second, in controlled laboratory settings, it is difficult to achieve ecological validity and account for real-life situational factors.
Virtual reality (VR) is a promising platform that allows for the examination of message effects in simulated, but realistic environments (Bonneterre, 2023; Schmälzle et al., 2023). In addition, the advancements in generative AI tools allow us to augment our message development processes (Lim & Schmälzle, 2023). This poster presents the highlights from a proof-of-concept study that demonstrates how immersive VR and generative AI technologies can be used to rigorously examine message effects. Specifically, we used generative AI to create health promotion messages and inserted them into a virtual city environment as billboards. Then we manipulate the amount of distraction participants experience in the virtual city by telling one group to freely drive around four city blocks and the other group to count the number of stop signs. Finally, we track where people look at (eye-tracking) when navigating in the virtual city. Our analysis focuses on how the distraction conditions influence which billboards people look at, and whether the fixations influence their subsequent recall and recognition of the messages.
Method. Our materials, equipment, and procedures resemble those of Schmälzle et al. (2023). We build the virtual city by importing the Classical City 3D model from Unity Asset into the Vizard VR platform. The health billboard messages were created in conjunction with Canva (for billboard templates), ChatGPT (for modifying the text of the message), and Midjourney (for the image that accompanies the text). Finally, we use the HP Reverb G2 Omnicept to collect physiological measurements such as eye-tracking. The setup is tested, piloted, and IRB-approved; we will be able to present first results during KCHC in April.
Implications. The VR city paradigm expands our capacity to observe people’s attention and its influence on memory by simulating realistic environments and precisely manipulating messaging strategies and the ways attention is deployed to the messages. Furthermore, this study expands upon existing studies that show how generative AI can augment the message design and campaign delivery process. Finally, just like the rise of social media has led to new forms of health communication, we can already expect that the shift towards the Metaverse will bring about new forms of health communication in new-media settings. In sum, this study showcases how the field can adapt to these developments and continue to decipher the timeless ingredients of and strategies for effective health communication.
Virtual reality (VR) is a promising platform that allows for the examination of message effects in simulated, but realistic environments (Bonneterre, 2023; Schmälzle et al., 2023). In addition, the advancements in generative AI tools allow us to augment our message development processes (Lim & Schmälzle, 2023). This poster presents the highlights from a proof-of-concept study that demonstrates how immersive VR and generative AI technologies can be used to rigorously examine message effects. Specifically, we used generative AI to create health promotion messages and inserted them into a virtual city environment as billboards. Then we manipulate the amount of distraction participants experience in the virtual city by telling one group to freely drive around four city blocks and the other group to count the number of stop signs. Finally, we track where people look at (eye-tracking) when navigating in the virtual city. Our analysis focuses on how the distraction conditions influence which billboards people look at, and whether the fixations influence their subsequent recall and recognition of the messages.
Method. Our materials, equipment, and procedures resemble those of Schmälzle et al. (2023). We build the virtual city by importing the Classical City 3D model from Unity Asset into the Vizard VR platform. The health billboard messages were created in conjunction with Canva (for billboard templates), ChatGPT (for modifying the text of the message), and Midjourney (for the image that accompanies the text). Finally, we use the HP Reverb G2 Omnicept to collect physiological measurements such as eye-tracking. The setup is tested, piloted, and IRB-approved; we will be able to present first results during KCHC in April.
Implications. The VR city paradigm expands our capacity to observe people’s attention and its influence on memory by simulating realistic environments and precisely manipulating messaging strategies and the ways attention is deployed to the messages. Furthermore, this study expands upon existing studies that show how generative AI can augment the message design and campaign delivery process. Finally, just like the rise of social media has led to new forms of health communication, we can already expect that the shift towards the Metaverse will bring about new forms of health communication in new-media settings. In sum, this study showcases how the field can adapt to these developments and continue to decipher the timeless ingredients of and strategies for effective health communication.