Sony Developing AI-Powered PlayStation Characters in New Experiment
Sony is testing artificial intelligence (AI) technology to bring PlayStation game characters to life, with a prototype featuring Aloy from Horizon Forbidden West reportedly in development.
An internal video from Sony’s PlayStation division, leaked to The Verge, showcased an AI-driven version of Aloy capable of engaging in voice-based conversations with players. The footage was narrated by Sharwin Raghoebardajal, a director of software engineering at Sony Interactive Entertainment, who specializes in AI, computer vision, and face technology.
According to the demonstration, the AI-powered Aloy can respond to spoken prompts using a synthesized voice and facial animations, both in a controlled setting and within the full game environment. Raghoebardajal described the project as an internal prototype developed alongside Guerrilla Games to explore new AI capabilities.
The system utilizes OpenAI’s Whisper for speech-to-text, GPT-4 and Llama 3 for conversations and decision-making, and Sony’s proprietary Emotional Voice Synthesis (EVS) for generating speech. Additionally, facial animations are powered by Sony’s Mockingbird technology. While the demo ran on PC, Sony has tested portions of the AI technology directly on the PlayStation 5, with minimal impact on performance.
Sony first showcased the concept internally over a year ago and later presented a more advanced version at the Sony Technology Exchange Fair (STEF) in Tokyo last November.
“This is just a glimpse of what is possible,” Raghoebardajal stated in the leaked video. However, the experiment raises broader questions about the role of AI in gaming, including its impact on voice actors, game developers, and the player experience. Notably, Aloy is a playable protagonist in Horizon Forbidden West, leading to speculation about how AI-driven interactions would fit within the gameplay structure.
Sony’s move aligns with industry trends, as Nvidia and Microsoft are also investing in AI-driven game characters. Nvidia’s Ace technology, developed with Inworld AI, enables players to converse freely with non-playable characters (NPCs). Meanwhile, Microsoft is working on generative AI-powered characters, storylines, and game environments through its collaboration with Inworld AI and its Muse AI model.
The use of AI in gaming remains a controversial topic, particularly as job cuts and automation concerns loom over the industry. A 2024 Game Developers Conference (GDC) survey revealed that 49% of game developers already use generative AI tools in their workplaces, with 31% personally implementing them in game development.
With GDC 2024 set to take place next week in San Francisco, Sony’s AI experiments are likely to fuel further debate on the future of AI in video game creation. Despite multiple inquiries, Sony has not commented on the leaked footage or its decision to remove the video from YouTube after a copyright claim.
