Supervisor

Dr Matthew Stephenson
Stephenson, Matthew (Dr)
matthew.stephenson@flinders.edu.au
View Flinders profile

Project description

Generative language models such as ChatGPT, Gemini and Claude are revolutionising the way that humans interact with AI on a regular basis. Beyond being used as personal assistants, tutors, editors and technical support, this cutting-edge AI technology can also provide new creative opportunities. Several prominent entertainment companies (Nvidia, Ubisoft, Electronic Arts, Xbox, Sony, etc.) alongside a suite of high-profile academic researchers, are already exploring the potential for generative AI to create dynamic characters that respond authentically to users. This project will focus on developing an interactive character conversation demonstrator, able to interpret a variety of input modalities. This demonstrator will utilise existing Generative AI technology, such as the GPT-4, GPT-vision and Whisper models provided by OpenAI, to produce a virtual character capable of taking both audio and visual inputs (from microphones and cameras) and producing its own generated speech and visuals in response. These functions can then be combined with real-time animation and lip-syncing software to create a fully realised virtual character, able to seamlessly interact with human users. This project will provide the following student learning experiences:
  • Research into the capabilities of Large Multimodal Models (LMMs) capable of interpreting both text and image inputs.
  • Development of skills working with the OpenAI API, including utilisation of GPT-4 models for text generation and image analysis (GPT-vision), as well as Text-to-Speech and Speechto-Text capabilities.
  • Utilising prompt engineering techniques to craft believable virtual characters that seamlessly integrate visual and audio inputs.

Assumed knowledge

A basic knowledge of Python (as covered in ENGR1721 or COMP2712) or equivalent understanding of an alternative programming language such as Java or C#.

Register interest in this project


Note: You need to register interest in projects from different supervisors (not a number of projects with the one supervisor).
You must also contact each supervisor directly to discuss both the project details and your suitability to undertake the project.