Playing Codenames with Large Language Models

Project description

• Codenames is one of the most popular board games in the world, currently being the eighth most rated game on popular website Board Game Geek. It is a cooperative team-based game of language understanding and communication, where players must work together to discern relationships between words based on clues given by their teammates. To play successfully, players need to be able to give and interpret semantic clues for a wide range of potential teammates, requiring both a multi-modal understanding of language and epistemic reasoning (theory-of-mind). This open-ended style of play makes the task of developing adaptive agents that can modify their strategies to suit a wide range of potential teammates very difficult using traditional game-playing AI techniques.
• The recent rise of large language models (LLMs) over the past few years may provide a potential solution to this challenge, having previously demonstrated their potential and flexibility in natural language understanding across many domains. This project will explore the capabilities of Large Language Models, including popular pre-trained models such as ChatGPT / GPT-4, Gemini, Claude and Llama, for playing Codenames against a variety of opponents playstyles. This will involve researching and implementing a variety of prompt engineering techniques, intended to improve the performance of the models. Investigating the impact of various prompting techniques on performance is more accessible to novice programmers, as it primarily involves using natural language without the need to train new models. The focus of this project will not only be on developing an effective Codenames player using LLMs, but also on adapting the clues and playstyles of the LLM agent to suit a wide range of teammate skill levels and playstyles.
• This project will provide the following student learning experiences:

1. Research into the capabilities of Large Language Models (LLMs) capable of reasoning about and generating text-based prompts.
2. Development skills working with the OpenAI API for GPT-4, along with the opportunity to explore other popular LLM models such as Gemini (Google), Claude (Anthropic) and Llama (Meta).
3. Understanding a variety of modern prompt engineering techniques, including Chain-of-Thought, Self-Refine, Few-Shot prompting, and Automatic-Prompt-Engineering.

Co-supervisors

Richard Leibbrandt

Assumed knowledge

A basic knowledge of Python (as covered in ENGR1721 or COMP2712) or equivalent understanding of an alternative programming language such as Java or C#.

Note: You need to register interest in projects from different supervisors (not a number of projects with the one supervisor).
You must also contact each supervisor directly to discuss both the project details and your suitability to undertake the project.