AI Agents in Games

"AI Agents", two words I’m convinced we’re going to hear a lot in 2025, are an exciting topic to explore. To kick off the year, I wanted to share some of the knowledge about that I’ve recently acquired and a small project I developed.

During my Master’s program in Artificial Intelligence, I had the incredible opportunity to learn about various techniques to develop AI agents. One of the most fascinating assignments was to create AI agents for a two-player board game called "The End of The Track." This game, while simpler than classics like chess or checkers, posed its own unique challenges and turned out to be an intriguing experience for me.

👾 Agents in Games

In the context of games, an agent is an autonomous entity designed to make decisions and perform actions to achieve specific goals. For games like chess, Go, or even simpler ones like "The End of The Track," agents aim to outsmart their opponents by analyzing the game state, predicting outcomes, and choosing optimal moves.

The field of AI in games has a rich history. In 1950, Claude Shannon published one of the earliest papers on computer chess, laying the groundwork for future research. In the 1990s, Deep Blue—a chess-playing AI developed by IBM—made history by defeating the reigning world champion, Garry Kasparov, in a six-game match in 1997. This victory marked the beginning of AI's dominance in classic board games.

More recently, in 2016, AlphaGo—developed by DeepMind—achieved a milestone in AI by defeating Lee Sedol, one of the world’s top Go players. Go, with its near-infinite number of possible board configurations, had long been considered a major challenge for AI. AlphaGo's success was attributed to advanced techniques like deep reinforcement learning and neural networks, showcasing how modern AI algorithms can excel in highly complex, strategic environments.

These milestones illustrate how AI has evolved over decades to tackle increasingly sophisticated challenges in games, pushing the boundaries of what autonomous agents can achieve.

For my project, I explored these concepts by focusing on a classic algorithmic approach: the minimax algorithm.

♟️ The End of The track

"The End of The Track" is a strategic two-player board game that combines movement and positioning with the goal of outmaneuvering your opponent. Each player starts the game with five blocks and one ball.

Goal: The objective is to pass your ball to the opposite side of the board before your opponent can do the same.

Gameplay: On their turn, players can choose between two actions:

1. Move a Block: Blocks move in an L-shape, similar to a knight in chess (two squares in one direction and then one square perpendicular). However, blocks cannot move if they are holding the ball.

2. Pass the Ball: The ball moves like a queen in chess, able to jump any number of squares in a straight line—horizontally, vertically, or diagonally—but only by landing on spaces occupied by the player’s blocks. The ball can chain multiple jumps in a single turn, provided the path is unobstructed by opponent pieces.

This simple yet engaging set of rules creates opportunities for strategic play, such as blocking your opponent’s movement, positioning your blocks to enable long ball passes, and planning ahead to trap your opponent. With its mix of tactical depth and straightforward mechanics, "The End of The Track" offers an exciting experience for players of all levels.

The original concept for "The End of The Track" was created by Gaya, who also sells the physical version of the game.

The End of The Track original game

📃 Minimax algorithm

The minimax algorithm is a fundamental technique used to determine the best move for a player, assuming that the opponent will also play optimally. The algorithm works by constructing a game tree, where each node represents a possible state of the game. The agent simulates all possible moves and counter-moves up to a certain depth and assigns a value to each game state based on how favorable it is for the agent.

The process alternates between maximizing the agent’s score (when it’s the agent’s turn) and minimizing the opponent’s score (when it’s the opponent’s turn). By the end of this recursive evaluation, the agent selects the move leading to the most favorable outcome, assuming optimal play from both sides.

However, constructing and evaluating every possible state of the game—even in simpler games—can be computationally prohibitive. To overcome this, I implemented two key optimizations:

• Heuristics: Heuristics serve as a way to estimate the value of a game state without fully simulating to the end of the game. For example, in "The End of The Track," I designed heuristics that evaluated the positioning of pieces, proximity to the goal, and potential threats from the opponent. These heuristics allowed the agent to focus on promising moves and avoid wasting computational resources on less relevant branches of the game tree.

• Alpha-Beta Pruning: This optimization technique significantly reduces the number of nodes evaluated in the game tree. It does so by keeping track of two values: alpha (the best score the maximizing player can guarantee) and beta (the best score the minimizing player can guarantee). If at any point it becomes clear that a branch cannot improve the outcome for the current player, that branch is pruned—or ignored—from further evaluation. This made the algorithm much more efficient while maintaining its optimality.

Minimax algorithm visualization

🎮 Bringing the Game to Life

One challenge I faced was the lack of an online version of "The End of The Track." Without a physical copy or digital counterpart, it was difficult to test strategies and visualize the AI’s decisions. So, I decided to create a web version of the game from scratch.

This web version served as a testing ground for my AI agent, but it also presented some unique challenges. Defining valuable heuristics was one of the most complex aspects. I had to play the game multiple times, experimenting with different strategies and building assumptions about what makes one move better than another and by how much. These heuristics played a vital role in helping the AI evaluate the board state effectively.

Another challenge was determining how many steps ahead the agent should visualize. For my homework assignment, limited computational resources forced me to keep this number low and focus on optimizing pruning techniques. However, for the digital version of the game, I had more resources at my disposal. This allowed me to focus on playability and user experience, striking a balance between an AI that was not too easy to beat and one that wasn’t overly challenging. It required fine-tuning to ensure the game remained engaging and fun.

As I built and refined the digital game, I realized it had the potential to be more than just a tool for my project. I added features such as:

• Solo Mode: Play against the AI agent I developed as part of my assignment.

• Online Multiplayer Mode: Challenge friends online, taking turns to make moves.

The game is now hosted on my personal website, where anyone can play for free—you just need a browser. Feel free to check it out and challenge the AI agent I developed. I hope you enjoy playing "The End of The Track" as much as I enjoyed creating it! 👇

Please send me your feedback and suggestions. I’m always looking to improve the game and make it more engaging for players.

The End of The Track digital version I developed

📓 Key Learnings

This project provided valuable insights into developing AI agents and building games:

• Exploring Possibility Spaces: Designing the agent required a deep understanding of the game’s mechanics and exploring the vast space of possible moves and strategies. This taught me how to identify and prioritize patterns that could enhance the agent’s decision-making process.

• Optimization in AI: Working within computational constraints emphasized the importance of efficient algorithms and optimizations. Techniques like heuristics and alpha-beta pruning were crucial in balancing performance and computational feasibility.

• The Joy of Game Development: Creating a digital version of "The End of The Track" was not just a technical exercise but also a creative endeavor. Building the game, refining its mechanics, and adding features like online multiplayer and solo modes was immensely rewarding and reinforced the idea that combining technology with creativity can result in engaging experiences.

This journey highlighted the interplay between strategy, optimization, and creativity in AI development. It also underscored how much fun it can be to bring a concept to life and share it with others.