AI-Generated Puzzle Hunt (ADA Project 2)

Use generative AI to create clues for an ever-changing interactive puzzle hunt! This project show you how to use the OpenAI Python API with the Adafruit MagTag (an ESP32 w/ e-ink display and some fun inputs) to create a puzzle.

To make the game interactive, we're using the 4 input buttons on the MagTag (left, up, down, and right). The AI creates a clue that corresponds to one of those four inputs and you, the player, must guess which one!

Here's a rundown of thegame:

1. The solution is randomly chosen from one of the four inputs: left, up, down, or right.

2. The solution goes to the AI model (along with some additional context for writing good clues). The AI model responds with a unique clue that provides a hint to the solution.

3. The player (you!) guesses the answer by pressing one of the input buttons.

4. If your input is correct, you win the game! And you get to play again, yay! If your input is incorrect, you get to guess again.

Currently, there's no way to "lose" because I had way too much fun getting new clues. And besides, puzzles are about the journey :)

What you'll learn:

New to AI? Awesome! This is a "102 project" of the "AI Display Assistant, " or ADA. These projects teach you how to program with AI using hardware! If you're feeling a lil' hesitant, check out the "101" project, a daily "Happy Facts" display.

This project dives deeper into prompt engineering for AI. Specifically, we'll learn how to write more complex prompts that include examples for the AI to replicate. You can think about this like getting the AI to respond in a structured way. By getting more structured responses, we can more easily use those responses to perform actions, like check if an answer to a puzzle is correct!

Since the goal of this project is to learn you AI coding skills, we'll only be using the built-in peripherals for the MagTag (which you can modify for whatever board you're using). That said, it is possible to adapt what you learn to do more complex things like turn on a light, send an email, etc.

But, one thing at a time! Onward to building the puzzle project!

What you'll need:

1. Adafruit MagTag & CircuitPython - if you don't have one on-hand, you can use another ESP32 board *or* any other WiFi-enabled device that supports CircuitPython.

To set these up, check out this guide: Getting started with MagTag + CircuitPython

2. OpenAI account- you can set one up for free but please keep in mind that it costs a lil' money to call the OpenAI models.

3. VS Code or another Python code editor of your choice!

4. Clone the Puzzle Hunt GitHub repo.

Note: this project assumes you're somewhat familiar with hardware and Python programming.

Build it: Hardware!

If you're using the Magtag: you don't need to do anything for the hardware! We'll use the four input buttons: left, right, up and down. Here's the code mappings:

• Left == button_a
• Up == button_b
• Down == button_c
• Right == button_d

If you have a different board with built-in buttons: check the pinout diagram to see how to map the words left, right, up, and down to the buttons. Then update the function check_answer().

If you have an ESP32 breakout board with no buttons: grab 4 pushbuttons and wire up (via breadboard or soldering) those 4 buttons to 4 inputs on the ESP32 I/O pins. Note that you may also need to add some button debounce code. When all your buttons are wired up, update the function check_answer().

Code it: Writing the AI Prompt!

For this second ADA project, we need a lot more context in our AI prompt than our first "Happy Facts" project. Because of this, I created a separate file, context.py, to store the lengthy prompt.

In our context.py file, first let's articulate the basics of what we want the AI to do:

Given a word, write a clue that is a hint to that word.

For my prompt, I elaborated on this basic ask and asked for short clues:

Embrace your role as a fun puzzle maestro! You are given a word. You then MUST write a fun clue that is a hint to that word. Use fewer than 20 words. Draw inspiration from these examples:

We also need to show the AI how we want it to write a clue (and tell it we want to use the examples, like above). For this, we give it some examples, like this:

EXAMPLE (Movie):
Word: up
Clue: In a tale where balloons aloft, an old man's dreams take flight so soft. To the skies, his house does bound; which cinematic wonder are we bound?

The examples that we give the AI model show it how we want it to respond, both in terms of language and formatting.

Finally, we end our prompt by, well, prompting the AI to give us a new clue with a word that we give it:

Use the following word to generate a fun, unique clue that is a hint to that word. Draw inspiration from the examples above. The word is:

In the main code loop, when we call the AI model API, add the randomly generated word (i.e., the answer to the puzzle) in the prompt. We also want to format it like we did in the examples:

prompt = prompt + "\nWord: " + answer + "\nClue: "

Where the prompt variable is the full contents of the context.py file and the answer variable is the single word that forms the basis of our clue.

Additional tips:

• "Talk" to the AI like you would a 7-year-old.
• Give the AI at least 4 - 5 examples. I found the best performance when I gave the AI 1 example per word.
• If you need the AI to do something super specific, or it's not doing what you ask it, you can "yell" at it by writing words in all caps, e.g. "you MUST write a clue..."
• Tell the AI how long/short to write its responses. Use word count or say things like "brief" (it doesn't 'understand' character count).
• You're charged per token, which includes whitespace. Remove excess whitespace where you don't need it.

Why are we randomly choosing an answer? Why can't we let the AI choose an answer?

For the first iteration I tried to have the AI choose a word and generate a clue. After experimenting, I opted to randomly choose the gameplay word and have the AI model use that to generate a clue for two main reasons:

1. The behavior of a Large Language Model (LLM) is not 100% predictable. Sometimes (~15% of the time) the AI chose a word not in the list I gave it in the prompt. I tried "yelling" that it had to choose from the list and that still didn't work.

2. The format of an LLM response can differ each time you call it, even with examples. This makes it really difficult (time- and code-intensive) to parse out pieces of a response. It was way easier to specify the answer and only get the clue.

Code it: Write a game loop!

Since this tutorial is focused on teaching you how to write more complex AI prompts, here's a quick TLDR of the game mechanics:

1. Randomly choose a word from a list of possible choices. This word is the "answer" to our puzzle.

2. Call the OpenAI Python API with the prompt + answer.

3. Display the AI model response on the MagTag screen.

4. Enter a while loop that waits until the player gets the correct answer (using a win_state variable that is False unless player presses the button that corresponds to the answer, e.g. left button for the word 'left').

5. For a correct answer: play the neopixel win sequence and update the screen to indicate that the player has won! Ask if they want to play a new game + restart the game loop (steps 1 - 4).

6. For an incorrect answer: play the neopixel lose sequence for 4 seconds. Then let the player guess again. No change to screen (keep the AI-generated clue displayed).

There's technically no way to 'lose' my game. You can keep the same type of game loop or you can add your own game mechanics!

Other Considerations

• Choosing an AI model: You need a 'completions' AI model (not a chat model) for this project. I used the OpenAI text-davinci-003 model. It is a lil' more expensive but wrote way better clues than text-davinci-002).
• Model temperature: This changes how much the model 'hallucinates' or makes things up. A higher temp == more hallucination. The OpenAI Python API allows for values between 0 and 2 - I set mine right in the middle at 1. Play around with this to see if you can get better (i.e., more fun) clues!
• Coming up with prompt examples: I used ChatGPT to help me write some examples that I included in the context.py file. You can also use it to write better examples or ask it to improve your prompt.

Going Forward

There are tons of ways you can update the game or adapt the project for different applications. Here are some ideas to get you started:

• Add more word options for clues! On the MagTag, you can use the light sensor ("light" and "dark") or connect other inputs via the JST connectors.
• Create a trivia game where you use the input buttons to select from generated answers. (Update the prompt to tell the AI to generate 4 possible answers where only 1 is correct. Remember to update the examples!)
• Create a scavenger hunt around your city! This might require a browser app so you can input coordinates or names of places.

Whatever you make, I'd love to see your creations! And of course, if you have questions or need troubleshooting help, please leave a comment.

Happy making!