What is multi-agent prompting?

Imagine a bunch of AI tools, each with its own skill set, working in harmony. It's like an orchestra of instruments where every musician contributes to the song by playing their own unique part.

By using this approach, you can tackle really ambitious projects using AI as a copilot. For instance, creating an entire website from scratch requires coding, graphic design, copywriting, search engine optimization, and promotion. There’s a tool for each of those tasks, and you can create a workflow that makes them all work in concert.

But getting to that point takes a thoughtful setup, a little bit of knowhow, and a lot of practice. Here’s how to do it.

1. Learn the nuances of different tools

Before you can compose your masterpiece, you've got to know your instruments—or in this case, your AI tools, which we’ll also call “agents.” Each tool has its limitations and strengths. Working with them before tackling your grand ambition can help you figure out where each excels and falls short. 

Check out the limitations of these tools—they all have specific constraints that might impact their performance, such as ChatGPT's context window or Midjourney's challenges with generating full-length character renders (Describe the shoes. Always describe the shoes). Figure out what they do best, and consider their best role in your project.

2. Choose your workstyle

Next, decide how you want to work with the tools. This involves considering your preferred workflow and the way you want to achieve the goals. For instance, are you a Centaur, who delegates specific tasks to the AI while doing some on your own, or are you a Cyborg, who works continuously and iteratively with the AI? 

(Don't know which you are? Figure out your style here.)

Experiment with different workstyles to see how you like them.

3. Articulate your goals

When you're dealing with one tool, goal-setting is relatively straightforward. But when you're working with multiple agents, things get way more complicated. It pays to sit down and map out what you hope to achieve. You may even benefit from flowcharts or process maps to visualize how each tool will contribute at various stages.

Think about what parts can be done in parallel and which ones need others to be worked out first. For example, if you're doing a rebranding exercise, you need to articulate your brand's voice and identify your ideal customer before you start doing any copywriting. But you might not need to have finalized your design for the "About Us" website page before you start on a "Services" page.

This roadmap is essential: it's easy to get lost when using so many tools. Put it front and center and use it throughout the process. If you're stuck, ask one of the AI tools for help. 

‍

4. Organize, organize, organize

It's easy to get overwhelmed by the sheer volume of content you'll generate, especially when you're going through multiple revisions and working iteratively with the tools to get the output right. Things will get lost if they’re not formatted and named correctly.

Set up an organizational framework that includes the in-progress and final versions of each tool’s output. Make sure you organize it as you go. If you're using text-generating AI, you might consider whether you want to specify the format of the output from the tool explicitly in your prompts or in your custom instructions. For instance, if you're generating code, ChatGPT likes to summarize it—if you want to put it directly into your file, you have to explicitly ask for it to write everything out.

If you're generating lots of different files, systematically name them so that you remember key information about each one and so you can find it later. If you haven't already, this is a great time to start creating your own prompt library.

Making these decisions early will help you avoid a mid-project meltdown.

5. Assemble your band

Once you know your goals, you can start "auditioning" your agents. You might choose a diverse ensemble of tools like ChatGPT for language tasks and Midjourney for images. Or maybe you'll go for a homogenous group, like multiple instances of ChatGPT each programmed to perform specific roles—think business strategist, website designer, or even customer persona.

Most likely your AI tools will need to be supported by other pieces of software and the work of real humans. You'll need to decide who does what when.

And if one of your tools is underperforming, you might need to kick it out of the band. 

‍

6. Get them talking

Communication between your agents is key. Most likely, you'll be linking them indirectly by feeding one's output into another's input. This won’t happen automatically, but on the plus side, this lets you review the results before they go into the next step. 

Often, this means that you’re going to have to explicitly generate output that can be used as input to another tool. Basically, you're asking a tool to create a whole or part of the prompt for another AI. But be careful: make sure you're using prompting best practices. For instance, ChatGPT doesn't always do a great job of generating Midjourney prompts, so while they might be a good first start, you’ll probably have to edit them.

For really big jobs, your agents can interact directly through APIs (Application Programming Interfaces). There are also specialized extensions that can help coordinate multi-agent activities. 

Whatever the method, the goal is synergy. You want to maximize the work you’re doing that's necessarily human-led—like deciding whether the output meets your needs or requires edits or a redo—rather than fussing around with formatting or copy-pasting (don't forget you can set up an agent to do the formatting for you).

7. Add a TL;DR tool

Even when you think you've set it up perfectly, you're going to encounter bumps along the way. Say you're deep into a content-generation task, and suddenly, the output starts going off-topic or you hit the context window limit. Or maybe the AI starts giving you results that are technically correct but miss the emotional tone you're aiming for. 

You need a summarizer tool. What's that? It's a prompt that generates a succinct few lines of text you can use if you need to "restart" a tool mid-way through.

Prompt template:

Please act as a [persona]. We are [describe task].

Here is [content 1]: [summary of content 1]

Here is [content 2]: [summary of content 2]

etc.

[Write your actual prompt]

‍

For example:

Please act as a professional copywriter. We are creating a website for a brand.

Here is the brand voice: The brand voice is engaging, upbeat, conversational, approachable. It uses simple language, short sentences, and humor. 

Here is a description of the service for the page we're working on: The service is….

Here is the copy we wrote for another service page: 

Please write the website copy for the service page that matches the brand voice….

‍

Getting into the habit of periodically asking for summaries or high-level project information makes it much easier to organize big projects.

8. Conduct your symphony

Finally, it's showtime! With all your agents in place and communicating well, you're ready to set them into action. But this phase isn’t just about hitting “start” and watching the magic unfold; it requires active management and possibly some on-the-fly adjustments.

Remember to stay flexible. For instance, if ChatGPT produces a product description that doesn't quite align with the visual created by Midjourney, you may need to tweak the input prompt for one or both tools. 

Having a good organizational method can make this much easier (you did make one, didn't you?) 

In the process, you might keep a spreadsheet that tracks which agent is responsible for what task, and the status of each. It acts like your conductor's score, guiding you through the performance.

Conclusion

Conducting a multi-agent AI symphony isn't just for tech savants—it's for anyone willing to understand the capabilities of different tools and how to harmonize them for more complex tasks. It might take a bit of time to get used to the baton, but once you do, you'll wonder how you ever got along with a solo act. So go ahead, assemble your band of AI agents, and start making some beautiful music.