The Misnomer of Prompt Engineering

Embracing the unpredictability of Generative AI

In software development, the term "engineering"* carries a weight of precision and predictability. When a coder writes a script in languages like C# or Python, there's an expectation of a specific, reliable output. This expectation stems from a fundamental principle of programming: the unwavering obedience of code to its written logic. (Note: I know, I know, we all agree that programming is a lot like golf. Expectations rarely meet reality)

Enter the world of prompt "engineering" in the context of generative AI, and you find yourself in totally unfamiliar territory. Here, the term "engineering" almost feels like a misnomer. Unlike the deterministic nature of traditional coding, interacting with generative AI models like GPT-4 is similar to conversing with a human - unpredictable, nuanced, and sometimes, delightfully surprising.

Why "Prompt Engineering" is not quite engineering

Prompt engineering, at its core, is an art of trial and error. It's about developing inputs (prompts) for AI in a way that nudges it toward the desired output. But here's the catch: the AI's response often varies significantly, influenced by a ton of factors, much like human responses. This unpredictability isn't a flaw; rather, it's a feature of the AI's design, mirroring the complexity of human thought and language.

The Human Element in AI

What makes this unpredictability interesting is the human-like traits that AI exhibits. The large language models are trained on VAST amounts of data that encompasses the full richness of human language and human interaction.  As a result, generative AI is a reflection of our unpredictability, biases, creativity, and even humor. Side note: I appreciate my company hasn't required me to expose my ChatGPT history with all of my "humor" requests 😜. IMO, prompt engineering is less about commanding and more about conversing with an entity that has its own understanding of the world. Am I the only one who feels comforted by ChatGPT's often encouraging and upbeat responses to prompts?

Navigating the unpredictability of generative AI

Despite its unpredictability, I've discovered some strategies to improve the consistency of outcomes when using generative AI. Here are my tips:

1. Be specific and detailed: The more detailed your prompt, the less room for AI to veer off course. Precision in language guides AI more effectively to a meaningful response. For example, I wanted a summary of a very long whitepaper so I typed into ChatGPT: "summarize this whitepaper". What I got was a very long response that didn't give me what I really wanted which was: "summarize this whitepaper, plus provide me with a summary of what's most important, plus give me any interesting data points." The response I got was much better the second time when I told ChatGPT exactly what I wanted.
2. Understand limitations of the model: Generative AI, while powerful, has limitations. It's not an oracle, but a reflection of the data it's been trained on. That's why asking it for detailed information on the very latest technology often yields lousy results. 
3. Refine, refine, refine: It's a rare occurrence for me to get it right the first time. If you treat AI as an iterative process you'll be surprised at how much your results improve by continously refining. Like debugging code, the more you refine, the better it gets.
4. Use examples as often as possible: Providing examples within your prompt acts as a guide for AI - setting a precedent for the kind of response you're looking for. One time I kept getting an unsatisfactory response to some sample code I wanted generated. Finally, I added some code that I'd gotten earlier from AI. It was amazing how my response immediately got better. 
5. Experiment & learn: Can we all agree that AI is kind of unpredictable? Sometimes that leads to serendipitous results - offering new insights you might never have considered. And sometimes it's just crazy stuff. Either way, the more you experiment the better it gets.

Embracing a new paradigm

Prompt engineering challenges our traditional understanding of "engineering". Being great at prompt engineering requires a blend of technical expertise, creativity, and a willingness to embrace uncertainty. As we all get better and more experienced we'll learn to communicate with an emerging form of digital consciousness - one that's as unpredictably human as the data it was trained on.

*Credit to Simon Willison whose blog post inspired me to write this post.