From Prompt Engineering to Eval Engineering

2026-08-02

Understanding the Modern AI Engineering Stack

Version 1.0

Published: August 2026

This article is a living document. AI engineering is evolving rapidly, and so is the language we use to describe it. As new research, tooling, and best practices emerge, this article will be revised and expanded. Major updates will be reflected in the version history.

Version History

v1.0 — Initial publication.

Part of a series: Read Part Two: Context Engineering

Prologue

The Day Prompt Engineering Wasn't Enough

In late 2022, something remarkable happened.

For the first time, millions of people could sit down in front of a computer, type a sentence into a chat box, and watch an artificial intelligence respond with startling fluency. It could write essays, explain calculus, debug code, summarize books, translate languages, draft business proposals, and even hold conversations that felt surprisingly human.

It wasn't just another software release. It was the first time many people experienced a computer that seemed to understand them.

Naturally, everyone began asking the same question:

"How do I get better answers?"

The answer, at least initially, was simple.

Write better prompts.

Soon, prompt engineering became one of the most talked-about skills in technology. Social media filled with threads promising "10 prompts that make ChatGPT smarter." Online courses appeared almost overnight. Job titles mentioning prompt engineering began showing up. Entire communities were dedicated to finding the perfect wording for interacting with large language models.

For a while, it seemed that prompts were everything.

If your AI produced poor results, the assumption was straightforward: you hadn't asked the question well enough.

And to some extent, that was true.

A carefully written prompt often produced dramatically better results than a vague one. Learning how to give clear instructions, provide examples, and specify the desired output became a valuable skill.

But then something changed.

Developers stopped asking AI to answer questions.

They started asking it to do work.

Instead of saying,

"Explain how recursion works."

they began saying,

"Open my GitHub repository, inspect the failing tests, identify the bug, write a fix, run the test suite, and open a pull request."

That single sentence represented a fundamental shift.

The model was no longer acting as a conversational assistant. It was becoming part of a larger system—a system expected to browse the web, search documentation, interact with databases, execute code, coordinate with external tools, recover from failures, and sometimes work autonomously for several minutes or even hours.

At that point, something became obvious.

No prompt, no matter how carefully written, could solve every problem.

Imagine hiring the smartest engineer you've ever met.

They arrive on their first day at work.

You give them a desk.

Then you take away their laptop.

No internet.

No documentation.

No access to the company's codebase.

No email.

No development tools.

No colleagues to ask for help.

Now ask them to fix a production bug.

They'll probably struggle—not because they lack intelligence, but because intelligence alone isn't enough. They need information. They need tools. They need feedback. They need an environment in which they can operate effectively.

Artificial intelligence is no different.

As language models became more capable, engineers discovered that the model itself was only one piece of the puzzle. Around it grew an entire ecosystem of new engineering disciplines, each addressing a different challenge in building reliable AI systems.

Some disciplines focused on how instructions were written.

Others focused on what information the model received.

Some focused on giving models access to tools.

Others explored how agents could iteratively improve their own work, coordinate complex workflows, or evaluate whether their outputs were actually correct.

Collectively, these ideas form what many practitioners now think of as the modern AI engineering stack.

It includes six closely related—but fundamentally different—disciplines:

Prompt Engineering
Context Engineering
Harness Engineering
Loop Engineering
Graph Engineering
Eval Engineering

These are not competing ideas.

They are complementary layers.

Each exists because the layer before it eventually reached its limits.

Understanding that evolution is the key to understanding how modern AI systems are actually built.

This article is the story of those layers.

A Note on Terminology

AI engineering is evolving rapidly, and so is its vocabulary. Terms such as Prompt Engineering are widely established, while others—including Harness Engineering, Loop Engineering, and Graph Engineering—are still emerging and may be defined differently across research groups, companies, and open-source communities. Rather than presenting these as rigid standards, this article uses them as a practical framework for understanding the different responsibilities involved in building reliable AI systems.

Chapter One

Prompt Engineering

Teaching Intelligence Through Language

Imagine that your company hires an exceptionally talented intern.

They graduated at the top of their class. They learn quickly, solve problems creatively, and possess an almost photographic memory. Everyone is excited to work with them.

On their first morning, you walk past their desk and say,

"Handle the client."

Then you leave for a meeting.

What exactly should they do?

Should they send an email?

Schedule a meeting?

Call the client?

Prepare a report?

Review the contract?

Follow up on last week's conversation?

They don't know.

Not because they're unintelligent.

Because your instruction is ambiguous.

Now imagine giving the same intern a different instruction.

"Please review Acme Corporation's support ticket from yesterday. Summarize the customer's concerns in three bullet points, identify whether the issue is technical or billing-related, and draft a polite email that offers the next available support appointment. Do not send the email—save it as a draft."

Notice what changed.

The intern didn't become smarter.

The instruction became clearer.

That, in its simplest form, is prompt engineering.

The Building Blocks of a Good Prompt

Although prompts vary depending on the task, most effective prompts contain some combination of the following elements:

Role

Who should the model act as?

For example:

A software engineer
A science teacher
A legal assistant
A travel planner

Assigning a role doesn't change what the model knows, but it influences how it organizes and presents that knowledge.

Task

What exactly should it do?

Tasks should be explicit and measurable whenever possible.

Compare:

Improve this.

with:

Rewrite this paragraph to improve clarity while preserving its meaning.

The second instruction leaves far less room for interpretation.

Context

What background information does the model need?

This might include previous conversation, project requirements, reference documents, or company policies.

We'll explore this in much greater depth in the next chapter because, as AI systems have evolved, managing context has become an engineering discipline of its own.

Constraints

What should the model avoid?

Examples include:

Maximum word count
Required output format
Tone of voice
Programming language
Citation requirements

Constraints are often just as important as the task itself because they define the boundaries within which the model should operate.

Examples

Sometimes, showing is more effective than telling.

Rather than describing the desired output, you can provide one or more examples. This technique—often called few-shot prompting—helps the model infer patterns from demonstration rather than instruction alone.

For instance, if you want consistent product descriptions, providing three examples of your preferred style is usually more effective than writing a long explanation of that style.

Prompt Engineering Is Specification Engineering

As software engineers, we write specifications for people.

Prompt engineering extends that idea to machines.

The quality of the outcome depends heavily on the clarity of the specification.

In fact, many experienced AI practitioners have stopped thinking about prompts as prompts.

They think about them as contracts.

A well-designed prompt answers questions before the model has to ask them internally.

What is my objective?

What should I avoid?

Who is the audience?

What format should I use?

What does success look like?

The fewer assumptions the model has to make, the more reliable its outputs tend to become.

The Limits of Prompt Engineering

For a time, it was tempting to believe that every AI problem could be solved with a better prompt.

If the answer wasn't good enough, perhaps the instructions needed refinement.

If the model hallucinated, perhaps the wording wasn't precise enough.

If the output lacked detail, perhaps more examples would help.

Prompt engineering undoubtedly improves results.

But eventually, every practitioner encounters the same limitation.

Imagine asking ChatGPT:

"Summarize the meeting that happened in my office this morning."

If the meeting transcript has never been provided, no prompt—regardless of how sophisticated—is capable of producing an accurate summary.

The model cannot reason over information it has never seen.

Likewise, no prompt can tell a coding assistant about the contents of a private repository it cannot access. No prompt can reveal yesterday's customer support tickets. No prompt can expose the latest entries in a company's database.

The limitation isn't the quality of the instructions.

It's the absence of information.

As AI systems became more deeply integrated into real-world workflows, engineers realized that asking better questions wasn't enough.

The more important question became:

How do we ensure the model has the right information at the right time?

Answering that question gave rise to the next discipline in the evolution of AI engineering.

** >Context Engineering.**

Coming Up in Part Two

In the next chapter, we'll explore why intelligence without information is fundamentally limited, how retrieval-augmented generation (RAG), memory, and dynamic context transformed AI systems, and why many practitioners now argue that context engineering has become more important than prompt engineering itself.