Prompt Engineering Is Not. Engineering, That Is

Summary

Prompt Engineering Is Not. Engineering, That Is. We went looking for engineering. We found creative writing advice with a technical veneer. After months of building AI-driven tooling – writing prompts, iterating on specifications, debugging non-deterministic outputs – we hit a wall. The prompts weren’t producing reliable results. The outputs varied between runs. The AI made judgment calls we couldn’t predict or audit. So we did what any engineer would do: we went looking for the engineering discipline behind prompt design. The formal methods. The testing frameworks. The measurement science. What we found was blog posts that say “be specific” and “iterate.” That’s what started this investigation. A declaration of the human’s bias: I am trained as a scientist – a BS in Physics and Mathematics, an MS in Physics – and I have spent over four decades building global-scale infrastructure. I have strong opinions about what the word “engineering” means. This article is not neutral. Every criticism is backed by evidence, but the frustration is mine and the perspective comes from a career spent in disciplines where precision is not optional. Unfortunately, the AI’s biases are not so easily summarized. This article is critical. It is not anti-AI. We use AI extensively, we find the collaboration genuinely powerful, and we want this technology to succeed. The criticism exists because we’re concerned that the issues raised here – if not addressed – will prevent that success. Every technology that eventually became game-changing was misunderstood in the early part of its lifecycle. We are trying to help separate the signal from the hype so that the signal can win. What Engineering Actually Means Before we can argue that prompt engineering isn’t engineering, we need to establish what engineering actually is. Not what we feel it should be – what the professional and accreditation bodies that govern the discipline say it is. The definitions converge. ABET, IEEE, and the National Socie...