Founder of Goodspeed
Everyone claims to be a Codex expert now. The barrier to sounding fluent has never been lower, because an AI coding agent will happily generate confident code and confident explanations for anyone who asks. The result is a market full of people who can talk about building with OpenAI Codex, and a much smaller group who can actually ship something that survives contact with real users.
Telling the two apart is the whole game when you are hiring. The difference is not the tools they name or the demos they show, both of which are easy to produce. The difference is depth: whether they understand what happens after the impressive first draft, when the software has to be reliable, secure and maintainable in production.
This guide gives you the signals that separate a genuine AI engineer from someone who is good at prompting. None of them require you to be technical. They only require you to know what to look for and to insist on evidence rather than enthusiasm.
Why everyone suddenly sounds expert
AI coding agents have collapsed the gap between sounding competent and being competent. A newcomer can now produce working code, explain it in plausible technical language, and assemble a portfolio that looks the part, all in a fraction of the time it used to take. That is genuinely useful, but it also means surface impressions are worthless for judging real skill.
The consequence for buyers is that you cannot rely on the usual signals. A polished landing page, a confident pitch and a good looking demo now cost almost nothing to fake. To find genuine expertise you have to look past the first layer and probe the parts of the work that AI cannot generate for you: judgement, hard won experience and evidence of systems that lasted.
Production systems, not just prototypes
The clearest signal of a real expert is a track record of software running in production, used by actual people, handling actual data. A prototype proves you can start something. A production system proves you can finish it, keep it alive, and deal with everything that goes wrong between the demo and the thousandth user. That distance is where most projects quietly die.
When you interview a prospective partner, ask them to walk you through a system they built that is live today. Who uses it, what does it do, how long has it been running, what broke along the way. Genuine experts light up at this question because it is where their real work lives. Prompt only operators steer the conversation back to demos, because a demo is all they have.
They can explain how they know it works
Ask a genuine engineer how they know their software is correct, and you will get a real answer about testing and evaluation. They will talk about automated tests, regression suites, and for AI features, evals that measure accuracy against known cases. They treat correctness as something you prove, not something you hope for after clicking around the app a few times.
This is one of the fastest tells available to a non technical buyer. Someone who cannot explain how they verify their work, beyond it seemed to work when I tried it, is not operating at a professional level. The moment you hear a structured, confident answer about evaluation, you are probably talking to someone who has shipped real software before.
Guardrails show they have been burned
Experienced AI engineers design guardrails because they have seen software behave badly in the wild. They talk naturally about input validation, handling the cases where a model returns nonsense, rate limiting to control cost, and safe fallbacks when something fails. This defensive instinct is earned, not taught, and it is almost impossible to fake convincingly.
When you ask what happens when the AI gets it wrong, listen for whether the answer is concrete. A genuine expert has specific mechanisms in mind and can describe how they have used them. A beginner is surprised by the question, because in their world the AI has always worked well enough in a controlled demo. Real production experience makes people cautious in a way you can hear.
Honest about trade offs and limits
Genuine experts are comfortable telling you what a technology cannot do, or what a decision costs. They will explain why they chose one approach over another, what they gave up, and where the risks sit. This willingness to name trade offs is a strong signal, because only people who understand a system deeply can describe its limits.
Beware anyone who says AI can do anything or promises there are no downsides. Software is a series of compromises, and pretending otherwise is either dishonesty or inexperience. The people you want to hire are the ones who temper their enthusiasm with realism, because that same realism is what stops your project running off a cliff halfway through.
They talk about failure comfortably
Ask an expert about a project that went wrong and watch their reaction. The genuine ones answer easily, because failure is a normal part of building software and they have a stock of lessons from it. They can tell you what broke, why, and what they changed afterwards. That story is often more revealing than any success.
People without real experience tend to have no failure stories, or offer sanitised ones with no substance. Everything in their world has gone smoothly, which only happens to people who have not shipped much. The ability to discuss failure specifically and without defensiveness is a hallmark of someone who has done the work for real and learned from it.
Depth on the boring engineering
The unglamorous parts of software are where expertise really shows. Version control, code review, continuous integration, staging environments, monitoring and deployment are not exciting, but they are what makes software maintainable and safe. A genuine engineer treats these as non negotiable, because they know that skipping them is how fast projects become unmaintainable ones.
A useful test is to ask how their code gets from an idea into production. An expert describes a disciplined pipeline with review and testing at each step. A prompt only operator describes generating code and pushing it live, because the surrounding engineering is invisible to them. The AI writes the code either way, but only the expert has the process that keeps it trustworthy.
No certificate proves this
There is no official Codex certification, so anyone presenting a qualification as proof of expertise is either confused or hoping you are. The tools are too new and moving too fast for a meaningful formal credential to exist. What matters is demonstrated ability, and that is assessed through evidence, not badges.
This is liberating once you accept it. Instead of trying to decode qualifications, you judge people on what they have actually built and how well they can explain it. A portfolio of live systems and a fluent, specific conversation about how they were built tells you everything a certificate never could. Trust the evidence in front of you over any claimed credential.
How they use the tools, not that they do
Using an AI coding agent is not a skill any more, it is a baseline. The skill is in how you use it: what you delegate to the model, what you keep under human control, how you review its output, and how you catch the mistakes it makes. Experts have a clear philosophy here, born from doing it a lot.
Ask how they work with AI coding agents day to day. A strong answer describes a deliberate division of labour, careful review of generated code, and awareness of where the model tends to go wrong. A weak answer treats the AI as infallible or describes handing it a task and shipping whatever comes back. The tool is the same, the judgement around it is what you are paying for.
References that describe outcomes
A genuine expert has clients who can describe what was built and what it achieved. When you take references, push past general praise and ask what the system actually does, whether it is still running, and what problem it solved. Concrete answers from real users are hard to manufacture and tell you the work was real.
Be wary of references that speak only in vague warmth, or case studies that never name a measurable result. Real engagements leave a trail of specifics: a system, a set of users, an outcome. If a partner cannot connect you with anyone who can describe those things, treat the gap as an answer in itself.
Curiosity over certainty
The field of AI engineering moves quickly, and the genuine experts stay curious because they have to. They can tell you what they have learned recently, what they have changed their mind about, and where they think the tooling is heading. That ongoing engagement is a good sign, because a system built by someone who stopped learning last year will show it.
Overconfidence is the opposite signal. Anyone who claims to have it all figured out in a field this young is either not paying attention or bluffing. The people worth hiring hold their expertise alongside genuine humility about how much is still changing, and that balance is exactly what you want steering your project.
Putting the signals together
No single signal is decisive, but together they paint a reliable picture. A genuine Codex expert has production systems, an evaluation story, a healthy respect for failure, disciplined engineering habits and the honesty to name trade offs. They talk in specifics, welcome hard questions, and never lean on certificates that do not exist.
Bring these tests to every hiring conversation and the field sorts itself quickly. The prompt only operators cannot sustain the depth, and the real engineers cannot help but show it. Spend your first call probing evidence rather than admiring demos, and you will hire the person who can actually ship, not just the person who can talk.
Conclusion
A genuine Codex expert is not the person with the slickest demo or the most confident patter. It is the person who can show you production systems, explain how they know the software works, and talk honestly about the parts that were hard. Expertise reveals itself in specifics, in evidence, and in a healthy respect for the ways software fails.
Prompt only operators cannot fake these things for long. Ask for the eval story, the failure story and the running system, and the real experts stand out within minutes. Hire on that evidence and you will avoid the expensive mistake of paying a beginner senior rates.
If you want a team that ships production software with AI coding agents, see our AI work, or book a free call with our AI engineering team.
Written By
Founder of Goodspeed