9 Generative AI Portfolio Red Flags to Watch For When Hiring

Gartner predicts that by 2028, as many as 1 in 4 candidate profiles globally could be fake or materially misleading, driven largely by generative AI tools that make polished, keyword-perfect applications trivial to produce. The problem is already here: Willo's 2026 hiring research found that 76.6% of hiring teams currently encounter AI-generated or AI-assisted applications, and separate research found more than 80% of organisations have received portfolios containing inflated experience or unverifiable claims.

If you're looking to hire generative AI developers, evaluating portfolios beyond polished demos and well-written case studies is increasingly essential for distinguishing genuine technical expertise from AI-assisted presentations. 

For generative AI portfolio red flags specifically, the stakes are higher than usual, because the exact skill being hired for, generating polished, confident-sounding content, is also the skill that makes a weak portfolio easy to fake. Here are nine patterns that separate a genuine GenAI builder from someone who's mostly demonstrated they can prompt an LLM to describe a project convincingly.

A genuine builder's projects differ in the specific technical choices they made, chunking strategy, model selection, and evaluation approach, because different problems actually demand different solutions. A portfolio where every case study follows an identical structure and makes near-identical claims ("significantly improved efficiency," "streamlined the process") suggests the descriptions were generated after the fact from a template, not written from genuine, project-specific memory.

Ask why they used GPT-4 instead of Claude, or LangChain instead of a direct API call, for a specific project. A real builder has an opinion tied to actual trade-offs they weighed, cost, latency, context window, specific capability gaps. A prompt tinkerer typically defaults to "it's what I know" or a generic answer about popularity, with no sign they seriously considered an alternative.

Anyone who has genuinely deployed an LLM system has run into hallucinated output and had to build around it. A portfolio or interview answer with zero mention of hallucination mitigation, grounding, confidence thresholds, output validation against source documents, either means the person hasn't shipped anything real, or hasn't thought seriously about the biggest reliability problem in the field.

Real generative AI projects involve dead ends: a fine-tune that underperformed, a chunking strategy that hurt retrieval quality. This prompt worked in testing but broke on real user input. A portfolio that presents every project as a clean, linear success, with no "what didn't work" section anywhere, reads like it was written to look impressive after the fact rather than documented as it actually happened.

"Improved response accuracy by 40 percent" means nothing without knowing 40 percent better than what, measured how. A genuine builder can name the baseline, the evaluation set, and the specific metric, and will often volunteer the limitation of their own measurement. A prompt tinkerer's claims tend to be precise-sounding numbers with no verifiable methodology behind them, which is a pattern serious enough that some hiring guides list it as the single most common inflation signal in AI portfolios.

If a project claims to use retrieval-augmented generation, ask specifically how chunk size was chosen, what embedding model they used and why, and how they measured retrieval quality before generation even entered the picture. LLM integration developer guide covers exactly this kind of production-specific competency to screen for. A candidate who can only describe RAG at the conceptual level, retrieve then generate, without engaging with any of the actual engineering decisions, likely assembled the project from a tutorial rather than solving a real retrieval problem.

Production LLM systems live under real constraints: unmanaged token spend can blow a budget fast, and latency directly affects whether users tolerate a feature. A genuine builder's case studies mention these constraints somewhere, prompt compression, caching, model routing to control cost, because they had to solve for them. A portfolio entirely silent on cost and latency, especially for a project claiming production deployment, suggests the deployment was smaller-scale or more theoretical than described.

A claimed solo build of a large-scale, multi-component generative AI system, data pipeline, fine-tuning infrastructure, production serving, and monitoring, all built and owned by one person, is possible but uncommon, and worth probing directly. Ask them to walk through the infrastructure layer specifically. If they can't explain a part of the system they claimed credit for, the honest answer, "a platform team handled that piece, I owned the model and evaluation," is actually a stronger and more credible signal than an inflated solo claim.

Clone the repository and try to set it up according to its own instructions. A surprising share of portfolio projects fail this basic test, missing dependencies, undocumented environment variables, or code that references a dataset that was never included. Beyond that, a single-commit history on a supposedly substantial project, no incremental development, no visible iteration, is a strong signal the code was generated or copied wholesale rather than built over time, the way real projects actually get built.

As AI tools make it easier to produce a convincing-sounding portfolio and harder to distinguish genuine depth from polished description, the interview and trial-project stage will keep mattering more than the portfolio itself, not less. The nine patterns above aren't about catching someone in a lie for its own sake; they're about finding the layer where real experience and fabricated experience actually diverge, which is usually two or three follow-up questions past the initial pitch. GenAI skills for hiring managers is a useful companion read if you want the skills checklist to pair with this red-flags list. When you're ready to skip the guesswork, ai and ml freelance developers with a documented, verifiable track record can save you the vetting cycle entirely.