The 10 Skills at a Glance: Verification Method and Signal
|
# |
Skill |
Verification Method |
What It Proves |
|
1 |
Prompt engineering |
Ask for a structured system prompt from a past project with reasoning |
Production-grade output control, not trial-and-error prompting |
|
2 |
LLM fine-tuning |
Request before/after evaluation results from a domain-specific fine-tune |
Ability to adapt foundation models to proprietary data and measure the result |
|
3 |
RAG pipeline design |
Ask to describe chunking strategy, retrieval tuning, and hallucination mitigation |
Can build grounded, accurate knowledge retrieval systems for production |
|
4 |
LangChain / LlamaIndex |
Ask why they chose the framework for a specific past project |
Framework selection is deliberate, not default; knows trade-offs |
|
5 |
Vector databases |
Ask which vector DB they used and why, with a specific use case |
Understands semantic search infrastructure, not just embedding API calls |
|
6 |
API integration |
Request a code sample showing multi-provider LLM API handling with error retry |
Can build resilient production integrations across OpenAI, Anthropic, Gemini |
|
7 |
Evaluation frameworks |
Ask what evaluation suite they ran before shipping a Gen AI feature |
Quantifies output quality; does not ship without measurement |
|
8 |
Cost optimisation |
Ask for a cost estimate methodology for a high-volume LLM feature |
Understands token economics; builds financially sustainable systems |
|
9 |
Safety and alignment basics |
Ask how they prevent prompt injection and handle off-topic inputs |
Production-ready Gen AI requires guardrails, not just capable outputs |
|
10 |
Cloud deployment (AWS/GCP) |
Ask for a deployed Gen AI endpoint they own and can describe the infra |
The model is running somewhere; can describe the stack that runs it |
Skill 1: Prompt Engineering
Prompt engineering is the most misunderstood skill on this list because it is simultaneously the most practised and the most poorly executed. Every developer who has used ChatGPT has written prompts. Fewer than a third of them have designed a structured prompt system that controls LLM output reliably across thousands of varied inputs in a production environment. The distinction between casual prompting and production prompt engineering is the difference between a developer who can get a good response from a model and one who can guarantee a consistent, parseable, safe response format across input distributions that were not part of the original design.
Production prompt engineering requires four components that are absent in most tutorial-level work: a system prompt that defines role, constraints, output format, and failure behaviour; a user prompt template that normalises variable inputs into a consistent structure the model can parse; a few-shot example set that anchors the model's output distribution to the expected format; and an output parser that validates the model's response against the expected schema before it reaches downstream systems. A developer who cannot produce all four for a past project has not designed a production prompt system. Shreyans Padmani's generative AI development services validate each prompt system against the client's specific use case and input distribution before deployment, which is the standard that production Gen AI work requires.
How to verify this skill
Ask the candidate to share a system prompt from a past production project, with a brief explanation of why each constraint was added. A developer who has designed a production prompt system will produce this in minutes and explain specific decisions: why they included a JSON output format constraint, how they handled the model's tendency to add unsolicited caveats, and what they did when the model violated the output schema. A developer who has not will describe the concept of system prompts accurately without being able to show a specific implementation.
Skill 2: LLM Fine-Tuning
LLM fine-tuning is the Gen AI skill with the widest gap between claimed experience and demonstrated capability, because the technical vocabulary of fine-tuning (LoRA, QLoRA, PEFT, instruction tuning, RLHF) is widely known from blog posts, but the practical experience of running a successful fine-tune on domain-specific proprietary data, evaluating the result against a held-out test set, and deploying the fine-tuned model to a production inference endpoint is comparatively rare. The reason the gap exists is that fine-tuning is expensive (GPU compute cost), data-intensive (high-quality annotated examples required), and time-consuming (multiple training runs with evaluation between each), which means it only gets done in real project contexts, not in online course exercises. Shreyans Padmani's custom AI model training service covers this full cycle, including data pipeline setup, training configuration, evaluation against domain-specific held-out sets, and production deployment of the fine-tuned model.
How to verify this skill
Ask for the before-and-after evaluation results from a specific fine-tuning project: the base model performance on the domain task, the fine-tuned model performance on the same evaluation set, and the evaluation metric used. A developer who has completed a real fine-tune will have these numbers because they are how fine-tuning success is measured. Ask which PEFT method they used and why: LoRA over full fine-tuning for compute efficiency, QLoRA for quantised training on smaller GPU budgets, full fine-tuning for small models where parameter count allows it. Ask how they curated the training examples and what quality criteria they applied. Answers to all three questions should be specific and consistent.
Skill 3: RAG Pipeline Design
Retrieval-Augmented Generation is the architecture pattern that enables generative AI systems to answer questions accurately about private or recent information without the hallucinations that occur when a model is asked to generate from parametric knowledge alone. It is also the most common source of production Gen AI failures, because RAG is architecturally simple to implement at a basic level and genuinely difficult to implement reliably at production quality. The basic version, embed documents, store in a vector database, retrieve on query similarity, insert into prompt, generate response, takes two hours to build. The production version, with accurate retrieval across heterogeneous document types, chunking strategies tuned to the specific content structure, hybrid BM25 and dense retrieval for recall, and confidence-based hallucination mitigation, takes weeks and requires domain-specific measurement to validate.
The NLP customer feedback classification and AI video summarisation systems documented in Shreyans Padmani's AI case studies both involve production retrieval pipelines where the quality of the knowledge retrieval layer directly determines output accuracy. The video summarisation system, which reduced meeting review time from 45 minutes per video to under 5 with no proprietary SaaS dependency, required a retrieval layer tuned specifically to the semantic structure of spoken meeting content, which differs materially from written document retrieval.
How to verify this skill
Ask the candidate to describe the chunking strategy they used in a past RAG project and why they chose it. Fixed-size chunking versus semantic chunking versus document-structure-aware chunking are different strategies with different strengths, and a developer who has built a production RAG system made a deliberate choice between them for a specific reason. Ask how they measured retrieval quality: MRR@k, NDCG, or recall@k on a held-out evaluation set. Ask how they handled the case where the retrieved context contained partially relevant and partially irrelevant information. A developer who cannot answer these questions has not debugged a RAG system under real usage conditions.
Skill 4: LangChain and LlamaIndex
LangChain and LlamaIndex are the two dominant orchestration frameworks for building Gen AI applications in 2026. LangChain is broader in scope, covering chains, agents, tool use, and memory with a large ecosystem of integrations. LlamaIndex is more focused on data ingestion, retrieval, and RAG-specific workflows. Neither is universally the better choice, and a developer who applies the same framework to every project regardless of requirements has not made a design decision. The correct verification question is not which framework they know but why they chose the one they used for a specific past project and what the trade-offs were against the alternative. Shreyans Padmani's AI agent development services use LangChain, AutoGen, and CrewAI for multi-agent orchestration, selected per project requirements rather than as a default stack, which reflects the framework-selection discipline that production Gen AI work requires.
How to verify this skill
Ask the candidate to describe a situation where they chose LangChain over a custom implementation, or chose a custom implementation over LangChain, and explain the reasoning. A developer who has made this decision deliberately will describe the specific trade-off that drove it: LangChain's abstraction overhead became a latency problem at high request volume, so the prompt chain was reimplemented with direct API calls and a custom output parser. Or: LlamaIndex's document index abstraction handled the PDF parsing complexity better than building a custom ingestion pipeline, saving two weeks of development time with acceptable abstraction cost. The candidate who cannot produce a specific reasoning for their framework choice has not encountered the production constraints that force that reasoning.
Skill 5: Vector Databases
Vector databases are the infrastructure layer that enables semantic search, RAG retrieval, and embedding-based recommendation in Gen AI applications. The developer skill here is not simply knowing how to call the Pinecone or Weaviate API: it is understanding the trade-offs between managed vector database services (Pinecone, Weaviate Cloud), self-hosted vector databases (Qdrant, Milvus), and relational database vector extensions (pgvector on PostgreSQL) for different use cases, and being able to make the right selection based on the project's data volume, query latency requirements, cost structure, and data residency constraints. Shreyans Padmani's NLP development services include semantic search pipeline construction using embedding models and vector retrieval, where the choice between Pinecone, pgvector, and FAISS is made based on the client's existing infrastructure and query volume rather than framework familiarity.
How to verify this skill
Ask the candidate which vector database they used in a past project and why they chose it over the alternatives. Ask what embedding model they used to generate the vectors and how they evaluated embedding quality on the specific content domain. Ask how they handled index updates as new documents were added after the initial index build. A developer who has operated a vector database in production will have answered all three questions under real conditions: they will have chosen pgvector because the client's data was already in PostgreSQL and the query volume did not justify a separate managed service, or chosen Pinecone because the managed infrastructure eliminated the operational overhead of self-hosting Qdrant at the team's scale. The reasoning is the verification signal, not the tool name.
Skill 6: API Integration
Generative AI applications in 2026 run across multiple LLM provider APIs: OpenAI, Anthropic, Google Gemini, Cohere, and open-source models served via Replicate, Together AI, or self-hosted vLLM. A production Gen AI system that relies on a single provider without fallback logic is a system that goes down when that provider has an outage. An API integration skill that covers only the happy path, the successful API call that returns a well-formed completion, has not been tested against the failure modes that define production reliability: rate limit errors that require exponential backoff and retry, context length errors that require prompt truncation or chunking, provider outages that require fallback to an alternative model, and malformed API responses that require defensive parsing before the output reaches downstream systems.
How to verify this skill
Ask the candidate to describe their error handling strategy for LLM API calls in a past production system. Specifically ask what happens when the API returns a rate limit error, when the response is malformed JSON despite a JSON output instruction in the system prompt, and when the primary provider is unreachable. A developer who has shipped a production Gen AI system has encountered all three failure modes and has code that handles them. Ask to see the relevant code section or describe the retry logic specifically. Exponential backoff with jitter for rate limit errors, output validation with re-prompt on malformed responses, and provider fallback with response format normalisation are the correct answers. Vague descriptions of error handling without these specifics indicate the failure modes have not been encountered in production.
Skill 7: Evaluation Frameworks
LLM output evaluation is the discipline that most clearly separates developers who have shipped Gen AI products from those who have built Gen AI prototypes. A prototype is evaluated by its creator, who reads sample outputs and judges them good. A production system is evaluated by a framework that runs automatically, catches regressions when the underlying model is updated, and provides quantitative evidence that the system performs at or above the required standard before changes are deployed. The absence of an evaluation framework in a Gen AI developer's portfolio is the single strongest signal that the developer has not maintained a live Gen AI system through a provider model update.
How to verify this skill
Ask the candidate what evaluation framework they used before shipping a Gen AI feature and how they used it after shipping when the upstream model was updated. Specific tools indicate production-level evaluation practice: RAGAS for RAG pipeline evaluation, LangSmith or LangFuse for tracing and debugging individual LLM calls, human evaluation rubrics with inter-annotator agreement for subjective output quality, and automated test suites that run against a held-out golden dataset. Ask specifically what metric they used to decide whether a model update was safe to deploy. A developer who cannot answer this question with a specific metric and threshold has not designed an evaluation process that would catch a quality regression before it reached users.
Skill 8: Cost Optimisation
LLM inference cost is the most common reason Gen AI features are economically unsustainable at scale, and it is the skill area where the gap between prototype development and production engineering is most financially consequential. A developer building a Gen AI feature at prototype scale, with ten to fifty API calls per day, never encounters the cost problem because the daily API bill is below the threshold of concern. The same developer building a feature at 50,000 requests per day, with a prompt averaging 800 tokens and a response averaging 400 tokens, is generating 60 million tokens per day at current GPT-4o pricing, which amounts to approximately 900 US dollars per day, or 27,000 US dollars per month before any infrastructure costs. Cost optimisation is not a post-launch concern: it is an architectural decision made at the system design stage.
How to verify this skill
Ask the candidate to estimate the monthly LLM inference cost for a described system: 100,000 user requests per day, each requiring one LLM call with a 600-token prompt and a 300-token response, using GPT-4o. A developer who understands LLM cost will produce an estimate within 20 percent of the correct figure and describe the optimisation strategies they would apply: prompt compression to reduce input token count, semantic caching in Redis to serve repeated queries without inference, model tiering where GPT-4o-mini handles simple classification and GPT-4o handles complex reasoning, and batching asynchronous requests to improve throughput efficiency. A developer who cannot produce the estimate without looking up the pricing page has not designed a cost-aware Gen AI system.
Skill 9: Safety and Alignment Basics
Safety and alignment basics are the Gen AI developer skills that the industry most consistently underweights during hiring and most consistently regrets underweighting after deployment. A Gen AI system deployed without guardrails is a system that users will find ways to exploit: prompt injection attacks that cause the system to ignore its operational constraints, jailbreaking attempts that elicit content the system was not designed to produce, and off-topic usage that creates legal or reputational exposure for the business deploying the system. None of these require sophisticated adversarial expertise: the most common prompt injection attacks are trivially simple, and a developer who has not designed defences against them has not thought about the adversarial surface of the system they built.
How to verify this skill
Ask the candidate how they prevent prompt injection in a RAG system where user input is inserted into a prompt that also contains retrieved document content. The correct answer describes at minimum three layers: input sanitisation that strips or escapes control characters and injection patterns before the user input enters the prompt; a system prompt constraint that instructs the model to ignore instructions embedded in retrieved context; and an output validation layer that checks the model's response for signs of instruction-following behaviour that should not have occurred. Ask how they handle a user input that is entirely off-topic for the system's defined scope. Ask whether they implement content filtering on outputs before they reach the end user. A developer who has thought about these questions has built a production Gen AI system. A developer who has not will be encountering these questions for the first time.
Skill 10: Cloud Deployment on AWS or GCP
Deployment is the skill that converts a working Gen AI model into a business asset. A generative AI system running in a Jupyter notebook on a developer's laptop is not a product. A generative AI system running as a containerised FastAPI endpoint on AWS Lambda with auto-scaling, a Redis prediction cache, CloudWatch monitoring for latency and error rate, and a CI/CD pipeline that validates the evaluation suite before each deployment is a product. The deployment skill for Gen AI in 2026 includes familiarity with serverless compute for cost-efficient low-to-medium traffic endpoints, GPU-backed EC2 or GCP Vertex AI for large model inference, container orchestration with Docker and Kubernetes for high-traffic production systems, and infrastructure-as-code practices that make deployment reproducible and auditable.
How to verify this skill
Ask the candidate to describe the production infrastructure for a Gen AI system they deployed: where the model inference runs, how the endpoint scales under traffic spikes, what monitoring is in place, and how they deploy updates without downtime. A developer who has built production Gen AI infrastructure will describe specific choices with specific reasoning: Lambda for a low-traffic document classification endpoint because the cost at 10,000 requests per day is under 30 US dollars monthly and cold starts are acceptable at that latency tolerance; EC2 GPU instance for a real-time RAG system requiring sub-200ms response times because Lambda's cold start eliminates it as an option; Fargate for a mid-traffic conversational AI endpoint because auto-scaling handles traffic variance without the operational overhead of managing EC2 instances. The specificity of the reasoning is the verification signal.
Putting the Verification Into Practice: The Pre-Hire Flow
The ten verification questions in this guide are not a sequential interview script. They are a filter applied at different stages of the hiring process to eliminate unqualified candidates as early as possible and minimise the time cost of discovering skill gaps late. The most efficient sequence is three stages: portfolio filter before the first call, technical screen covering five of the ten skills in the first conversation, and paid trial project as the final verification.
The portfolio filter requires only a deployed Gen AI endpoint or a published case study with a quantified business outcome. A developer who cannot produce either has not shipped a production system. This single filter eliminates the majority of candidates who list Gen AI skills without the production track record those skills require in a 2026 hiring context. Shreyans Padmani's profile demonstrates what this looks like: 12 published case studies across generative AI, ML, NLP, computer vision, and AI agent development, a 100 percent Upwork job success score across paid client engagements, and a Microsoft AI certification providing independent skills verification. The case studies cover systems that reduced hiring review time by 70 percent, cut video review time from 45 minutes to under 5, saved 30 hours per week in content creation, and improved defect detection speed by 80 percent on a manufacturing line. Each outcome is specific, measurable, and verifiable.
The technical screen applies the five highest-signal questions from this guide: RAG chunking strategy and retrieval quality measurement (Skill 3), cost estimation for a described high-volume system (Skill 8), evaluation framework used before and after a model update (Skill 7), prompt injection defence in a RAG system (Skill 9), and the production infrastructure description for a deployed endpoint (Skill 10). These five questions cannot be answered correctly by a developer who has not shipped production Gen AI systems, which makes them a reliable filter for the gap between the claimed experience on a CV and the demonstrated experience in a portfolio. For a deeper look at LLM vs Generative AI distinctions and how each maps to developer skill requirements, the AI predictions post on shreyans.tech covers the architectural evolution from general LLMs to vertical and domain-specific generative AI systems.
Frequently Asked Questions
The best generative ai developers in 2026 demonstrate production competency across ten skill areas: structured prompt engineering with output validation, LLM fine-tuning using PEFT methods (LoRA, QLoRA) with domain-specific evaluation, RAG pipeline design with measurable retrieval quality, LangChain or LlamaIndex framework selection based on architectural requirements, vector database selection and operation (Pinecone, pgvector, Weaviate), multi-provider LLM API integration with error handling and fallback logic, automated Gen AI evaluation frameworks (RAGAS, LangSmith), inference cost optimisation through prompt compression, caching, and model tiering, safety and prompt injection defence, and cloud deployment on AWS or GCP with monitoring and CI/CD integration.
The three most reliable verification steps are: require a deployed Gen AI endpoint or a published case study with a quantified business outcome before scheduling a technical interview; ask five targeted technical questions covering RAG retrieval quality measurement, cost estimation for a high-volume system, evaluation framework design, prompt injection defence, and production infrastructure description; and run a paid trial project costing 500 to 1,500 US dollars on a representative subset of the actual project data. A developer who passes all three filters has demonstrated production Gen AI capability across the dimensions that matter most for project delivery.
The terms are often used interchangeably, but they carry a useful distinction. An LLM developer focuses specifically on large language model integration, fine-tuning, and deployment, working primarily with text-based models and the orchestration frameworks that connect them to applications. A generative AI developer covers a broader scope that includes text (LLMs), image generation (DALL-E, Stable Diffusion, Midjourney API), audio generation, video generation, and multimodal models that combine modalities. In practice, most generative AI development work in 2026 is LLM-centric, and the two titles refer to overlapping skill sets. The architectural distinctions between general LLMs and vertical domain-adapted models, and between prompt-based and fine-tuned approaches, are where the meaningful technical differences lie.
Rates for generative ai developers in 2026 range from 60 to 110 US dollars per hour for India-based specialists with verified production portfolios to 160 to 280 US dollars per hour for US-based senior LLM engineers. Fixed-price Gen AI project costs run 5,000 to 20,000 US dollars for a focused integration with RAG and deployment, and 15,000 to 50,000 US dollars for a full multi-agent or fine-tuning engagement. Monthly dedicated contracts for a senior India-based Gen AI developer run approximately 10,000 to 18,000 US dollars, compared to 25,000 to 40,000 US dollars for a US equivalent at the same experience level.
The most reliable sourcing channels for verified Gen AI developers are Upwork's Expert Vetted and Top Rated tiers, where the job success score and public contract history provide accountability not available on portfolio-only platforms; Toptal, which pre-screens candidates through a multi-stage technical interview; and personal referrals from technical founders or CTOs for IP-sensitive projects. The verification framework in this guide applies regardless of sourcing channel: require a deployed system, run the five technical questions, and use a paid trial project to validate before committing to a full contract. To hire generative ai developers with a verified production track record, Shreyans Padmani's generative AI development services provide the case studies and engagement options that make the verification straightforward.
Cost optimisation is the most important first verification because it is the skill whose absence most reliably produces a business-critical failure at scale. A Gen AI system with weak prompt engineering produces inconsistent outputs that can be identified and corrected during quality assurance. A Gen AI system with no cost optimisation designed in produces monthly inference bills that scale to financial unsustainability before the problem is detected. The cost estimation question described in Skill 8 takes five minutes to ask and immediately reveals whether the developer has thought about the token economics of the system they are being asked to build.
Hire Production-Ready Generative AI Developers?
Build secure, scalable, and cost-effective AI solutions with proven expertise in LLMs, RAG, AI agents, and custom Generative AI development.
Book a Free Consultation