Generative AI and Large Language Models Questions
The capabilities and behavior of modern generative and large language models. Covers how LLMs are pretrained, in-context learning and few-shot prompting, generative model families (autoregressive, diffusion), context windows, and tokenization and sampling. Emphasizes understanding what generative models can and cannot do and how they differ from discriminative ML.
What is the difference between prompting an LLM and fine-tuning it to change its behavior? In what scenarios is prompting a sufficient first approach, and when does it become insufficient, requiring fine-tuning instead?
Sample Answer
Direct answer
Prompting an LLM means steering its behavior at inference time through instructions and examples in the input, with no change to the model's weights; fine-tuning changes the weights themselves through additional training. Prompting is the right first approach whenever the base model's existing knowledge and capabilities are already sufficient and you just need to steer HOW it uses them; fine-tuning becomes necessary when the required behavior can't reliably be specified through instructions alone, either because it needs knowledge the model doesn't have, or because it needs a level of consistency prompting can't guarantee.
Structured elaboration
What prompting can and can't do. A prompt can specify style, format, persona, and task framing, and can supply new information directly in context (few-shot examples, retrieved documents). What it cannot do is permanently change what the model "knows" or reliably guarantee a specific behavior across every possible input, since prompt-following is itself a learned, imperfect capability; a sufficiently unusual or adversarial input can still cause the model to ignore or misapply prompt instructions.
When prompting suffices. Style and format control ("respond in JSON," "use a formal tone"), task framing for capabilities the base model already broadly has (summarization, translation, general Q&A), and situations where you can supply the needed facts directly in the prompt (via retrieval) rather than needing them baked into the weights.
When it becomes insufficient. Three concrete triggers: (1) the task requires domain knowledge that's absent or too sparse in the model's pre-training data and too voluminous to supply per-request via retrieval or examples; (2) the required consistency is higher than prompting reliably delivers, e.g., a classification task that must NEVER misfire on adversarial inputs; (3) the per-request cost of achieving the behavior via prompting (long instructions, many few-shot examples) becomes larger than the one-time cost of fine-tuning would be, once volume is high enough.
Trade-offs & pitfalls
The common mistake is reaching for fine-tuning as a default "make the model better at X" move before actually testing whether a well-designed prompt (possibly with retrieval or few-shot examples) already solves the problem; fine-tuning has real fixed costs (data collection, training compute, evaluation, and a slower iteration loop since every behavior change requires retraining) that prompting doesn't. The inverse mistake also happens: sticking with an increasingly baroque, ever-longer prompt to force consistent behavior on a task that has clearly outgrown what prompting can reliably deliver, when a modest fine-tuning investment would produce a shorter, cheaper, more reliable system. The practical discipline is to start with prompting, measure where and how often it fails, and let that concrete failure pattern justify the fine-tuning investment rather than assuming it up front.
You must decide between two third-party LLM options for a knowledge assistant: a faster, cheaper model with slightly lower factual accuracy, versus a slower, costlier model with better factuality. How would you evaluate and choose, and how might you combine both to meet product goals?
Sample Answer
Direct answer
You must decide between two third-party LLMs for a knowledge assistant: a faster, cheaper model with slightly lower factual accuracy versus a slower, costlier model with better factuality, and possibly using both together. The right approach is to define a small set of measurable evaluation criteria tied to the actual product requirement, benchmark both models against real (or realistic) queries on those criteria, and then decide whether a single model suffices or whether a routing/fallback strategy combining both is worth the added complexity.
Structured elaboration
How to evaluate. Build a held-out evaluation set of realistic queries with known-correct answers (or human-graded rubrics for open-ended ones), and measure each candidate model on: factual accuracy (does it get the answer right, and does it hallucinate on out-of-scope questions), latency (p50/p95 response time under realistic load), cost per query at your expected volume, and any user-experience metrics that matter for the product (helpfulness ratings, task completion rate in A/B tests). Benchmarks alone are not sufficient; a model can score well on a public benchmark and still perform differently on your specific domain and query distribution, so the evaluation set must reflect your actual traffic.
Vendor/integration risk. Beyond raw model quality, evaluate SLA guarantees, rate limits, data-handling and privacy terms, and how exposed you are if the vendor changes pricing, deprecates the model, or has an outage, since a third-party dependency carries operational risk that a benchmark score alone won't capture.
Combining both models. A common production pattern is routing: use the fast, cheap model for the majority of queries (the ones it handles well), and escalate to the slower, more accurate model only for queries flagged as higher-risk or lower-confidence, e.g., via the fast model's own confidence signal, query complexity heuristics, or a lightweight classifier trained on where the fast model tends to fail. This captures most of the cost savings of the cheap model while limiting factual-accuracy risk to the harder subset of queries that actually need it.
Worked example
Say the cheap model costs $0.20 per 1,000 queries and the accurate model costs $2.00 per 1,000 queries, roughly a 10x cost difference, and evaluation shows the cheap model is factually correct 92% of the time versus 98% for the expensive model on your held-out set. If a routing classifier can reliably identify the roughly 20% of queries where the cheap model is most likely to be wrong (say, questions requiring precise numeric facts or recent information) and escalate only those to the expensive model, the blended cost is 0.8×$0.20+0.2×$2.00=$0.16+$0.40=$0.56 per 1,000 queries, about a 72% cost reduction from always using the expensive model, while capturing most of its accuracy benefit specifically where it matters most.
Trade-offs & pitfalls
A routing strategy only pays off if the signal for "this query needs the accurate model" is genuinely predictive; a poorly calibrated router either escalates too much (eroding the cost savings) or too little (letting factuality-sensitive queries slip through to the cheap model). It also adds real engineering and operational complexity: two vendor integrations, two SLAs to monitor, and a routing component that itself needs to be evaluated and maintained over time. For an early-stage product without the traffic volume or engineering capacity to build and maintain a router well, a single well-chosen model, even if slightly suboptimal on cost or accuracy, is often the more pragmatic starting point, with routing revisited once volume and evaluation infrastructure justify the added complexity.
Define 'hallucination' in the context of large generative models. What concrete forms can it take, and why do these occur from a training and objective-function perspective? At a high level, what categories of mitigation exist?
Sample Answer
Direct answer
Hallucination is when a generative model produces output that is fluent and confident-sounding but not actually grounded in truth or in its source material; it happens fundamentally because the model's training objective rewards producing plausible-sounding continuations, not verified-true ones.
Structured elaboration
Concrete forms it can take. A model can state an outright factual error, confidently asserting something false as if it were true. It can produce a plausible-sounding fabrication, inventing a specific detail, citation, statistic, or event that sounds exactly like the kind of specific detail a correct answer would contain, but that simply does not exist. And it can make an incorrect attribution, correctly stating a real fact but attributing it to the wrong source, person, or context.
Why this happens from a training and objective-function perspective. The model is trained to predict the next token that is most probable given everything it has seen, both during pre-training (predicting the next token of real text) and typically further tuned to produce helpful, fluent, confident-sounding responses. Nothing in that objective directly rewards "say I don't know when uncertain" or "only state things you can verify." A fluent, specific, confident-sounding answer is exactly the kind of continuation the training objective rewards, whether or not it is actually true, because fluency and specificity are also what genuinely correct training-data answers look like. The model has no strong internal signal distinguishing "this specific detail is real" from "this specific detail merely has the surface form of a real one."
Categories of mitigation, at a high level. Grounding techniques anchor the model's output in externally verifiable evidence, for example retrieval-augmented generation or requiring the model to cite a source, rather than relying purely on what it recalls from training. Calibration techniques try to make the model's expressed confidence track its actual likelihood of being correct, so it becomes more willing to hedge or decline when genuinely uncertain instead of always answering in the same confident tone regardless of accuracy. Verification techniques check the model's output after the fact, either automatically by cross-referencing claims against a trusted source, or with a human in the loop, before the output reaches the end user.
Worked example
Consider asking an LLM "who won the Nobel Prize in Physics in a given recent year, and for what work." If the model's training data cut off before that year's announcement, a well-behaved model would say it does not know. A hallucinating model instead often produces a specific, plausible-sounding name and a specific, plausible-sounding citation for the discovery, both of which read exactly like a correct answer would, because the model is pattern-completing "Nobel Prize announcements have this shape" rather than retrieving a verified fact it does not have. Grounding this query in a live retrieval step (searching for the actual announcement and feeding the result into the prompt) directly fixes this specific failure mode, because the model is now completing a pattern anchored in real retrieved text rather than free-associating from a gap in its training data.
Trade-offs & pitfalls
No single mitigation category eliminates hallucination outright. Grounding only helps to the extent the retrieved evidence is itself relevant and correctly used, since the model can still ignore or misread grounded evidence. Calibration is imperfect, and models can still be confidently wrong. Verification adds latency and cost and does not scale to checking every single output in a high-volume product. In practice, teams typically combine categories, for example retrieval-grounded generation plus a lighter-weight automated verification pass, rather than expecting any one technique alone to solve the problem, and treat hallucination rate as a metric to continuously monitor and reduce rather than a bug to be fixed once.
That is every published Generative AI and Large Language Models question for Product Manager so far. Browse the other topics in this category, or practice this one interactively.