AI prototypes are seductive because the first version works. Ninety percent of a hard problem appears solved in an afternoon. The remaining ten percent is where products live and die.
Build the evaluation set before the feature. A few hundred labelled real examples turn subjective argument into measurable regression. Without one, you cannot safely change a prompt, a model or a retrieval strategy ever again.
Then design failure. What does the product do when confidence is low, the context is missing, or the provider is down? A graceful refusal with a path to a human is a feature. A confident wrong answer is a liability.
Finally, model the cost per interaction before launch and put a ceiling on it. Unbounded inference spend has killed more AI features than poor accuracy.