subtopic

Compare with the base model and the big model

Run Project 1's golden test set (120 emails) through three models with the same prompt: the small base model, your fine-tuned small model, and the big API model. Compare valid JSON %, category accuracy, field accuracy, seconds per email and cost per 1,000 emails. Look at 20 errors of the fine-tuned model.

Builds on: S11 · Build: Model Comparison, Error Analysis & Interpretation