Leaderboards tell you which models are strong in general. They cannot tell you which one is best at your job: answering your customers, in your language, with your documents. For that you need a small test of your own. It takes an afternoon, and it saves you from choosing on reputation alone.
1. Collect 30 real examples
Take real inputs from your product or your work: customer emails, documents, questions people actually asked. Thirty is enough to see clear differences; with fewer, luck plays too big a part. Include a few hard cases and a few odd ones, because that is where models differ most.
2. Write down what a good answer looks like
Before running anything, decide what you are checking. Is the answer correct? Is it in the right language and tone? Does it follow the format? Does it invent anything? A checklist of three or four points keeps you honest when two answers both look fine at first sight.
3. Pick three candidates
Choose one strong model, one mid-priced and one cheap, using the rankings or the model finder. Three is enough; with more, the comparison gets tiring and less careful.
4. Run the same prompt on all of them
Use exactly the same instructions and settings for each model, and save every answer with the model’s name hidden. Most providers have a playground where you can do this by hand, and a short script works too.
5. Score blind
Read the answers without knowing which model wrote them, and score each one against your checklist. Blind scoring matters more than it sounds: people tend to rate the famous model higher. If a colleague can score as well, even better.
6. Put quality next to cost
Add up the scores, then set them beside the monthly cost of each model at your volume; the cost calculator does the sums. Often the cheaper model is close enough, or even wins on your task. Sometimes the gap is real and worth paying for. Either way, you now know instead of guessing.
7. Keep the test
Save the examples, the checklist and the scores. When a new model or a price cut comes out, you can run the test again in an hour and decide with the same yardstick.