Choosing the Right LLM for Each Task

Choosing the Right LLM for Each Task

Picking one model for everything is simple, and it is also the quickest way to overspend or underdeliver. A better habit is to match each request to the smallest model that does the job well, and reserve the big models for the work that needs them.

Start with the shape of the task

• Formatting, classification, extraction: short, well specified jobs. A fast, low cost model handles these reliably.
• Chat and drafting: a balanced model gives good tone and speed.
• Coding: small diffs suit mid size models. Architecture questions and hard debugging reward stronger reasoning.
• Planning, math, hard decisions: choose a reasoning model and give it room to think.
• Images and screenshots: choose a model with vision input.

Read the catalogue numbers

Every model in our catalogue lists its context window and its price per million input and output tokens. For example, gpt-4o-mini is listed at $0.15 input and $0.60 output with a 128K window, while gpt-5.4 is listed at $2.50 and $15.00 with a 1M window. Output tokens usually cost more than input tokens, so tasks that return short answers over long inputs are the cheapest to run.

Let the auto profiles route for you

Text Generator ships semantic auto models. Set the model name once and each request is routed by what it asks for:

• text-generator/auto balances quality, speed, and cost.
• text-generator/auto-fast targets interactive assistants and quick agent loops.
• text-generator/auto-cheap targets formatting, classification, and simple edits.
• text-generator/auto-code routes coding prompts by complexity.
• text-generator/auto-reasoning picks the model and thinking level for hard problems.
• text-generator/auto-vision handles image understanding and OCR.

from openai import OpenAI
            
            client = OpenAI(base_url="https://api.text-generator.io/v1", api_key=API_KEY)
            r = client.chat.completions.create(
                model="text-generator/auto-cheap",
                messages=[{"role": "user", "content": "Label this ticket as billing, bug, or feature: 'My invoice shows two charges'"}],
            )
            print(r.choices[0].message.content)

A simple evaluation loop

1. Collect 20 real prompts from your product.
2. Run them through the cheap profile and the balanced profile.
3. Compare answers side by side and note where the cheaper model already meets your bar.
4. Move that task type to the cheaper profile and keep the stronger one for the rest.

Repeat when new models arrive. Because the API is OpenAI compatible, switching is a one line change to the model name.

Keep fallbacks in mind

Catalogue entries list fallback models, so a request can continue on a similar model if a provider has trouble. Set a generous max_tokens for long outputs and log which model served each request, so your evaluation data stays current.

Browse the full list at /models and try a few profiles on your own prompts today.

Get started free