In AI inference, token types typically distinguish between input tokens (the prompt or context fed to a model) and output tokens (the generated response), which are often metered or priced differently.
Output tokens are usually more compute-intensive to generate than input tokens are to process, since generation happens sequentially, one token at a time, while input can often be processed in parallel.
Understanding the input/output distinction matters for accurately pricing or forecasting inference workloads, since a job with a short prompt but a long generated response has a very different cost profile than the reverse.
We use cookies for analytics and advertising. Privacy policy