What are ChatGPT tokens, and how do they work?

Large language models like the one powering ChatGPT can generate thousands of words within a single minute. Likewise, they can quickly make sense of long inputs. This may seem like magic, but unlike humans, the chatbot doesn’t process text as individual sentences or even words. Instead, ChatGPT uses tokens to decode and output human languages like English, Spanish, and others. So in this article, let’s answer the basics, namely how ChatGPT tokens work, why they are necessary, and how they affect your chatting experience.

What are ChatGPT tokens? Context window explained

Tokens represent the basic building blocks of any ChatGPT text response. While we tend to group text based on the number of words, the GPT language model doesn’t work the same way. Instead, it looks for predictable combinations of letters and groups them together to form a token.

ChatGPT tokens may seem like an abstract concept, so let’s understand how it works with an example. The word “air” is a word used frequently in everyday language. The model has likely come across it several times while scanning its training data. Both of these factors mean that “air” forms a single token.

However, if you take a longer and comparatively less common word like “airline”, you’ll find that the language model treats “air” and “line” as two independent tokens.

Tokens don’t matter to most of the chatbot’s users, but they do play a role in determining ChatGPT’s character limit. Not just that, the language model also has a limitation on the number of tokens it can hold in memory. So if you tell ChatGPT a fact early on in a conversation, it will eventually forget it after processing a few thousand tokens. This is known as the language model’s “context window”.

How to count tokens in ChatGPT?

Common words in the English dictionary are more likely to correspond to a single token. However, complex words don’t get the same treatment and may actually consist of several tokens at once. Here’s a quick table that illustrates how you can count the number of tokens for a given piece of text:

While the above table serves as a rough guideline, it’s not always possible to predict how many tokens a particular word or phrase will contain. Take numbers, for example. Common groups like “123” and “333” will only correspond to one token. However, longer sequences of numbers will split into multiple tokens.

As a rule of thumb, ChatGPT creator OpenAI says that 100 tokens will approximately equal 75 words. However, that correlation between word length and token count only holds true for English words.

Words in foreign languages, especially less common ones, will take up more tokens. For example, the 17 characters in the Vietnamese text “Bãi đậu xe ở đâu?” corresponds to an astounding 13 tokens.

If you’d like to check how many tokens are in a particular piece of text, check out OpenAI’s free Tokenizer tool. It highlights different tokens in a given piece of text.

What is the token limit in ChatGPT?

The token limit in ChatGPT depends on the model you use and whether you use the chatbot or interact with the model programmatically. The latter only applies if you’re a programmer and implementing the GPT model within your own app. If you’re this kind of API user, expect a token limit ranging between 4,096 to 128,000. Newer models like GPT-4 Turbo have higher limits, but will also cost more per query to use.

If you’re not a programmer, though, you’ll simply interact with chatbot through the website or app. In this case, many users have reported that ChatGPT’s token limit is fixed at 4,096. However, OpenAI hasn’t publicly stated the actual limit, so this can change without notice.

The token limit notably doesn’t change even if you pay for ChatGPT Plus, which grants you access to the latest GPT-4 language model.

How much is a ChatGPT token worth?

As with the limit, the cost per ChatGPT token depends on the model of your choice. As you’d expect, the latest models cost significantly more than the last-gen GPT-3.5 that was released in late 2022. Likewise, OpenAI often releases minor language model updates that reduce computational costs and, in turn, the price of each token generated.

With that disclaimer out of the way, here’s how much you will have to pay for ChatGPT tokens on a model-by-model basis:

1. GPT-3.5 Turbo: As the oldest model still in use for the free version of ChatGPT, it’s one of the cheaper options. Developers pay $0.0010 per 1,000 input tokens and $0.0020 per 1,000 output tokens.
2. GPT-4: Released in early 2023 shortly after ChatGPT, the GPT-4 language model offers higher-quality responses. It’s also available in two varieties, one with an 8,096 token limit and another with 32,000. The former costs $0.03 per 1,000 input tokens and $0.06 per 1,000 output tokens.
3. GPT-4 Turbo: Even though it’s the latest language model, GPT-4 Turbo aims to be significantly more computationally efficient than GPT-4. This is why it costs $0.01 per 1,000 input tokens and $0.03 per 1,000 output tokens. It’s still an order of magnitude higher than GPT-3.5 Turbo but the improved logical abilities may make it worthwhile for some.

How to buy ChatGPT tokens? OpenAI Playground explained

If you’re a ChatGPT user, you don’t have to deal with tokens. However, developers and those looking to experiment can also interact with the underlying language model directly.

The OpenAI Playground offers a similar interface to ChatGPT, but you cannot use it for free. Each and every message you send and receive will be billed to your payment method, as per the costs outlined in the previous section. It can get expensive, but it’s currently the only way to interact with the full-fledged language model without arbitrary token limits.

FAQs