Technology and innovation

Language models can now read thousand-page documents

New language models keep a whole contract or textbook in memory and find the paragraph you need. Long texts are still slower and more expensive, though.

For you: You can find the right spot in a long document in a minute, but check the answer against the page number yourself.

Language models can now read thousand-page documents

Two years ago a chatbot would “forget” the start of a long text by the middle. Give it a long contract, ask about a clause on page one, and it would get confused or make up an answer. So you had to cut documents into pieces and feed them in one by one.

Now large models take in hundreds of thousands of tokens at once, which is the size of a thick book. Here is what that means, who it helps, and what to keep in mind when you work with long documents.

What a token is

A model reads text not in words but in tokens. A token is a small piece of text: a short word, part of a word or a punctuation mark. One word can take up one token or several.

The amount of text a model can “see” at once is called its context window. Think of it as the model's working memory. If the window is small, the start of a long text falls out of it, and the model “forgets” it. Once the window grew to hundreds of thousands of tokens, a whole book, a long contract or a textbook could fit into a single request.

What this gives you in practice

Being able to read a long document in one go helps many kinds of people:

  • Lawyers upload whole contracts and ask where the penalties hide. Before, finding those clauses meant reading the entire text by hand.
  • Students upload an entire textbook to find a definition with its page number. That saves time when writing a term paper or preparing for an exam.
  • Business owners can check a lease, a supplier's offer or a long manual.
  • Journalists can quickly find the key figures and statements in a long report or document.

The most useful trick is to ask for the page number. If the model points to the spot, you can find it in the original right away and check it.

The limit that remains

One limit remains: the longer the text, the slower and pricier the answer. With every question, the model processes the whole text again. Even a short question about a thousand-page document takes a lot of computing.

In paid services this affects the price directly, since you usually pay by the number of tokens. In free versions, your daily limit may run out sooner.

So short questions are still cheaper to ask about the relevant chapter. If your question concerns one section, upload just that section rather than the whole book. The answer will come faster and cost less.

The window grew, but the model did not become a perfect reader. It can mix up two similar clauses, misread a number in a table or miss a small caveat in a long text. So do not fully trust a “the document does not say that” answer either: if the question matters, check it yourself with a text search.

File quality matters too. A scanned document is often stored as an image rather than text, and some services read such files poorly or not at all. If you can, upload the original file whose text can be copied. Documents with numbered pages are also easier to work with: when the model points to a spot, you find it quickly.

How to ask the right way

With a long document, a precise question matters even more. Ask the model not only for the answer but also for where it is in the document. For example:

Read this document. [Your question, for example: find every clause about penalties.] For each answer, give the page or clause number and quote the exact text. If the document does not contain the answer, write “not in the document”.

The last sentence matters: it makes it less likely that the model will invent an answer.

What you can do now

Models that read long documents save time both in study and at work. To use them well:

  1. Ask for the page number and a quote. Check every important answer against the original.
  2. Upload only the part you need. If your question is about one chapter, do not hand over the whole book.
  3. Be careful with confidential documents. Before uploading a contract with personal data, trade secrets or client details, read the service's rules.
  4. Do not use the model instead of a lawyer. It will find the right clause quickly, but before signing something important, talk to a professional.
  5. Use it to study, not instead of studying. Once you have found a definition in a textbook, read that page yourself. The model shows you where to look; understanding is your job.

Models have learned to hold a long text in mind. The key skill now is asking them the right question and checking the answer.

Ad Your ad hereA banner on the homepage, in stories and sections. Formats and prices on our Advertise page.

Comments

No comments yet. Be the first.