• NEWS FEATURE

The AI revolution is running out of data. What can researchers do?

AI developers are rapidly picking the Internet clean to train large language models such as those behind ChatGPT. Here’s how they are trying to get around the problem.
By
  1. Nicola Jones

    1. Nicola Jones is a freelance writer in Pemberton, British Columbia.

    View author publications

    You can also search for this author in PubMed  Google Scholar

Illustration of a parrot composed of colourful dots to represent artificial intelligence, eating a small cluster of fruit from an otherwise barren tree.

Illustration: Adrià Voltà

Access through your institution
Buy or subscribe

The Internet is a vast ocean of human knowledge, but it isn’t infinite. And artificial intelligence (AI) researchers have nearly sucked it dry.

Access options

Access through your institution

Rent or buy this article

Prices vary by article type

from$1.95

to$39.95

Learn more

Prices may be subject to local taxes which are calculated during checkout

Nature 636, 290-292 (2024)

doi: https://doi.org/10.1038/d41586-024-03990-2

References

  1. Villalobos, P. et al. Proc. Mach. Learn. Res. 235, 49523–49544 (2024).

    Google Scholar 

  2. Longpre, S. et al. Preprint at arXiv https://doi.org/10.48550/arXiv.2407.14933 (2024).

  3. Trinh, T. H., Wu, Y., Le, Q. V., He, H. & Luong, T. Nature 625, 476–482 (2024).

    Article  PubMed  Google Scholar 

  4. Alemohammad, S. et al. Preprint at arXiv https://doi.org/10.48550/arXiv.2307.01850 (2023).

  5. Ho, A. et al. Preprint at arXiv https://doi.org/10.48550/arXiv.2403.05812 (2024).

  6. Muennighoff, N. et al. Preprint at arXiv https://doi.org/10.48550/arXiv.2305.16264 (2023).

Download references

Subjects

Latest on:

Nature Careers

Jobs

Nature Briefing

Sign up for the Nature Briefing newsletter — what matters in science, free to your inbox daily.

Get the most important science stories of the day, free in your inbox. Sign up for Nature Briefing