"Models all the way down"

If you want to make a really big AI model — the kind that can generate images or do your homework, or build this website, or fake a moon landing — you start by finding a really big training set. Images and words, harvested by the billions from the internet, material to build the world that your AI model will reflect back to you...

...In December, researchers from Stanford's Internet Observatory identified more than 1,000 images categorized as Child Sexual Abuse Material (CSAM) in one of the most influential AI training sets of the moment: LAION-5B...

...LAION-5B is a really big, open-source dataset of images and text captions scraped from the internet, designed for large AI models...

...The stated goal of the project to create LAION-5B was to conduct basic research into dataset curation. Specifically, its authors wanted to create an image training set with purely automated methods - with no humans in the mix. The resulting "hands-off" dataset has been used in hundreds of academic projects. The paper announcing LAION-5B has been cited 1,331 times...

...Midjourney and Stable Diffusion, two large models for which some of the data sources are known, are both trained in part on LAION-5B. It’s likely that many other commercial models - perhaps hundreds - have been trained on the set. Models that power chat bots and image generators and have hundreds of thousands of users...

...LAION-5B has, since the CSAM findings in December, been unavailable for download. The developers say they are working on remediating it.