I already know it’s not that easy and that the Internet is constantly expanding, but I really wonder if it’s possible to very roughly estimate its size
I already know it’s not that easy and that the Internet is constantly expanding, but I really wonder if it’s possible to very roughly estimate its size
Something I always find kind of puts the scale into perspective, is that all these AI companies have a huge interest in storing as much of the internet’s data as possible, because they need access for training and it ensures they continue to have access to relatively high-quality (it’s only going to get worse with more and more content on the internet being generated).
And yet, even with their trillions of investor money, they evidently don’t have it all stored. Because they still regularly hammer all kinds of webpages with an insane number of requests, when they do their trainings.
They’re checking the websites for change. They’re apparently too stupid to tell websites that change often from ones that are more static.
They’re not using it live for training. That’d be far to slow for their training.
That’s a good point! I forgot about AI companies stealing as much as data as they can…