I already know it’s not that easy and that the Internet is constantly expanding, but I really wonder if it’s possible to very roughly estimate its size
My friend works at AWS and has single customers with hundreds to thousands of petabytes.
In whole sum, I think it would be pretty unfathomable.
Define “Internet” to answer your question more meaningfully.
Are you counting cached copies of data (Cloudflare etc), or only the original copy? Only the source data or the data copied on the receiving end (webbrowser etc) too? Data at rest on a computer hard drive or data being transmitted or both? Are you only including world wide web (websites) or any data (gaming, streaming, fileshare). Humans or bots or both? Public web or dark web? Any computer physically connected to a network or only those data accessible publicly?
Would it be easier to be simply defined as any data on any storage device that is connected to the internet and accessible that way? This would include things such as those CloudFlare cached copies and anything beyond simply WWW and make no distinction between humans and bots or dark or public. I guess that would make it pretty hard to make any even remotely educated guess though since so much it would not be accessible to anyon trying to count it.
42 of course.
Found the frood dude☝️
Goosnargh
IDC forecasts that the amount of data generated each year will grow to 175 zettabytes by 2025. It further estimates that a total of 22 zettabytes of digital storage will be shipped across all storage media types between 2018 and 2025, with nearly 59 percent of this capacity being provided by the hard drive industry.
Interesting, thanks for your response!
What’s carrying the remaining 41% if not hard drives??
Tape drives are one thing that comes to mind.
Lto tapes and ssd
TIL about automated tape libraries. Very cool, very slow, very cheap. 2 cents/gig!
Idk but I’m about to eclipse 200TB myself! 😅
- 6x 3TB in RAID10 = 18TB raw (9TB usable)
- 10x 8TB in RAID5E = 80TB raw (64TB usable)
- 8x 16TB in RAID5 = 128TB raw (112TB usable)
AKA 226TB raw or 185TB usable
Yes, you can estimate anything.
Follow up question: what percentage of that do you think is porn?
Not enough
Yes
This was asked less then a week ago
https://lemmy.world/post/53033969but depending on where you draw the line on what is the internet then we are likely in the Exobyte territory.
Oh I didn’t see, thanks for letting me know!
this feels like a perfect llm question. It may have it wrong. It quite likely will be wrong or at least not exact. But it will give you a feel good answer that is going to be correct with enough proximity tolerance.
Something I always find kind of puts the scale into perspective, is that all these AI companies have a huge interest in storing as much of the internet’s data as possible, because they need access for training and it ensures they continue to have access to relatively high-quality (it’s only going to get worse with more and more content on the internet being generated).
And yet, even with their trillions of investor money, they evidently don’t have it all stored. Because they still regularly hammer all kinds of webpages with an insane number of requests, when they do their trainings.
They’re checking the websites for change. They’re apparently too stupid to tell websites that change often from ones that are more static.
They’re not using it live for training. That’d be far to slow for their training.
That’s a good point! I forgot about AI companies stealing as much as data as they can…
lmgtfy? estimates vary a lot of course but currently tend to range from tens to hundreds of zettabytes.
A Zettabyte is 1021 bytes. That is trillions of gigabytes, or to put it another way: tens of terabytes per living person.
That’s really impressive!
