You must log in or register to comment.
This is getting ridiculous. Can someone please ban AI? Or at least regulate it somehow?
The problem is, how? I can set it up on my own computer using open source models and some of my own code. It’s really rough to regulate that.
Jokes on them. I’m going to use AI to estimate the value of content, and now I’ll get the kind of content I want, though fake, that they will have to generate.
Should have called it “Black ICE”.
Definitely falls under the category of a Trap ICE card.
People complain about AI possibly being unreliable, then actively root for things that are designed to make them unreliable.
Here’s the key distinction:
This only makes AI models unreliable if they ignore “don’t scrape my site” requests. If they respect the requests of the sites they’re profiting from using the data from, then there’s no issue.
People want AI models to not be unreliable, but they also want them to operate with integrity in the first place, and not profit from people’s work who explicitly opt-out their work from training.
I’m a person.
I dont want AI, period.
We cant even handle humans going psycho. Last thing I want is an AI losing its shit due from being overworked producing goblin tentacle porn and going full skynet judgement day.
Got enough on my plate dealing with a semi-sentient olestra stain trying to recreate the third reich, as is.
We cant even handle humans going psycho. Last thing I want is an AI losing its shit due from being overworked producing goblin tentacle porn and going full skynet judgement day.
That is simply not how “AI” models today are structured, and that is entirely a fabrication based on science fiction related media.
The series of matrix multiplication problems that an LLM is, and runs the tokens from a query through does not have the capability to be overworked, to know if it’s been used before (outside of its context window, which itself is just previous stored tokens added to the math problem), to change itself, or to arbitrarily access any system resources.
You must be fun at parties.
- Say something blatantly uninformed on an online forum
- Get corrected on it
- Make reference to how someone is perceived at parties, an entirely different atmosphere from an online forum, and think you made a point
Good job.
- See someone make a comment about a AI going rogue after being forced to produce too much goblin tentacle porn
- Get way to serious over the factual capabilities of a goblin tentacle porn generating AI.
- Act holier than thou over it while being completely oblivious to comedic hyperbole.
Good job.
Whats next? Call me a fool for thinking Olestra stains are capable of sentience and thats not how Olestra works?
This will only make models of bad actors who don’t follow the rules worse quality. You want to sell a good quality AI model trained on real content instead of other misleading AI output? Just follow the rules ;)
Doesn’t sound too bad to me.
i mean this is just designed to thwart ai bots that refuse to follow robots.txt rules of people who specifically blocked them.
I find this amusing, had a conversation with an older relative who asked about AI because I am “the computer guy” he knows. Explained basically how I understand LLMs to operate, that they are pattern matching to guess what the next token should be based on a statistical probability. Explained that they sometimes hallucinate, or go of on wild tangents due to this and that they can be really good at aping and regurgitating things but there is no understanding simply respinning fragments to try to generate a response that pleases the asker.
He observed, “oh we are creating computer religions, just without the practical aspects of having to operate in the mundane world that have to exist before a real religion can get started. That’s good, religions that have become untethered from day to day practical life have never caused problems for anyone.”
Which I found scarily insightful.
Oh good.
now I can add digital jihad by hallucinating AI to the list of my existential terrors.
Thank your relative for me.
Not if we go butlerian jihad on them first
lol, I was gonna say a reverse butlerian jihad but i didnt think many people would get the reference :p
Maybe it will learn discretion and what sarcasm are instead of being a front loaded google search of 90% ads and 10% forums. It has no way of knowing if what it’s copy pasting is full of shit.
I have no idea why the makers of LLM crawlers think it’s a good idea to ignore bot rules. The rules are there for a reason and the reasons are often more complex than “well, we just don’t want you to do that”. They’re usually more like “why would you even do that?”
Ultimately you have to trust what the site owners say. The reason why, say, your favourite search engine returns the relevant Wikipedia pages and not bazillion random old page revisions from ages ago is that Wikipedia said “please crawl the most recent versions using canonical page names, and do not follow the links to the technical pages (including history)”. Again: Why would anyone index those?
They want everything, does it exist, but it’s not in their dataset? Then they want it.
They want their ai to answer any question you could possibly ask it. Filtering out what is and isn’t useful doesn’t achieve that
Because it takes work to obey the rules, and you get less data for it. The theoretical comoetutor could get more ignoring those and get some vague advantage for it.
I’d not be surprised if the crawlers they used were bare-basic utilities set up to just grab everything without worrying about rule and the like.
Because you are coming from the perspective of a reasonable person
These people are billionaires who expect to get everything for free. Rules are for the plebs, just take it already
I’m glad we’re burning the forests even faster in the name of identity politics.
Do you identify as retarded?
What has this anything to do with identity politics?
Don’t feed the troll
I’m imagining a sci-fi spin on this where AI generators are used to keep AI crawlers in a loop, and they accidentally end up creating some unique AI culture or relationship in the process.
I guess this is what the first iteration of the Blackwall looks like.
Gotta say “AI Labyrinth” sounds almost as cool.
“I used the AI to destroy the AI”
And consumed the power output of a medium country to do it.
Yeah, great job! 👍
We truly are getting dumber as a species. We’re facing climate change but running some of the most power hungry processers in the world to spit out cooking recipes and homework answers for millions of people. All to better collect their data to sell products to them that will distract them from the climate disaster our corporations have caused. It’s really fun to watch if it wasn’t so sad.
We had to kill the internet, to save the internet.
We have to kill the Internet, to save humanity.
Surprised at the level of negativity here. Having had my sites repeatedly DDOSed offline by Claudebot and others scraping the same damned thing over and over again, thousands of times a second, I welcome any measures to help.
thousands of times a second
Modify your Nginx (or whatever web server you use) config to rate limit requests to dynamic pages, and cache them. For Nginx, you’d use either fastcgi_cache or proxy_cache depending on how the site is configured. Even if the pages change a lot, a cache with a short TTL (say 1 minute) can still help reduce load quite a bit while not letting them get too outdated.
Static content (and cached content) shouldn’t cause issues even if requested thousands of times per second. Following best practices like pre-compressing content using gzip, Brotli, and zstd helps a lot, too :)
Of course, this advice is just for “unintentional” DDoS attacks, not intentionally malicious ones. Those are often much larger and need different protection - often some protection on the network or load balancer before it even hits the server.
Already done, along with a bunch of other stuff including cloudflare WAF and rate limiting rules.
I am still annoyed that it took me over a day’ of my life to finally (so far) restrict these things. And several other days to offload the problem to Cloudflare pages for sites that I previous self hosted but my rural link couldn’t support.
this advice is just for “unintentional” DDoS attacks, not intentionally malicious ones.
And I don’t think these high volume AI scrapes are unintentional DDOS attacks. I consider them entirely intentional. Not deliberrately malicious, but negligent to the point of criminality. (Especially in requesting the same pages again so frequently, and all of them ignoring robots.txt)
I think the negativity is around the unfortunate fact that solutions like this shouldn’t be necessary.
Considering how many false positives Cloudflare serves i see nothing but misery coming from this.
Lol I work in healthcare and Cloudflare regularly blocks incoming electronic orders because the clinical notes “resemble” SQL injection. Nurses type all sorts of random stuff in their notes so there’s no managing that. Drives me insane!
In terms of Lemmy instances, if your instance is behind cloudflare and you turn on AI protection, federation breaks. So their tools are not very helpful for fighting the AI scraping.
Can’t you configure exceptions for behaviours?
I’m not sure what can be done at the free tier. There is a switch to turn on AI not blocking, and it breaks federation.
You can’t whitelist domains because federation could come from and domain. Maybe you could somehow whitelist /inbox for the ActivityPub communication, but I’m not sure how to do that in Cloudflare.
Burning 29 acres of rainforest a day to do nothing
It certainly sounds like they generate the fake content once and serve it from cache every time: “Rather than creating this content on-demand (which could impact performance), we implemented a pre-generation pipeline that sanitizes the content to prevent any XSS vulnerabilities, and stores it in R2 for faster retrieval.”
Bitcoin?
You have Thirteen hours in which to solve this labyrinth before your baby AI becomes one of us, forever.
And soon, the already AI-flooded net will be filled with so much nonsense that it becomes impossible for anyone to get some real work done. Sigh.
Some of us are only here to crank hog.
AROOO!
I introduce to you, the Trace Buster Buster!
If you’ve never seen the movie The Big Hit, it’s great.
