Also not the person you responded to but in my experience, yes. I don’t tell it to “save it in its memory files” because I don’t think that’s a thing, but including “do not actually edit any files, only propose changes” somewhere in the initial prompt of a conversation has worked well for me.
That’s cool but some guy who read a lot of Harry Potter fan fiction about it says we’re all gonna die. /s in case it’s needed lol.
The hardware requirements are substantial. If companies or individuals want to run models locally, especially at the speed you’d expect from the big name models, then everyone needs a pretty serious GPU on their work station. That or the company needs to build and maintain a server capable of running whatever loads they expect to see.
None of this is impossible of course. But you pay for the tokens one way or another. Either you pay the upfront cost of the hardware and continuing cost of the electricity to run it, or you pay with the inconvenience of running a model very slowly on lower end hardware, or you pay a subscription for someone else to worry about all that.
This is why local models are still niche. Only hobbyists nerds are willing to go through the trouble of setting up and running a local model. Most companies still are not sure if AI is actually worth it for them. So it’s much easier to pay for tokens than the giant upfront cost required to build you own infrastructure.
If the numbers the AI companies are putting up as far as infrastructure cost vs revenue are anything to go by, it probably isn’t worth it to most companies.
I am in software and I don’t think it’s that clear cut tbh. There’s definitely a right and wrong way to use AI. I think a lot of companies are moving fast and breaking things right now, but a lot of them are going to find out that the thing they broke is the product and they won’t be able to slop their way out of a problem they slopped their way into.
At a micro scale, AI is pretty good at writing workable code quickly. This obviously boosts productivity at the individual level, especially in the short term. At a macro scale, good maintainable software requires higher order thinking that transformer based models simply cannot do. So a lot of the stochastic micro decisions an AI makes lead to technical debt that is not trivial to remove, and can’t really be resolved by the AI itself, at least not in a way that won’t contribute to the problem.
These issues can be curbed by the programer at the point of origin, before it gets out of hand, but obviously that requires more judicious use which cuts into the productivity gains. Because writing code has never been the hard part of software.
TL;DR: fully embracing the slop machine will eventually poison your product in a way that is not really easily recoverable. It is useful, but not “too good to not use.”