I hope this post is allowed, I feel if benefit not only me but more people.
Which model you’d recommend for non-coding questions?
- If I have 48GB of unified memory (MacBook Pro M5) which is a good model I can ask questions about China, socialism and similar from a “comrade” framing?
- Which models you’d recommend in different bands?
- If I wanted to improve my setup (build a better rig to run better local models) what are my options?
- How can I tune the models I’m using?
In the last 6 months I started to use more AI tools because of (I was actually pressured by) work. I’m becoming better at using it as a tool in my day to day life and it has helped to take steps to delete my digital footprint, improve my privacy settings and limit my dependance on “search engines”.
I’m building a bridge to a raspberry Pi that has a hard drive plugged in, cutting my dependency to services of photos, files and music. This same bridge access this work computer (which I’ll leave plugged in permanently, 48GB of unified memory) and I made it so I can load models through the bridge and ask questions.
Some of the ones I have take too long, eating tokens and sometimes getting stuck in loops; they fill up quickly (run out of memory due to the context?).
Someone here must know: I’m a lurker and you, comrades, are on the edge. I’m also interested in looking for models that provide unfiltered info. You can dm and we can chat. I’m not a fed, you can go check the educational Marxist content I’ve been doing.
Edit: I must add that I’ve been using Claude Code A LOT. I’m aware of the implications and I’d like to take steps to reduce that usage.


I think most concerns around LLMs really boil down to imperialist ownership and control of said technology which often gets confused for AI as a technology generally. That’s why I like open source models better.
So yeah, I might actually be one of the few comrades on Lemmy who support LLMs as a technology or at least doesn’t actively oppose them on the condition that we control how our data is used. I’m not much of an expert otherwise, let alone on self-hosted LLMs.
in the context of local models i’m still concerned about hallucinations and brainrot
I don’t think “brainrot” has ever really been empirically proven. People mostly just cherry pick information from one specific MIT study to the point that the authors had to add a forward saying AI “psychosis” isn’t what is being asserted or proven in the paper. I’m old enough to remember when there was a similar moral panic over search engines and Wikipedia “making people dumber”. What is “dumber” and how do you scientifically quantify that? There never seems to be a real answer because you’d have to define intelligence first. The answer seems to just be people cheating on their homework which like?? 🤷♂️
As for the hallucination thing, yeah you shouldn’t ever use one single source for everything, that principle predates LLMs. I learned that in grade 9.
There seems to be significant evidence that students using AI significantly degrades their scores when they can’t use AI. I wouldn’t boil your opposition down to one study. A recent study of 30,000 children in China paints a pretty bad picture:
Student use of AI per week directly correlates with reduced exam scores, too:
Yes, most of the results here are correlative (there’s a mix of methods), but significantly strongly so, to a point where it’s hard to think an explanation other than “using AI degrades your mental capabilities, directly or indirectly” could possibly account for it. (e.g. It can’t be that only already-low-scoring students started using AI, as we know their exam scores were roughly average before using AI.)
The paper also found that after children had stopped using AI, most students took over a year to return to their previously expected scores, some students never (in the timeframe of the study) recovered to their previously expected levels.
Outsource your thinking at your own risk. Full paper: https://papers.ssrn.com/sol3/papers.cfm?abstract_id=6868618
When their exam scores are lower, they’re worse at the exam. Do exam scores measure “brainrot”?
A consistent decline in your ability to complete any of your exams definitely feels like a fair definition of brainrot. In fact test scores is almost the only practical method we have of reasonably objectively measuring brainrot.
If the brainrot hasn’t been “emprically proven”, I’ve felt it from experience. Writing, coding, and other matters expressed in text often only improve with practice. At work, when I go a long stretch of time working on my projects by just conversing with a chatbot because I have something I can’t be assed to figure out that day, I find my skills degrade quite naturally for the next couple days after. It’s a normal phenomena outside of AI usage; contact books made people not have to remember phone numbers, GPS lets people not know how to navigate a place independently, etc. It’s honestly really reasonable that substituting one’s own judgement or skills for an LLM would naturally lead to a degradation of one’s own judgement or skills.
You always learn. You just learned something different.
Example: when you write C, you don’t learn assembly, but you learn C.