I admit my read of the article was partly skimming, so maybe they covered this point, but from everything I’ve seen with LLMs, I’m skeptical their impact is going to change much, unless it’s to make things shittier by forcing them where they aren’t ready. AI as a whole could change a lot that is hard to predict because AI is kinda synonymous with automation and could be many developments of many different kinds of technologies. But the current crop of AI hype and what it’s capable of? Where I see it most taking over is the capitalistic “content churn” industry. For anything that needs to be thinking beyond “cash in and move onto the next one”, I don’t see how it gets integrated very effectively.
Part of what makes me doubt it is efficiency. Although there are some notable advances in efficiency, such as Deepseek’s cost reduction in training, generative AI is overall a resource-heavy technology. Both training and inference are costly (environmentally, in GPUs, etc., not just in price tag). Another point is competence. The more complicated a task is, the easier it is for the AI to make mistakes, some of which only an expert in the related subject matter would pick up on, which makes it a high competence task just to evaluate the AI’s results and make sure it isn’t doing more harm than good. Another is learning. You could look at the competence example and say, a human in training needs similar evaluation, but the human in training will usually learn from their mistakes, with correction, and not make them as often in the future. The AI won’t unless you retrain it and then it is still highly limited due to its statistical and tokenizing nature. Another element is trust. The western market has much more of a vested interest than, say, China, in selling the idea that AI as it is now will work and will integrate and therefore will make a profit; otherwise, its house of cards gold rush investments go to waste and the industry tanks (the fragility of that seen already in how easily Deepseek upset the equilibrium, or lack thereof).
I think programmers and programming as a field is in more danger (or danger of change, depending on how you want to look at it) from capitalists than from generative AI. The field already zipped past a phase where I can still remember reading about someone talking about a fizz buzz example as a test of basic programming competence, to the internet being stuffed to the brim with coding bootcamp stuff and “master algorithms and data structures” doctrine. And that change happened before generative AI. I don’t know what the hard numbers are, so I could be deceived on it, but by all appearances, programming became much more saturated via all the “learn to code” stuff, coupled with more companies cutting jobs in general, resulting in it being a field that is significantly harder to get into and harder to stay in. And again, all of that before generative AI.
I don’t mean this toward you, Yogthos, of course, but I think there is a certain amount of programmers being in denial about the field being touched by capitalism in general. This sort of unspoken belief that because programming is so important, the trend will just sort of continue that way and it will continue to be a lucrative and cozy ivory tower to hang out in. But that won’t stop capitalists from trying to reduce payroll as much as possible, whether it truly makes rational sense or not.
It seems like AI is a very polarizing topic, and people tend to either think it’ll do everything or reject it as pure hype. Typically, the reality of the usefulness of new tech tends to lie somewhere in between. I don’t expect that programmers will disappear as a profession in the foreseeable future. My view is that LLMs are becoming a genuinely useful tool, and they will be increasingly able to take care of writing boilerplate freeing up developers to do more interesting things.
For example, just the other day I had to create a SQL schema for an API endpoint, and I was able to throw sample JSON into DeepSeek R1 to get a reasonable schema out of it that needed practically no modifications. It probably would’ve taken me a couple of hours of work to design and write it. I also find you can generally figure out how to do something quicker with these tools than by searching sites like stack overflow or random blogs. Even if it doesn’t give a correct solution, it can point you in the right direction. Another use I can see is having it search through code bases finding where specific functionality is. This would be very helpful with finding your way around large projects. So, my experience is that there are already a lot of legitimate time saving uses for this tech. And as you note it’s hard to say where we start getting into diminishing returns territory.
Efficiency of these things is still a valid concern, but I don’t think we’ve really tried optimizing things much yet. The fact that DeepSeek was able to get such a huge improvement makes me think that there are a lot of other low hanging fruit to be plucked in the near future. I also think it’s highly likely we’ll be combining LLMs with other types of AI such as symbolic logic. This is already being tried with neurosymbolic systems. Different types of machine learning algorithms could tackle different types of problems more efficiently. There are also interesting things happening on the hardware side with stuff like analog chips showing up. Making the chip analog is way more efficient for this stuff since we’re currently emulating analog systems on top digital ones.
I very much agree regarding the point of capitalism being a huge negative factor here. AI being used abusively is just another reason to fight against this system.
You make fair points and in case it’s not clear, I don’t personally see it as “pure hype.” I’ve been around a particular AI service for… I don’t know, maybe more than a couple of years now? Some of my feel on it comes from using AI, some of it comes from discussing it with others (sometimes like beating a dead horse conversations that have been had a hundred times), and some of it comes from my technical understanding of AI, which is certainly not on the level of an ML engineer, but is quite a lot more than a layperson.
My skepticism is more about degrees than anything else. I know it can be useful in some capacities (though it is informative to hear more about the programming end, as I’ve not personally experimented with it in that capacity - my info on that more comes from others who have). But widespread adoption doesn’t seem as inevitable as some people make it out to be, or if it is, it seems like it’ll be more a consequence of capital forcing it to happen, regardless of whether it makes sense. To focus on programming as example since that’s the main subject here, comparing to other forms of advancement in programming, the benefits seem much more clearly weighted in favor of benefits in those other cases, with little known drawbacks. The main drawback along the way being one already brought up in this thread, like the dwindling knowledge of lower level system operations, especially in older languages. With generative AI, maybe it’s just cause of how much time I’ve spent around it, but I see much more obvious and difficult-to-contend-with drawbacks, like those mentioned in my previous post.
I still think it can have a place as an assistive tool, but I’m not sure it’s going to be as simple of an overall improvement as some things that are considered advancements. And to your point about evolution of the tech, where my mind goes in part on that, is the field may need breakthroughs beyond the transformer architecture before it can reach useful widespread adoption. The hallucination factor alone makes it borderline useless and at best untrustworthy for certain kinds of tasks and skill levels (e.g. not having the knowledge to know when it’s wrong, or the awareness to cross-reference with something or someone who is not an LLM). Now if there was a breakthrough (well more like multiple breakthroughs) in infrastructure and design where it could link up with a more logic-based system, with a connection to the internet, and cross-reference on things, and show its sources, and utilize an evaluative system to learn from its mistakes (which I understand is a lot, but just saying) then I think it could much more easily be seen as an overall benefit. Or even without the self-learning, if it could do all those other things (making it more trustworthy, more reasoned (not just a lingual mimic of reasoning), and more “show your work”) and it could be manually added onto with new knowledge where needed without needing a huge budget and team of professionals, then I think it could more feasibly reach widespread adoption (without being forced on people). It’s possible it will be pushed on people anyway, at least in the capitalist west, and capital will say “catch up or starve”, but there is already some not-insignificant AI hate out there and people may not make it easy for them.
Right, the reality is going to be nuanced. There will be niches where this tool will be helpful, and others where it doesn’t really make sense. We’re in a hype phase right now, and people are still figuring out good uses for it. It’s also worth noting that people are already actively working on solutions for the hallucination problem and doing actual reasoning. The most interesting approach I’ve seen so far is neurosymbolics. It combines a deep neural net with a symbolic logic engine. The neural net does what it’s good at which is parsing raw input data and classifying, and symbolic logic system operates on the classified data. This way you can have the system actually reason through a problem, explain the steps, correct it, etc. This is a fun read about it https://arxiv.org/abs/2305.00813
I do think the AI might present a problem for the capitalist system as a whole because if vast majority of work really can be automated going forward, then the whole model of working for a living will fall apart. It will be very interesting to see how the capitalist world grapples with this assuming it lasts that long to begin with.
Efficiency problems aside (hopefully R1 keeps us focused on increasing efficiency while still being useful), I find it super useful when you set a pattern and let it fill it out for you.
On a side project, I built out 10 or 15 structs and then implemented one of them in a particular pattern and I just asked it to finish off the rest. I did like 10% of the work, but because I set the pattern, it finished everything else flawlessly.
Oh yeah, I noticed that too. Once you give it a few examples, it’s good at iterating on that. And this is precisely the kind of drudgery I want to automate. There is a lot of code you end up having to write that’s just glue that holds things together, and it’s basically just a repetitive task that LLMs can automate.
I admit my read of the article was partly skimming, so maybe they covered this point, but from everything I’ve seen with LLMs, I’m skeptical their impact is going to change much, unless it’s to make things shittier by forcing them where they aren’t ready. AI as a whole could change a lot that is hard to predict because AI is kinda synonymous with automation and could be many developments of many different kinds of technologies. But the current crop of AI hype and what it’s capable of? Where I see it most taking over is the capitalistic “content churn” industry. For anything that needs to be thinking beyond “cash in and move onto the next one”, I don’t see how it gets integrated very effectively.
Part of what makes me doubt it is efficiency. Although there are some notable advances in efficiency, such as Deepseek’s cost reduction in training, generative AI is overall a resource-heavy technology. Both training and inference are costly (environmentally, in GPUs, etc., not just in price tag). Another point is competence. The more complicated a task is, the easier it is for the AI to make mistakes, some of which only an expert in the related subject matter would pick up on, which makes it a high competence task just to evaluate the AI’s results and make sure it isn’t doing more harm than good. Another is learning. You could look at the competence example and say, a human in training needs similar evaluation, but the human in training will usually learn from their mistakes, with correction, and not make them as often in the future. The AI won’t unless you retrain it and then it is still highly limited due to its statistical and tokenizing nature. Another element is trust. The western market has much more of a vested interest than, say, China, in selling the idea that AI as it is now will work and will integrate and therefore will make a profit; otherwise, its house of cards gold rush investments go to waste and the industry tanks (the fragility of that seen already in how easily Deepseek upset the equilibrium, or lack thereof).
I think programmers and programming as a field is in more danger (or danger of change, depending on how you want to look at it) from capitalists than from generative AI. The field already zipped past a phase where I can still remember reading about someone talking about a fizz buzz example as a test of basic programming competence, to the internet being stuffed to the brim with coding bootcamp stuff and “master algorithms and data structures” doctrine. And that change happened before generative AI. I don’t know what the hard numbers are, so I could be deceived on it, but by all appearances, programming became much more saturated via all the “learn to code” stuff, coupled with more companies cutting jobs in general, resulting in it being a field that is significantly harder to get into and harder to stay in. And again, all of that before generative AI.
I don’t mean this toward you, Yogthos, of course, but I think there is a certain amount of programmers being in denial about the field being touched by capitalism in general. This sort of unspoken belief that because programming is so important, the trend will just sort of continue that way and it will continue to be a lucrative and cozy ivory tower to hang out in. But that won’t stop capitalists from trying to reduce payroll as much as possible, whether it truly makes rational sense or not.
It seems like AI is a very polarizing topic, and people tend to either think it’ll do everything or reject it as pure hype. Typically, the reality of the usefulness of new tech tends to lie somewhere in between. I don’t expect that programmers will disappear as a profession in the foreseeable future. My view is that LLMs are becoming a genuinely useful tool, and they will be increasingly able to take care of writing boilerplate freeing up developers to do more interesting things.
For example, just the other day I had to create a SQL schema for an API endpoint, and I was able to throw sample JSON into DeepSeek R1 to get a reasonable schema out of it that needed practically no modifications. It probably would’ve taken me a couple of hours of work to design and write it. I also find you can generally figure out how to do something quicker with these tools than by searching sites like stack overflow or random blogs. Even if it doesn’t give a correct solution, it can point you in the right direction. Another use I can see is having it search through code bases finding where specific functionality is. This would be very helpful with finding your way around large projects. So, my experience is that there are already a lot of legitimate time saving uses for this tech. And as you note it’s hard to say where we start getting into diminishing returns territory.
Efficiency of these things is still a valid concern, but I don’t think we’ve really tried optimizing things much yet. The fact that DeepSeek was able to get such a huge improvement makes me think that there are a lot of other low hanging fruit to be plucked in the near future. I also think it’s highly likely we’ll be combining LLMs with other types of AI such as symbolic logic. This is already being tried with neurosymbolic systems. Different types of machine learning algorithms could tackle different types of problems more efficiently. There are also interesting things happening on the hardware side with stuff like analog chips showing up. Making the chip analog is way more efficient for this stuff since we’re currently emulating analog systems on top digital ones.
I very much agree regarding the point of capitalism being a huge negative factor here. AI being used abusively is just another reason to fight against this system.
You make fair points and in case it’s not clear, I don’t personally see it as “pure hype.” I’ve been around a particular AI service for… I don’t know, maybe more than a couple of years now? Some of my feel on it comes from using AI, some of it comes from discussing it with others (sometimes like beating a dead horse conversations that have been had a hundred times), and some of it comes from my technical understanding of AI, which is certainly not on the level of an ML engineer, but is quite a lot more than a layperson.
My skepticism is more about degrees than anything else. I know it can be useful in some capacities (though it is informative to hear more about the programming end, as I’ve not personally experimented with it in that capacity - my info on that more comes from others who have). But widespread adoption doesn’t seem as inevitable as some people make it out to be, or if it is, it seems like it’ll be more a consequence of capital forcing it to happen, regardless of whether it makes sense. To focus on programming as example since that’s the main subject here, comparing to other forms of advancement in programming, the benefits seem much more clearly weighted in favor of benefits in those other cases, with little known drawbacks. The main drawback along the way being one already brought up in this thread, like the dwindling knowledge of lower level system operations, especially in older languages. With generative AI, maybe it’s just cause of how much time I’ve spent around it, but I see much more obvious and difficult-to-contend-with drawbacks, like those mentioned in my previous post.
I still think it can have a place as an assistive tool, but I’m not sure it’s going to be as simple of an overall improvement as some things that are considered advancements. And to your point about evolution of the tech, where my mind goes in part on that, is the field may need breakthroughs beyond the transformer architecture before it can reach useful widespread adoption. The hallucination factor alone makes it borderline useless and at best untrustworthy for certain kinds of tasks and skill levels (e.g. not having the knowledge to know when it’s wrong, or the awareness to cross-reference with something or someone who is not an LLM). Now if there was a breakthrough (well more like multiple breakthroughs) in infrastructure and design where it could link up with a more logic-based system, with a connection to the internet, and cross-reference on things, and show its sources, and utilize an evaluative system to learn from its mistakes (which I understand is a lot, but just saying) then I think it could much more easily be seen as an overall benefit. Or even without the self-learning, if it could do all those other things (making it more trustworthy, more reasoned (not just a lingual mimic of reasoning), and more “show your work”) and it could be manually added onto with new knowledge where needed without needing a huge budget and team of professionals, then I think it could more feasibly reach widespread adoption (without being forced on people). It’s possible it will be pushed on people anyway, at least in the capitalist west, and capital will say “catch up or starve”, but there is already some not-insignificant AI hate out there and people may not make it easy for them.
Right, the reality is going to be nuanced. There will be niches where this tool will be helpful, and others where it doesn’t really make sense. We’re in a hype phase right now, and people are still figuring out good uses for it. It’s also worth noting that people are already actively working on solutions for the hallucination problem and doing actual reasoning. The most interesting approach I’ve seen so far is neurosymbolics. It combines a deep neural net with a symbolic logic engine. The neural net does what it’s good at which is parsing raw input data and classifying, and symbolic logic system operates on the classified data. This way you can have the system actually reason through a problem, explain the steps, correct it, etc. This is a fun read about it https://arxiv.org/abs/2305.00813
I do think the AI might present a problem for the capitalist system as a whole because if vast majority of work really can be automated going forward, then the whole model of working for a living will fall apart. It will be very interesting to see how the capitalist world grapples with this assuming it lasts that long to begin with.
Efficiency problems aside (hopefully R1 keeps us focused on increasing efficiency while still being useful), I find it super useful when you set a pattern and let it fill it out for you.
On a side project, I built out 10 or 15 structs and then implemented one of them in a particular pattern and I just asked it to finish off the rest. I did like 10% of the work, but because I set the pattern, it finished everything else flawlessly.
Oh yeah, I noticed that too. Once you give it a few examples, it’s good at iterating on that. And this is precisely the kind of drudgery I want to automate. There is a lot of code you end up having to write that’s just glue that holds things together, and it’s basically just a repetitive task that LLMs can automate.