For me the worry is what happens if the practice of using LLMs to generate code becomes so ubiquitous that in idk, 50-100 years people forget how to actually write code themselves? We already have this problem thanks to higher level programming languages that an ever dwindling number of people have any kind of competencies in reading let alone writing Assembly code. Compilers do all that for us. So there is this danger of computers literally becoming black boxes that nobody understands anymore because we’ve abstracted the tools we use to interface with them so much that what is really going on at the most basic level just looks like magic to us.
If nothing else this could create some really weird social phenomena where people start to develop all sorts of superstitions and unscientific beliefs about computers because even the people working on them professionally just don’t understand them. I’m a bit anxious that all of this is pointing to how our societies, rather than adopting a more materialist and scientific world view, will instead just be entering a new age of obscurantism.
And what happens when something goes wrong and you need to debug something on a more fundamental level? What happens when only computers will be able to “understand” how computers actually work?
This is my biggest worry about AI. Not that it will try to “take over the world” or any of that other sci-fi apocalypse stuff, but simply how it will negatively affect humans on a social and psychological level, change how we relate to technology, knowledge and skills…and even to each other.
If you don’t really need to acquire and train these kinds of skills anymore because you always have an “AI” do your work for you, will we all just become incapable of doing these things ourselves? If someone always gives you the answers to your math homework, how are you supposed to learn? I wonder if this how people thought about industrialization in the 19th century? Why do i feel like i’m turning into those people who were saying much the same things about the Internet in the early 90s?
I think we’re already largely there. Nobody really knows how the full computing stack works anymore. The whole thing is just too big to fit in your head. So, there is a lot of software out there that’s already effectively a black box. There’s a whole joke how large legacy systems are basically like generation ships where new devs have no idea how or why the system was built that way, and they just plug holes as they go.
However, even if people forget how to write code, it’s not like it’s a skill that can’t be learned if it becomes needed again. And if we do get to the point where LLMs are good enough that people forget how to write code, then it means the LLMs just become the way people write code. I don’t see how it’s different from people who only know how to use a high level language today. A Js dev will not know how to work with pointers, do manual memory management and so on. You can even take it up a level and look at it from a perspective of a non technical person asking a developer to write a program for them. They’re already in this exact scenario, and that’s vast majority of the population.
And given the specification writing approach I described, I don’t actually see that much of a problem with the code being a black box. You would basically create contracts and LLM will fill them, and this way you have some guarantees about the behavior of the system.
It’s possible people start developing mysticism about software, but at this point most people already treat technology like magic. I expect there will always be people who have an inclination towards a scientific view of the world, and who enjoy understanding how things work. I don’t think LLMs are going to change that.
Personally, I kind of see a synthesis between AI tools and humans going forward. We’ll be using this tech augment our abilities, and we’ll just focus on solving bigger problems together. I don’t expect there’s going to be some sort of intellectual collapse, rather the opposite could happen where people start tacking problems on the scale that seems unimaginable today.
For me the worry is what happens if the practice of using LLMs to generate code becomes so ubiquitous that in idk, 50-100 years people forget how to actually write code themselves? We already have this problem thanks to higher level programming languages that an ever dwindling number of people have any kind of competencies in reading let alone writing Assembly code. Compilers do all that for us. So there is this danger of computers literally becoming black boxes that nobody understands anymore because we’ve abstracted the tools we use to interface with them so much that what is really going on at the most basic level just looks like magic to us.
If nothing else this could create some really weird social phenomena where people start to develop all sorts of superstitions and unscientific beliefs about computers because even the people working on them professionally just don’t understand them. I’m a bit anxious that all of this is pointing to how our societies, rather than adopting a more materialist and scientific world view, will instead just be entering a new age of obscurantism.
And what happens when something goes wrong and you need to debug something on a more fundamental level? What happens when only computers will be able to “understand” how computers actually work?
This is my biggest worry about AI. Not that it will try to “take over the world” or any of that other sci-fi apocalypse stuff, but simply how it will negatively affect humans on a social and psychological level, change how we relate to technology, knowledge and skills…and even to each other.
If you don’t really need to acquire and train these kinds of skills anymore because you always have an “AI” do your work for you, will we all just become incapable of doing these things ourselves? If someone always gives you the answers to your math homework, how are you supposed to learn? I wonder if this how people thought about industrialization in the 19th century? Why do i feel like i’m turning into those people who were saying much the same things about the Internet in the early 90s?
I think we’re already largely there. Nobody really knows how the full computing stack works anymore. The whole thing is just too big to fit in your head. So, there is a lot of software out there that’s already effectively a black box. There’s a whole joke how large legacy systems are basically like generation ships where new devs have no idea how or why the system was built that way, and they just plug holes as they go.
However, even if people forget how to write code, it’s not like it’s a skill that can’t be learned if it becomes needed again. And if we do get to the point where LLMs are good enough that people forget how to write code, then it means the LLMs just become the way people write code. I don’t see how it’s different from people who only know how to use a high level language today. A Js dev will not know how to work with pointers, do manual memory management and so on. You can even take it up a level and look at it from a perspective of a non technical person asking a developer to write a program for them. They’re already in this exact scenario, and that’s vast majority of the population.
And given the specification writing approach I described, I don’t actually see that much of a problem with the code being a black box. You would basically create contracts and LLM will fill them, and this way you have some guarantees about the behavior of the system.
It’s possible people start developing mysticism about software, but at this point most people already treat technology like magic. I expect there will always be people who have an inclination towards a scientific view of the world, and who enjoy understanding how things work. I don’t think LLMs are going to change that.
Personally, I kind of see a synthesis between AI tools and humans going forward. We’ll be using this tech augment our abilities, and we’ll just focus on solving bigger problems together. I don’t expect there’s going to be some sort of intellectual collapse, rather the opposite could happen where people start tacking problems on the scale that seems unimaginable today.