If you can’t take 10 - 30 minutes to search for, read, and comprehend information on stack overflow or docs
A lot of the time this is just looking for syntax though; you know what you want to do, and it’s simple, but it is gated behind busywork. This is to me the most useful part about ChatGPT, it knows all the syntax and will write it out for you and answer clarifying questions so you can remain in a mental state of thinking about the actual problem instead of digging through piles of junk for a bit of information.
Somehow you hit an unpopular opinion landmine with the greybeard devs.
For the greybeard devs:
Try asking ChatGPT to write you some Arduino code to do a specific task. Even if you don’t know how to write code for an Arduino, ChatGPT will get you 95% of the way there with the proper libraries and syntax.
No way in hell I’m digging through forums and code repos for hours to blink an led and send out a notification through a web hook when a sensor gets triggered if AI can do it for me in 30 seconds.
AI obviously can’t do everything for you if you’ve never coded anything before, but it can do a damn good job of translating your knowledge of one programming language into every other programming language available.
It’s great for jumping into something you’re very unfamiliar with. Unfortunately, if you often find yourself very unfamiliar with day to day tasks, you’re probably incompetent. (Or maybe a butterfly who gets paid to learn new things every day.)
Did ChatGPT just hallucinate it? Does it exist but it isn’t used like ChatGPT says? Does it exist but it doesn’t do what ChatGPT thinks it does?
I use ChatGPT sometimes to help out with stuff at home (I’ve tried it for work stuff but the stuff I work on is niche enough that it purely hallucinates), and I’ve ended up running in circles for hours because the answer I got ended up in this uncanny valley: Correct enough that it isn’t immediately obviously wrong, but incorrect enough that it won’t work, it can’t work, and you’re going to really have to put a lot of work in to figure that out.
The more you grow in experience the more you’re going to realize that syntax and organization is the majority of programming work.
When you first start out, it feels like the hardest part is figuring out how to get from a to b on a conceptual level. Eventually that will become far easier.
You break the big problem down into discrete steps, then figure out the besy way to do each step. It takes little skill to say “the computer just needs to do this”. The trick is knowing how to speak to the computer in a way that can make sense to the computer, to you, and to the others who will eventually have to work with your code.
You’re doing the equivalent of a painter saying “I’ve done the hard part of envisioning it in my head! I’m just going to pay some guy on fiver to move the brush for me”
This is difficult to put into words, as it’s also not about memorization of every language specific syntax pattern. But there’s a difference between looking up documentation or at previous code for syntax, and trying to have chatGPT turn your psuedocode notes into working code.
I’m a pretty senior dev and have chat gpt open for quick searches. It’s great for helping me figure out what to Google in the cases where I can’t think of the name of a pattern or type I’m looking for. It also helps quite a bit with learning about obscure functions and keywords in SQL that I can do more research on
Hell, I use Copilot daily. Its auto complete is top-tier
Copilot is good for tedious stuff like writing enums. But otherwise I more often than not need to only accept tne suggested line or particular words, since in multiline snippets it can do stupid things, like exiting outside of main() or skipping error checks.
I’ve been programming for decades, I’m not actually a beginner. A mistake I made early on was thinking that everything I learn will be worth the time to learn it, and will always increase my overall skill level. But (particularly as relates to syntax) it’s not and it doesn’t; something I only use once or rarely, something that isn’t closely connected with the rest of what I often do, I’ll just forget it after a while. I greatly prefer being broadly capable of making things happen to having a finely honed specialization, so I run into that sort of thing a lot, there is an ocean of information out there and many very different things a programmer can be doing.
I think it is an important and valuable lesson to know when to get over yourself and take shortcuts. There are situations where you absolutely should never do that, but they are rare. There are many situations where not taking shortcuts is a huge mistake and will result in piles of abandoned code and not finishing what you set out to do. AI is an incredibly powerful source of shortcuts.
You’re doing the equivalent of a painter saying “I’ve done the hard part of envisioning it in my head! I’m just going to pay some guy on fiver to move the brush for me”
More like you’ve coded the functionality for a webapp, have a visual mockup, and pay some guy on fiver to write the CSS for you, because doing it yourself is an inefficient use of your time and you don’t specialize in CSS.
As for the issue of a new programmer ending up with problems because they rely too much on AI and somehow fail to learn how to model the structure of programs in their head, that’s probably real, but I can’t imagine how that will go because all I had to go on when I was learning was google and IRC and it’s totally different. Hope it works out for them.
TBF that’s how many master artists worked in the past. The big art producers had one master painter guiding a bunch of apprentices who did the actual legwork.
That the exact opposite problem for most people though. Syntax is hard at first because you are unfamiliar and gets more natural to you overtime. Algorithms are easier to think about conceptually as a person with no programming experience as they are not programming specific. If you are an experienced developer struggling over syntax but breezing through difficult data structure and algorithm problems (Eg. Thinking about the most efficient way to upload constantly updating vertex data to the gpu) you are definitely the anomaly.
There’s a 5 hour interview with John Carmack on YouTube where he talks about transitioning from really caring deeply about algorithms and the like to deeply caring about how to make a sustainable and maintainable codebase you can have an entire team work on.
Often, a solution that is completely correct if all you’re doing is solving that problem is completely incorrect in the greater context of the codebase you’re working within, like if you wanted to add a dog to the Mona Lisa, you can’t just draw a detailed line art dog or a cartoon dog and expect it to work – you’d need to find someone who can paint a dog similar to the art style of the piece and properly get it to mesh with the painting.
I write a lot of bash and I still have to check syntax every day, but the answer to that is not chatGPT but a proper linter like shell check that you can trust because it’s based on a rigid set of rules, not the black box of a LLM.
I can understand the syntax justification for obscure languages that don’t have a well written linter, but if anything that gives me less confidence about CHATGPT because it’s training material for an obscure language is likely smaller.
Less checking syntax and more like “what was this function name again ?”
“Which library has that ?”
“Do I need to instance this or is it static ?”
All of theses can be answered by documentation, but who want to read the docs when you can ask chatgpt. (Copilot is better in my experience btw)
Never ask ChatGPT to write code that you plan to actually use, and never take it as a source of truth. I use it to put me on a possible right path when I’m totally lost and lack the vocabulary to accurately describe what I need. Sometimes I’ll ask it for an example of how sometimes works so that I can learn it myself. It’s an incredibly useful tool, but you’re out of your damn mind if you’re just regularly copying code it spits out. You need to error check everything it does, and if you don’t know the syntax well enough to write it yourself, how the hell do you plan to reliably error check it?
ChatGPT cannot explain, because it doesn’t understand. It will simply string together a likely sequence of characters. I’ve tried to use it multiple times for programming tasks and found each time that it doesn’t save much time, compared to an IDE. ChatGPT regularly makes up methods or entire libraries. I do like it for creating longer texts that I then manually polish, but any LLM is awful for factual information.
ChatGPT regularly makes up methods or entire libraries
I think that when it is doing that, it is normally a sign that what you are asking for does not exist and you are on the wrong track.
ChatGPT cannot explain, because it doesn’t understand
I often get good explanations that seem to reflect understanding, which often would be difficult to look up otherwise. For example when I asked about the code generated, {myVariable} , and how it could be a valid function parameter in javascript, it responded that it is the equivalent of {"myVariable":myVariable}, and “When using object literal property value shorthand, if you’re setting a property value to a variable of the same name, you can simply use the variable name.”
If ChatGPT gives you correct information you’re either lucky or just didn’t realize it was making shit up. That’s a simple fact. LLMs absolutely have their uses, but facts ain’t one of them.
you can remain in a mental state of thinking about the actual problem
more like you’ll end up wasting a significant amount of time debugging not only the problem, but also chatGPT, trying to correct the bullshit it spews out, often ignoring parts of your prompt
It might be wrong even if you provide extensive context to make it more accurate in it’s heuristics. And providing extensive context is pretty time consuming at times.
I mean it might be good at helping you when you’re stuck, but sometimes it misses simple issues such as typos and for one issue resolved, it might introduce another if you’re not careful.
A lot of the time this is just looking for syntax though; you know what you want to do, and it’s simple, but it is gated behind busywork. This is to me the most useful part about ChatGPT, it knows all the syntax and will write it out for you and answer clarifying questions so you can remain in a mental state of thinking about the actual problem instead of digging through piles of junk for a bit of information.
Somehow you hit an unpopular opinion landmine with the greybeard devs.
For the greybeard devs: Try asking ChatGPT to write you some Arduino code to do a specific task. Even if you don’t know how to write code for an Arduino, ChatGPT will get you 95% of the way there with the proper libraries and syntax.
No way in hell I’m digging through forums and code repos for hours to blink an led and send out a notification through a web hook when a sensor gets triggered if AI can do it for me in 30 seconds. AI obviously can’t do everything for you if you’ve never coded anything before, but it can do a damn good job of translating your knowledge of one programming language into every other programming language available.
It’s great for jumping into something you’re very unfamiliar with. Unfortunately, if you often find yourself very unfamiliar with day to day tasks, you’re probably incompetent. (Or maybe a butterfly who gets paid to learn new things every day.)
Getting paid to learn new things everyday at work is a dream, I don’t see the issues
The issue is it’s a dream.
BIIIIG problem: The last 5%.
Did ChatGPT just hallucinate it? Does it exist but it isn’t used like ChatGPT says? Does it exist but it doesn’t do what ChatGPT thinks it does?
I use ChatGPT sometimes to help out with stuff at home (I’ve tried it for work stuff but the stuff I work on is niche enough that it purely hallucinates), and I’ve ended up running in circles for hours because the answer I got ended up in this uncanny valley: Correct enough that it isn’t immediately obviously wrong, but incorrect enough that it won’t work, it can’t work, and you’re going to really have to put a lot of work in to figure that out.
Just a few days ago I read an article on the newest features of Kotlin 1.9. Zero of it was true.
Internet is littered with stuff like this.
If model is correct, you are correct. If model is not correct, you are working on false assumptions.
No difference there, either way your information may be wrong or misleading. Running code and seeing what it does is the solution.
The more you grow in experience the more you’re going to realize that syntax and organization is the majority of programming work.
When you first start out, it feels like the hardest part is figuring out how to get from a to b on a conceptual level. Eventually that will become far easier.
You break the big problem down into discrete steps, then figure out the besy way to do each step. It takes little skill to say “the computer just needs to do this”. The trick is knowing how to speak to the computer in a way that can make sense to the computer, to you, and to the others who will eventually have to work with your code.
You’re doing the equivalent of a painter saying “I’ve done the hard part of envisioning it in my head! I’m just going to pay some guy on fiver to move the brush for me”
This is difficult to put into words, as it’s also not about memorization of every language specific syntax pattern. But there’s a difference between looking up documentation or at previous code for syntax, and trying to have chatGPT turn your psuedocode notes into working code.
organization, absolutely - but syntax? c’mon…
I’m a pretty senior dev and have chat gpt open for quick searches. It’s great for helping me figure out what to Google in the cases where I can’t think of the name of a pattern or type I’m looking for. It also helps quite a bit with learning about obscure functions and keywords in SQL that I can do more research on
Hell, I use Copilot daily. Its auto complete is top-tier
Copilot is good for tedious stuff like writing enums. But otherwise I more often than not need to only accept tne suggested line or particular words, since in multiline snippets it can do stupid things, like exiting outside of
main()or skipping error checks.I’ve been programming for decades, I’m not actually a beginner. A mistake I made early on was thinking that everything I learn will be worth the time to learn it, and will always increase my overall skill level. But (particularly as relates to syntax) it’s not and it doesn’t; something I only use once or rarely, something that isn’t closely connected with the rest of what I often do, I’ll just forget it after a while. I greatly prefer being broadly capable of making things happen to having a finely honed specialization, so I run into that sort of thing a lot, there is an ocean of information out there and many very different things a programmer can be doing.
I think it is an important and valuable lesson to know when to get over yourself and take shortcuts. There are situations where you absolutely should never do that, but they are rare. There are many situations where not taking shortcuts is a huge mistake and will result in piles of abandoned code and not finishing what you set out to do. AI is an incredibly powerful source of shortcuts.
More like you’ve coded the functionality for a webapp, have a visual mockup, and pay some guy on fiver to write the CSS for you, because doing it yourself is an inefficient use of your time and you don’t specialize in CSS.
As for the issue of a new programmer ending up with problems because they rely too much on AI and somehow fail to learn how to model the structure of programs in their head, that’s probably real, but I can’t imagine how that will go because all I had to go on when I was learning was google and IRC and it’s totally different. Hope it works out for them.
TBF that’s how many master artists worked in the past. The big art producers had one master painter guiding a bunch of apprentices who did the actual legwork.
And senior devs guide junior devs in the same way. The point is the masters already did their time in the trenches. That how they became masters.
That the exact opposite problem for most people though. Syntax is hard at first because you are unfamiliar and gets more natural to you overtime. Algorithms are easier to think about conceptually as a person with no programming experience as they are not programming specific. If you are an experienced developer struggling over syntax but breezing through difficult data structure and algorithm problems (Eg. Thinking about the most efficient way to upload constantly updating vertex data to the gpu) you are definitely the anomaly.
There’s a 5 hour interview with John Carmack on YouTube where he talks about transitioning from really caring deeply about algorithms and the like to deeply caring about how to make a sustainable and maintainable codebase you can have an entire team work on.
Often, a solution that is completely correct if all you’re doing is solving that problem is completely incorrect in the greater context of the codebase you’re working within, like if you wanted to add a dog to the Mona Lisa, you can’t just draw a detailed line art dog or a cartoon dog and expect it to work – you’d need to find someone who can paint a dog similar to the art style of the piece and properly get it to mesh with the painting.
I write a lot of bash and I still have to check syntax every day, but the answer to that is not chatGPT but a proper linter like shell check that you can trust because it’s based on a rigid set of rules, not the black box of a LLM.
I can understand the syntax justification for obscure languages that don’t have a well written linter, but if anything that gives me less confidence about CHATGPT because it’s training material for an obscure language is likely smaller.
Less checking syntax and more like “what was this function name again ?”
“Which library has that ?”
“Do I need to instance this or is it static ?”
All of theses can be answered by documentation, but who want to read the docs when you can ask chatgpt. (Copilot is better in my experience btw)
Never ask ChatGPT to write code that you plan to actually use, and never take it as a source of truth. I use it to put me on a possible right path when I’m totally lost and lack the vocabulary to accurately describe what I need. Sometimes I’ll ask it for an example of how sometimes works so that I can learn it myself. It’s an incredibly useful tool, but you’re out of your damn mind if you’re just regularly copying code it spits out. You need to error check everything it does, and if you don’t know the syntax well enough to write it yourself, how the hell do you plan to reliably error check it?
ChatGPT cannot explain, because it doesn’t understand. It will simply string together a likely sequence of characters. I’ve tried to use it multiple times for programming tasks and found each time that it doesn’t save much time, compared to an IDE. ChatGPT regularly makes up methods or entire libraries. I do like it for creating longer texts that I then manually polish, but any LLM is awful for factual information.
I think that when it is doing that, it is normally a sign that what you are asking for does not exist and you are on the wrong track.
I often get good explanations that seem to reflect understanding, which often would be difficult to look up otherwise. For example when I asked about the code generated,
{myVariable}, and how it could be a valid function parameter in javascript, it responded that it is the equivalent of{"myVariable":myVariable}, and “When using object literal property value shorthand, if you’re setting a property value to a variable of the same name, you can simply use the variable name.”If ChatGPT gives you correct information you’re either lucky or just didn’t realize it was making shit up. That’s a simple fact. LLMs absolutely have their uses, but facts ain’t one of them.
more like you’ll end up wasting a significant amount of time debugging not only the problem, but also chatGPT, trying to correct the bullshit it spews out, often ignoring parts of your prompt
That hasn’t been my experience. How are you trying to use it?
It might be wrong even if you provide extensive context to make it more accurate in it’s heuristics. And providing extensive context is pretty time consuming at times.
I think it would help me organize my thoughts to write that all out anyway even without a LLM.
I mean it might be good at helping you when you’re stuck, but sometimes it misses simple issues such as typos and for one issue resolved, it might introduce another if you’re not careful.