LLMs are just computerized puppies that are really good at performing tricks for treats. They’ll still do incredibly stupid things pretty frequently.

I’m a software engineer, and I am not at all worried about my career in the long run.

In the short term… who fucking knows. The C-suite and MBA circlejerk seems to have decided they can fire all the engineers because wE CAn rEpLAcE tHeM WitH AI 🤡 and then the companies will have a couple absolutely catastrophic years because they got rid of all of their domain experts.

Dunno. I’d expect to have to make several attempts to coax a working snippet from the ai, then spending the rest of the time trying to figure out what it’s done and debugging the result. Faster to do it myself.

E.g. I once coded Tetris on a whim (45 min) and thought it’d be a good test for ui/ game developer, given the multi disciplinary nature of the game (user interaction, real time engine, data structures, etc) Asked copilot to give it a shot and while the basic framework was there, the code simply didn’t work as intended. I figured if we went into each of the elements separately, it would have taken me longer than if i’d done it from scratch anyway.

Yes and no. GPT usually gives me clever solutions I wouldn’t have thought of. Very often GPT also screws up, and I need to fine tune variable names, function parameters and such.

I think the best thing about GPTis that it knows the documentation of every function, so I can ask technical questions. For example, can this function really handle dataframes, or will it internally convert the variable into a matrix and then spit out a dataframe as if nothing happened? Such conversions tend to screw up the data, which explains some strange errors I bump into. You could read all of the documentation to find out, or you could just ask GPT about it. Alternatively, you could show how badly the data got screwed up after a particular function, and GPT would tell that it’s because this function uses matrices internally, even though it looks like it works with dataframes.

I think of GPT as an assistant painter some famous artists had. The artist tells the assistant to paint the boring trees in the background and the rough shape of the main subject. Once that’s done, the artist can work on the fine details, sign the painting, send it to the local king and charge a thousand gold coins.

I’ve tried Copilot and to be honest, most of the time it’s a coin toss, even for short snippets. In one scenario it might try to autocomplete a unit test I’m writing and get it pretty much spot on, but it’s also equally likely to spit out complete garbage that won’t even compile, never mind being semantically correct.

To have any chance of producing decent output, even for quite simple tasks, you will need to give an LLM an extremely specific prompt, detailing the precise behaviour you want and what the code should do in each scenario, including failure cases (hmm…there used to be a term for this…)

Even then, there are no guarantees it won’t just spit out hallucinated nonsense. And for larger, enterprise scale applications? Forget it.

The LLM can type the Code, but you need to know what you want / how you want to solve it.

Yes, in small bits, after several tries, with human supervision. For now.

No in large amounts, too hard to human review, they’re still doing it anyway.

Great question.

is there any legit reason anyone should learn advanced coding techniques?

Don’t buy the hype. LLMs can produce all kinds of useful things but they don’t know anything at all.

No LLM has ever engineered anything. And there’s no sparse (concession to a good point made in response) current evidence that any AI ever will.

Current learning models are like trained animals in a circus. They can learn to do any impressive thing you an imagine, by sheer rote repetition.

That means they can engineer a solution to any problem that has already been solved millions of times already. As long as the work has very little new/novel value and requires no innovation whatsoever, learning models do great work.

Horses and LLMs that solve advanced algebra don’t understand algebra at all. It’s a clever trick.

Understanding the problem and understanding how to politely ask the computer to do the right thing has always been the core job of a computer programmer.

The bit about “politely asking the computer to do the right thing” makes massive strides in convenience every decade or so. Learning models are another such massive stride. This is great. Hooray!

The bit about “understanding the problem” isn’t within the capabilities of any current learning model or AI, and there’s no current evidence that it ever will be.

Someday they will call the job “prompt engineering” and on that day it will still be the same exact job it is today, just with different bullshit to wade through to get it done.

my dad uses this LLM python code generation quite routinely, he says the output’s mostly fine.

Its the most ok’est coder with the attention span of a 5 year old.

This question is basically the same as asking “Are 2d6 capable of rolling a 9?”

A broken clock is right twice a day.

For small boilerplate or very common small pieces of code, for instance a famous algorithm implementation. Yes. As they are just probably giving you the top stack overflow answer for a classic question.

Anything that the LLM would need to mix or refactor would be terrible.

For basic boiler plate like routes for an API, an etl script from sample data to DB tables, or other similar basics, yeah, it’s perfectly acceptable. You’ll need to swap out dummy addresses, and maybe change a choice or two, but it’s fine.

But when you’re trying to organize more complicated business logic or debug complicated dependencies it falls over

That all depends on where the data set comes from. The code you’ll get out of an LLM is the average code of the data set. If it’s scraped from the internet (which is very likely) the code you’ll get will be an amalgam of concise examples from one website, incorrect examples from another, bits from blogs with all the typos and all the gunk and garbage that’s out there.

Getting LLM code to work well takes an understanding of what the code it gives you actually does and why it’s bad. It will always be bad because it cannot be better than the dataset and in order for a dataset to be big enough to train an LLM it’ll have to have everything they can get including all the trash. But it can be good for providing you a framework to start with. It is however never going to replace actual programming and understanding of programming. The talk of LLMs completely replacing programers is mostly coming from people who do not understand coding or LLMs at all.

I think your wording is something to consider. If you want something that’s written professionally, by definition it needs to be written by a professional. So that’s clearly not what you’re asking for, but that’s what you wrote. And that kind of detail does matter, because LLMs are very good at getting part of the format correct and then messing up small details in random places, which makes them precisely useless on their own. But if you want to use them to produce templates that you’re later going to modify, of course you can do that.

I’m not clear what you think an advanced coding technique would be. But if your system breaks and you don’t understand it well enough to fix it, then I sure hope a competent programmer is on staff who can help you.

Finally, if you rely on automation to write your programs for you and somehow they magically seem to work most of the time, how do you know that they actually work all of the time? If they’re giving you numbers, can you believe the numbers? When? Why? Who is guaranteeing you quality in product? Of course nobody is.