• @[email protected]
    link
    fedilink
    English
    184 days ago

    Software and computers are a joke at this point.

    Computers no longer solve real problems and are now just used to solve the problems that overly complex software running on monstrous cheap hardware create.

    “Hey I’d like to run a simple electronics schematic program like we had in the DOS days, it ran in 640K and responded instantly!”

    “OK sure first you’ll need the latest Windows 11 with 64G of RAM and 2TB of storage, running on at least 24 cores, then you need to install a container for the Docker for the VM for the flatpak for the library for the framework because the programmer liked the blue icon, then make sure you are always connected to the internet for updates or it won’t run, and somehow the program will still just look like a 16 bit VB app from 1995.”

    “Well that sounds complicated, where’s the support webpage for installing the program in Windows 7?”

    “Do you have the latest AI agents installed in your web browser?”

    “It’s asking me to click OK but I didn’t install the 1GB mouse driver that sends my porn browsing habits to Amazon…”

    “Just click OK on all the EULAs so you lose the right to the work you’ll create with this software, then install a few more dependencies, languages, entire VMs written in byte code compiled to HTML to run on JAVA, then make sure you have a PON from your ISP otherwise how can you expect to have a few kilobytes of data be processed on your computer? This is all in the cloud, baby!”

  • @[email protected]
    link
    fedilink
    English
    855 days ago

    Devs are famously bad at estimating how long a software project will take.

    No, highly complex creative work is inherently extremely difficult to estimate.

    Anyway, not shocked at all by the results. This is a great start that begs for larger and more rigorous studies.

    • @[email protected]
      link
      fedilink
      English
      245 days ago

      You’re absolutely correct that the angle approach that statement is bullshit. There is also that they want to think making software is not highly complex creative work but somehow is just working an assembly line and the software devs are gatekeepers that don’t deserve respect.

    • nickwitha_k (he/him)
      cake
      link
      fedilink
      English
      2
      edit-2
      4 days ago

      “Devs are famously bad at estimating how long a software project will take.”

      No, highly complex creative work is inherently extremely difficult to estimate.

      Akshually… I’m on a dev team where about 60% of us are diagnosed with ADHD. So, at least in our case, it’s both.

        • @[email protected]
          link
          fedilink
          English
          54 days ago

          We’re the only ones that can get hyper focused and also hyper fixated on why a switch statement is failing when it includes a for loop until finding out there’s actually a compiler bug, and if you leave a space after the bracket it somehow works correctly.

          That was a fun afternoon.

  • @[email protected]
    link
    fedilink
    English
    455 days ago

    As someone that has had to double check peoples code before, especially those that don’t comment appropriately, I’d rather just write it all again myself than try and decipher what the fuck they were even doing.

  • Charlie Stross
    link
    fedilink
    134 days ago

    @dgerard What fascinates me is *why* coders who use LLMs think they’re more productive. Is the complexity of their prompt interaction misleading them as to how effective the outputs it results in are? Or something else?

      • @[email protected]
        link
        fedilink
        English
        54 days ago

        The reward mechanism in the brain is triggered when you bet. I think it also triggers a second time when you do win, but I’m not sure. So, yeah, sometimes the LLM spits out something good, and your brain rewards you already when you ask it. Hence, you probably do feel better, because you constantly get hits dopamine.

    • @[email protected]
      link
      fedilink
      English
      134 days ago

      Here’s a random guess. They are thinking less, so time seems to go by quicker. Think about how long 2 hours of calculus homework seems vs 2 hours sitting on the beach.

      • @[email protected]
        link
        fedilink
        English
        84 days ago

        This is such a wild example to me because sitting at beach is extremely boring and takes forever whereas doing calculus is at least engaging so time flies reasonably quick.

        Like when I think what takes the longest in my life I don’t think “those times when I’m actively solving problems”, I think “those times I sit in a waiting room at the doctors with nothing to do” or “commuting, ditto”.

        • @[email protected]
          link
          fedilink
          English
          34 days ago

          I know what you mean. If I’m absorbed in something I find interesting time flies. Solving integrals is not one those for me.

    • @[email protected]
      link
      fedilink
      English
      24 days ago

      Most people want to do the least possible work with the least possible effort and AI is the vehicle for that. They say whatever words make AI sound good. There’s no reason to take their words at face value.

  • @[email protected]
    link
    fedilink
    English
    215 days ago

    I have an LLM usage mandate in my performance review now. I can’t trust it to do anything important, so I’ll get it to do incredibly noddy things like deleting a clause (that I literally always have highlighted) or generate documentation that’s more long-winded than just reading the code and then go to the bathroom while it happens.

  • @[email protected]
    link
    fedilink
    English
    465 days ago

    Megacorp LLM death spiral:

    1. Megacorp managers at all levels introduce new LLM usage policies.
    2. Productivity goes down (see study linked in post)
    3. Managers make the excuse that this is due to a transitional period in LLM policies.
    4. Policies become mandates. Beatings begin and/or intensify.
    5. Repeat from 1.
    • @[email protected]
      link
      fedilink
      English
      165 days ago

      I’ve been through the hellscape where managers used missed metrics as evidence for why we didn’t need increased headcount on an internal IT helpdesk.

      That sort of fuckery is common when management gets the idea in their head that they can save money on people somehow without sacrificing output/quality.

      I’m pretty certain they were trying to find an excuse to outsource us, as this was long before the LLM bubble we’re in now.

      • @[email protected]
        link
        fedilink
        English
        155 days ago

        oh, absolutely. I mean you could sub out “LLM” with any bullshit that management can easily spring on their understaff. Agile, standups, return to office, the list goes on. Management can get fucked

  • @[email protected]
    link
    fedilink
    English
    155 days ago

    From the blog post referenced:

    We do not provide evidence that:

    AI systems do not currently speed up many or most software developers

    Seems the article should be titled “16 AI coders think they’re 20% faster — but they’re actually 19% slower” - though I guess making us think it was intended to be a statistically relevant finding was the point.

    That all said, this was genuinely interesting and is in-line with my understanding of the human psychology that’s at play. It would be nice to see this at a wider scale, broken down across different methodologies / toolsets and models.

    • David GerardOPM
      link
      fedilink
      English
      165 days ago

      these are stock images! Which are surprisingly cheap. By Valeriy Kachaev, who puts stuff up as Studiostoks on a pile of stock image sites. His pics are bizarre and keep being the perfect thing.

      • @[email protected]
        link
        fedilink
        English
        54 days ago

        I’m not sure how much this observation can be generalized, but I’ve also wondered how much the people who overestimate the usefulness of AI image generators underestimate the chances of licensing decent artwork from real creatives with just a few clicks and at low cost. For example, if I’m looking for an illustration for a PowerPoint presentation, I’ll usually find something suitable fairly quickly in Canva’s library. That’s why I don’t understand why so many people believe they absolutely need AI-generated slop for this. Of course, however, Canva is participating in the AI hype now as well. I guess they have to keep their investors happy.

        • David GerardOPM
          link
          fedilink
          English
          54 days ago

          all the stock sites are. use case: an image that’s almost perfect but you wanna tweak it

          LEARN PAINT YOU GHOULS

  • @[email protected]
    link
    fedilink
    English
    205 days ago

    Anyone who has had to unfuck someone else’s work knows it would have been faster to do the work correctly from scratch the first time.

  • Tar_Alcaran
    link
    fedilink
    English
    205 days ago

    I just want to point out that every single heavily downvoted, idiotic pro-AI reply on this post is from a .ml user (with one programming.dev thrown in).

    I wonder which way the causation flows.

      • @[email protected]
        link
        fedilink
        English
        204 days ago

        It’s kind of the opposite, GenAI is downstream of machine learning which is how artificial neural networks rebranded after the previous AI winter ended.

        Also after taking a look there I don’t think lemmy.ml has anything in particular to do with machine learning, it looks more like a straight attempt at a /r/all clone.

  • hankg
    link
    fedilink
    415 days ago

    @dgerard I normally consider myself a 10x developer. With the 10x speedup of AI I now consider myself a 100x developer. I can replace an entire small business worth of developers with just myself and my LLM bot assistance. Just pay me $100 million up front no strings and I’ll prove it to you! /s

    • @[email protected]
      link
      fedilink
      English
      245 days ago

      I have the deal of a lifetime for you.

      I represent a group of investors in possession of a truly unique NFT that has been recently valued at over $100M. We will invest this NFT in your 100x business - in return you transfer us the difference between the $100M investment and the excess value of the NFT. Standard rich people stuff, don’t worry about it.

      Let me know when you’re ready to unlock your 100x potential and I’ll make our investment available via a suitable escrow service.

      • Logi
        link
        fedilink
        English
        84 days ago

        Don’t be silly. Mark Zuckerberg already knows our location.

  • David GerardOPM
    link
    fedilink
    English
    245 days ago

    ahahaha holy shit. I knew METR smelled a bit like AI doomsday cultists and took money from OpenPhil, but those “open source” projects and engineers? One of them was LessWrong.

    Here’s a LW site dev whining about the study, he was in it and i think he thinks it was unfair to AI

    I think if people are citing in another 3 months time, they’ll be making a mistake

    dude $NEXT_VERSION will be so cool

    so anyway, this study has gone mainstream! It was on CNBC! I urge you not to watch that unless you have a yearning need to know what the normies are hearing about this shit. In summary, they are hearing that AI coding isn’t all that actually and may not do what the captains of industry want.

    around 2:30 the two talking heads ran out of information and just started incorrecting each other on the fabulous AI future, like the worst work lunchroom debate ever but it’s about AI becoming superhuman

    the key takeaway for the non techie businessmen and investors who take CNBC seriously ever: the bubble starts not going so great

    • @[email protected]
      link
      fedilink
      English
      64 days ago

      I think if people are citing in another 3 months time, they’ll be making a mistake

      In 3 months they’ll think they’re 40% faster while being 38% slower. And sometime in 2026 they will be exactly 100% slower - the moment referred to as “technological singularity”.

    • @[email protected]
      link
      fedilink
      English
      64 days ago

      Here’s a LW site dev whining about the study, he was in it and i think he thinks it was unfair to AI

      There a complete lack of introspection. It seems like the obvious conclusion to draw from a study showing people’s subjective estimates of their productivity with LLMs were the exact opposite of right would inspire him to question his subjectively felt intuitions and experience but instead he doubles down and insists the study must be wrong and surely with the latest model and best use of it it would be a big improvement.

    • @[email protected]
      link
      fedilink
      English
      10
      edit-2
      5 days ago

      Yeah, METR was the group that made the infamous AI IS DOUBLING EVERY 4-7 MONTHS GRAPH where the measurement was 50% success at SWE tasks based on the time it took a human to complete it. Extremely arbitrary success rate, very suspicious imo. They are fanatics trying to pinpoint when the robo god recursive self improvement loop starts.

  • Photuris
    link
    fedilink
    English
    75 days ago

    The point isn’t to increase employee productivity (or employee happiness, obviously).

    The point is to replace most employees.

    In order for that to happen, LLM-assisted entry-level developers merely need to be half as good as expert human unassisted developers, at scale, at a lower aggregate cost.

    • @[email protected]
      link
      fedilink
      English
      115 days ago

      This is a very “nine women can make a baby in one month”.

      The idea that there can even be two half as good developers is a misunderstanding of how anything works. If it worked like that, the study would be a dud because people could just run two AIs for 160% productivity.

    • @[email protected]
      link
      fedilink
      English
      245 days ago

      Are these entry-level developers that are merely half as good as expert human unassisted developers in the room with us right now?

    • @[email protected]
      link
      fedilink
      English
      205 days ago

      LLM-assisted entry-level developers merely need to be half as good as expert human unassisted developers

      1. This isn’t even close to existing.
      2. The theoretical cyborg-developer at that skill level would surely be introducing horrible security bugs or brittle features that don’t stand up to change
      3. Sadly i think this is exactly what many CEOs are thinking is going to happen because they’ve been sold on openai and anthropic lies that it’s just around the corner
      • @[email protected]
        link
        fedilink
        English
        85 days ago

        “I’m not scared a LLM is going to be able to replace me. I’m scared that CEO are going to think that”

        • David GerardOPM
          link
          fedilink
          English
          64 days ago

          AI->cocaine filter: Cocaine isn’t going to replace you. Someone using cocaine is going to replace you.

      • Photuris
        link
        fedilink
        English
        24 days ago

        Point 3. is what I’m getting at. They desperately want to unload expensive developers, so bad. They’ll try it, and keep trying it, even as products suffer, in the hopes of driving down labor costs.

    • @[email protected]
      link
      fedilink
      English
      155 days ago

      That’s a fool’s errand. Only those highly skilled experienced devs are going to spot certain subtle issues or architectural issues that will bite everyone the moment the product grows larger.

      • @[email protected]
        link
        fedilink
        English
        75 days ago

        as one of the people representing the “hero group” (for lack of a better term) your comment references: eh. I didn’t start out with all this knowledge and experience. it built up over time.

        it’s more about the mode of thinking and how to engage with a problem, than it is about specific “highly skilled” stuff. the skill and experience help/contribute, they refine, they assist in filtering

        the reason I make this comment is because I think it’s valuable that anyone who can do the job well gets to do the thing, and that it’s never good to gatekeep people out. let’s not unnecessarily contribute to imposter syndrome

      • Photuris
        link
        fedilink
        English
        34 days ago

        That’s a fool’s errand

        Of course it is.

        They’re still gonna do it.

    • @[email protected]
      link
      fedilink
      English
      145 days ago

      Entry-level devs ain’t replacing anyone. One senior dev is going to be doing the work of a whole team

      • Photuris
        link
        fedilink
        English
        15 days ago

        For now.

        But when a mid-tier or entry level dev can do 60% of what a senior can do, it’ll be a great way to cut costs.

        I don’t think we’re there now. It’s just that that’s the ultimate goal - employ fewer people, and pay the remaining people you do employ less.

        • @[email protected]
          link
          fedilink
          English
          205 days ago

          But when a mid-tier or entry level dev can do 60% of what a senior can do

          This simply isn’t how software development skill levels work. You can’t give a tool to a new dev and have them do things experienced devs can do that new devs can’t. You can maybe get faster low tier output (though low tier output demands more review work from experienced devs so the utility of that is questionable). I’m sorry but you clearly don’t understand the topic you’re making these bold claims about.

          • @[email protected]
            link
            fedilink
            English
            7
            edit-2
            4 days ago

            I think more low tier output would be a disaster.

            Even pre AI I had to deal with a project where they shoved testing and compliance at juniors for a long time. What a fucking mess it was. I had to go through every commit mentioning Coverity because they had a junior fixing coverity flagged “issues”. I spent at least 2 days debugging a memory corruption crash caused by such “fix”, and then I had to spend who knows how long reviewing every such “fix”.

            And don’t get me started on tests. 200+ tests, of them none caught several regressions in handling of parameters that are shown early in the frigging how-to. Not some obscure corner case, the stuff you immediately run into if you just follow the documentation.

            With AI all the numbers would be much larger - more commits “fixing coverity issues” (and worse yet fixing “issues” that LLM sees in code), more so called “tests” that don’t actually flag any real regressions, etc.

        • @[email protected]
          link
          fedilink
          English
          16
          edit-2
          5 days ago

          half as good as expert human

          60% of what a senior can do

          is there like a character sheet somewhere so i can know where i fall on this developer spectrum

          • @[email protected]
            link
            fedilink
            English
            115 days ago

            It’s going to be your INT bonus modifier, but you can get a feat that also adds the WIS modifier

            For prolonged coding sessions you do need CON saving throws, but you can get advantage from drinking coffee (once per short rest)

            • @[email protected]
              link
              fedilink
              English
              85 days ago

              but you can get advantage from drinking coffee (once per short rest)

              I must have picked up a feat somewhere because I hit that shit way more than once per short rest

        • @[email protected]
          link
          fedilink
          English
          22
          edit-2
          5 days ago

          But when a mid-tier or entry level dev can do 60% of what a senior can do, it’ll be a great way to cut costs.

          Same as how an entry level architect can build a building 60% as tall, and that’ll last 60% as long, right?

          Edit: And an entry level aerospace engineer with AI assistance will build a plane that’s 60% as good at not crashing.

          I’m not looking forward to the world I believe is coming…

          • @[email protected]
            link
            fedilink
            English
            95 days ago

            Get 2 and the plane will be 120% as good!

            In fact if children with AI are a mere 1% as good, a school with 150 children can build 150% as good!

            I am sure this is how project management works, and if it is not maybe Elon can get Grok to claim that it is. (When not busy praising Hitler.)

            • @[email protected]
              link
              fedilink
              English
              55 days ago

              this brooks no argument and it’s clear we should immediately throw all available resources at ai so as to get infinite improvement!!~

              (I even heard some UN policy wonk spout the AGI line recently 🙄)

        • @[email protected]
          link
          fedilink
          English
          3
          edit-2
          4 days ago

          Yeah, the glorious future where every half-as-good-as-expert developer is now only 25% as good as an expert (a level of performance also known as being “completely shit at it”), but he’s writing 10x the amount of unusable shitcode.

          • Photuris
            link
            fedilink
            English
            44 days ago

            You scoff, but this is exactly the future CEOs and upper management absolutely want.

            Why? Because labor is too expensive and “entitled” (wanting things like time off, health insurance, remote work, and so on).

            They will do everything they can to squeeze labor and disempower the bargaining capability that knowledge workers have.

            Why do you think Microsoft has been trying to screengrab everything that knowledge workers do? To train their models to do that work instead. Why did they just lay off thousands of workers and direct something like $80 billion towards AI investments?

            You know why.

            Quality doesn’t matter. Income minus expenses matter. You are an expense. They will do everything they can to replace you as soon as they catch even a whiff of economic viability (or even before), because even if it’s more costly right now, it can drive down labor costs by putting the squeeze on employees.

            And that is the point.

            • @[email protected]
              link
              fedilink
              English
              3
              edit-2
              4 days ago

              Okay but that is different from the argument that entry developers only need to be half as good to deliver a working product