• pseudo@jlai.lu
    link
    fedilink
    arrow-up
    50
    ·
    5 days ago

    When you delegate, to a person, a tool or a process, you check the result. You make sure that the delegated tasks get done and correctly and that the results are what is expected.

    Finding that it is not the case after months by luck shows incompetence. Look for the incompetent.

    • flying_sheep@lemmy.ml
      link
      fedilink
      arrow-up
      12
      ·
      edit-2
      5 days ago

      Yeah. Trust is also a thing, like if you delegate to a person that you’ve seen getting the job done multiple times before, you won’t check as closely.

      But this person asked to verify and was told not to. Insane.

    • Tja@programming.dev
      link
      fedilink
      arrow-up
      7
      ·
      5 days ago

      100%

      Hallucinations are widely known, this is a collective failure of the whole chain of leadership.

    • jj4211@lemmy.world
      link
      fedilink
      arrow-up
      5
      ·
      5 days ago

      Problem being is that whoever is checking the result in this case had to do the work anyway, and in such a case… why bother with the LLM that can’t be trusted to pull the data anyway?

      I suppose they could take the facts and figures that a human pulled and have an LLM verbose it up for people who for whatever reason want needlessly verbose BS. Or maybe an LLM can do a review of the human generated report to help identify potential awkward writing or inconsistencies. But delegating work that you have to do anyway to double check the work seems pointless.

      • pseudo@jlai.lu
        link
        fedilink
        arrow-up
        1
        ·
        5 days ago

        Like someone here said “trust is also thing”. Once you check a few time that the process is right and the result are right, you don’t need to check more than ponctually. Unfortunatly, that’s not what happened in this story.

  • mr_sunburn@lemmy.ml
    link
    fedilink
    arrow-up
    86
    ·
    edit-2
    6 days ago

    I raised this as a concern at the corporate role I work in when an AI tool that was being distributed and encouraged for usage showed two hallucinated data points that were cited in a large group setting. I happened to know my area well, the data was not just marginally wrong but way off, and I was able to quickly check the figures. I corrected it in the room after verifying on my laptop and the reaction in the room was sort of a harmless whoops. The rest of the presentation continued without a seeming acknowledgement that the rest of the figures should be checked.

    When I approached the head of the team that constructed the tool after the meeting and shared the inaccuracies and my concerns, he told me that he’d rather have more data fluency through the ease of the tool and that inaccuracies were acceptable because of the convenience and widespread usage.

    I suspect stories like this are happening across my industry. Meanwhile, the company put out a press release about our AI efforts (literally using Gemini’s Gem tool and custom ChatGPTs seeded with Google Drive) as something investors should be very excited about.

    • squaresinger@lemmy.world
      link
      fedilink
      arrow-up
      77
      ·
      6 days ago

      When I approached the head of the team that constructed the tool after the meeting and shared the inaccuracies and my concerns, he told me that he’d rather have more data fluency through the ease of the tool and that inaccuracies were acceptable because of the convenience and widespread usage.

      “I prefer more data that’s completely made up over less data that is actually accurate.”

      This tells you everything you need to know about your company’s marketing and data analysis department and the whole corporate leadership.

      Potemkin leadership.

      • whoisearth@lemmy.ca
        link
        fedilink
        arrow-up
        26
        ·
        6 days ago

        Honestly this is not a new problem and is a further expression of the larger problem.

        “Leadership” becomes removed from the day to day operations that run the organization and by nature the “cream” that rises tend to be sycophantic in nature. Our internal biases at work so it’s no fault of the individual.

        Humanity is their own worst enemy lol

        • squaresinger@lemmy.world
          link
          fedilink
          arrow-up
          16
          ·
          6 days ago

          It is not a new problem and that has been the case for a long time. But it’s a good visualization of it.

          Everyone in a company has their own goals, from the lowly actual worker who just wants to pay the bills and spend as little effort on it as possible, to departments which want to justify their useless existence, to leadership who mainly wants to look good towards the investors to get a nice bonus.

          That some companies end up actually making products that ship and that people want to use is more of an unintended side effect than the intended purpose of anyone’s work.

      • sp3ctr4l@lemmy.dbzer0.com
        link
        fedilink
        English
        arrow-up
        10
        ·
        edit-2
        6 days ago

        You’re thinking like a person who values accurate information more than feeling some kind of ‘cool’ and ‘trendy’ because now you can vibe code and we are a forward thinking company that embraces new paradigms and synergizes our expectations with the potential reality our market disprupting innovations could bring.

        … sorry, I lapsed back into corpo / had a stroke.

    • chiliedogg@lemmy.world
      link
      fedilink
      arrow-up
      13
      arrow-down
      2
      ·
      6 days ago

      The board room is more concerned with the presentation than the data, because presentations make sales.

      What a lot of people fail.to understand is that for the C-Suite, the product isn’t what’s being manufactured, or the service being sold. The product is the stock, and anything that makes the number go up in the short term is good.

      Lots of them have fiduciary duties, meaning they’re legally prohibited from doing anything that doesn’t maximize the value of the stock from moment to moment.

      • Sprocketfree@sh.itjust.works
        link
        fedilink
        English
        arrow-up
        18
        ·
        6 days ago

        Someone please show me the criminal lawsuit against the CEO that made the moral decision and the stock went down! I’m so sick of the term fiduciary duty being used as a bullshit shield for bad behavior. When Tesla stock tanked because musk threw a Nazi salute, where were the fiduciary duty people!?

      • jj4211@lemmy.world
        link
        fedilink
        arrow-up
        9
        ·
        6 days ago

        Further, as you hinted, long term is not their problem. They get a bump, cash in a few million dollars worth of RSUs, and either saddle the next guy with the fallout, it of they haven’t left yet “whoopsie, but I can blame the LLM and I was just following best practices in the industry at the time”. Either way they have enough to not even pretend to work another day of their life, even ignoring previous grifts, and they’ll go on and do the same thing to some other company when they bail or the company falls over.

        • cogman@lemmy.world
          link
          fedilink
          arrow-up
          9
          ·
          6 days ago

          At the moment, nothing will be done. There’s no way the current SEC chair will give a fuck about this sort of stuff.

          But assuming a competent chair ever gets in charge, I expect there to be a shit show of lawsuits. It really doesn’t matter that “the LLM did it” lying on those mandatory reports can lead to big fines.

      • aesthelete@lemmy.world
        link
        fedilink
        arrow-up
        8
        ·
        edit-2
        6 days ago

        Lots of them have fiduciary duties, meaning they’re legally prohibited from doing anything that doesn’t maximize the value of the stock from moment to moment.

        Overall, I agree with you that stock price is their motivation, but the notion of shareholder supremacy binding their hands and preventing them from doing things that they want to otherwise do is incorrect. For one, they aren’t actually mandated to do this by law, and secondarily, even if they were – which to reiterate, they aren’t – just about any action they take on any single issue can be portrayed as them attempting to maximize company value.

        https://pluralistic.net/2024/09/18/falsifiability/#figleaves-not-rubrics

          • aesthelete@lemmy.world
            link
            fedilink
            arrow-up
            5
            ·
            edit-2
            5 days ago

            Sure, but since it’s an unfalsifiable proposition, good luck proving it in court for any specific action.

          • WoodScientist@lemmy.world
            link
            fedilink
            arrow-up
            2
            ·
            5 days ago

            Not really, no. This is mostly a myth. Unless the executives are deliberately causing the company to lose money, they really can’t be sued based on this fiduciary duty to shareholders. They have to act in the shareholders’ best interest, but “shareholder interest” is entirely up to interpretation. For example, it’s perfectly fine to say, “we’re going to lose money over the next five years because we believe it will ensure maximum profits over the long term.” In order to sue a CEO for failing to protect shareholders, they would have to be doing something deliberately and undeniably against shareholder interest. Like if they embezzle money into their own bank account, or if they hold a Joker-style literal money burning.

            If it were that easy to sue executives for violating their fiduciary duty to shareholders, golden parachutes and inflated executive compensation packages wouldn’t exist. But good luck suing a CEO because he’s paid too much. He can just claim in court that his compensation will ensure the company attracts the best talent to perform the best they can.

            Executives are given wide latitude in how they define the best financial interest of shareholders. Shareholders ultimately do have the ability to remove executives from their positions. This is supposed to be the default way of dealing with incompetent executives. As shareholders already possess the ability to fire a CEO at any time, there is a very high bar to clear before shareholders can also sue executives. It’s generally assumed if they really are doing that bad a job, you should just fire them.

            • Snowclone@lemmy.world
              link
              fedilink
              arrow-up
              1
              ·
              4 days ago

              Yes, that’s correct, it’s not an issue of legal liability, it’s an issue of their interests converging. the CEO holds stock, he is a stock holder, and the execs are stock holders, they don’t need any motivation to put the stocks first, they know where their interests converge, and precious few executives make more money in salary than they take away in stocks, in practicality, every corporation that offers stocks is stock focused, the is why we had daily and weekly meetings in retail stores on the absolute bottom of the ladder to talk primarily about stock prices, and why the main information displayed on price guns is the sale price/cost/ current quarter sales/last quarter sales/and last year to date quarter sales, and the sales numbers daily/monthly/quarterly are what you see posted around the office, it’s always about beating last year to date numbers, and last quarter’s numbers, and what always drove me fucking nuts is that the store made TWENTY TWO MILLION FUCKING DOLLARS in profit, but "your store is failing because you didn’t make twenty two million and one penny. they don’t care that we were making money hand over fist, because that’s not the game. that game is dead. don’t worry. they’ll still cut payroll, and you can’t like… spend that money or keep that money, but it doesn’t matter. it only maters if it makes the stocks move. it’s stocks all the way down. because that’s where the interests converge. also as a side note, golden parachutes are an internal security measure against hostile take over, it means if someone does successfully raid your business and performs a hostile takeover, they have to pay your executives staff when they fire them and loot the company more money than the company could be looted for. it’s never actually intended to be paid out.

      • Snowclone@lemmy.world
        link
        fedilink
        arrow-up
        8
        arrow-down
        1
        ·
        6 days ago

        it’s why capitalism is over. they do not care about making a profit at all. they only care about the stocks. there is only one outcome to this approach, and that’s dissolving the company slowly until it fails because your willing to saw your legs off for a small spike in quarterly earning. You eventually run out of legs to saw off.

    • cub Gucci@lemmy.today
      link
      fedilink
      arrow-up
      7
      ·
      6 days ago

      Coming from science to industry taught me one thing: numbers (and rationality as a whole) serves only one goal. And the goal is to persuade the opponents: colleagues, investors, regulators.

      In this broken sense, your head of the team is right: hallucinations are acceptable if supervisors believe the output.

    • Buddahriffic@lemmy.world
      link
      fedilink
      arrow-up
      6
      ·
      6 days ago

      Sounds like the people who are realistic about AI are going to end up having a huge advantage over people who use it naively.

      Like with statistics, there are a lot of tools out there that can handle them perfectly accurately, you just don’t want an LLM doing “analysis” because the NN isn’t encoded for that. Consider how often our own NNs get addicted to gambling while not being fully specialized for processing language. An LLM might not get caught up in a gambler’s fallacy, but that’s more on account of being too simple than being smarter.

      I wonder if this will break the trust in MBAs because LLMs are deceptively incompetent and from the sound of this comment and other things I’ve seen, that deception works well enough that their ego around being involved in the tool’s development clashes with the experts telling them it’s not as useful as it seems.

    • ScoffingLizard@lemmy.dbzer0.com
      link
      fedilink
      arrow-up
      1
      ·
      6 days ago

      You should have ask what would happen if the figures were wrong, let them make an excuse and then eat shit later. AI is taking our jobs. Never interrupt an enemy making a mistake.

  • MuteDog@lemmy.world
    link
    fedilink
    arrow-up
    21
    arrow-down
    1
    ·
    5 days ago

    Apparently that reddit post itself was generated with AI. Using AI to bash AI is an interesting flex.

    • Crozekiel@lemmy.zip
      link
      fedilink
      English
      arrow-up
      4
      ·
      4 days ago

      Have any evidence of that? The only thing I saw was commentors in that thread (who were obvious AI-bros) claiming it must be AI generated because “it just wouldn’t happen”…

  • Anna@lemmy.ml
    link
    fedilink
    arrow-up
    66
    arrow-down
    1
    ·
    6 days ago

    I work in a regulated sector and our higher ups are pushing AI so much. And there response to AI hallucinations is to just put a banner on all internal AI tools to cross verify and have some quarterly stupid “trainings” but almost everyone I know never checks and verifies the output. And I know of atleast 2 instances where because AI hallucinated some numbers we sent out extra money to a third party.

    • wizardbeard@lemmy.dbzer0.com
      link
      fedilink
      English
      arrow-up
      32
      ·
      6 days ago

      My workplace (finance company) bought out an investments company for a steal because they were having legal troubles, managed to pin it on a few individuals, then fired the individuals under scrutiny.

      Our leadership thought the income and amount of assets they controlled was worth the risk.

      This new group has been the biggest pain in the ass. Complete refusal to actually fold into the company culture, standards, even IT coverage. Kept trying to sidestep even basic stuff like returning old laptops after upgrades.

      When I was still tech support, I had two particularly fun interactions with them. One was when it was discovered that one of their top earners got fired for shady shit, then they discovered a month later that he had set his mailbox to autoreply to every email pointing his former clients to his personal email. Then, they hired back this guy and he lasted a whole day before they caught him trying to steal as much private company info as he could grab. The other incident was when I got a call from this poor intern they hired, then dumped the responsibility for this awful home grown mess of Microsoft Access, Excel, and Word docs all linked over ODBC on this kid. Our side of IT refused to support it and kept asking them to meet with project management and our internal developers to get it brought up into this century. They refused to let us help them.

      In the back half of last year, our circus of an Infosec Department finally locked down access to unapproved LLMs and AI tools. Officially we had been restricted to one specific one by written policy, signed by all employees, for over a year but it took someone getting caught by their coworker putting private info into a free public chatbot for them to enforce it.

      Guess what sub-company is hundreds of thousands of dollars into a shadow IT project that has went through literally none of the proper channels to start using an explicitly disallowed LLM to process private customer data?

      • ChickenLadyLovesLife@lemmy.world
        link
        fedilink
        English
        arrow-up
        11
        ·
        5 days ago

        My last job was with a very large west coast tech giant (its name is a homonym with an equally-large food services company). The mandatory information security training was a series of animated shorts featuring talking bears which you could fast-forward through and still get credit for completing. Not surprisingly, we had major data thefts every few months – or more accurately we admitted to major data thefts that often.

    • SocialMediaRefugee@lemmy.world
      link
      fedilink
      arrow-up
      10
      ·
      edit-2
      5 days ago

      It reminds me of when the internet exploded in the 90s and everyone “needed” a website. Even my corner gas station had a web presence for some reason. Then with smartphones everyone needed their own app. Now with AI everyone MUST use AI everywhere! If you don’t you are a fool and going to get left behind! Do you know what you actually need it for? Not really but some article you read said you could fire 50% of your staff if you do.

      • NannerBanner@literature.cafe
        link
        fedilink
        arrow-up
        12
        ·
        5 days ago

        I would quite honestly prefer every place to have their own web site instead of the ginormous amount of places that have facebook pages.

    • jj4211@lemmy.world
      link
      fedilink
      arrow-up
      2
      ·
      5 days ago

      If they have to verify the results every time, what is the point?

      have some quarterly stupid “trainings”

      Feeling this in my bones, executive just sent out a plan for ‘fixing’ the fact that the AI tools they are paying for us to use are getting roasted for sucking, they are giving the vendor more money to provide 200 hours of mandatory training for us to take. That’s more training than they have required for anything before, and using LLM tools isn’t exactly a difficulty problem.

  • stoy@lemmy.zip
    link
    fedilink
    English
    arrow-up
    95
    arrow-down
    3
    ·
    6 days ago

    I suspect this will happen all over with in a few years, AI was good enough at first, but over time reality and the AI started drifting apart

    • Kirp123@lemmy.world
      link
      fedilink
      arrow-up
      89
      arrow-down
      1
      ·
      6 days ago

      AI is literally trained to get the right answer but not actually perform the steps to get to the answer. It’s like those people that trained dogs to carry explosives and run under tanks, they thought they were doing great until the first battle they used them in they realized that the dogs would run under their own tanks instead of the enemy ones, because that’s what they were trained with.

      • merc@sh.itjust.works
        link
        fedilink
        arrow-up
        4
        ·
        5 days ago

        It’s not trained to get the right answer. It’s trained to know what sequence of words tends to follow another sequence of words, and then a little noise is added to that function to make it a bit creative. So, if you ask it to make a legal document, it has been trained on millions of legal documents, so it knows exactly what sequences of words are likely. But, it has no concept of whether or not those words are “correct”. It’s basically making a movie prop legal document that will look really good on camera, but should never be taken into court.

    • jj4211@lemmy.world
      link
      fedilink
      arrow-up
      26
      ·
      6 days ago

      They haven’t drifted apart, they were never close in the first place. People have been increasingly confident in the models because they’ve increasingly sounded more convincing, but the tenuous connection to reality has been consistently off.

      • jballs@sh.itjust.works
        link
        fedilink
        English
        arrow-up
        4
        ·
        5 days ago

        Yeah it’s not even drift. It’s just smoke and mirrors that looks convincing if you don’t know what you’re talking about. It’s why you see writers say “AI is great at coding, but not writing” and then you see coders say “AI is great at writing, but not coding.”

        If you have any idea what good looks like, you can immediately recognize AI ain’t it.

        For a fun example, at my company we had a POC done by a very well known AI company. It was supposed to analyze a MS Project schedule, then compare tasks in that schedule to various data sources related to to tasks, then flag potential schedule risks. In the demo to the COO, they showed the AI look at a project schedule and say “Task XYZ could be at risk due to vendor quality issues or potential supply chain issues.”

        The COO was amazed. Wow it looked through all this data and came back with such great insight. Later I dug under the hood and found that it wasn’t looking at any data behind the scenes at all. It was just answering specifically “what could make a project task at risk?” and then giving a hypothetical answer.

        Anyone using AI to make any sort of decision is basically doing the equivalent of Googling your issue and then taking the top response as gospel. Yeah that might work for a few basic things, but anything important that requires any thought whatsoever is going to fail spectacularly.

        • jj4211@lemmy.world
          link
          fedilink
          arrow-up
          5
          ·
          5 days ago

          It’s why you see writers say “AI is great at coding, but not writing” and then you see coders say “AI is great at writing, but not coding.”

          I’ve always thought of this as being just like Hollywood. If you have expertise in whatever field they present an expert in, it’s painful how off they are but it lookks fine for everyone outside the field of expertise.

        • merc@sh.itjust.works
          link
          fedilink
          arrow-up
          2
          ·
          5 days ago

          It was just answering specifically “what could make a project task at risk?” and then giving a hypothetical answer

          It wasn’t even doing that. It was “looking” at training data for what a an analysis like that might look like, and then inventing a sequence of words that matched that training data. Maybe “vendor quality issues” is something that appears in the training data, so it’s a good thing to put in its output.

      • Rothe@piefed.social
        link
        fedilink
        English
        arrow-up
        4
        arrow-down
        3
        ·
        6 days ago

        It is a thing that is happening, but the OP instance probably didn’t, since it is just a reddit post.

    • rozodru@piefed.world
      link
      fedilink
      English
      arrow-up
      29
      ·
      6 days ago

      As someone who has to deal with LLMs/AI daily in my work in order to fix the messes they create, this tracks.

      AI’s sole purpose is to provide you a positive solution. That’s it. Now that positive solution doesn’t even need to be accurate or even exist. It’s built to provide a positive “right” solution without taking the steps to get to that “right” solution thus the majority of the time that solution is going to be a hallucination.

      you see it all the time. you can ask it something tech related and in order to get to that positive right solution it’ll hallucinate libraries that don’t exist, or programs that don’t even do what it claims they do. Because logically to the LLM this is the positive right solution WITHOUT utilizing any steps to confirm that this solution even exists.

      So in the case of OPs post I can see it happening. They told the LLM they wanted analytics for 3 months and rather than take the steps to get to an accurate solution it ignored said steps and decided to provide positive solution.

      Don’t use AI/LLMs for your day to day problem solving. you’re wasting your time. OpenAI, Anthropic, Google, etc have all programmed these things to provide you with “positive” solutions so you’ll keep using them. they just hope you’re not savvy enough to call out their LLM’s when they’re clearly and frequently wrong.

      • jj4211@lemmy.world
        link
        fedilink
        arrow-up
        22
        ·
        edit-2
        6 days ago

        Probably the skepticism is around someone actually trusting the LLM this hard rather than the LLM doing it this badly. To that I will add that based on my experience with LLM enthusiasts, I believe that too.

        I have talked to multiple people who recognize the hallucination problem, but think they have solved it because they are good “prompt engineers”. They always include a sentence like “Do not hallucinate” and thinks that works.

        The gaslighting from the LLM companies is really bad.

          • wizardbeard@lemmy.dbzer0.com
            link
            fedilink
            English
            arrow-up
            4
            arrow-down
            1
            ·
            edit-2
            5 days ago

            There are ways to get more relevant info (when using terms that have different meanings based on context), to reduce the needless ass kissing, and to help ensure you get response in formats more useful to you. But being able to provide it context is not some magic fix for the underlying problems of the way this tech is constructed and its limitations. It will never be trustworthy.

            Edit: God forbid anyone want our criticism to be based of an understanding of this shit rather than pure vitriol and hot takes.

    • fizzle@quokk.au
      link
      fedilink
      English
      arrow-up
      10
      ·
      6 days ago

      Yeah.

      Kinda surprised there isn’t already a term for submitting / presenting AI slop without reviewing and confirming.

    • joostjakob@lemmy.world
      link
      fedilink
      arrow-up
      7
      ·
      6 days ago

      Having worked in departments providing data all my career, I’m not surprised in the slightest that people do not care in any way about where the numbers they got come from.

      • wizardbeard@lemmy.dbzer0.com
        link
        fedilink
        English
        arrow-up
        3
        ·
        5 days ago

        Base line level of trust in co-worker comperence combined with either too much workload to give everything a fine toothed comb through, or too much laziness to bother.

        Presented by F, slide deck created by E, based off conclusions made by D, from data formatted to look good to them by C, from work that they asked B to do, which was ultimately done by low man on the totem pole A.

        All it takes is for one person in that chain to be considered trustworthy for every level above it to consider it trustworthy info by default.

  • db_null@lemmy.dbzer0.com
    link
    fedilink
    arrow-up
    17
    ·
    5 days ago

    I guarantee you this is how several, if not most, fortune 500 companies currently operate. The 50k DOW is not just propped up by the circlejerk spending on imaginary RAM. There are bullshit reports being generated and presented every day.

    I patiently wait. There is a diligent bureaucrat sitting somewhere going through fiscal reports line by line. It won’t add up… receipts will be requested… bubble goes pop

  • sp3ctr4l@lemmy.dbzer0.com
    link
    fedilink
    English
    arrow-up
    54
    ·
    edit-2
    6 days ago

    As an unemployed data analyst / econometrician:

    lol, rofl, perhaps even… lmao.

    Nah though, its really fine, my quality of life is enormously superior barely surviving off of SSDI and not having to explain data analytics to thumb sucking morons (VPs, 90% of other team leads), and either fix or cover all their mistakes.

    Yeah, sure, just have the AI do it, go nuts.

    I am enjoying my unexpected early retirement.

  • Bubbaonthebeach@lemmy.ca
    link
    fedilink
    English
    arrow-up
    36
    ·
    5 days ago

    To everyone I’ve talked to about AI, I’ve suggested a test. Take a subject that they know they are an expert at. Then ask AI questions that they already know the answers to. See what percentage AI gets right, if any. Often they find that plausible sounding answers are produced however, if you know the subject, you know that it isn’t quite fact that is produced. A recovery from an injury might be listed as 3 weeks when it is average 6-8 or similar. Someone who did not already know the correct information, could be damaged by the “guessed” response of AI. AI can have uses but it needs to be heavily scrutinized before passing on anything it generates. If you are good at something, that usually means you have to waste time in order to use AI.

    • NABDad@lemmy.world
      link
      fedilink
      English
      arrow-up
      16
      ·
      5 days ago

      I had a very simple script. All it does is trigger an action on a monthly schedule.

      I passed the script to Copilot to review.

      It caught some typos. It also said the logic of the script was flawed and it wouldn’t work as intended.

      I didn’t need it to check the logic of the script. I knew the logic was sound because it was a port of a script I was already using. I asked because I was curious about what it would say.

      After restating the prompt several times, I was able to get it to confirm that the logic was not flawed, but the process did not inspire any confidence in Copilot’s abilities.

    • laranis@lemmy.zip
      link
      fedilink
      arrow-up
      8
      ·
      5 days ago

      Happy cake day, and this absolutely. I figured out its game the first time I asked it a spec for an automotive project I was working on. I asked it the torque specs for some head bolts and it gave me the wrong answer. But not just the wrong number, the wrong procedure altogether. Modern engines have torque to yield specs, meaning essentially you torque them to a number and then add additional rotation to permanently distort the threads to lock it in. This car was absolutely not that and when I explained back to it the error it had made IT DID IT AGAIN. It sounded very plausible but someone following those directions would have likely ruined the engine.

      So, yeah, test it and see how dumb it really is.

    • Tja@programming.dev
      link
      fedilink
      arrow-up
      1
      ·
      5 days ago

      Do the same to any person online, most blogs by experts, or journalists.

      Even apparently easy to find data, like the specs of a car. Sucking and lying is not exclusive to LLMs.

    • GalacticSushi@lemmy.blahaj.zone
      link
      fedilink
      arrow-up
      26
      ·
      5 days ago

      Bro, just give us a few trillion dollars, bro. I swear bro. It’ll be AGI this time next year, bro. We’re so close, bro. I just need need some money, bro. Some money and some god-damned faith, bro.

      • vaderaj@lemmy.world
        link
        fedilink
        arrow-up
        5
        ·
        edit-2
        5 days ago

        User: Hi big corp AI(LLM), do this task

        Big Corp AI: Here is output

        User: Hi big corp your AI’s output is not up to standard I guess it’s a waste of…

        Big Corp: use this agent which ensures correct output (for more energy)

        User: it still doesn’t work…guess I was wrong all along let me retry…

        And the loop continues until they get a few trillion dollars

    • rumba@lemmy.zip
      link
      fedilink
      English
      arrow-up
      8
      ·
      5 days ago

      You can make something AI based that does this, but it’s not cheap or easy. You have to make agents that handle data retrieval and programmatically make the LLM to chose the right agent. We set one up at work, it took months. If it can’t find the data with a high certainty, it tells you to ask the analytics dept.

  • tover153@lemmy.world
    link
    fedilink
    arrow-up
    47
    arrow-down
    1
    ·
    6 days ago

    Before anything else: whether the specific story in the linked post is literally true doesn’t actually matter. The following observation about AI holds either way. If this example were wrong, ten others just like it would still make the same point.

    What keeps jumping out at me in these AI threads is how consistently the conversation skips over the real constraint.

    We keep hearing that AI will “increase productivity” or “accelerate thinking.” But in most large organizations, thinking is not the scarce resource. Permission to think is. Demand for thought is. The bottleneck was never how fast someone could draft an email or summarize a document. It was whether anyone actually wanted a careful answer in the first place.

    A lot of companies mistook faster output for more value. They ran a pilot, saw emails go out quicker, reports get longer, slide decks look more polished, and assumed that meant something important had been solved. But scaling speed only helps if the organization needs more thinking. Most don’t. They already operate at the minimum level of reflection they’re willing to tolerate.

    So what AI mostly does in practice is amplify performative cognition. It makes things look smarter without requiring anyone to be smarter. You get confident prose, plausible explanations, and lots of words where a short “yes,” “no,” or “we don’t know yet” would have been more honest and cheaper.

    That’s why so many deployments feel disappointing once the novelty wears off. The technology didn’t fail. The assumption did. If an institution doesn’t value judgment, uncertainty, or dissent, no amount of machine assistance will conjure those qualities into existence. You can’t automate curiosity into a system that actively suppresses it.

    Which leaves us with a technology in search of a problem that isn’t already constrained elsewhere. It’s very good at accelerating surfaces. It’s much less effective at deepening decisions, because depth was never in demand.

    If you’re interested, I write more about this here: https://tover153.substack.com/

    Not selling anything. Just thinking out loud, slowly, while that’s still allowed.

    • plenipotentprotogod@lemmy.world
      link
      fedilink
      arrow-up
      8
      arrow-down
      1
      ·
      5 days ago

      Very well put. This is a dimension to the ongoing AI nonsense that I haven’t seen brought up before, but it certainly rings true. May I say also that “They already operate at the minimum level of reflection that they’re willing to tolerate.” Is a hell of a sentence and I’m a little jealous that I didn’t come up with it.

      • tover153@lemmy.world
        link
        fedilink
        arrow-up
        7
        arrow-down
        1
        ·
        5 days ago

        Thanks, I really appreciate that. I’ve been getting a little grief this weekend because some of my posts are adapted from essays I’ve been working on for Substack, and apparently careful editing now makes you suspect as an actual person.

        I’m very real, just flu-ridden and overthinking in public. Glad the line landed for you.

  • mudkip@lemdro.id
    link
    fedilink
    English
    arrow-up
    21
    ·
    5 days ago

    Ah yes, what a surprise. The random word generator gave you random numbers that aren’t actually real.

  • Jankatarch@lemmy.world
    link
    fedilink
    arrow-up
    19
    ·
    5 days ago

    Tbf at this point corporate economy is made up anyway so as long as investors are gambling their endless generational wealth does it matter?

    • wabasso@lemmy.ca
      link
      fedilink
      English
      arrow-up
      9
      ·
      5 days ago

      This is how I’m starting to see it too. Stock market is just the gambling statistics of the ownership class. Line goes down and we’re supposed to pretend it’s harder to grow food and build houses all of a sudden.

      • jj4211@lemmy.world
        link
        fedilink
        arrow-up
        6
        ·
        5 days ago

        There’s a difference. If I go and gamble away my life savings, then I’m on the street. If they gamble away their investments, the government will say ‘poor thing’ and give them money to keep the economy ok.

    • sp3ctr4l@lemmy.dbzer0.com
      link
      fedilink
      English
      arrow-up
      10
      ·
      edit-2
      6 days ago

      I have been saying for years now that the kind of work that LLMs are best suited for replacing and also would by far be their most cost effective use case from a business stand point is…

      Well its the most expensive employees who basically just spend most of their time having meetings or writing emails about things they only understand at a very birds eye view level.

      You know, C Suite, upper management.