I love to show that kind of shit to AI boosters. (In case you’re wondering, the numbers were chosen randomly and the answer is incorrect).
They go waaa waaa its not a calculator, and then I can point out that it got the leading 6 digits and the last digit correct, which is a lot better than it did on the “softer” parts of the test.
Kagis Multi-Step assistant does that:
I can imagine the reason is that Wolfram alpha does calculations but also much more, so they can throw a whole bunch of shit at it.
Mistral.ai easily gets it right too
under no circumstances is “my favorite stochastic parrot got it right / I reran the prompt and it worked this time hmmmm wonder why” an interesting post