It looks like Nintendo have finished testing every game.
Summary:
| category | count | percentage |
|---|---|---|
| cannot be used | 6 | ≈ 0.04% |
| issues that prevent progress or startup issues | 162 | ≈ 1.07% |
| issues that have been resolved, or are planned to be | 185 | ≈ 1.22% |
| require JoyCon 1 | 10 | ≈ 0.07% |
| no known issues | ≈ 14759 | ≈ 97.60% |
| total | ≈15122 | 100% |
I was skeptical about how complete this effort would be, but I’ll admit when I’m wrong. I’ve got to say that Nintendo has impressed me with their thoroughness and the fact that they completed this audit by launch. It must have been an enormous undertaking.
15K games is a lot to test thoroughly, for sure.
I’m sure they have some kind of CI/CD pipeline that can run any game through a gamut of automated tests. They let it rip on a development switch or SDK that has access to the whole software repo and spit out the results in a CSV, with human investigation on games that flag negatively.
Hah. Sure. So does everybody else, even the games that launch on absolute fire.
Some of the stuff in the broken/to be corrected file is fairly intricate. They almost certainly didn’t flag a handful of corrupted textures or a crash in later levels in a game via that process.
Best guess they got all of these tested manually to some degree. There’s a decent chance that they got in touch with teams who had dev kits and asked them to self-report issues, too, but who knows how much testing was in-house and how much contributed by third party publishers. Either way it’s a TON of games to go through, even to hook for automation.