A few years ago I designed a way to detect bit-flips in Firefox crash reports and last year we deployed an actual memory tester that runs on user machines after the browser crashes. Today I was looking at the data that comes out of these tests and now I'm 100% positive that the heuristic is sound and a lot of the crashes we see are from users with bad memory or similarly flaky hardware. Here's a few numbers to give you an idea of how large the problem is. 🧵 1/5

@gabrielesvelto Like at a minimum have it date >> ~/.mozilla/firefox/memtest_fail or something, come on!

@purpleidea well, yes, we could definitely do that, or store it so that it can be reached from about:support

@gabrielesvelto if you want some field data: I have a PC with 4 memory slots. When populated with 2x16GB cards overclocked to their XMP speed, all memory tests pass. When populated with 4x16GB cards overclocked to their XMP speed (all the exact same model), memory tests fails consistently on a very high-address bit. I had to slightly under-volt them below their XMP speed to get tests to pass. I would only ever get crashes on games and other heavy-load processes.

@gabrielesvelto my guess is that a lot of people who enable XMP never test their memory and assume it should "just work." My guess is that it's my chipset which isn't good enough to handle all that memory bandwidth, as tests fail on the same address bit regardless of the permutation of RAM boards. So I am not surprised at all that there's plenty of weird memory bugs out in the field, especially for a popular program.

@highvoltage and at that time you could still buy memory with parity! There was still an expectation that client systems would have the option of having at least a check that everything was fine