I'm getting back into the swing of things with my statistical analysis certifications. I remember there's a rule about sample size (n > 30) for using a Z-test, but does that still apply if I don't know the population standard deviation? Can I ever truly use a Z-test in real-world data science?
3 answers
In the real world, you almost never know the population standard deviation ($\sigma$), so the T-test is your best friend. The "n > 30" rule is a bit of an old-school textbook simplification. While the T-distribution converges to the Z-distribution as $n$ gets larger, modern software like Python or R will just run a T-test by default because it's more conservative. If you have a huge sample (thousands of rows), the results of a Z-test and T-test will be identical anyway. Stick to the T-test; it accounts for the extra uncertainty of estimating the standard deviation from your sample.
Are you working with proportions or means? If you are doing an A/B test on conversion rates (proportions), a Z-test is actually quite standard and very frequently used.
Basically, if you have to ask, use a T-test. It’s the safer choice for almost every scenario where you’re dealing with sample-based data in a professional setting.
I agree with Jason. There’s really no downside to using a T-test even with large samples, as it naturally adjusts to the Z-distribution as your degrees of freedom increase.
Brian, I am mostly working with average order value (means). Does the proportion rule only apply because the variance of a proportion is directly tied to the mean, making the "known $\sigma$" requirement easier to satisfy in that specific case?