A statistical significance test between two models
Finding out whether the difference between two models' accuracies across cross-validation folds is statistically significant.
Goal: Computing the mean and standard deviation and normalising columns without Python loops.
import numpy as np
rng = np.random.default_rng(42)
X = rng.normal(loc=[10, 50, 100], scale=[1, 5, 20], size=(5, 3))
mean = X.mean(axis=0)
std = X.std(axis=0)
Z = (X - mean) / std # broadcasting: (5,3) - (3,)
print("Shape:", X.shape)
print("Means:", np.round(mean, 2))
print("After standardisation, each column's mean ≈ 0:", np.round(Z.mean(axis=0), 6))
print("and its standard deviation = 1:", np.round(Z.std(axis=0), 6))
We create a random array with a fixed-seed generator so the result is reproducible, then compute each column's statistics with axis=0, and normalise the data (Z-score) in a single operation thanks to broadcasting between a 5×3 array and a vector of length 3.
Shape: (5, 3) Means: [ 10.12 48.7 102.71] After standardisation, each column's mean ≈ 0: [-0. -0. -0.] and its standard deviation = 1: [1. 1. 1.]
This example was actually run and tested on 21 November 2026 on a CPU — numpy 2.5.5, python 3.13.18
Finding out whether the difference between two models' accuracies across cross-validation folds is statistically significant.
Sections
No section has this name. Press “Search” to search all site content.
Favourites
My results & certificates
My academy progressMy quiz resultsKids’ progressAcademy certificatesQuiz certificatesStart any lesson or quiz and it will show up here so you can get back to it quickly.
Settings
I am the AI Frameworks & Libraries Assistant. Describe your project or ask me about a library, a comparison or a code example, and I will answer from the encyclopedia's database with links and sources.
Answers from the encyclopedia's database only, without a language model and without sending your question to any third party.
The assistant can make mistakes, so check the attached sources.