scalib.metrics.ttest

The Welch’s \(t\)-test can be used to highlight a difference in the means of two distributions. To do so, a t statistic is derived following the expression:

\[t = \frac{\mu_0 - \mu_1}{\sqrt{\frac{v_0}{n_0} + \frac{v_1}{n_1}}}\]

where \(\mu_0\) (resp. \(\mu_1\)) is the estimated moment of the first (resp.second) population and \(\frac{v_0}{n_0}\) the variance of its estimate from \(n_0\) samples. In the context of side-channel analysis, many of these statistical tests are performed independently for each point of the traces. See [1] for additional details.

In this module, the definition of \(\mu\) and \(v\) are adapted to perform higher-order univariate and multivariate \(t\)-test to compare higher-order moments of two distributions:

• Higher-order t-tests pre-process the populations by elevating the traces at the \(d\)-th power, after subtracting the mean (for \(d>1\)) and normalizing the variance (for \(d>2\)).

• Multivariate t-test are a variant of higher-order t-tests where a product of \(d\) samples of the trace is use instead of the \(d\)-th power of a single sample. Mean and variance normalization are applied similarly to higher-order t-tests.

Ttest	Univariate (higher order) \(t\)-test.
MTtest	Multivariate \(t\)-test.

Warning

Ttest should not be used alone as a standalone evaluation tool because of its qualitative nature. See [2][3] for cautionary notes.

Implementations Details

In order to enable both efficient one-core and parallelized performance of the \(t\)-test implementation, SCALib uses the one-pass formula for estimation arbitrary order statistical moment from [4] and its application to side-channel context in [1].

Concretely, the implementations first compute an estimation of the required statistical moments using a two-passes algorithms (first pass to compute the mean and the variances, and a second pass to compute the centered products). This new estimation is then used to update the current estimation using the merging rule from [4]. To enable multi-threading, SCALib internally divides the fresh traces into smaller independent chunks and then merges the output of each threads using [4].

As a conclusion, the performance of the SCALib improves if the two-passes algorithm can be used on large chunks. Hence, it is recommended to feed a large enough amount of data for every call to fit_u().

References

[1] (1,2)

Tobias Schneider and Amir Moradi. Leakage assessment methodology - extended version. J. Cryptogr. Eng., 6(2):85–99, 2016.

[2]

François-Xavier Standaert. How (not) to use welch’s t-test in side-channel security evaluations. In CARDIS, volume 11389 of Lecture Notes in Computer Science, 65–79. Springer, 2018.

[3]

Carolyn Whitnall and Elisabeth Oswald. A critical analysis of ISO 17825 (‘testing methods for the mitigation of non-invasive attack classes against cryptographic modules’). In ASIACRYPT (3), volume 11923 of Lecture Notes in Computer Science, 256–284. Springer, 2019.

[4] (1,2,3)

Philippe Pierre Pebay. Formulas for robust, one-pass parallel computation of covariances and arbitrary-order statistical moments. Technical Report, Sandia National Laboratories (SNL), Albuquerque, NM, and Livermore, CA, 2008.