U.S. flag   An unofficial archive of your favorite United States government website
Dot gov

Official websites do not use .rip
We are an unofficial archive, replace .rip by .gov in the URL to access the official website. Access our document index here.


Secure websites use HTTPS
A lock (Dot gov) or https:// means you've safely connected to our website. Please do not share sensitive information with us.

This is an archive
(replace .gov by .rip)

White Paper (Draft)

Combination Frequency Differencing

Date Published: December 6, 2021
Comments Due: February 7, 2022
Email Comments to: cfdwp@nist.gov


Richard Kuhn (NIST), M S Raunak (NIST), Raghu Kacker (NIST)


Combinatorial coverage measures have been defined and applied to a wide range of problems, including fault location and evaluating the adequacy of test inputs and input space models. More recently, methods applying coverage measures have been used in applications of artificial intelligence and machine learning for explainability and analyzing aspects of transfer learning. These methods have been developed using measures that depend on the inclusion or absence of t-tuples of values in inputs and test cases.

This paper introduces a new method related to combinatorial testing and measurement, combination frequency differencing (CFD), and illustrates the use of CFD in machine learning applications. This method is particularly well-suited to artificial intelligence and machine learning applications, where training data sets used in learning systems are dependent on the prevalence of various attributes of elements of class and non-class sets. This paper illustrates the use of this method by applying it to analyzing physical unclonable functions (PUFs) for bit combinations that have a disproportionately strong influence on PUF response bit values. Additionally, it is shown that combination frequency differences provide a simple but effective algorithm for classification problems.



combinatorial coverage; combination frequency difference; combinatorial testing; physical unclonable function (PUF); unclonable
Control Families

None selected


White Paper (DOI)

Supplemental Material:
Draft White Paper (pdf)

Document History:
12/06/21: White Paper (Draft)