U.S. flag   An unofficial archive of your favorite United States government website
Dot gov

Official websites do not use .rip
We are an unofficial archive, replace .rip by .gov in the URL to access the official website. Access our document index here.

Https

We are building a provable archive!
A lock (Dot gov) or https:// don't prove our archive is authentic, only that you securely accessed it. Note that we are working to fix that :)

This is an archive
(replace .gov by .rip)

Combinatorial Testing

Assured Autonomy and Explainable AI Papers

Our conference and journal papers on assured autonomy and explainable AI.  We try to include links to the full papers, but for those not yet linked, please contact us for a copy:  kuhn@nist.gov. 

Papers

  1. DR Kuhn, D Yaga, R Kacker, Y Lei, V Hu, Pseudo-Exhaustive Verification of Rule Based Systems, 30th Intl Conf on Software Engineering and Knowledge Engineering, July 2018.
  2. R. Kuhn, R. Kacker, An Application of Combinatorial Methods for Explainability in Artificial Intelligence and Machine LearningNIST Cybersecurity Whitepaper, May 22, 2019. 
  3. Chandrasekaran, J., Feng, H., Lei, Y., Kacker, R., & Kuhn, D. R. (2020, August). Effectiveness of dataset reduction in testing machine learning algorithms. In 2020 IEEE International Conference On Artificial Intelligence Testing (AITest) (pp. 133-140). IEEE.
  4. DR Kuhn, R Kacker, Y Lei, D Simos, "Combinatorial Methods for Explainable AI", Intl Workshop on Combinatorial Testing, Porto, Portugal, March 23-27, 2020.
  5. D.R. Kuhn, I. Dominguez, R.N. Kacker and Y. Lei. "Combinatorial Coverage Measurement Concepts and Applications", 2nd Intl Workshop on Combinatorial Testing, Luxembourg, IWCT2013, IEEE, Mar. 2013.
  6. Kuhn, R., Kacker, R. N., Lei, Y., & Simos, D. (2020). Input Space Coverage Matters. Computer, 53(1), 37-44.
  7. Kuhn, D. R., Raunak, M. S., & Kacker, R. N. (2021). Combinatorial Coverage Difference Measurement. NIST Cybersecurity Whitepaper.
  8. Chandrasekaran, J., Lei, Y., Kacker, R., & Kuhn, D. R. (2021, April). A Combinatorial Approach to Explaining Image Classifiers. In 2021 IEEE International Conference on Software Testing, Verification and Validation Workshops (ICSTW) (pp. 35-43). IEEE. 
  9. Lanus, E., Freeman, L. J., Kuhn, D. R., & Kacker, R. N. (2021, April). Combinatorial Testing Metrics for Machine Learning. In 2021 IEEE International Conference on Software Testing, Verification and Validation Workshops (ICSTW) (pp. 81-84). IEEE.
  10. Chandrasekaran, J., Lei, Y., Kacker, R., & Kuhn, D. R. (2021, April). A Combinatorial Approach to Testing Deep Neural Network-based Autonomous Driving Systems. In 2021 IEEE International Conference on Software Testing, Verification and Validation Workshops (ICSTW) (pp. 57-66). IEEE.
  11. Toohey, J. R., Raunak, M. S., & Binkley, D. (2021). From Neuron Coverage to Steering Angle: Testing Autonomous Vehicles EffectivelyComputer, 54(8), 77-85.
  12. Kampel, L., Simos, D. E., Kuhn, D. R., & Kacker, R. N. (2021). An exploration of combinatorial testing-based approaches to fault localization for explainable AIAnnals of Mathematics and Artificial Intelligence, 1-14.
  13. Raunak, M. S., & Kuhn, R. (2021). Explainable Artificial Intelligence and Machine Learning. IEEE Computer, 54(10), 25-27.

Presentations

  1. R. Kuhn, R. Kacker, Explainable AI, NIST presentation.  PDF Explainable AI   PPT Explainable AI  
  2. D R Kuhn, R Kacker, Risk, Assurance, and Explainability for Autonomous SystemsAdvancements in Test and Evaluation of Autonomous Systems (ATEAS) workshop, Dayton, OH, Oct, 2019. NIST presentation. 
  3. R. Kuhn, Assured Autonomy - Problems and Possible Solutions, Hill AFB, Aug, 2021. 

Created May 24, 2016, Updated October 12, 2021