Computer Security Resource Center

Computer Security Resource Center

Computer Security
Resource Center

This is an archive
(replace .gov by .rip)

Automated Combinatorial Testing for Software

Commercial ACTS Users by Industry

Pie Chart Graph showing Commercial ACTS Users by different Industries


User Feedback

Users have been very positive, and are applying ACTS to a wide variety of software.

  • "At present I am working with a development team to enable automated testing of a complex rating algorithm under development within a property/casualty insurance system. The system component under test accepts several hundred independent input variables which are used to produce rating factors as output. I have had some opportunity to use [NIST-ACTS] as a means to calculate test coverage and identify missing test cases in our current population of test data. I'm pleased to report that the combinatorial coverage data has proven useful to us in identifying gaps. Eventually, I would like to use this data to extend our test coverage as well -- at coverage strength 3 we're producing a volume of test cases that should be manageable when paired with automated test data generation. I can also foresee other uses which would pair automated testing tools with [NIST-ACTS] output to improve our test coverage for other systems."
  • "The tool is pretty easy to use, and has already reduced our planned number of tests by 20% or so."
  • "Nice... the XML configuration can be used for both CLI and application."
  • "Our feedback on [NIST-ACTS] is extremely positive. The tool implement in a very efficient way the algorithm of N-Wise reduction we were looking at. In fact, we would be interested in including your technology in our tool."
  • "[NIST-ACTS] is a good application. It was easy to figure out and use. ... I would use this application again."
  • "I've fund it intuitive and greatly helped to reduce the test size. I'll continue to use this tool at next opportunity."
  • "I think that [NIST-ACTS] has been very helpful in determining test coverage for simple test cases with lots of variables."
  • "Not only does [NIST-ACTS] handle phenomenal combinatorial complexity without breaking a sweat, it is also very easy and straightforward to use. The user interface is very well thought out, intuitive, and delightfully uncluttered."


Created May 24, 2016, Updated July 26, 2018