Measuring the Effectiveness of U.S. Government Security Awareness Programs: A Mixed-Methods Study (Short Paper)

The goal of organizational security awareness programs is to positively influence employee security behaviors. However, organizations may struggle to determine program effectiveness, often relying on training policy compliance metrics (training completion rates) rather than measuring actual impact. Few studies have begun to discover approaches and challenges to measuring security awareness program effectiveness, particularly within compliance-focused sectors such as the U.S. government. To address this gap, we conducted a mixed-methods research study that leveraged both focus group and survey methodologies focused on U.S. government organizations. We discovered that organizations do indeed place emphasis on compliance metrics and are challenged in determining other ways to gauge success. Our results can inform guidance and other initiatives to aid organizations in measuring the effectiveness of their security awareness programs. While the research focused on the U.S. government, our findings may also have implications for other sectors and countries.