Cybersecurity Assessment Tools
Department of Defense funded: BAA-00-15, NSA H982301510273 and CNAP-CAE Grant# H98230-17-1-03473
NSF funded: DGE 1820531.
In the coming years, America will need to educate an increasing number of cybersecurity professionals. But how will we know if the preparation courses are effective? Presently there is no rigorous research-based method for measuring the quality of instruction. Using a panel of experts, this project has first developed quantitative rankings for two sets of fundamental concepts and topics: one appropriate for students in a first course in cybersecurity, the other appropriate for students graduating from college headed for a career in cybersecurity. Next, this project will develop assessment tools for these two student populations, to measure how well students understand these basic concepts and how well prepared they are to begin work in the field.
This project will provide infrastructure for a rigorous evidence-based improvement of cybersecurity education by developing the first Cybersecurity Assessment Tools (CATs) targeted at measuring the quality of instruction. The first CAT will be a Cybersecurity Concept Inventory (CCI) that measures how well students understand basic concepts in cybersecurity (especially adversarial thinking) after a first course in the field. The second CAT will be a Cybersecurity Curriculum Assessment (CCA) that measures how well curricula prepared students graduating from college on fundamentals needed for careers in cybersecurity. Each CAT will be a multiple-choice test with approximately thirty questions. Finally, the project will perform psychometric evaluations of these two CATs to demonstrate their quality and value.
Illinois PI: Geoffrey L. Herman
UMBC PI: Alan T. Sherman
UMBC Co-PI: Linda Oliva
UMBC Co-PI: Dhananjay Phatak
UMBC Grad Students: Travis Scheponik
Development of Concept Inventories for Computer Science
NSF Funded – Grant Number: CCLI 0618589
This project was primarily focused on creating three assessment tools for courses in computer science. These assessment tools took the form of concept inventories: multiple-choice tests that force examinees to choose between common misconceptions and accepted conceptions. The Digital Logic Concept Inventory has been developed, deployed, and validated on a national scale.
The development of the concept inventories began by interviewing students as they solved traditional homework problems. We analyzed these interviews to document students’ misconceptions and conceptual difficulties. These documented misconceptions were then used to create multiple-choice questions that assess students’ conceptual understanding.
Craig Zilles (University of Illinois)
Michael Loui (University of Illinois)
Cinda Heeren (University of Illinois)
Geoffrey Herman (University of Illinois)
Lisa Kaczmarczyk (Independent Researcher)
|Innovation in Engineering and STEM Education
||How Students Learn Engineering and Computer Science
|Designing Educational Assessment Tools
||Intrinsic Motivation Course Conversions