Science of Human Circumvention of Security

Investigators: Tao Xie, Jim Blythe, Ross Koppel, and Sean Smith

Well-intentioned human users continually circumvent security controls. The pandemic/ubiquitous fact of this circumvention undermines the effectiveness of security designs that implicitly assume circumvention never happens. We seek to develop metrics to enable security engineers and other stakeholders to make meaningful, quantifiable comparisons, decisions, and evaluations of proposed security controls in light of what really happens when these controls are deployed.

This project builds on foundations of human-computer-interface in security and the preliminary research the investigators have been working on already: Blythe, Koppel, and Smith, studying workers’ reasons for and methods of circumvention along with Xie, studying techniques for assisting mobile-app users (who can be enterprise workers) to conduct security controls on apps to be installed on their mobile devices. Research conducted in large enterprise systems increasingly finds that such apps are a major source of malware invasions into those larger systems. Similarly, with the expanded use of BYOD (bring your own device), such dangers are pandemic without security controls and without users’ ability to understand and follow those controls. Security-control circumvention by enterprise workers as mobile app users is reflected by their acceptance to install apps without sufficiently assessing their risk.

Hard Problems Addressed

Publications

  1. Jim Blythe, Ross Koppel, and Sean W. Smith, “Circumvention of Security: Good Users Do Bad Things”, IEEE Security & Privacy, volume 11, issue 5, September – October 2013. [full text]
  2. Vijay Kothari, Jim Blythe, Sean W. Smith, and Ross Koppel, “Agent-Based Modeling of User Circumvention of Security”, 1st International Workshop on Agents and CyberSecurity (ACySE), Paris, France, May 5, 2014. [full text]
  3. Jim Blythe, Ross Koppel, Vijay Kothari and Sean Smith, “Ethnography of Computer Security Evasions in Healthcare Settings: Circumvention as the Norm”, 2014 USENIX Summit on Health Information Technologies, San Diego, CA, August 19, 2014. [video]
  4. Ross Koppel, Sean Smith, James Blythe, and Vijay Kothari, “Workarounds to Computer Access in Healthcare Organizations: You Want My Password or a Dead Patient?” Information Technology and Communications in Health (ITCH 2015), February – March 2015. [full text]
  5. Ross Koppel, Sean Smith, Jim Blythe and Vijay Kothari, “Workarounds to Computer Access in Healthcare Organizations: You Want My Password or a Dead Patient?” Driving Quality in Informatics: Fulfilling the Promise, Series on Technology and Informatics, volume 208, February – March 2015. [link]
  6. Sean Smith, Ross Koppel, Jim Blythe and Vijay Kothari, “Mismorphism: A Semiotic Model of Computer Security Circumvention”, Technical Report TR2015-768, Dartmouth College, March 2015. [full text]
  7. Vijay Kothari, Jim Blythe, Sean Smith and Ross Koppel, “Measuring the Security Impacts of Password Policies Using Cognitive Behavioral Agent Based Modeling”, Symposium and Bootcamp on the Science of Security (HotSoS), Urbana, IL, April 21-22, 2015. [full text]
  8. Sean Smith, Ross Koppel, Jim Blythe and Vijay Kothari, “Mismorphism: A Semiotic Model of Computer Security Circumvention”, Symposium and Bootcamp on the Science of Security (HotSoS), Urbana, IL, April 21-22, 2015. [full text]
  9. T. Xie, J. Bishop, N. TIllmann and J. de Halleux, “Gamifying Software Security Education and Training via Secure Coding Duels in Code Hunt”, Symposium and Bootcamp on the Science of Security (HotSoS), Urbana, IL, April 21-22, 2015. [full text]
  10. Wei Yang, Xusheng Xiao, Benjamin Andow, Sihan Li, Tao Xie, and William Enck, “AppContext: Differentiating Malicious and Benign Mobile App Behavior Under Context”, 37th International Conference on Software Engineering (ICSE 2015), Florence, Italy, May 16-24, 2015. [full text]
  11. Sean Smith, Ross Koppel, Jim Blythe, and Vijay Kothari, “Mismorphism: A Semiotic Model of Computer Security and Circumvention”, 9th International Symposium on Human Aspects of Information Security and Assurance (HAISA 2015), Levos, Greece, July 1-3, 2015.
  12. Xusheng Xiao, Nikolai Tillmann, Manuel Fahndrich, Jonathan de Halleux, Michal Moskal, and Tao Xie, “User-Aware Privacy Control via Extended Static-Information-Flow Analysis”, Automated Software Engineering Journal, volume 22, issue 3, September 2015. [full text]
  13. Huoran Li, Xuanzhe Liu, Tao Xie, Kaigui Bian, Xuan Lu, Felix Xiaozhu Lin, Qiaozhu Mei, and Feng Feng, “Characterizing Smartphone Usage Patterns from Millions of Android Users”, 2015 Internet Measurement Conference (IMC 2015), Tokyo, Japan, October 28-30, 2015. [full text]
  14. Harold Thimbleby and Ross Koppel, “The Healthtech Declaration”, IEEE Security and Privacy, volume 13, issue 6, pages 82-84, November/December 2015. [full text]
  15. Sihan Li, Xusheng Xiao, Blake Bassett, Tao Xie, and Nikolai Tillmann, “Measuring Code Behavioral Similarity for Programming and Software Engineering Education”, 38th International Conference on Software Engineering (ICSE 2016), Software Education and Training track, Austin, TX, May 14-22, 2016. [full text]
  16. Benjamin Andow, Adwait Nadkarni, Blake Bassett, William Enck, and Tao Xie, “A Study of Grayware on Google Play”, Workshop on Mobile Security Technologies (MoST 2016), held in conjunction with IEEE Symposium on Security and Privacy, San Jose, CA, May 26, 2016. [full text]
  17. Ross Koppel, Jim Blythe, Vijay Kothari, and Sean Smith, “Beliefs about Cybersecurity Rules and Passwords: A Comparison of Two Survey Samples of Cybersecurity Professionals Versus Regular Users”, 12th Symposium On Usable Privacy and Security (SOUPS 2016), Denver, CO, June 22-24, 2016. [full text]
  18. Pierre McCauley, Brandon Nsiah-Ababio, Joshua Reed, Faramola Isiaka, and Tao Xie, “Preliminary Analysis of Code Hunt Data Set from a Contest”, 2nd International Code Hunt Workshop on Educational Software Engineering (CHESE 2016), Seattle, WA, November 14, 2016. [full text]
  19. Xia Zeng, Dengfeng Li, Wuijie Zheng, Fan Xia, Yuetang Deng, Wing Lam, and Tao Xie, “Automated Test Input Generation for Android: Are We Really There Yet in an Industrial Case?”, 24th ACM SIGSOFT International Symposium on the Foundations of Software Engineering (FSE 2016), Seattle, WA, November 13-18, 2016. [full text]
  20. Ross Koppel, Vijay Kothari, Sean W. Smith, and Jim Blythe, “Beyond Pleading With or Restricting Users to Achieve Cyber Security Goals: Approaches to Understanding and Responding to Circumvention”, CRA CCC Sociotechnical Cybersecurity Workshop, College Park, MD, December 12-13, 2016. [full text]
  21. Sean W. Smith, Vijay Kothari, Jim Blythe, and Ross Koppel, “Flawed Mental Models Lead to Bad Cyber Security Decisions: Let’s Do a Better Job”, CRA CCC Sociotechnical Cybersecurity Workshop, College Park, MD, December 12-13, 2016. [full text]
  22. Ross Koppel, Jim Blythe, Vijay Kothari, and Sean Smith, “Password Logbooks and What Their Amazon Reviews Reveal About the Users’ Motivations, Beliefs, and Behaviors”, 2nd European Workshop on Useable Security (EuroUSEC 2017), Paris, France, April 29, 2017. [full text]
  23. Ross Koppel and Harold Thimbleby, “Lessons from the 100 Nation Ransomware Attack”, The Healthcare Blog (THCB), May 14, 2017. [link]
  24. Haibing Zheng, Dengfeng Li, Xia Zeng, Beihai Liang, Wujie Zheng, Yuetang Deng, Wing Lam, Wei Yang, and Tao Xie, “Automated Test Input Generation for Android: Towards Getting There in an Industrial Case”, 39th International Conference on Software Engineering (ICSE 2017), Software Engineering in Practice (SEIP), Buenos Aires, Argentina, May 20-28, 2017. [full text]
  25. Christopher Novak, Jim Blythe, Ross Koppel, Vijay Kothari, and Sean Smith, “Modeling Aggregate Security with User Agents that Employ Password Memorization Techniques”, Who Are You?! Adventures in Authentication (WAY 2017), workshop in conjunction with Symposium On Usable Privacy and Security (SOUPS 2017), July 12-14, 2017, Santa Clara, CA. [full text]
  26. Benjamin Andow, Akhil Acharya, Dengfeng Li, William Enck, Kapil Singh, and Tao Xie, “UiRef: Analysis of Sensitive User Inputs in Android Applications”, 10th ACM Conference on Security and Privacy in Wireless and Mobile Networks (WiSec 2017), Boston, MA, July 18-20, 2017. [full text]

Presentations

  1.  August 2014, 2014 USENIX Summit on Health Information Technologies, Keynote, Ross Koppel: Software Loved by its Vendors and Disliked by 70% of its Users: Two Trillion Dollars of Healthcare Information Technology’s Promises and Disappointments
  2.  August 2014, European Sociological Association Midterm Conference, presentation, Ross Koppel: Ethnography of Computer Security Evasions in Healthcare Organizations: Circumvention and Cyber Controls
  3. December 2014, Rutgers University, Department of Electrical and Computer Engineering Colloquium, Sean Smith: Circumvention: Why Do Good People Do Bad Things, and What Can We Do About It?
  4. January 2015, NSA SoS Bi-weekly Meeting, Tao Xie: AppContext: Differentiating Malicious and Benign Mobile App Behavior Under Contexts [slides]
  5. February 2015, NSA SoS Bi-weekly Meeting, Tao Xie: Science of Human Circumvention of Security [slides]
  6. February 2015, Royal College of Physicians (Edinburgh), Keynote, Ross Koppel: Healthcare Software Usability and the Influence on Compliance with Cybersecurity Rules
  7. February 2015, Wales Health Trust at Prince of Wales Hospital, invited seminar, Ross Koppel: Healthcare Software Usability and the Influence on Compliance with Cybersecurity Rules
  8. April 2015, Symposium and Bootcamp on the Science of Security (HotSoS), invited tutorial, Jim Blythe and Sean Smith: Understanding and Accounting for Human Behavior [slides]
  9. April 2015, Human Factors and Ergonomics in Health Care: Improving Outcomes (HFES), invited talk, Ross Koppel, Sean Smith and Harold Thimbleby: What you See Is What You See: Misinforming Displays in Electronic Health Care Records and Medical Devices
  10. April 2015, Dagstuhl: Assuring Resilience, Security and Privacy for Flexible Networked Systems and Organizations, Sean Smith: Trust Challenges in Massive Multi-organization Distributed Systems
  11. May 2015, 37th International Conference on Software Engineering (ICSE 2015), Wei Yang: AppContext: Differentiating Malicious and Benign Mobile App Behavior Under Context
  12. July 2015, NSA SoS Quarterly Meeting, Ross Koppel: Progress, Problems, Publications, Plans and Promises of the Group Studying Passwords and Cyber Security Circumvention [slides]
  13. July 2015, International Symposium on Human Aspects of Information Security and Assurance, Sean Smith: Mismorphism: A Semiotic Model of Computer Security Circumvention
  14. September 2015, New England Security Day, University of Massachusetts, Amherst, Vijay Kothari: Mismorphism and Circumvention
  15. October 2015, invited tutorial, NSF Interdisciplinary Workshop on Statistical NLP and Software Engineering, Tao Xie: Software Mining and Software Datasets
  16. October 2015, tutorial, 2015 Annual ACM Conference on Systems, Programming Languages, and Applications: Software for Humanity, Tao Xie: Software Analytics: Achievements and Challenges
  17. November 2015, St. Lawrence University, Sean Smith: Circumvention: Why Do Good People Do Bad Things, and What Can We Do About It
  18. November 2015, invited talk, Washington State University, Tao Xie: Text Analytics for Mobile App Security and Beyond
  19. December 2015, Holy Cross College, Sean Smith: Circumvention: Why Do Good People Do Bad Things, and What Can We Do About It
  20. December 2015, Society for Risk Analysis, Jim Blythe: A Toolkit for Exploring the Impact of Human Behavior on Cybersecurity through Multi-agent Simulations
  21. January 2016, ITI Joint Trust and Security/Science of Security Seminar, Tao Xie: User Expectations in Mobile App Security [Media Link]
  22. March 2016, ITI Joint Trust and Security/Science of Security Seminar, Wing Lam: Towards Preserving Mobile Users’ Privacy in the Context of Utility Apps
  23. April 2016, Symposium and Bootcamp on the Science of Security (HotSoS 2016), invited tutorial, Tao Xie and William Enck: Text Analytics for Security [slides]
  24. May 2016, 38th International Conference on Software Engineering (ICSE 2016), Tao Xie: Measuring Code Behavioral Similarity for Programming and Software Engineering
  25. June 2016: University of Central Florida, invited talk, Tao Xie: User Expectations in Mobile App Security
  26. July 2016, NSA SoS Quarterly Meeting, poster presentation, Jim Blythe, Vijay Kothari, Ross Koppel, and Sean Smith: Modeling Human Security Behavior: Recent Results on Understanding Compliance [poster]
  27. July 2016, NSA SoS Quarterly Meeting, poster presentation, Ross Koppel, Jim Blythe, Vijay Kothari, and Sean Smith: Beliefs about Cybersecurity Rules and Passwords: A Comparison of Two Survey Samples of Cybersecurity Professionals Verses Regular Users [poster]
  28. July 2016, NSA SoS Quarterly Meeting, poster presentation, Sean Smith, Ross Koppel, Jim Blythe, and Vijay Kothari: Reasons for Cybersecurity Circumvention: A Study and a Model [poster]
  29. July 2016, NSA SoS Quarterly Meeting, poster presentation, Wing Lam, Dengfeng Li, Wei Yang, and Tao Xie: User-Centric Mobile Security Assessment [poster]
  30. November 2016, NSA SoS Quarterly Meeting, poster presentation, Jim Blythe, Christopher Novak, Vijay Kothari, Ross Koppel, and Sean Smith: Modeling Human Security Behavior: Recent Results on Understanding Compliance [poster]
  31. November 2016, NSA SoS Quarterly Meeting, poster presentation, Ross Koppel, David Harmon, Sean Smith, Jim Blythe, and Vijay Kothari: Beliefs about Cybersecurity Rules and Passwords: Comparing Two Survey Samples of Cybersecurity Professionals and General Users and Future Data Collection Experiments [poster]
  32. November 2016, NSA SoS Quarterly Meeting, poster presentation, Sean Smith, Ross Koppel, Jim Blythe, and Vijay Kothari: Flawed Mental Models Lead to Bad Cyber Security Decisions: Let’s Do a Better Job! [poster]
  33. November 2016, NSA SoS Quarterly Meeting, poster presentation, Dengfeng Li, Wei Yang, Wing Lam, and Tao Xie: User-Centric Mobile Security Assessment [poster]
  34. February 2017, invited seminar, University of Buffalo, Jim Blythe: Modeling Human Behavior to Improve Cyber Security
  35. February 2017, Monthly UIUC/R2 Presentation, Wing Lam, Dengfeng Li, and Wei Yang: Towards Privacy-Preserving Mobile Utility Apps: A Balancing Act [slides]
  36. March 2017, invited seminar, IEEE Rochester Section CS/CIS Joint Chapters/Department of Computing Security, Rochester Institute of Technology, Tao Xie: User Expectations in Mobile App Security
  37. March 2017, Monthly UIUC/R2 Presentation, Jim Blythe, Ross Koppel, Sean Smith, Vijay Kothari, David Harmon, and Christopher Novak: A Cross-Disciplinary Study of User Circumvention of Security [slides]
  38. April 2017, Symposium and Bootcamp in the Science of Security (HotSoS 2017), poster session, Jim Blythe, Sean Smith, Ross Koppel, Christopher Novak, and Vijay Kothari: FARM: Finding the Appropriate level of Realism for Modeling [poster]
  39. April 2017, Symposium and Bootcamp in the Science of Security (HotSoS 2017), poster session, Dengfeng Li, Wing Lam, Wei Yang, Zhengkai Wu, Xusheng Xiao, and Tao Xie: Towards Privacy-Preserving Mobile Apps: A Balancing Act [poster]
  40. April 2017, Symposium and Bootcamp in the Science of Security (HotSoS 2017), poster session, Jim Blythe, Ross Koppel, Sean Smith, and Vijay Kothari: Analysis of Two Parallel Surveys on Cybersecurity: Users and Security Administrators — Notable Similarities and Differences [poster]
  41. April 2017, Symposium and Bootcamp in the Science of Security (HotSoS 2017), poster session, Sean Smith, Ross Koppel, Jim Blythe, and Vijay Kothari: Flawed Mental Models Lead to Bad Cybersecurity Decisions: Let’s Do a Better Job! [poster]