Audit Mechanisms for Provable Risk Management and Accountable Data Governance

  • Jeremiah Blocki
  • Nicolas Christin
  • Anupam Datta
  • Arunesh Sinha
Part of the Lecture Notes in Computer Science book series (LNCS, volume 7638)


Organizations that collect and use large volumes of personal information are expected under the principle of accountable data governance to take measures to protect data subjects from risks that arise from inapproriate uses of this information. In this paper, we focus on a specific class of mechanisms—audits to identify policy violators coupled with punishments—that organizations such as hospitals, financial institutions, and Web services companies may adopt to protect data subjects from privacy and security risks stemming from inappropriate information use by insiders. We model the interaction between the organization (defender) and an insider (adversary) during the audit process as a repeated game. We then present an audit strategy for the defender. The strategy requires the defender to commit to its action and when paired with the adversary’s best response to it, provably yields an asymmetric subgame perfect equilibrium. We then present two mechanisms for allocating the total audit budget for inspections across all games the organization plays with different insiders. The first mechanism allocates budget to maximize the utility of the organization. Observing that this mechanism protects the organization’s interests but may not protect data subjects, we introduce an accountable data governance property, which requires the organization to conduct thorough audits and impose punishments on violators. The second mechanism we present achieves this property. We provide evidence that a number of parameters in the game model can be estimated from prior empirical studies and suggest specific studies that can help estimate other parameters. Finally, we use our model to predict observed practices in industry (e.g., differences in punishment rates of doctors and nurses for the same violation) and the effectiveness of policy interventions (e.g., data breach notification laws and government audits) in encouraging organizations to adopt accountable data governance practices.


Expected Utility Repeated Game Subgame Perfect Equilibrium Public Signal Inside Attack 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Center for Information Policy Leadership: Accountability-Based Privacy Governance Project (accessed May 1, 2012)Google Scholar
  2. 2.
    The White House: Consumer Data Privacy in a Networked World: A Framework for Protecting Privacy and Promoting Innovation in the Global Digital Economy (accessed May 1, 2012)Google Scholar
  3. 3.
    Fairwarning: Industry Best Practices for Patient Privacy in Electronic Health Records (April 2011)Google Scholar
  4. 4.
    Hulme, G.: Steady Bleed: State of HealthCare Data Breaches. InformationWeek (September 2010)Google Scholar
  5. 5.
    U.S. Department of Health & Human Services: HIPAA enforcement (accessed May 1,2012)Google Scholar
  6. 6.
    Ornstein, C.: Breaches in privacy cost Kaiser, (May 2009)
  7. 7.
    Picard, K.: Are Drug-Stealing Nurses Punished More Than Doctors? (2012)Google Scholar
  8. 8.
    Blocki, J., Christin, N., Datta, A., Sinha, A.: Regret minimizing audits: A learning-theoretic basis for privacy protection. In: Computer Security Foundations Symposium, pp. 312–327 (2011)Google Scholar
  9. 9.
    Fudenberg, D., Tirole, J.: Game Theory. The MIT Press (1991)Google Scholar
  10. 10.
    PricewaterhouseCoopers: A practical guide to risk assessment (December 2008)Google Scholar
  11. 11.
    Vellani, K.H.: Strategic Healthcare Security, Risk Assessments in the Environment of Care, Report for Wisconsin Healthcare Engineering Association (2008)Google Scholar
  12. 12.
    NIST: Guide for Conducting Risk Assessments (September 2011)Google Scholar
  13. 13.
    Cheng, P.-C., Rohatgi, P.: IT Security as Risk Management: A Reserach Perspective. IBM Research Report (April 2008)Google Scholar
  14. 14.
    Petrochko, C.: DHC: EHR Data Target for Identity Thieves (December 2011)Google Scholar
  15. 15.
    American National Standards Institute(ANSI)/The Santa Fe Group/Internet Security Alliance: The financial impact of breached protected health information (accessed May 1,2012)Google Scholar
  16. 16.
    Verizon: 2012 Data Breach Investigations Report (2012)Google Scholar
  17. 17.
    Ponemon Institute, LLC: Benchmark Study on Patient Privacy and Data Security (November 2010)Google Scholar
  18. 18.
    Ponemon Institute, LLC: 2011 Cost of Data Breach Study: United States (March 2012)Google Scholar
  19. 19.
    Ponemon Institute, LLC: 2010 Annual Study: U.S. Cost of a Data Breach (March 2011)Google Scholar
  20. 20.
    Ichniowski, C., Shaw, K., Prennushi, G.: The Effects of Human Resource Management Practices on Productivity. Technical Report 5333, National Bureau of Economic Research (November 1995)Google Scholar
  21. 21.
    Hanushek, E.A.: Statistical Methods for Social Scientists. Academic Press, New York (1977)Google Scholar
  22. 22.
    Mailath, G.J., Samuelson, L.: Repeated Games and Reputations: Long-Run Relationships. Oxford University Press, USA (2006)CrossRefGoogle Scholar
  23. 23.
    Varian, H.: System reliability and free riding. In: Economics of Information Security (Advances in Information Security), vol. 12, pp. 1–15 (2004)Google Scholar
  24. 24.
    Grossklags, J., Christin, N., Chuang, J.: Secure or insure? A game-theoretic analysis of information security games. In: World Wide Web Conference (WWW 2008), pp. 209–218 (2008)Google Scholar
  25. 25.
    Saltzer, J.H., Schroeder, M.D.: The protection of information in computer systems. Proceedings of the IEEE 63(9), 1278–1308 (1975)CrossRefGoogle Scholar
  26. 26.
    U.S. Department of Health & Human Services: HIPAA Privacy and Security Audit ProgramGoogle Scholar
  27. 27.
    Ponemon Institute, LLC: Second Annual Benchmark Study on Patient Privacy and Data Security (December 2011)Google Scholar
  28. 28.
    Romanosky, S., Hoffman, D., Acquisti, A.: Empirical analysis of data breach litigation. In: International Conference on Information Systems (2011)Google Scholar
  29. 29.
    MedAssets: MedAssets Case Sudy: Stanford hospital takes charge of its charge capture process, increasing net revenue by 4 million (2011)Google Scholar
  30. 30.
    Barth, A., Datta, A., Mitchell, J.C., Nissenbaum, H.: Privacy and contextual integrity: Framework and applications. In: IEEE Symposium on Security and Privacy, pp. 184–198 (2006)Google Scholar
  31. 31.
    Basin, D., Klaedtke, F., Müller, S.: Policy Monitoring in First-Order Temporal Logic. In: Touili, T., Cook, B., Jackson, P. (eds.) CAV 2010. LNCS, vol. 6174, pp. 1–18. Springer, Heidelberg (2010)CrossRefGoogle Scholar
  32. 32.
    Garg, D., Jia, L., Datta, A.: Policy auditing over incomplete logs: theory, implementation and applications. In: ACM Computer and Communications Security (CCS), pp. 151–162 (2011)Google Scholar
  33. 33.
    Tschantz, M.C., Datta, A., Wing, J.M.: Formalizing and enforcing purpose requirements in privacy policies. In: IEEE Symposium on Security and Privacy (2012)Google Scholar
  34. 34.
    Backes, M., Datta, A., Derek, A., Mitchell, J.C., Turuani, M.: Compositional analysis of contract-signing protocols. Theor. Comput. Sci. 367(1-2), 33–56 (2006)MathSciNetzbMATHCrossRefGoogle Scholar
  35. 35.
    Barth, A., Datta, A., Mitchell, J.C., Sundaram, S.: Privacy and utility in business processes. In: Computer Security Foundations Symposium (CSF), pp. 279–294 (2007)Google Scholar
  36. 36.
    Jagadeesan, R., Jeffrey, A., Pitcher, C., Riely, J.: Towards a Theory of Accountability and Audit. In: Backes, M., Ning, P. (eds.) ESORICS 2009. LNCS, vol. 5789, pp. 152–167. Springer, Heidelberg (2009)CrossRefGoogle Scholar
  37. 37.
    Küsters, R., Truderung, T., Vogt, A.: Accountability: definition and relationship to verifiability. In: ACM Conference on Computer and Communications Security, pp. 526–535 (2010)Google Scholar
  38. 38.
    Feigenbaum, J., Jaggard, A.D., Wright, R.N.: Towards a formal model of accountability. In: Proceedings of the 2011 Workshop on New Security Paradigms Workshop (2011)Google Scholar
  39. 39.
    Bauer, L., Garriss, S., Reiter, M.K.: Detecting and resolving policy misconfigurations in access-control systems. In: Symposium on Access Control Models and Technologies (SACMAT), pp. 185–194 (2008)Google Scholar
  40. 40.
    Vaughan, J.A., Jia, L., Mazurak, K., Zdancewic, S.: Evidence-based audit. In: Computer Security Foundations Symposium (CSF), pp. 177–191 (2008)Google Scholar
  41. 41.
    Lampson, B.W.: Computer security in the real world. IEEE Computer 37(6), 37–46 (2004)CrossRefGoogle Scholar
  42. 42.
    Cederquist, J.G., Corin, R., Dekker, M.A.C., Etalle, S., den Hartog, J.I., Lenzini, G.: Audit-based compliance control. Int. J. Inf. Sec. 6(2-3), 133–151 (2007)CrossRefGoogle Scholar
  43. 43.
    Cheng, P.C., Rohatgi, P., Keser, C., Karger, P.A., Wagner, G.M., Reninger, A.S.: Fuzzy Multi-Level Security: An Experiment on Quantified Risk-Adaptive Access Control. In: Proceedings of the IEEE Symposium on Security and Privacy (2007)Google Scholar
  44. 44.
    Cheng, P.C., Rohatgi, P.: IT Security as Risk Management: A Research Perspective. IBM Research Report RC24529 (April 2008)Google Scholar
  45. 45.
    Zhao, X., Johnson, M.E.: Access governance: Flexibility with escalation and audit. In: Hawaii International International Conference on Systems Science (HICSS), pp. 1–13 (2010)Google Scholar
  46. 46.
    Zhang, N., Yu, W., Fu, X., Das, S.K.: Towards effective defense against insider attacks: The establishment of defender’s reputation. In: IEEE International Conference on Parallel and Distributed Systems, pp. 501–508 (2008)Google Scholar
  47. 47.
    Band, S.R., Cappelli, D.M., Fischer, L.F., Moore, A.P., Shaw, E.D., Trzeciak, R.F.: Comparing Insider IT Sabotage and Espionage: A Model-Based Analysis. Technical Report CMU/SEI-2006-TR-026, Carnegie Mellon University (December 2006)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2012

Authors and Affiliations

  • Jeremiah Blocki
    • 1
  • Nicolas Christin
    • 1
  • Anupam Datta
    • 1
  • Arunesh Sinha
    • 1
  1. 1.Carnegie Mellon UniversityPittsburghUSA

Personalised recommendations