Advertisement

Finding Unexpected Patterns in Data

  • Balaji Padmanabhan
  • Alexander Tuzhilin
Part of the Studies in Fuzziness and Soft Computing book series (STUDFUZZ, volume 95)

Abstract

Many pattern discovery methods in the KDD literature have the drawbacks of (1) discovering too many obvious or irrelevant patterns and (2) not using prior knowledge systematically. In this chapter we present an approach that addresses these drawbacks. In particular we present an approach to characterizing the unexpectedness of patterns based on prior background knowledge in the form of beliefs. Based on this characterization of unexpectedness we present an algorithm, ZoomUR, for discovering unexpected patterns in data.

Keywords

Association Rule Knowledge Discovery Atomic Condition Minimum Support Mining Association Rule 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. [AIS93]
    Agrawal, R., Imielinski, T. and Swami, A., 1993. Mining Association Rules Between Sets of Items in Large Databases. In Proc. of the ACM SIGMOD Conference on Management of Data, pp. 207–216.Google Scholar
  2. [AMS+95]
    Agrawal, R., Mannila, H., Srikant, R., Toivonen, H. and Verkamo,A.I., 1995. Fast Discovery of Association Rules. In Fayyad, U.M., PiatetskyShapiro, G., Smyth, P., and Uthurusamy, R. eds., Advances in Knowledge Discovery and Data Mining. AAAI Press.Google Scholar
  3. AT97] Adomavicius, G., and Tuzhilin, A., 1997. Discovery of Actionable Patterns in Databases: The Action Hierarchy Approach. In Proc. of the Third Intl. Conference on Knowledge Discovery and Data Mining (KDD 97). Google Scholar
  4. [BF78]
    Buchanan, B.G. and E.A. Feigenbaum. DENDRAL and METADENDRAL: Their Applications Dimensions. Artificial Intelligence, 11:5 — 24, 1978.Google Scholar
  5. [BMU+97]
    Brin, S., Motwani, R., Ullman, J.D., and Tsur, S., 1997. Dynamic Itemset Counting and Implication Rules for Market Basket Data. Procs. ACM SIGMOD Int. Conf. on Mgmt. of Data, pp. 255–264.Google Scholar
  6. [DT93]
    Dhar, V., and Tuzhilin, A., 1993. Abstract-Driven Pattern Discovery in Databases. IEEE Transactions on Knowledge and Data Engineering, v.5, no. 6 December 1993.Google Scholar
  7. [F97]
    Forbes Magazine, Sep. 8, 1997. Believe in yourself, believe in the merchandise, pp.118–124.Google Scholar
  8. [FPM91]
    Frawley, W.J., Piatetsky-Shapiro, G. and Matheus, C.J., 1991. Knowledge Discovery in Databases: An Overview. In Piatetsky-Shapiro, G. and Frawley, W.J. eds., Know. Disc. in Databases. AAAI/MIT Press, 1991.Google Scholar
  9. [FPS96]
    Fayyad, U.M., Piatetsky-Shapiro, G., Smyth, P., 1996. From Data Mining to Knowledge Discovery: An Overview. In Fayyad, U.M.,Piatetsky-Shapiro, G., Smyth, P., and Uthurusamy, R. eds., Advances in Knowledge Discovery and Data Mining. AAAI/MIT Press.Google Scholar
  10. [KMR+94]
    Klemettinen, M., Mannila, H., Ronkainen, P., Toivonen, H. and Verkamo, A.I., 1994. Finding Interesting Rules from Large Sets of Discovered Association Rules. In Proc. of the Third International Conference on Information and Knowledge Management, pp. 401–407.Google Scholar
  11. [LB84]
    D.B. Lenat and J.S. Brown. Why AM and EURISKO appear to work. Artificial Intelligence, 23 (3): 269–294. 1984.CrossRefGoogle Scholar
  12. [LBA98]
    Y. Lee, B.G. Buchanan, and J.M. Aronis. Knowledge-based Learning in Exploratory Science: Learning Rules to Predict Rodent Carcinogenicity. Machine Learning, 30: 217–240. 1998.CrossRefGoogle Scholar
  13. [Len83]
    D.B. Lenat. AM: Discovery in Mathematics as Heuristic Search. In R. Davis and D. Lenat, editors. Knowledge-Based Systems in Artificial Intelligence. McGraw-Hill. 1983.Google Scholar
  14. [LH96]
    Liu, B. and Hsu, W., 1996. Post-Analysis of Learned Rules. In Proc. of the Thirteenth National Conf. on Artificial Intelligence (AAAI ‘86), pp. 828–834.Google Scholar
  15. [LHC97]
    Liu, B., Hsu, W. and Chen, S, 1997. Using General Impressions to Analyze Discovered Classification Rules. In Proc. of the Third Intl. Conf. on Knowledge Discovery and Data Mining (KDD 97 ), pp. 31–36.Google Scholar
  16. [M80]
    Mitchell, T. The need for biases in learning generalizations. Technical Report CBM-TR-117, Dept. of Computer Science, Rutgers University, 1980.Google Scholar
  17. [MK97]
    Michalski, R.S. and Kaufman, K.A. Data Mining and Knowledge Discovery: A Review of Issues and a Multistrategy Approach. Technical Report P97–3 MLI 97–2, Machine Learning and Inference Laboratory, George Mason University, 1997.Google Scholar
  18. [P99]
    Padmanabhan, B, 1999. Discovering Unexpected Patterns in Data Mining Applications. Doctoral dissertation, Department of Information Systems, Stern School of Business, New York University.Google Scholar
  19. [PK92]
    Pazzani, M. and Kibler, D. “The Utility of Knowledge in Inductive Learning.” Machine Learning, 9 (1): 57–94, 1992.Google Scholar
  20. [PSM94]
    Piatetsky-Shapiro, G. and Matheus, C.J., 1994. The Interestingness of Deviations. In Proc. of AAAI-94 Workshop on Know. Discovery in Databases, pp. 25–36.Google Scholar
  21. [PT97]
    Padmanabhan, B. and Tuzhilin, A., 1997. On the Discovery of Unexpected Rules in Data Mining Applications. In Procs. of the Workshop on Information Technology and Systems (WITS ‘87), pp. 81–90.Google Scholar
  22. [S97]
    Stedman, C., 1997. Data Mining for Fool’s Gold. Computerworld, Vol. 31,No. 48, Dec. 1997.Google Scholar
  23. [SL90]
    Shrager, J. and P. Langley. Computational Models of Scientific Discovery and Theory Formation. San Mateo, CA: Morgan Kaufmann, 1990.Google Scholar
  24. [ST95]
    Silberschatz, A. and Tuzhilin, A., 1995. On Subjective Measures of Interestingness in Knowledge Discovery. In Proc. of the First International Conference on Knowledge Discovery and Data Mining, pp. 275–281.Google Scholar
  25. [ST96a]
    Silberschatz, A. and Tuzhilin, A., 1996. What Makes Patterns Interesting in Knowledge Discovery Systems. IEEE Trans. on Know. and Data Engineering. Spec. Issue on Data Mining, v. 5, no. 6, pp. 970–974.CrossRefGoogle Scholar
  26. [ST96b]
    Silberschatz, A. and Tuzhilin, A., 1996. A Belief-Driven Discovery Framework Based on Data Monitoring and Triggering. Working Paper #IS-96–26, Dept. of Information Systems, Stern School of Business, NYU.Google Scholar
  27. [Suz97]
    Suzuki, E., 1997. Autonomous Discovery of Reliable Exception Rules. In Proc. of the Third International Conference on Knowledge Discovery and Data Mining, pp. 259–262.Google Scholar
  28. [SVA97]
    Srikant, R., Vu, Q. and Agrawal, R. Mining Association Rules with Item Constraints. In Proc. of the Third International Conference on Knowledge Discovery and Data Mining (KDD 97), pp. 67–73.Google Scholar
  29. [ZZH90]
    Zytkow, J., J. Zhu, and A. Hussam. Automated Discovery in Chemistry Laboratory. Proceedings of the Eighth National Conference on Artificial Intelligence. pp 889–894, 1990.Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2002

Authors and Affiliations

  • Balaji Padmanabhan
    • 1
  • Alexander Tuzhilin
    • 2
  1. 1.Operations and Information Management Department, The Wharton SchoolUniversity of PennsylvaniaUSA
  2. 2.Information Systems Department, Stern School of BusinessNew York UniversityUSA

Personalised recommendations