Skip to main content
SpringerLink
Log in

CaTabRa: Efficient Analysis and Predictive Modeling of Tabular Data

  • Conference paper
  • First Online:
Artificial Intelligence Applications and Innovations (AIAI 2023)

Part of the book series: IFIP Advances in Information and Communication Technology ((IFIPAICT,volume 676))

Abstract

We present CaTabRa, a novel open-source Python package for the efficient and largely automated analysis of tabular data. It combines a variety of established frameworks and libraries for data processing, automated machine learning, explainable AI and out-of-distribution detection into one coherent system. Thanks to its simple user interface, CaTabRa can be used by practitioners who want to quickly gain insights into their data and the potential of predictive modeling, but it also provides added value for data-science experts through its function library. We demonstrate CaTabRa’s usefulness in two example applications.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 109.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Hardcover Book
USD 139.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Notes

  1. 1.

    Scientific publications describing these projects are currently in preparation.

References

  1. Ali, M., et al.: PyCaret. https://pycaret.org/. Accessed 30 Mar 2023

  2. Alnegheimish, S., et al.: Cardea: an open automated machine learning framework for electronic health records. In: 2020 IEEE 7th International Conference on Data Science and Advanced Analytics (DSAA), pp. 536–545 (2020). https://doi.org/10.1109/DSAA49011.2020.00068

  3. Bai, Y., Li, Y., Shen, Y., Yang, M., Zhang, W., Cui, B.: AutoDC: an automatic machine learning framework for disease classification. Bioinformatics 38(13), 3415–3421 (2022). https://doi.org/10.1093/bioinformatics/btac334

    Article  Google Scholar 

  4. Bergstra, J., Yamins, D., Cox, D.: Making a science of model search: hyperparameter optimization in hundreds of dimensions for vision architectures. In: Dasgupta, S., McAllester, D. (eds.) Proceedings of the 30th International Conference on Machine Learning. Proceedings of Machine Learning Research, Atlanta, Georgia, USA, vol. 28, pp. 115–123. PMLR (2013). https://proceedings.mlr.press/v28/bergstra13.html

  5. Caicedo-Torres, W., Gutierrez, J.: ISeeU: visually interpretable deep learning for mortality prediction inside the ICU. J. Biomed. Inform. 98, 103269 (2019). https://doi.org/10.1016/j.jbi.2019.103269

    Article  Google Scholar 

  6. Chen, T., Guestrin, C.: XGBoost: a scalable tree boosting system. In: Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, KDD 2016, pp. 785–794. ACM, New York (2016). https://doi.org/10.1145/2939672.2939785

  7. Christ, M., Braun, N., Neuffer, J., Kempa-Liehr, A.W.: Time series feature extraction on basis of scalable hypothesis tests (tsfresh - a python package). Neurocomputing 307, 72–77 (2018). https://doi.org/10.1016/j.neucom.2018.03.067

    Article  Google Scholar 

  8. Dask Development Team: Dask: Library for dynamic task scheduling (2016). https://dask.org. Accessed 30 Mar 2023

  9. Efron, B.: Bootstrap methods: another look at the jackknife. Ann. Stat. 7, 1–26 (1979). https://doi.org/10.1214/aos/1176344552

    Article  MathSciNet  MATH  Google Scholar 

  10. Feurer, M., Eggensperger, K., Falkner, S., Lindauer, M., Hutter, F.: Auto-Sklearn 2.0: Hands-free AutoML via Meta-Learning. Technical report. arXiv:2007.04074 (2021)

  11. Feurer, M., Klein, A., Eggensperger, K., Springenberg, J., Blum, M., Hutter, F.: Efficient and robust automated machine learning. In: Advances in Neural Information Processing Systems, vol. 28. Curran Associates Inc. (2015). https://papers.neurips.cc/paper/2015/hash/11d0e6287202fced83f79975ec59a3a6-Abstract.html

  12. Garouani, M., Ahmad, A., Bouneffa, M., Hamlich, M.: AMLBID: an auto-explained automated machine learning tool for big industrial data. SoftwareX 17, 100919 (2022). https://doi.org/10.1016/j.softx.2021.100919

    Article  Google Scholar 

  13. Google: Vertex AI. https://cloud.google.com/vertex-ai?hl=en. Accessed 30 Mar 2023

  14. Hatib, F., et al.: Machine-learning algorithm to predict hypotension based on high-fidelity arterial pressure waveform analysis. Anesthesiology 129(4), 663–674 (2018). https://doi.org/10.1097/ALN.0000000000002300

    Article  Google Scholar 

  15. Jarrett, D., Bica, I., Ercole, A., Yoon, J., Qian, Z., van der Schaar, M.: Clairvoyance: a pipeline toolkit for medical time series. In: Proceedings of ICLR 2021, p. 32 (2021)

    Google Scholar 

  16. Johnson, A.E.W., Stone, D.J., Celi, L.A., Pollard, T.J.: The MIMIC code repository: enabling reproducibility in critical care research. J. Am. Med. Inform. Assoc. 25(1), 32–39 (2018). https://doi.org/10.1093/jamia/ocx084

  17. Kanter, J.M., Veeramachaneni, K.: Deep feature synthesis: towards automating data science endeavors. In: 2015 IEEE International Conference on Data Science and Advanced Analytics (DSAA), pp. 1–10 (2015). https://doi.org/10.1109/DSAA.2015.7344858

  18. Lundberg, S.M., Lee, S.I.: A unified approach to interpreting model predictions. In: Guyon, I., et al. (eds.) Advances in Neural Information Processing Systems, vol. 30, pp. 4765–4774. Curran Associates Inc. (2017). https://papers.nips.cc/paper/7062-a-unified-approach-to-interpreting-model-predictions.pdf

  19. Maletzky, A., et al.: Lifting hospital electronic health record data treasures: challenges and opportunities. JMIR Med. Inform. 10(10), e38557 (2022). https://doi.org/10.2196/38557

  20. McKinney, W.: Data structures for statistical computing in python. In: van der Walt, S., Millman, J. (eds.) Proceedings of the 9th Python in Science Conference, pp. 56–61 (2010). https://doi.org/10.25080/Majora-92bf1922-00a

  21. Microsoft: Azure Machine Learning - ML as a Service. https://azure.microsoft.com/en-us/products/machine-learning. Accessed 30 Mar 2023

  22. Müller, F., Botache, D., Huseljic, D., Heidecker, F., Bieshaar, M., Sick, B.: Out-of-distribution Detection and Generation using Soft Brownian Offset Sampling and Autoencoders. arXiv:2105.02965 (2021)

  23. Niculescu-Mizil, A., Caruana, R.: Predicting good probabilities with supervised learning. In: Proceedings of the 22nd International Conference on Machine Learning, ICML 2005, pp. 625–632. Association for Computing Machinery, New York (2005). https://doi.org/10.1145/1102351.1102430

  24. Pedregosa, F., et al.: Scikit-learn: machine learning in Python. J. Mach. Learn. Res. 12, 2825–2830 (2011)

    MathSciNet  MATH  Google Scholar 

  25. Perkins, N.J., Schisterman, E.F.: The inconsistency of ‘optimal’ cutpoints obtained using two criteria based on the receiver operating characteristic curve. Am. J. Epidemiol. 163, 670–675 (2006). https://doi.org/10.1093/aje/kwj063

    Article  Google Scholar 

  26. Płońska, A., Płoński, P.: Mljar: state-of-the-art automated machine learning framework for tabular data. version 0.10.3 (2021). https://github.com/mljar/mljar-supervised. Accessed 30 Mar 2023

  27. Plotly Technologies Inc.: Collaborative data science (2015). https://plotly.com. Accessed 30 Mar 2023

  28. Reith, F.C.M., Van den Brande, R., Synnot, A., Gruen, R., Maas, A.I.R.: The reliability of the Glasgow Coma Scale: a systematic review. Intensive Care Med. 42(1), 3–15 (2015). https://doi.org/10.1007/s00134-015-4124-3

    Article  Google Scholar 

  29. Roland, T., et al.: Domain shifts in machine learning based Covid-19 diagnosis from blood tests. J. Med. Syst. 46(5), 1–12 (2022). https://doi.org/10.1007/s10916-022-01807-1

    Article  Google Scholar 

  30. Sculley, D., et al.: Hidden technical debt in machine learning systems. In: Proceedings of the 28th International Conference on Neural Information Processing Systems - Volume 2, NIPS 2015, pp. 2503–2511. MIT Press, Cambridge (2015)

    Google Scholar 

  31. Smirnov, N.V.: Estimate of deviation between empirical distribution functions in two independent samples. Bull. Moscow Univ. 2(2), 3–16 (1939)

    Google Scholar 

  32. Smith, M.J., Sala, C., Kanter, J.M., Veeramachaneni, K.: The machine learning bazaar: harnessing the ML ecosystem for effective system development. In: Proceedings of the 2020 ACM SIGMOD International Conference on Management of Data, SIGMOD 2020, pp. 785–800. Association for Computing Machinery, New York (2020). https://doi.org/10.1145/3318464.3386146

  33. Zhao, Y., Nasrullah, Z., Li, Z.: Pyod: a python toolbox for scalable outlier detection. J. Mach. Learn. Res. 20(96), 1–7 (2019). https://jmlr.org/papers/v20/19-011.html

  34. Zhou, Y.: Rethinking reconstruction autoencoder-based out-of-distribution detection. In: 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), New Orleans, LA, USA, pp. 7369–7377. IEEE (2022). https://doi.org/10.1109/CVPR52688.2022.00723

Download references

Acknowledgements

This project is financed by research subsidies granted by the government of Upper Austria. RISC Software GmbH is Member of UAR (Upper Austrian Research) Innovation Network.

We thank the anonymous reviewers for their valuable comments.

Author information

Authors and Affiliations

  1. Research Department Medical Informatics, RISC Software GmbH, Softwarepark 32a, 4232, Hagenberg, Austria

    Alexander Maletzky, Sophie Kaltenleithner, Philipp Moser & Michael Giretzlehner

Authors
  1. Alexander Maletzky
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Sophie Kaltenleithner
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Philipp Moser
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Michael Giretzlehner
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Alexander Maletzky .

Editor information

Editors and Affiliations

  1. University of Piraeus, Piraeus, Greece

    Ilias Maglogiannis

  2. Democritus University of Thrace, Xanthi, Greece

    Lazaros Iliadis

  3. University of Sunderland, Sunderland, UK

    John MacIntyre

  4. University of Leon, León, Spain

    Manuel Dominguez

Rights and permissions

Reprints and permissions

Copyright information

© 2023 IFIP International Federation for Information Processing

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Maletzky, A., Kaltenleithner, S., Moser, P., Giretzlehner, M. (2023). CaTabRa: Efficient Analysis and Predictive Modeling of Tabular Data. In: Maglogiannis, I., Iliadis, L., MacIntyre, J., Dominguez, M. (eds) Artificial Intelligence Applications and Innovations. AIAI 2023. IFIP Advances in Information and Communication Technology, vol 676. Springer, Cham. https://doi.org/10.1007/978-3-031-34107-6_5

Download citation

  • DOI: https://doi.org/10.1007/978-3-031-34107-6_5

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-031-34106-9

  • Online ISBN: 978-3-031-34107-6

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Societies and partnerships

Search

Navigation