Reading Big Data(bases)

Abstract

Now that you understand how to manage data inside R, let’s consider where data is found. While smaller data is found in comma-separated values (CSV) files or files easily converted to such, larger data tends to live in other places. This chapter deals with big data, or at least data that may be big. What is the challenge with big data? R works in memory, random access memory, not hard drives. A quick check of your system settings should reveal the amount of memory you have. We, the authors, use between 4 and 32 gigabytes in our real-world systems, with the larger number being a somewhat expensive habit. R cannot analyze data larger than available RAM.