How to subset RData file that is too large for memory?

 

I have an RData file that has become too large to load on my computer using the load() command. It contains a data frame with ~3 million observations and ~100 variables.

I want to

  • load the file, maybe using some bigdata or ff package (but I have no idea how these work)
  • get rid of some variables to reduce file size
  • save it as as an Rdata file with a regular data frame (not some weird "bigdata" type object)
  • use load() to load the condensed file and resume regular R operations

    • How should I go about this?

    last week Jared Lander (author of the book "R for Everyone") helped is in creating a R tint on a bigboards.io hex. We installed RStudio Server on it and we use the whole storage/processing capabilities of the hex. Ceph is used to distribute the data on the nodes

    Anyway, it feels like a solution for your question.

    链接地址: http://www.djcxy.com/p/38338.html

    上一篇: 如何从R包加载.Rdata？

    下一篇: 如何对RData文件进行子集分配过大？