8
Python 的 pickle 加载大于内存上限的内容时会分段载入吗？应该怎么处理大文件读取？
source link: https://www.v2ex.com/t/821016
Go to the source link to view the article. You can view the picture content, updated content and better typesetting reading experience. If the link is broken, please click the button below to view the snapshot at that time.
要处理很大的内容，比如视频裸流之类的东西，如果全放到内存里内存会撑爆。目前对储存和读取都有疑问，python 的 pickle 模块能简单的解决这个问题吗？就是随便 pickle.dump 一下，比如 dump 了 100G ，然后 load 的时候它会自动分段读吗？还是一次性把 100G 都读进内存之后才能解码？
Recommend
About Joyk
Aggregate valuable and interesting links.
Joyk means Joy of geeK