List S3 objects in Juice client #4077
zhanghongzhou
started this conversation in
General
Replies: 3 comments 5 replies
-
In JuiceFS, files are split to multiple objects (doc), so it makes no sense to show object list in readdir. You can inspect all objects related to a file with the |
Beta Was this translation helpful? Give feedback.
0 replies
-
这里有个常见场景:海量的存量数据在S3,预热的部分(真子集)在juicefs。用户想全量glob出来,区分哪些需要预热到juicefs 从juicefs写入S3的object在listobject的时候过滤掉,展示给客户端的是从juicefs写入的文件+从S3写入的存量文件 |
Beta Was this translation helpful? Give feedback.
0 replies
-
感觉这个应该属于业务侧的事情,不应该在文件系统里面实现,否则会导致各种不可修复的性能和一致性问题。 |
Beta Was this translation helpful? Give feedback.
5 replies
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
-
What would you like to be added:
像S3FS、Goofys一样,在readdir 目录的时候,把juicefs的该目录的文件,和S3的对应前缀的object list,合并展示出来
Why is this needed:
全量数据的glob
Beta Was this translation helpful? Give feedback.
All reactions