CORTX and Erasure #361
-
|
What are the data protection options we will be allowing / recommending / assuming with CORTX community edition? We can't expect everyone to exclusively utilize Seagate RBOD platforms as we are doing with our reference designs such as the Lyve Drive Rack R1 and follow-on scale out R2 clustered object store. If I'm building a minimum CORTX solution with a handful of nodes and storage local to each node (either embedded or in node-dedicated JBODs), will CORTX perform EC on top of those drives for me? Should I assume that I need to utilize mdraid / zfs / hardware RAID FIRST and then feed the protected volumes to CORTX? What level of redundancy are we planning for the scale-out reference clustered configuration (R2)? We will have the underlying EC performed within the RBOD enclosure, but what about across nodes? We will have to be doing 2nd level EC across the nodes (first level is within RBOD, second level is across nodes). If I'm building a larger cluster using CORTX community edition, will CORTX itself perform both the lower level EC across the HDDs within a given node in addition to the higher level EC across nodes? Or (again) will we assume that the lower level data protection (EC or R6 or mirroring) will be handled by some other solution? |
Beta Was this translation helpful? Give feedback.
Replies: 1 comment 1 reply
-
|
Hey Paul, Thanks for the great question! Yeah, CORTX itself currently doesn't know whether it is running on JBOD or RBOD although we have plans to integrate more tightly with RBOD in the future. Our modeling has shown that taking the same amount of capacity overhead and splitting it across RBOD and network adds one additional nine of durability. Therefore, we certainly do recommend running CORTX across RBOD. Note that current scale-out CORTX is work-in-progress. Works in happy path and some failure scenarios but not all. That's why the first released reference architecture relies solely on RBOD for durability. The picture below shows how we think data durability mechanisms have evolved over the years (usually due to increasing rebuild times resulting from larger devices) and how we envision a scale-out CORTX tightly integrated with RBOD to be the next upcoming evolution. And we're open sourcing CORTX so that other object stores out there can similarly integrate with RBOD in the same fashion.
John |
Beta Was this translation helpful? Give feedback.
-
|
Paul, my email above mentions the reference architecture that Seagate will be building from CORTX. Here's a link to that reference architecture: https://www.seagate.com/products/storage/object-storage-solutions/lyve-drive-rack/ |
Beta Was this translation helpful? Give feedback.
Hey Paul,
Thanks for the great question!
Yeah, CORTX itself currently doesn't know whether it is running on JBOD or RBOD although we have plans to integrate more tightly with RBOD in the future. Our modeling has shown that taking the same amount of capacity overhead and splitting it across RBOD and network adds one additional nine of durability. Therefore, we certainly do recommend running CORTX across RBOD. Note that current scale-out CORTX is work-in-progress. Works in happy path and some failure scenarios but not all. That's why the first released reference architecture relies solely on RBOD for durability. The picture below shows how we think data durability mechanisms have evolved ov…