TY - JOUR
T1 - Secure Distributed Storage Orchestration on Heterogeneous Cloud-Edge Infrastructures
AU - Kontodimas, Konstantinos
AU - Soumplis, Polyzois
AU - Kretsis, Aristotelis
AU - Kokkinos, Panagiotis
AU - Fehér, Marcell
AU - Lucani Rötter, Daniel Enrique
AU - Varvarigos, Manos
PY - 2023/10/1
Y1 - 2023/10/1
N2 - Distributed storage systems spanning across different cloud data centers have substantially improved availability and flexibility for data storage and retrieval operations. However, stringent latency requirements of emerging applications necessitate optimized selection of storage resources that exhibit smaller delay. Introducing edge resources into distributed storage systems enables data placement closer to its source, but simultaneously increases the complexity of decision-making and orchestration processes for optimal data placement. In this work, we develop mechanisms for storing data across an infrastructure that includes both edge and cloud resources. Our approach focuses on optimizing data integrity, longevity, security, and cost, while leveraging erasure coding when performing the resource allocation. We first present a comprehensive mixed integer linear programming formulation of the storage resource orchestration problem. As the search space for the optimal solution can be vast and the execution time prohibitively large for real size problems, we also propose an innovative multi-Agent heuristic approach that uses the rollout, a reinforcement based policy, to balance performance and execution time efficiently. Through various simulation experiments, we evaluate the developed mechanisms and trade-offs involved in our approach. By incorporating data from a multi-cloud provider, we further enhance the validity of the simulations and the conclusions drawn.
AB - Distributed storage systems spanning across different cloud data centers have substantially improved availability and flexibility for data storage and retrieval operations. However, stringent latency requirements of emerging applications necessitate optimized selection of storage resources that exhibit smaller delay. Introducing edge resources into distributed storage systems enables data placement closer to its source, but simultaneously increases the complexity of decision-making and orchestration processes for optimal data placement. In this work, we develop mechanisms for storing data across an infrastructure that includes both edge and cloud resources. Our approach focuses on optimizing data integrity, longevity, security, and cost, while leveraging erasure coding when performing the resource allocation. We first present a comprehensive mixed integer linear programming formulation of the storage resource orchestration problem. As the search space for the optimal solution can be vast and the execution time prohibitively large for real size problems, we also propose an innovative multi-Agent heuristic approach that uses the rollout, a reinforcement based policy, to balance performance and execution time efficiently. Through various simulation experiments, we evaluate the developed mechanisms and trade-offs involved in our approach. By incorporating data from a multi-cloud provider, we further enhance the validity of the simulations and the conclusions drawn.
KW - Distributed storage
KW - cloud-edge continuum
KW - data availability
KW - data security
KW - erasure coding
KW - resource allocation
UR - http://www.scopus.com/inward/record.url?scp=85162926320&partnerID=8YFLogxK
U2 - 10.1109/TCC.2023.3287653
DO - 10.1109/TCC.2023.3287653
M3 - Journal article
VL - 11
SP - 3407
EP - 3425
JO - IEEE Transactions on Cloud Computing
JF - IEEE Transactions on Cloud Computing
IS - 4
ER -