Novel cloud storage ecosystem for efficient and secured multimedia services

Abstract

Since massive numbers of images are now being communicated from, and stored in different cloud systems, faster retrieval has become extremely important. This is more relevant, especially after COVID-19 in bandwidth-constrained environments. However, to the best of our knowledge, a coherent solution to overcome this problem is yet to be investigated in the literature. Hence, by customizing the Progressive JPEG method, we propose a new Scan Script and a new lossy PJPEG architecture to reduce the file size as a solution to overcome our Scan Script’s drawback. We improve the scanning of Progressive JPEG’s picture payloads to ensure Faster Image Retrieval using the change in bit pixels of distinct Luma and Chroma components (Y , Cb, and Cr). The orchestration improves user experience even in bandwidth-constrained cases. We evaluate our proposed orchestration in a real-world setting across two continents encompassing a private cloud. Compared to existing alternatives, our proposed orchestration can improve user waiting time by up to 54% and decrease image size by up to 27%. Our proposed work is tested in cutting-edge cloud apps, ensuring up to 69% quicker loading time. In addition, the demand for a robust as well as a highly-available surveillance system with efficient media sharing capability has considerably risen in recent times. To face these challenges, we propose a new methodology to utilize OpenStack Swift’s object storage to efficiently store and archive media data. Our method leverages expanding the cloud file sharing capabilities from storing media files to also processing and archiving them along with performing encryption. Our proposed approach first segments, encodes, and transcodes the videos according to several resolutions for covering diversified remote devices. Then, we store the processed video footage in the storage server of OpenStack Swift. Afterwards, we perform necessary media encryption-decryption, compress the files containing the video data, and archive them using an archive server. We carry out rigorous experiments over a real setup comprising machines deployed in two different countries (Canada and Bangladesh), located over two different continents, to validate the efficacy and efficiency of our proposed architecture and methodology. Experimental results demonstrate substantial performance improvement using our approach over conventional alternative solutions. Moreover, we propose a novel Content-Based Searching (CoBS) architecture to extract additional information from images and documents and store it in an Elasticsearch-enabled database, which helps us to search for our desired data based on its contents. This approach works in two sequential stages. First, it will be uploaded to a classifier that will select the data type and send it to the specific model for the data. The images that are being uploaded are sent to our trained model for object detection, and the documents are sent for keyword extraction. Next, the extracted information is sent to Elasticsearch, which enables searching based on the contents. We train our models with comprehensive datasets (Microsoft COCO Dataset) for multimedia data and SemEval2017 Dataset for document data. Besides, we propose a generalized architecture for smooth and efficient management as well as retrieval of multimedia data in cloud systems. Here, we demonstrate that video segment download time improves by up to 30% when segmentation is done in the object server rather than in the proxy server. After, we present a generalized architecture named ‘RemOrphan’ for detecting the orphan garbage data using OpenStack Swift hash Ring and scripts. We deploy a private media cloud SPMS and find that around 35% data can be orphan garbage data. Due to the huge amount of orphan data, rsync replication needs higher time and more network overhead which hampers the system sustainability. We lower around 25% sync delay and 30% network overhead after deploying a deletion daemon to remove the orphan garbage data. Furthermore, we propose a test-driven automated architecture for load testing, named as ‘svLoad’ to compare the performance of cache and backend servers. Here, we designed test cases considering diversified real scenarios such as different protocol types, same or different URLs, with or without load, cache hit or miss, etc. using tools namely JMeter, Ansible, and some custom utility bash scripts. To validate the efficacy of our proposed methodology, we conduct a set of experiments by running these test cases over a real private cloud development setup using two open source projects - Varnish as the cache server and OpenStack Swift as the backend server. Our focus is also to find out bottlenecks of Varnish and Swift by executing load requests, and then tune the system based on our load test analysis. After successfully tuning the Swift, Varnish, and network system, based on our test analysis, we were able to improve the response time by up to 80%.