Companies all across the world are shifting to cloud computing, and this change is happening at a rapid pace. According to a report by Statista, the public cloud services market is expected to reach nearly USD 266.4 billion in 2020 and is expected to exceed USD 350 billion by 2022.
FREMONT, CA: Analytics Stack Performance leader, Pepperdata, announced the release of its inaugural "Big Data Performance Report" for 2020. The report was compiled after reviewing comprehensive data on the applications contained in the company's most significant enterprise customer clusters, representing nearly 400 petabytes of data on 5000 nodes. The data collected approximately 4.5 million applications running in a 30-day time frame. The report provides insights into the enormous compute waste that occurs with big data applications in the cloud.
"When we analyzed the data, we were amazed to see how much underutilization and other wasted resources there were—unnecessarily driving costs up," said Joel Stewart, VP, Customer Success, Pepperdata. "The failure to optimize means companies are leaving a tremendous amount of money on the table—funds that could be reinvested in the business or drop straight to the bottom line. Unfortunately, many companies just don't have the visibility they need to recapture the waste and increase utilization."
The company's research shows how IT operations teams are dealing with this challenge. The new "Big Data Performance Report" reveals that, within enterprise data applications that are not optimized by solutions that allow for observability and continuous tuning, there exists enormous waste and tremendous potential to optimize and reduce that waste.
Companies all across the world are shifting to cloud computing, and this change is happening at a rapid pace. According to a report by Statista, the public cloud services market is expected to reach nearly USD 266.4 billion in 2020 and is expected to exceed USD 350 billion by 2022. However, as the cloud expands, so does cloud wastage. As more sophisticated big data applications migrate, the likelihood of resource misallocation rises. Gartner's research suggests that by the end of 2024, nearly all legacy applications migrated to public cloud infrastructure as a service (IaaS) will require optimization to become more cost-effective. Without this optimization, the data highlights there will be overspend.
Pepperdata's research further shed light on the nature of cloud wastage. The study shows that spark clusters and jobs are dominating spend across clusters. This is where the highest amount of net wastage was found. When it comes to wastage, failures are necessary. Job failures cause severe performance degradation and consume significant computational resources.