Thank you for Subscribing to CIO Applications Weekly Brief
Thank you for Subscribing to CIO Applications Weekly Brief
A long-standing attention-grabbing power word that continues to incentivize customers and dictate many important decisions in the world of business. What hasn’t changed about this trigger word over the years is that most brands use it to improve conversion rates but struggle to deliver on the promise.
Take for example the exponential deluge of data in the enterprise coming from artificial intelligence/machine learning (AI/ML) initiatives, IoT, and life sciences research to name a few. The amount of data generated by these use cases today seems truly limitless—many organizations are planning for exabyte-sized data capacities—but if data is trapped on storage media that isn’t immediately accessible to the applications that need it, then engineers can’t innovate from it, analysts can’t analyze it, and researchers can’t deduce any useful conclusions from it. The commonalities of these use cases are that the data is diverse, unstructured, and dynamic, thereby presenting unique challenges that legacy storage architectures like NAS, SAN, and DAS cannot solve. Furthermore, disruptive technologies such as cloud, flash, and compute acceleration technologies [GPUs and FPGAs] are significantly impacting how enterprise companies store and manage their data today. These technologies are at the forefront of digital transformation and adoption because they satisfy the need for agility and speed, however legacy storage architectures were not built to enable their inherent benefits. So, the question is, as more enterprise CIO’s learn to leverage their most valuable asset—their data—and launch ML initiatives how do they plan their data center to overcome the limitations of their storage systems so as not to impede innovation?
This is precisely where WekaIO™ (Weka) is pushing the ‘limits’ and is making storage ‘limitless’ in every sense of the word. Heralding a new era, WekaIO brings an innovative approach to data management and is committed to delivering three valuable benefits: Simplicity, Speed, and Scale. WekaIO’s revolutionary storage architecture is purpose-built for diverse modern-day application requirements being flash-optimized, cloud-native,and enabling a future-ready, agile, and accelerated data center. Used by eight of the Fortune 50 enterprise organizations today, Weka’s Limitless Data Platform empowers organizations to gain competitive advantage, get products to market faster, achieve breakthroughs in life-saving research, and create innovative business opportunities.
In a recent interview with CIO Applications, Liran Zvibel, Co-Founder and CEO at WekaIO, and seasoned storage technology veteran, touches upon his company’s innovative products, storage engineering expertise, and many more.
Briefly touch upon the inception story of WekaIO and the value that it delivers in the machine learning space.
Since we launched the company, Weka has been solving long-standing storage challenges for our customers. Our co-founders have a long legacy of storage expertise, which goes back to working for top storage companies like IBM. If you were buying storage from IBM in the last decade, you probably were buying the technology we developed. We launched Weka on the premise that data would become an organizations’ most valuable asset, but that legacy storage architectures aren’t equipped to keep pace with the demands of emerging applications that demand scale and performance. At the forefront of digital transformation and IT modernization to support ML workloads are flash-native infrastructure, cloud-first strategy, and the GPU. While flash isn’t new, how to cost-effectively leverage flash to accelerate applications has improved through multiple generations of technologies: first-generation made an existing product slightly better by adding flash to it; the second generationarchitected structured data around storage to run flash; and the third-generation re-architect’s unstructured data storage, or filer, to run flash making filers even better than block storage. Weka falls into the third-generation category.
What are the trends and challenges existing in the storage space today?
While the COVID-19 pandemic has significantly accelerated cloud adoption, Weka predicted this increased demand back in 2014 when we launched the company, we built our storage system on the Cloud. For now, most enterprise organizations supporting mixed workflows utilize legacy service providers to run the bulk of the workflows in their efficient on-premises infrastructure, where they control the cost and ensure the on-premise infrastructure is fully utilized, and the Cloud is then leveraged for cost-efficient capacity planning, such as to support bursty workloads, or to satisfy a disaster recovery (DR) strategy. They use the different storage platforms to satisfy the different requirements of their mixed workflows. Customers who use Weka can run their mixed workflows on the same storage platform, either on-premises or in the Cloud. They get all the benefits of managing their data on-premises, but then can extend the file namespace to the Cloud to gain all the benefits that platform has to offer—burst their workload for on-demand capacity or cost-effective backup, DR, or data lake.
The Weka Limitless Data Platform is the only solution available for ML workflows today that delivers simplicity, speed, and scale with no compromise, for better business outcomes
What is your company’s role in data center digital transformation, and what is the value that your product brings to the table?
Weka is a true scale-out NAS, unbounded from the capacity limitations of legacy NAS. Our storage solution delivers utmost scalability and is ‘limitless’ in the real sense. Customers with ML workflows love our storage because they can access their massive data sets in one file system namespace—managing hundreds of billions of files in a single directory. In that sense, you could draw similarities to efficiencies of object storage, but with the added benefits of high-performance and ease-of-use.In contrast to first-generation flash-based solutions that introduced half of the storage as flash connected through PCI to increase performance, Weka demonstrates how a shared storage architecture can deliverbetter speed than block (SAN) storage. And while single application workloads work well with traditional second-generation flash solutions organizationsmust makechoices around tuning [capacity, performance] parameters and when new applications are introduced the system ceases to work effectively. Weka eliminates the need for tuning parameters and instantly adapt to any new workload—without any knobs to tune or expertise.
Weka eliminates the data silos persistent in legacy storage architectures—if the application can’t access the data then it offers no value to the organization. Wherever your data lives, Weka manages that data in a global namespace ensuring the applications have access to all the data for better business outcomes. In addition, you can manage your data anyway you want because Weka is deployable on-premises, in the Cloud, or a hybrid—it’s your data, managed your way with no vendor lock-in.
Weka is software-only and available on any server vendor of your choosing; it’s acquired as a total solution through partners in the Weka Innovation Network™ (WIN). Again, this eliminates vendor lock-in while delivering all the enterprise features, including synchronized data security over Windows and Linux. In essence, we are the most flexible storage solution on the market today that delivers enterprise simplicity, speed, and scale that complex ML projects require.
While GPUs are an important aspect of such projects, we focus on the most important component—the data. We ensure that our clients can store the right amount of data and drive the right amount of data into the GPU. For example, an electric vehicle is creating a difference not because of the electric engine that has been in use since the eighties but for the advances in battery technology. Similarly, the AI and machine learning data is more than the machine; if you just had the engine, there is no drive and no learning. Weka is like the battery that enables the vehicle to view data as an effective component of learning.
Please share a case study or two to help understand the benefits that Weka brings to its clients.
Weka was approached by an organization in the autonomous driving vehicle market. The client was using a second-generation flash product from a well-known company and it wasn’t meeting their expectations with regards to speed and scale—it was taking too long to complete their training epochs and delaying time-to-market. With Weka, the clientshortened their single training cycles from two weeks to a few hours. This allowed them to achieve their goals and accelerate their product development within a week; such impressive results allowed the client to maintain a competitive edge in the market. And without have to retune the system, the client was running run several different applications on the same storage platform efficiently.
Another client with a similar use case was looking at us to help improve their GPU utilization and reduce their model training time. The client had decent compute capacity, running on a steady state. However, once every week, the company aimed to run a longer and bigger training model over more resources. Weka was able to save the client money with an on-premises implementation, while also producing snapshot copies and replicating the data to AWS S3. This workflow allowed the client to run large-scale model training projects by just migrating their workloads to AWS. The workflow with Weka provided scale, speed, and elasticity that did not introduce latency or additional time to move the data. As a result, the client achieved better business outcomes at a much faster speed and better cost economics. This client was also able to grow top line revenue by accepting more business; due to the accelerated speed with which training cycles were completed they were able to accept more jobs.
What are some of the significant differentiating factors that keep Weka ahead of the competition?
We have a convergence of an extremely talented and experienced engineering team who have architected a highly innovative solution and the fact that we are software-only. We are unique solution that offers the most flexible deployment options—choose your server and choose your platform because Weka works anywhere. Users can expand capacity without any limitations while the system is running, with no performance degradation, and there is no proprietary hardware.
What does the future look like for Weka?
We envision both the use of Kubernetes for containers and the Cloud to increase in user adoption—and so for us from a technology perspective, we will invest in R&D resources to build complementary features for both use cases. We are leveraging Kubernetes to unify orchestration and make the run time environment transparent to applications. And we will diversify our Cloud solutions to support both GCP and Azure users. And we announced our partnership with Hitachi Vantara last year, the product will GA this year, which supports our global expansion strategy.