Petuum: A New Platform for Distributed Machine Learning on Big Data

How can one build a distributed framework that allows efficient deployment of a wide spectrum of modern advanced machine learning (ML) programs for industrial-scale problems using Big Models (100s of billions of parameters) on Big Data (terabytes or petabytes)? Contemporary parallelization strategies employ fine-grained operations and scheduling beyond the classic bulk-synchronous processing paradigm popularized by MapReduce, or even specialized operators relying on graphical representations of ML programs. The variety of approaches tends to pull systems and algorithms design in different directions, and it remains difficult to find a universal platform applicable to a wide range of different ML programs at scale. We propose a general-purpose framework that systematically addresses data- and model-parallel challenges in large-scale ML.

Shibashis Bal completed his B.Sc. (Mathematics) at the University of Manitoba. He has obtained the MCSE, CNE, CISSP, and CISA designations. After using his skills in some far corners of the world, he currently owns and operates 6279040 Canada Inc., an information security, audit and analysis company. When not engaged directly in IT security audit, he is applying statistical techniques to health, fitness and financial analysis.