Owl Scientific Computing

Owl Scientific Computing is a software system for scientific and engineering computing developed in the Department of Computer Science and Technology, University of Cambridge.[2] The System Research Group (SRG) in the department recognises Owl as one of the representative systems developed in SRG in the 2010s.[3] The source code is licensed under the MIT License and can be accessed from the GitHub repository.[4]

Original author(s)Liang Wang
Developer(s)Community project
Initial release2016 (2016)
Stable release
1.0.2 / 14 February 2022 (2022-02-14)[1]
Repositoryhttps://github.com/owlbarn/owl
Written inOCaml, C
Operating systemCross-platform
TypeNumerical analysis
LicenseMIT
Websitehttps://ocaml.xyz

The library is mostly designed and developed in the functional programming language OCaml. As a unique functional programming language, OCaml offers runtime efficiency, flexible module system, static type checking, intelligent garbage collector, and powerful type inference. Owl inherits these features directly from OCaml. With Owl, users can write succinct type-safe numerical applications in a concise functional language without sacrificing performance. It speeds up the development life-cycle, and reduces the cost from prototype to production use. The system serves as the de facto tool for computation intensive tasks in OCaml.[5]

History

Owl was developed when Dr. Liang Wang was working as a Post-Doc in the OCaml Labs.[6] Owl originated from a research project which studied the design of synchronous parallel machines for large-scale distributed computing in July 2016. Back then the libraries for numerical computing in OCaml ecosystem were very limited and the tooling was fragmented at that time. In order to test various analytical applications, many numerical functions had to be implemented, from very low level algebra and random number generators to the high level stuff like algorithmic differentiation and deep neural networks. These code snippets started accumulating. These functions were later taken out and wrapped into a standalone library named Owl.

Owl's architecture undertook at least a dozen of iterations in the beginning, and some of the architectural changes are quite drastic. After one-year intensive development, Owl was capable of doing many complicated numerical tasks (e.g. image classification). Dr. Liang Wang held a tutorial at the CUFP 2017 to demonstrate data science in OCaml.[7] In 2018, Prof. Richard Mortier gave a talk about Owl in the Alan Turing Institute.[8] To further promote OCaml and functional programming in data science, Owl provides abundant learning materials in the form of a details manual.[9]

Design and Features

Owl has implemented many advanced numerical functions atop of its implementation of n-dimensional arrays. Compared to other numerical libraries, Owl is unique in many perspectives, e.g. algorithmic differentiation and distributed computing have been included as integral components in the core system to maximise developers' productivity. The figure below gives a bird view of Owl's system architecture. The subsystem on the left part is Owl's Numerical system. The modules contained in this subsystem fall into three categories.

The architecture of the Owl numerical library.

The first is core modules contains basic data structures, i.e., N-dimensional array (Ndarray) in both dense and sparse forms. The Ndarray module supports various number types: float32, float64, complex32, complex64, int16, int32, etc. Also, the core module provide foreign function interfaces to other low level numerical libraries, such as CBLAS and LAPACK. These libraries are fully interfaced to the Linear Algebra module.

The second category is the classic analytics modules. This part contains basic mathematical and statistical functions, linear algebra, regression, optimisation, plotting, etc. Advanced math and statistics functions such as statistical hypothesis testing and Markov chain Monte Carlo are also included. As a core functionality, Owl provides the algorithmic differentiation (or automatic differentiation) and dynamic computation graph modules.

The highest level in the Owl architecture includes modules more advanced numerical applications such as neural network, natural language processing, data processing etc. The Zoo system is used for efficient scripting and code sharing. The modules in the second category, especially the algorithmic differentiation, make the code at this level quite concise.

The subsystem on the right is called Actor Subsystem which extends Owl's capability to parallel and distributed computing. The core idea is to transform a user application from sequential execution mode into parallel mode (using various computation engines) with minimal efforts. The method is to compose two subsystems together with functors to generate the parallel version of the module defined in the numerical subsystem.

Besides what have been mentioned in this figure, there are several other features in Owl. For example, the JavaScript and unikernel backends, integration with other frameworks such as TensorFlow and PyTorch, utilising GPU and other accelerator frameworks via symbolic graph, etc.

Research

The Owl project is research oriented, and supports research of numerical computing in multiple related topics. Some of its research topics are listed below.

  • Synchronous parallel distributed machine learning design. Owl is the first to propose using sampling to synchronise nodes in iterative algorithms. The work published on arxiv comes with solid mathematical proof.[10] This idea proves to be advanced and was later proposed in top Machine Learning conferences.[11]
  • One of the factors that contribute to the small code base of Owl is that it builds advanced analytical functions around the algorithmic differentiation. This idea was also proves to be popular and develops into the paradigm of Differentiable programming. It is now being used in popular numerical packages such as JuliaDiff.[12]
  • Using the computation graph offers another dimension optimization to the computation in Owl. Besides, the computation graph also bridges Owl application and hardware accelerators such as GPU and TPU. Later, the computation graph becomes a de facto intermediate representation. Standards such as the Open Neural Network Exchange and Neural Network Exchange Format are now widely supported by various deep learning frameworks such as TensorFlow and PyTorch.
  • The idea of service-level composition and serving was investigated in the Zoo subsystem of Owl. The prototype demonstrates the streamlining various stages in the code development including composition, test, distribution, validation, and deployment. It is very similar to the later MLOps concepts. Recently this topic attracts attention in top system conferences such as OSDI.[13]

As result of research following part of these directions, Owl produces several publications. In 2018, a paper titled Data Analytics Service Composition and Deployment on Edge Devices is accepted at the ACM SIGCOMM 2018 Workshop on Big Data Analytics and Machine Learning for Data Communication Networks.[14] Two talks are also accepted at the OCaml Workshop of the International Conference on Functional Programming 2019, on the topics of numerical ordinary differential equation solving,[15] and executing Owl computation on GPUs.[16] An internship in the OCaml Labs investigates the topic of image segmentation and related memory optimisation in Owl.[17] In 2022, the book <<OCaml Scientific Computing>> was published by Springer.[18]

See also

References
  1. "Releases – owlbarn/owl". Retrieved 2020-11-11 – via GitHub.
  2. "Owl, Scientific Computing for OCaml". Retrieved 2020-11-11.
  3. "System Research Group". Retrieved 2020-11-11.
  4. Owlbarn GitHub repository, https://github.com/owlbarn/owl. Retrieved 2020-11-01.
  5. "OCamlverse: Machine Learning, Scientific Computing and Data Science". Retrieved 2020-11-05.
  6. "OCaml Labs". Retrieved 2020-11-01.
  7. "Owl: Data Science in OCaml". CUFP Tutorials. 2017. Retrieved 2020-11-01.
  8. "The design of functional numerical software". Alan Turing Institute. 2018. Retrieved 2020-11-05.
  9. "OCaml Scientific Computing: Functional Programming Meets Data Science". Owl Team. 2020. Retrieved 2020-11-18.
  10. Wang, Liang; Catterall, Ben; Mortier, Richard (2017). "Probabilistic Synchronous Parallel". arXiv:1709.07772 [cs.DC].
  11. "Distributed Learning over Unreliable Networks". Proceedings of Machine Learning Research. 2019. Retrieved 2020-11-18.
  12. "JuliaDiff". Julia. 2019. Retrieved 2020-11-18.
  13. "Serving DNNs like Clockwork: Performance Predictability from the Bottom Up". Usenix. 2020. Retrieved 2020-11-18.
  14. "BIG-DAMA Workshop 2018 Programme". ACM. 2018. Retrieved 2020-11-05.
  15. "OwlDE: making ODEs first-class Owl citizens". OCaml Workshop, ICFP 2019. 2019. Retrieved 2020-11-11.
  16. "Executing Owl Computation on GPU and TPU". OCaml Workshop, ICFP 2019. 2019. Retrieved 2020-11-11.
  17. "Personal Webpage, Pierre Vandenhove". 2018. Retrieved 2020-11-05.
  18. "OCaml Scientific Computing - Functional Programming in Data Science and Artificial Intelligence". Springer. 2022. Retrieved 2022-02-15.
This article is issued from Wikipedia. The text is licensed under Creative Commons - Attribution - Sharealike. Additional terms may apply for the media files.