3 years ago, when Machine Learning became more widely used in the industry, we realised that there were very few tools that could help experimenting and prototyping ML models quickly. Not only that, but in the industry, ML was facing issues that didn’t exist in research, such as authorisations, security, or all the caveats that come to light when trying to productise a proof-of-concept. That’s why we developed Machine Learning Lab and we’re happy to announce that since September 2020, it is available as an open-source project for everybody to enjoy!
ML Lab started as a simple, light-weight solution to cover these needs. The approach we took was to bring the tool to the data and not the other way around. To achieve this, ML Lab leveraged containerized technologies on the basis of Docker and Kubernetes(K8s). This allows us to deploy the platform quickly and easily in any server or cluster where Docker is available. Going hand-in-hand with containerization, the backend and web app were written with having cloud-native principles in mind from the beginning.
ML Lab launches a modified version of the open-source project ML Workspace for each user. With the workspace, the user has some tools and features at hand to support with the data science process in an isolated compute environment. For example, it has tools such as VNC or VS Code installed as well as Python or R for programming or ssh for remote access. The Lab-flavor has some Lab-specifics installed. As an example, it comes pre-installed with the ML Lab Python client with which data, models, and experiments of an ML Lab project can be accessed, up-or downloaded, or tracked, respectively.
One of the biggest problems in ML is reproducibility of the experiments. ML lab offers an experiments dashboard to help you keep track of the different runs. By using the for ML Lab, data scientists are able to keep track of metrics, artifacts, and dependencies of their experiments, making it easy to reproduce results and collaborate between different team members working on a project.
Models and datasets
Machine Learning Lab also offers a simple storage system to keep track of your datasets and Machine Learning models. Every file is versioned and you’re able to integrate into different systems and tools by using the provided API.
More-over, ML Lab comes with a custom python library: which allows you to have one-click deployments from your model.
You have experimented with different architectures, you have done some hyperparameter optimization, and now you want to expose your model as a REST API, but how you can do this? ML Lab offers the possibility of spinning-up Docker containers, so you can create your image and then use ML Lab to serve this image exposing your model as a web service.
Moreover, we facilitate the step of creating the docker image and the web service by providing some helpful tools such as the , which will allow you to go from a Python script into a web service in a couple of commands!
We’re continuously improving the code-base and we already have some features in the making, just to mention two:
- We are working in a feature to be able to spawn different workspaces with customisable compute resources. This will allows the different workspaces to bring their own docker images to used as a base for their workspaces with the necessary libraries already pre-installed.
- Enablement of different authentication mechanisms so that users can configure them such as login with Google or GitHub.
Are you curious? Do you want to try it out?
Check out the repository: https://github.com/SAP/machine-learning-lab PR’s are welcome!
The documentation: https://sap.github.io/machine-learning-lab/
Shout out to Benjamin Raethlein, who was contributing to this blog post!