Build an Event Driven Machine Learning Pipeline on Kubernetes
Join us for Kubernetes Forums Seoul, Sydney, Bengaluru and Delhi - learn more at kubecon.io
Don’t miss KubeCon + CloudNativeCon 2020 events in Amsterdam March 30 - April 2, Shanghai July 28-30 and Boston November 17-20! Learn more at kubecon.io. The conference features presentations from developers and end users of Kubernetes, Prometheus, Envoy, and all of the other CNCF-hosted projects
Build an Event Driven Machine Learning Pipeline on Kubernetes - Animesh Singh & Hou Gang, IBM AIOps as a field is becoming the need of the hour. With various Machine Learning capabilities coming in different open source projects, and pipelines being built, having a transparent AI pipeline which can notify users of any data drift, bias detection, model accuracy loss etc. is becoming key. In addition, we need capabilities to build a Data Scientists code from source, orchestrate the code, and automate the pipeline. In this talk we will leverage Kubernetes components like build, eventing, serving and pipelines to show how to build an end to end AI pipeline which we detect any events happening, notify and take actions, can build and run data scientists code, do A/B testing, dark launch, and orchestrate the whole workflow from Model training, validation, serving, and operations. We will focus primarily on eventing and pipeline CRDs from Kubernetes to show this can be automated.