Distributed training on Kubernetes

This project sits exactly where my two worlds meet: machine learning and cloud-native infrastructure. It demonstrates distributed model training with TensorFlow’s ParameterServerStrategy running on a Kubernetes cluster.

Problem

Training a model on one machine is a notebook cell; training it across a cluster is an infrastructure problem: role assignment, service discovery, networking and failure isolation. This project works through that problem in its minimal, readable form.

Architecture

• A parameter server pod coordinates model variables and training state.
• Worker pods run the training steps, connecting to the parameter server over gRPC.
• Each pod discovers its role and the cluster topology through a TF_CONFIG environment variable defined in the Kubernetes manifest.
• Network policies restrict communication to exactly the paths the training topology needs.

What it shows

That distributed training is fundamentally a cluster-infrastructure design problem (role assignment, service discovery, networking and failure isolation), not just a machine-learning one.