Machine Learning on the JVM: Java and Kotlin Inference That Survives Production
Java and Kotlin are strongest for integrating trained models into JVM products, batch pipelines, and governed enterprise services. They are rarely the default environment for exploratory model training, where Python's library and notebook ecosystem dominates. A robust design trains where the model is best supported, exports a versioned artifact, and validates JVM inference against a reference implementation.
Choose an inference boundary before a framework
ONNX Runtime provides a Java API for executing ONNX models; its official Java guide covers the runtime dependency and session model. Deep Java Library offers a higher-level engine-neutral API and model-loading facilities described in the DJL quick start. Select by supported operators, hardware backend, packaging, licence, and model format—not by a framework popularity list.
In-process inference reduces network latency but shares memory and failure fate with the application. A separate model service scales and deploys independently but adds serialization, network, authentication, and timeout behavior. Benchmark both with production-shaped batches and concurrency. Never assume a model that loads on a developer laptop fits the container memory limit.
Classical ML and distributed pipelines have different tools
Tribuo supplies typed Java APIs for classification, regression, clustering, provenance, and model evaluation; the official documentation shows its reproducibility model. Spark MLlib is appropriate when feature transformation and training already run on Spark data, not as a dependency for a small online predictor. Weka remains valuable for education and analysis but should not be treated as the universal production answer.
Kotlin can call these Java libraries naturally, yet Kotlin wrappers may lag the underlying engine. Test Java interoperability, tensor ownership, native-library extraction, target architecture, GPU drivers, and shading in the actual build. Keep model code separate from web controllers so the engine can be replaced without rewriting the product.
Operate the model as a versioned dependency
Store the model hash, training-data version, feature schema, preprocessing parameters, runtime and provider versions, evaluation metrics, and approval record. At startup, reject an incompatible feature or model version loudly. Run golden-input tests comparing Python and JVM outputs within a justified numeric tolerance, plus malformed, empty, oversized, and adversarial inputs.
Monitor latency, errors, input drift, output distribution, and business outcomes without logging sensitive raw features. Define rollback and a fallback when the model or accelerator fails. The Spark ML guide is useful for pipeline mechanics, but governance, privacy, bias review, and human escalation remain product responsibilities. A JVM framework does not make a model production-ready by itself.
Published · Updated