This hands-on workshop teaches you how to build and deploy secure, self-hosted AI applications using Java and Spring AI with local Large Language Models on Kubernetes. Learn how to create confidential, scalable AI solutions without third-party data exposure.
Workshop Link: Devoxx Belgium 2025 - Building Private, Scalable AI Applications
When: Monday, October 6, 2025, 09:30-12:30
Where: Devoxx Belgium, Kinepolis - BOF 2
Level: Beginner
Track: Architecture
Tools like ChatGPT, Claude, and Copilot are powerful, but relying on them means giving up control of your data to third-party providers. For organizations bound by confidentiality or regulatory requirements, self-hosted models such as Llama, Mistral, Phi, and Qwen provide a secure alternative, running locally or on Kubernetes clusters.
In this tutorial, you'll learn how to:
- ✅ Build a sample AI application using Java and Spring AI
- ✅ Integrate the application with a self-hosted LLM
- ✅ Deploy and scale both the application and the LLM on Kubernetes
- ✅ Implement an asynchronous architecture using message queues
- ✅ Ensure production readiness with resource requests, autoscaling, and real-time metrics
- ✅ Dive deeper into AI capabilities by creating an MCP Server and Client
Note: While the example application uses Java, all concepts around LLM deployment and scaling are fully language-agnostic, making this workshop valuable for developers of all backgrounds.
Slides are available here.
Understand the fundamentals of Generative AI, Large Language Models, and why self-hosted solutions matter for enterprise applications.
Get hands-on with self-hosted LLMs using Ollama, and build your first AI-powered Spring Boot application.
Learn to build production-grade AI applications with RAG, vector databases, streaming responses, and advanced Spring AI features.
Deploy and scale AI applications on Kubernetes with asynchronous processing, autoscaling, and production monitoring.
Master advanced techniques for customizing LLMs including prompt engineering, fine-tuning, quantization, and model optimization.
Build autonomous AI agents that can plan, use tools, and accomplish complex multi-step tasks using the Model Context Protocol.
Recommended Setup: GitHub Codespaces (Everything pre-configured! ✨)
This workshop is designed to work seamlessly with GitHub Codespaces. Simply click the button below to get started with a fully configured development environment:
The Codespace includes:
- ✅ Java 25 with Maven and Gradle 9
- ✅ Docker-in-Docker support
- ✅ Kubernetes (kubectl, Helm, Minikube)
- ✅ Ollama pre-installed for running LLMs
- ✅ All necessary development tools
If you prefer to run locally, ensure you have:
- Java 17 or higher - Download
- Docker Desktop - Download
- kubectl - Install Guide
- A Kubernetes cluster (local Minikube/Kind or cloud-based)
- Ollama - Install Guide
- Git - Download
- Your favorite IDE (IntelliJ IDEA, VS Code, Eclipse, etc.)
- Basic knowledge of Java and Spring Boot
-
Launch Codespace:
-
Wait for setup to complete (about 2-3 minutes)
-
Verify your environment:
java -version docker --version kubectl version --client ollama --version
-
Start learning: Begin with Module 0: (Generative) AI Fundamentals
-
Clone this repository:
git clone https://github.com/SandraAhlgrimm/springai-kubernetes-workshop.git cd springai-kubernetes-workshop -
Verify your setup: Follow the instructions in Module 0: (Generative) AI Fundamentals
-
Start learning: Begin with Module 0 and progress through each module sequentially.
Principal Solutions Engineer at Broadcom
Timo Salm is a Principal Solutions Engineer at VMware Tanzu by Broadcom with over a decade of experience in customer-facing roles, modern applications, DevSecOps, and AI. He ensures that the most strategic customers in the EMEA region achieve their goals with VMware Tanzu's developer platform, data, AI, and commercial Spring products.
- 🐦 Twitter: @salmto
- 💼 LinkedIn: timosalm
- 🦋 Bluesky: timosalm.bsky.social
Senior Cloud Advocate at Microsoft
Sandra Ahlgrimm is a Senior Cloud Advocate at Microsoft, specializing in supporting Java Developers. With over a decade of experience as a Java developer, she brings a wealth of knowledge to her role. Sandra is passionate about containers and has recently learned to love AI. She actively contributes to the Berlin Java User Group (JUG) and the Berlin Docker MeetUp.
- 🐦 Twitter: @skriemhild
- 💼 LinkedIn: sandraahlgrimm
- 🦋 Bluesky: sandraahlgrimm
- Spring Boot - Application framework
- Spring AI - AI integration for Spring
- Java 17+ - Programming language
- Kubernetes - Container orchestration
- Docker - Containerization
- Self-hosted LLMs - Llama, Mistral, Phi, Qwen
- Message Queues - RabbitMQ or Kafka
- Prometheus & Grafana - Monitoring
✨ Data Privacy: Keep your data within your infrastructure
🔒 Compliance: Meet confidentiality and regulatory requirements
💰 Cost Control: Avoid per-token pricing of cloud AI services
🎯 Customization: Fine-tune models for your specific domain
📈 Scalability: Learn to scale AI workloads on Kubernetes
🏗️ Architecture: Understand Domain-Driven Design with small language models
This workshop material is licensed under the MIT License. See LICENSE file for details.
We welcome feedback and contributions! Please open an issue or submit a pull request if you find any problems or have suggestions for improvements.
Ready to build secure, scalable AI solutions that keep your data under control? 🚀
Start with Module 0: (Generative) AI Fundamentals →