Spring batch migration to kubernetes platform

1
votes

I have my current application batches developed on Spring Batch framework. Now I need to forklift the same to a Kubernetes platform for cloud nativity. Please help me with the following queries.

  1. How do I achieve auto scaling (HPA) for the spring batches.
  2. Whether spring batch remote partitioning is the recommended approach for auto scaling in k8s platform and any best practices for this approach. Like how to effectively scale-up and scale-down etc.
  3. What are the advantages of refactoring the current spring batch over Spring Cloud Task. Is this a best practice for cloud compliance.

Thanks

UPDATE
while choosing spring batch remote partitioning - whether worker containers should be configured over k8s deployment(pods) or k8s jobs. Any recommended approach ?

do we have hpa/autoscaling for k8s jobs ?

spring batch remote partitioning over k8s platform which is better - using MessagingPartitionHandler+k8s jobs(worker queue pattern) or (DeployerPartitionHandler+KubernetesTaskLauncher) ?

1
kubernetesspring-batchspring-cloudspring-cloud-task
have you been checking docs on Jobs kubernetes.io/docs/concepts/workloads/controllers/job ? And speakerdeck.com/benas/spring-batch-kubernetes?slide=23 – Nick
hi nick, I had gone through these articles but my concerns/doubts are on these specific areas (Updated the above question) – Balu R

1 Answers

1
votes

do we have hpa/autoscaling for k8s jobs ?

No. There is a concept of Jobs Parallelism, but it is not quite the HPA.

If you have a continuous stream of background processing work to run (so work queue Jobs don't fit you), then consider running your background workers with a ReplicaSet instead, and consider running a background processing library such as https://github.com/resque/resque.

A ReplicaSet can also be a target for Horizontal Pod Autoscalers (HPA). That is, a ReplicaSet can be auto-scaled by an HPA.