top of page

Cron job with Horizontal Scaling

Writer's picture: Heeth JainHeeth Jain


A client needed a cron job in their backend server (with auto horizontal scaling).


This makes a cron job complicated.


Horizontal scaling in brief is having multiple copies of the server running together so that the load is divided between each server.


If I were to run the cron job inside the server, then with horizontal scaling, the cron will run in every server instance.


Let's say we have to call Service B when a cron event occurs. Now if we have 5 server instances, then 5 crons would run and Service B would be called 5 times (duplicates).


I solved this problem with a simple solution by having an API to start the cron job.


A lambda cron would call the API, and because of the load balancer, the API would hit only one single server instance (not all) at any time, thus Service B will only be called only once.


Thus, solving the duplication issue!

bottom of page