Friday 20 July 2018

Job Scheduler - YAML for writing job definition?

In our legacy job scheduling software (built on top of crontab), we are using apache config format for writing a job definition and we are using perl config general to parse the config files. This software is highly customized and have functionalities like running commands in my job after checking if dependency of that command is met or not, rescheduling jobs in case command fails, supporting custom notifications etc.

We are now planning to rewrite this software in python and considering options like YAML instead of apache config to write job definition. Is YAML good for writing such dynamic configurations?

Example of job definition (run this job at 2AM daily, check if it is tuesday and not holiday in India, if yes reserve my flight and send notification):

// python function to check if it is tuesday
checkIfTuesdayAndNotHoliday()

<job> 
    calendar: indian

        <dependency: arbitrary_python_code: checkIfTuesdayAndNotHoliday()>
        <command>  
            check availability of flight
        </command>

        <success: notify: email: agrawall/>
        <failure: notify: email: ops>
        <command>
            some command to book my flight
        </command>
</job>

<crontab> 0 2 * * * </crontab>

I am struggling to understand what format should I use to define job (YAML, Apache Config, XML, JSON etc). Note that this job definition will be converted to job object inside my python script.



from Job Scheduler - YAML for writing job definition?

No comments:

Post a Comment