I have an API which updates yaml file. I need to periodically update a yaml file which is deployed in a pipeline

Question

I have an API which updates yml file. I need to periodically update a yaml file which is deployed in a pipeline. Following set of shell commands need to be run after every 7 days.

Git pull the package.
Run a script which makes call to an API and updates the package.
Git push this package back to pipeline.

For this I need to find a service to schedule jobs(cron jobs). I am thinking of AWS Lambdas or DJS for this. Please suggest any better alternative and how could I use git in it.

davidverweij davidverweij · Accepted Answer · 2022-02-04T13:19:05

If you are hosting the yaml file on GitHub, I'd suggest looking at GitHub Workflows. You can set GitHub actions to (next to respond to events) run on a schedule (see here). Then, the workflow has access to that repository (your package), can make API calls and push back changes. Something like this:

NOTE: I did not test this workflow

name: Trigger API update

on:
  schedule:
    - cron:  "0 6 * * *" # run everyday at six

jobs:
  ping_api_and_update:
    runs-on: ubuntu-latest
    steps:

    - uses: actions/checkout@v2
      with:
        ref: ${{ github.head_ref }}

    - name: Webhook
      uses: distributhor/workflow-webhook@v2
      env:
        webhook_url: ${{ secrets.API_URL }}
        webhook_secret: ${{ secrets.API_SECRET }}

    - name: do something with output
      run: |
        command based on API result
        maybe another command
        edit the yaml file in this repo
      shell: bash 

    - name: Commit changes
      uses: stefanzweifel/git-auto-commit-action@v4
      id: commit
      with:
        commit_message: Auto-update YAML
        commit_author: CI/CD <[email protected]>

Alternatively, most (e.g. linux) servers can handle cron-jobs. You could let that cron-job trigger a script to handle all the steps. If you want to update a git repository then, you could use git on your server, and the specific repository as a git submodule. The script could then push the changes to the submodule to its respective repository.

As with most things, if it is private, you'll need to handle authentication and all that.

Hope any of these help your thinking process! Cheers.

I have an API which updates yaml file. I need to periodically update a yaml file which is deployed in a pipeline

1 Answers