New SaturnCI Job Configuration: Child Jobs, Hooks, and Decoupled Environments

A change has been made to the SaturnCI configuration API which addresses the following weaknesses:

1. There was no good way to trigger child jobs from parent jobs.
2. When a job got rerun, the rerun would illogically fail to rerun its child jobs.
3. There was no clear distinction between behavior that happened inside of jobs and behavior that happened between jobs.
4. The definition of a job was conflated with the definition of the environment for that job, hurting clarity and limiting the reusability of environments across different jobs.

Consequently, I've more or less redesigned the SaturnCI configuration API from the ground up.

SaturnCI's conceptual building blocks

The core unit in SaturnCI is a Job, from which are triggered Job Runs. Every job runs in an Environment, which is literally just a Docker Compose environment. Below is an example of a SaturnCI configuration which defines four types of jobs and three types of environments.

.saturnci
├── environments                                                                                                                                                                             
│   ├── rspec                         
│   │   ├── Dockerfile
│   │   ├── database.yml
│   │   └── docker-compose.yml
│   ├── production_image_build                                                                                                                                                               
│   │   ├── Dockerfile         
│   │   └── docker-compose.yml    
│   └── default                                                                                                                                                                              
│       ├── Dockerfile                                                                                                                                                                       
│       └── docker-compose.yml                                                                                                                                                               
└── jobs
    ├── github_push
    │   ├── config.yml
    │   └── github_push_job.rb
    ├── production_image_build
    │   ├── config.yml
    │   └── production_image_build_job.rb
    ├── test_suite
    │   ├── config.yml
    │   └── test_suite_job.rb
    └── deploy
        ├── Dockerfile
        ├── apply
        ├── docker-compose.yml
        ├── k8s
        │   ├── deployment.yaml
        │   ├── migration-job.yaml
        │   └── worker-deployment.yaml
        └── migrate

A job's (extremely brief) configuration file specifies the environment in which the job runs.

# .saturnci/jobs/test_suite/config.yml
environment: rspec

If you're familiar with SaturnCI's previous configuration API, the concept of an entrypoint, which would get triggered upon every GitHub webhook event, is gone. It has been replaced with a GitHubPushJob which gets triggered upon every GitHub push. We'll look at this job but first let's look at how child jobs get triggered.

Triggering child jobs

Every job can take an after hook. The after hook returns an array of child jobs to be run. Below is a job which triggers a deploy if and only if the test suite has passed and we're on the main branch.

# .saturnci/jobs/test_suite/test_suite_job.rb

class TestSuiteJob
  def after(job_run)
    return if job_run.status != "Passed"
    return if job_run.branch_name != "main"

    [{ job_name: "deploy" }]
  end
end

The Child Job Specification always requires, at minimum, a job_name, which corresponds to the name of the folder where that job is defined. If a job_name of deploy is given, as in the above example, SaturnCI will go looking for a job defined in .saturnci/jobs/deploy.

You can see that an after hook gets, as an argument, a job_run. This is, as you may imagine, the job which just ran. Having access to this job run can be useful if running a child job depends on things that happened with its parent job run.

A job can trigger multiple children as follows:

# .saturnci/jobs/test_suite/test_suite_job.rb

class TestSuiteJob
  def after(job_run)
    return if job_run.status != "Passed"
    return if job_run.branch_name != "main"

    [
      { job_name: "deploy" },
      { job_name: "deliver_test_suite_run_success_email" }
    ]
  end
end

Since after hooks are written in plain Ruby, you can add whatever conditions (or other behavior) you like.

You can find more information on SaturnCI's configuration API in the SaturnCI configuration documentation.