One Jenkins Seed Job, Multiple Pipeline Jobs: A Practical DevOps Guide
Introduction
In previous blog, we have seen how to create jobs declaratively, In this blog we will see how we can create a Parent Job(also called as seedjob) that creates other Jobs.
Why we need parent/seedjob ?
Remember we created our Jenkins Job declaratively by updating our helm values.
Now this is the values yaml which is the core for my Jenkins installation. To create another Job pipeline, I can have another -script block and add a new pipeline in the same values.yaml which gets messy over time. Instead I can create a master job pipeline that would call another repository which would execute groovy script to create more pipelines and it works at scale very well and we can have segregation based on various usecases.
Current setup.
How it works:
- Update the Helm values with the Jenkins job definition.
- Deploy the updated Helm configuration. Jenkins creates the job automatically.
- The job checks out another Git repository containing the Terraform script in
JenkinsFile. - When we run the job, Jenkins executes the pipeline defined in that repository, performing the Terraform operations.
Flow:
Helm Values updated→ Jenkins Job Created → Terraform Pipeline Repository → Terraform Execution
Create Parent and Child Jenkins Jobs
How it works:
- Update the Helm values to define the parent job.
- Jenkins creates the parent job.
- The parent job accesses a separate Git repository containing the configuration script (jobdsl script) for creating another Jenkins job.
- The child job is created using that script.
- The child job checks out a third repository containing the Terraform
Jenkinsfile. - When the child job runs, Jenkins executes the Terraform pipeline. (When the parent Job runs, it executes the Child job creation pipeline)
Helm Values → Parent Job Created → Child Job Configuration Repository → Child Job Created → Terraform Pipeline Repository → Terraform Execution
So using this way, we can configure a parent job which calls a repository where customers can define their own jobs. And they can run parent job which will create the child jobs.
Creating Parent job with Helm
Defining the Job inside my values.yaml
Ref: https://github.com/anandambitionv2/jenkins-config/blob/main/mycustomvalues/override-values.yaml
Get Anand Rao - Microsoft MVP’s stories in your inbox
Join Medium for free to get updates from this writer.
Now this Parent Job is targeting a JenkinsFile in a repository (https://github.com/anandambitionv2/Jobs-repo) where I have a script to create Child Jobs.
LEts look at the repository that creates Child Jobs — https://github.com/anandambitionv2/Jobs-repo\
The script has got the step JobDSL which is to create Jenkins Job definition
So now lets run the parent job which calls the child job repo and executes the pipeline script that creates child jobs
To create parent job, lets update our helm. Am using argocd, so I just push changes for my helm values and the Jenkins Master job gets created.
Master job got created
Now to create our Child job, we run the masterjob pipeline which will call the script (jobdsl) from another repo and create child job.
Lets run a build from masterjob
Before this, we need to approve script from UI
AFter approving, running the masterjob.
Masterjob executed successfully
This created another job
(Note: Now running the child job calls another repository which is having the script to execute terraform runs.)
Now to create another child job, I can update the Childjob repository and add a new Jobdsl step.
Push the changes on the child repo and running the master job pipeline will execute the updated script from the child repo(do the approval from the UI again if it asks)
Pipeline successful and created the second job
Now this child job can be parameterized and ran to create pipelines in a targeted way. Also the individual child jobs can be placed in separate Jenkinsfile (jenkinsfile1, jenkinsfile2) and it gets more complex and advanced.
