Say we have a job in Jenkins (publish_artifact) that archives an artifact in Jenkins. Every time this job succeeds, we want to auto-schedule a Go pipeline (targetPipeline) that will then download the artifact and use it. There are two ways to do this. One using Go API to notify Go and another that uses a git repo that Go then polls. The first approach is simpler but the second one has the added benefit of some reverse traceability.
1. Go API Approach
Jenkins Set up
publish_artifact is a job in Jenkins that archives an artifact (artifact.txt) as a post build action (see image below). We make use of the parameterized build trigger plugin to add another post build action that triggers the handing-over job in Jenkins (talk2go).
Note that talk2Go needs to declare itself as a parameterized build to complete the binding.
talk2Go simply uses the following shell command to schedule the pipeline in Go. Refer Go Pipeline scheduling API for details.
(some newlines added for readability, remove if copy/pasting)
curl -u user:passwd -d "variables[JENKINS_URL]=${JENKINS_URL}& variables[JENKINS_JOB_NAME]=publish-artifact& variables[JENKINS_JOB_NUMBER]=${PUBLISH_JOB_NUM}& variables[JENKINS_ARTIFACT]=artifact.txt" http://goserver:8153/go/api/pipelines/targetPipeline/schedule
|
Go set up
For the above call to work, we need to define pipeline level environment variables in the target Go pipeline as follows.
The Go pipeline also needs a material. Ideally this would be the repo containing deployment scripts (such as the one below - shown as an inline task here for readability). Finally, we add a Go job that downloads the artifact from Jenkins.
When set up correctly, we should see output from the Go pipeline as follows. The artifact gets saved into downloadedFile under the pipeline working directory.
[go] Start to execute task: <exec command="/bin/bash" >
<arg>-c</arg>
<arg>curl -o downloadedFile
${JENKINS_URL}job/${JENKINS_JOB_NAME}/${JENKINS_JOB_NUMBER}/artifact/${JENKINS_ARTIFACT}</arg>
</exec>.
…
[go] setting environment variable 'JENKINS_JOB_NAME' to value ''
[go] setting environment variable 'JENKINS_ARTIFACT' to value ''
[go] setting environment variable 'JENKINS_JOB_NUMBER' to value ''
[go] setting environment variable 'JENKINS_URL' to value ''
[go] overriding environment variable 'JENKINS_URL' with value 'http://jenkins-server:8080/'
[go] overriding environment variable 'JENKINS_JOB_NAME' with value 'publish-artifact'
[go] overriding environment variable 'JENKINS_JOB_NUMBER' with value '7'
[go] overriding environment variable 'JENKINS_ARTIFACT' with value 'artifact.txt'
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
2. Material Approach
Jenkins set up
In this approach, we make talk2Go provide the required information to Go via a git repo rather than via API. Say we have a git repo called testrepo with a file jenkins.properties containing the following:
$ cat jenkins.properties
JENKINS_URL=http://jenkins-server:8080/
JENKINS_JOB_NAME=publish-artifact
JENKINS_JOB_NUMBER=10
JENKINS_ARTIFACT=artifact.txt
talk2Go updates and pushes this file to git as follows. Note how we write the git checkin comment to avail of reverse traceability from Go.
#!/bin/bash
cat >/var/lib/jenkins/testrepo/jenkins.properties <<EOL
JENKINS_URL=$JENKINS_URL
JENKINS_JOB_NAME=publish-artifact
JENKINS_JOB_NUMBER=$PUBLISH_JOB_NUM
JENKINS_ARTIFACT=artifact.txt
EOL
# newline added for readability, remove if copy/pasting
cd /var/lib/jenkins/testrepo;git commit jenkins.properties -m "#$PUBLISH_JOB_NUM checkin by jenkins";
git push origin master
Go set up
The Go pipeline has testrepo as a material. It gets scheduled in due course, runs a job that exports environment variables from jenkins.properties and then makes the usual Jenkins fetch artifact API call.
/bin/bash -c
#newline added for readability, remove if copy/pasting
while read line; do export $line; done <jenkins.properties;curl -o downloadedFile
${JENKINS_URL}job/${JENKINS_JOB_NAME}/${JENKINS_JOB_NUMBER}/artifact/${JENKINS_ARTIFACT}
This approach no longer requires us to pre-define environment variables at the pipeline level. To get reverse traceability, we configure Go to link checkin comments back to Jenkins
That's it, now when publish_artifact runs in Jenkins, it then triggers talk2Go which pushes to testrepo. Go polls testrepo, detects the changes and schedules the target pipeline. The Changes section now has a helpful link back to the triggering job in Jenkins!
Handover external artifacts from Jenkins to Go
What if the artifact was generated but not stored in Jenkins? Simply use a different parameter to trigger talk2Go from publish_artifact.
ARTIFACT_URL=<path to artifact in external repo>
Soon Go pipelines will have the ability to poll package repositories. That will give us another option in addition to the above notification approach.
If, for some reason, you happen to be storing the artifacts in version control instead, you could simply create a Go pipeline to poll the VCS (material). In case of VCS, there is also the option to use a non-specific notification that just says material changed without reference to a specific pipeline.
http://www.thoughtworks-studios.com/docs/go/current/help/materials_api.html
Comments
0 comments
Please sign in to leave a comment.