Go’s support for job level properties can be used to pass variables to other pipelines. Before, we get into properties, let’s see what is already available.
What’s available out-of-the-box
As documented, the following dependency variables are available to child pipeline tasks as environment variables.
GO_DEPENDENCY_LABEL_PARENT e.g. value: parent-123
GO_DEPENDENCY_LOCATOR_PARENT e.g. value: parent/123/stage2/1
By child pipeline, we mean a pipeline that is immediately downstream to a parent pipeline i.e. the child has the parent configured as a pipeline dependency material. What if we want the label and locator of the grandparent? Or if we just want to pass a value generated by a task in one pipeline to another pipeline?
Passing via Properties
Publishing Properties
note: Properties serve a different purpose from environment variables or parameters.
A property is a key value pair. To set a property (e.g. appversion=1.2.3) at runtime for the current job, just use this API call from a task.
curl -u user:passwd -d "value=1.2.3" http://go-server:8153/go/properties/${ GO_PIPELINE_NAME}/${GO_PIPELINE_COUNTER}/${GO_STAGE_NAME}/${GO_STAGE_COUNTER}/${GO_JOB_NAME}/appversion
Fetching properties
To fetch the above property from a dependent pipeline, we use:
curl -u user:passwd http://go-server:8153/go/properties/${GO_DEPENDENCY_LOCATOR_PARENT}/job-name/appversion 2>/dev/null|sed -n '2p'
The API returns the property key-value in CSV format with key on line 1 and value on line 2. The sed command above returns just the value (line 2).
To fetch properties from an arbitrary pipeline, we’ll need to expand the URL section provided by GO_DEPENDENCY_LOCATOR ourselves. i.e. pipeline-name/pipeline-counter/stage-name/stage-counter
Passing via artifacts
Another approach is to stick the key value pairs into a properties file and publish it as a metadata artifact. For dynamic publishing, use the create or append API. A downstream pipeline can now fetch this metadata file using fetch ancestor artifact. Other arbitrary pipelines could use the artifact API.
Comments
5 comments
Could you elaborate on how to save the values fetched from the upstream/dependent pipeline so they can be used in the current one?
Rather than saving, could you do with fetch and use when needed? In the following examples, I fetch and pass the property as an argument to myscript.sh (regular material)
<exec command="/bin/bash" > <arg>-c</arg> <arg>path/to/myscript.sh `curl http://go-server:8153/go/properties/${GO_DEPENDENCY_LOCATOR_XX}/job-name/appversion 2>/dev/null|sed -n '2p'`</arg> </exec>
OR
<exec command="/bin/bash" > <arg>-c</arg> <arg>prop=$(curl http://go-server:8153/go/properties/${GO_DEPENDENCY_LOCATOR_XX}/job-name/appversion 2>/dev/null|sed -n '2p');/path/to/myscript.sh $prop</arg> </exec>Except all this is very OS-specific based on where the Agent runs when there is nothing OS specific about what I am trying to do.
Specifically, I am trying to change the profile that is used for my Maven command. In one case, I want to run:
mvn install -PlatestSnapshot
and in the other case I want to run:
mvn install -PlatestRelease
In my custom command I prefer something like:
mvn install -P${ARTIFACT_TYPE}
But I can't set properties in the scheduler (that I have found). So I set the property on the parent Pipeline thinking I could reference it from the downstream pipeline. However, I don't think there is a way to save the result of the CURL command to a property for use elsewhere without OS-specific scripts. Not to mention again that the CURL command is very OS-specific anyway.
In an attempt to use environment variables, I've tried:
mvn install -P%ARTIFACT_TYPE%
and then passing the environment variable in via the REST call to schedule the job. But it only works on Windows due to the %%. If I make it Linux-specific and do:
mvn install -P$ARTIFACT_TYPE
it doesn't work at all because $ARTIFACT_TYPE is not evaluated properly on Linux, and even if it did, it is still OS-specific.
Is there not a way to pass in a property when scheduling a job that can be set and used as any other property without writing OS-specific scripts?
Hmm. I'm afraid it is OS specific when it comes to dereferencing an environment variable. The only other way I can think of is to use parameters but they are statically bound so I'm not sure if it will serve your purpose. Do take a look at : http://support.thoughtworks.com/entries/23694116-Parameters-and-Environment-Variables
>$ARTIFACT_TYPE is not evaluated properly on Linux
Can you try with ${ARTIFACT_TYPE}
So, following is the curl command to get the variable:
curl -u user:passwd http://go-server:8153/go/properties/${GO_DEPENDENCY_LOCATOR_PARENT}/job-name/appversionI believe this will be executed from the downstream pipeline (having upstream pipeline as a material). However, to resolve ${GO_DEPENDENCY_LOCATOR_PARENT}, I need to know the name of the parent pipeline (i.e. PARENT in this example). How can I know that? Can't see any environment variable for it.
Please sign in to leave a comment.