A dependency is created when a pipeline is configured to depend on either an SCM or another pipeline. The domain concept for this is called a material. If a pipeline has a dependency it has been configured to have either an SCM material or a pipeline material.
SCM Dependency
 |
Pipeline 1 depends on an SCM.
Pipeline 1 will trigger each time it polls the SCM and detects a new revision.
|
Pipeline Dependency
 |
Pipeline 2 depends on Pipeline 1.
Pipeline 2 will trigger each time Pipeline 1 successfully completes.
|
Pipeline and SCM Dependency
 |
Pipeline 2 depends on Pipeline 1 and an SCM.
The expectation may be that a new revision will trigger both Pipeline 1 and 2. That’s not the case.
Instead, Pipeline 2 will only trigger if a new revision has successfully passed through Pipeline 1.
Real world example:
- SCM contains application code and acceptance tests
- Both pipelines depend on the SCM
- Pipeline 1 compiles code and creates an artifact
- Pipeline 2 fetches the build artifact and runs the tests that were written for the compiled code
- Pipeline 2 has to use the same acceptance tests committed with the code
- Someone commits code and tests as part of the same SCM commit.
|
Fan-out
 |
Pipelines 1 and 2 depend on the same SCM.
A new revision will trigger both Pipeline 1 and 2.
Why fan-out?
- Speed up the feedback loop and enable a team to fail and learn faster.
|
Fan-in - Pipeline
 |
Pipelines 1 and 2 depend on an SCM.
Pipeline 3 depends on Pipelines 1 and 2.
The expectation may be that any successful run of Pipeline 1 or Pipeline 2 will trigger Pipeline 3. That’s not the case.
Instead, Pipeline 3 will only trigger if a new revision (SCM) has successfully passed through both Pipeline 1 and Pipeline 2.
Why fan-in?
- Faster feedback - You can fan-out to get significantly faster feedback now that Go handles the fan-in.
- Eliminates spurious builds - Test software only when it's necessary and don’t waste resources.
- Meaningful feedback - You won't have false-positives or false-negatives.
|
Fan-in - Pipeline and SCM
 |
Pipelines 1 and 2 depend on SCM 1.
Pipeline 3 depends on Pipelines 1 and 2, and SCM 2.
Pipeline 3 will trigger:
- if a new revision (SCM 1) has successfully passed through both Pipeline 1 and Pipeline 2
- each time it polls SCM 2 and detects a new revision
|
Fan-in - Pipeline with different originating SCM
 |
Pipelines 1 and 2 depend on SCM 1.
Pipeline 3 depends on SCM 2.
Pipeline 4 depends on Pipelines 1, 2 and 3.
Pipeline 4 will trigger:
- if a new revision (SCM 1) has successfully passed through Pipelines 1 and 2.
- each time Pipeline 3 successfully completes (SCM 2)
|
------------------------------------------------------------------------------------------------------------------------------------
For each diagram:
- SCM is a Go supported SCM
- each rectangle is a Go pipeline
- assume that 2 different developers check in code at different times
- Go polls the SCM for changes automatically (default behavior)
- There are no manual gateways
------------------------------------------------------------------------------------------------------------------------------------
Once you understand how Go manages dependencies your team can model a Build-Test-Release workflow like this.
Comments
3 comments
For other readers, you can find the same information with images here: http://www.thoughtworks.com/products/docs/go/13.4/help/concepts_in_go.html
The images don't seem to load for me on this page. The image requests (e.g. https://lh5.googleusercontent.com/ebKGWNJ8_oKdrY9ZPIda60p_8yOG8RvQ0drVUPpNyC902yh3Ndd_CRdqbjpYFeFO6mNxXvL0-iZKdL0VyMaYjd1hNcKdx1ix8k8VyB5VZhg3ryoRMJE) get a 403 response back from Google.
Your client does not have permission to get URL
/ebKGWNJ8_oKdrY9ZPIda60p_8yOG8RvQ0drVUPpNyC902yh3Ndd_CRdqbjpYFeFO6mNxXvL0-iZKdL0VyMaYjd1hNcKdx1ix8k8VyB5VZhg3ryoRMJEfrom this server. (Client IP address: XXX.XXX.XXX.XXX)ACL Denied That's all we know.
Thanks for reporting this Mark. We have fixed the links now.
Cheers,
Jyoti
Please sign in to leave a comment.