Join Pipeline allows the user to combine the steps of two pipelines into a single pipeline.
This section covers the following topics:
We are going to use the example project from Split Pipeline, which was derived from the Sydney Buses example.
After creating the project and splitting the ingest pipeline we have this in our Network Diagram:
Currently, a pipeline is only a candidate to be joined to its upstream pipeline if all the following are true:
If any of these conditions does not hold, the Join menu option will not be available.
We cannot simply re-join ingest and ingest_continued. The ingest_continued pipeline has a sink (nexus) which disables Join.
First we need to disconnect the nexus sink from ingest_continued by removing the route-to-sink step.
Now we have an example of the kind of pipelines we can combine, we can go back to the Network Diagram and right click on ingest_continued, then select Join:
You will notice that:
We can now disconnect the categorized_buses sink from ingest by removing the route to sink step:
And we can re-route ingest to the nexus sink (be sure to select nexus as the sink):
And finally, from the Network Diagram you can delete the categorized_buses sink that was added when we split the pipelines originally. We are now back to where we started before the Split Pipeline: