Merged in DCD-621-parallelize-builds (pull request #20)

DCD-621 parallelize builds

* DCD-621: Parallelizes tests ny invoking the parralelize script

* DCD-621: Removes leading / to start jira command in product_startup role

* DCD-621: Runs a scenario by name instead of using the all parameter

* DCD-621: A better find all scenarios function to ensure we look at relevant molecule scenarios. Better logging too

* DCD-510: Parallelizes tests further, running in 26 batches on 1 each. The pipeline file is now generated using Jinja

* DCD-510: Adds a pre check stage to verify if number of batches match actual number of scenarios. This test will fail (deliberately)

* DCD-510: Adds a pre check stage to verify if number of batches match actual number of scenarios. Fixes 'test' failure

* DCD-590: Adds note to development.md on how to generate a pipeline file. Adds a makefile and updates a few script commands

* DCD-590: Better documentation in development README document. Updates YML with instructions on how to fix issues with the prevalidation stage

Approved-by: Steve Smith <ssmith@atlassian.com>
Approved-by: Ben Partridge <bpartridge@atlassian.com>

pipeline_generator/templates/bitbucket-pipelines.yml.j2

@@ -0,0 +1,39 @@
+---
+
+image: atlassian/default-image:2
+options:
+  size: 2x
+
+definitions:
+  services:
+    docker:
+      memory: 4096
+
+pipelines:
+  default:
+    - step:
+        name: Pre Parallelization stage
+        script:
+          - echo "Running tests in {{ parallel_steps|length }} batches"
+    - step:
+        name: Check if number of batches match actual number of scenarios
+        script:
+          - |
+              actual_scenario_count=$(find ./roles -type f -name "molecule.yml" -exec dirname {} ';' | wc -l |  sed -e 's/^[[:space:]]*//')
+              grep  "Running tests in ${actual_scenario_count} batches" bitbucket-pipelines.yml
+              GREP_RETURN_CODE=$?
+              if [[ $GREP_RETURN_CODE -ne 0 ]]; then
+                  echo "Mismatch between expected and actual number [${actual_scenario_count}] of scenarios. Please look at https://bitbucket.org/atlassian/dc-deployments-automation/src/master/DEVELOPMENT.md for instructions on how to fix this error."
+                  exit $GREP_RETURN_CODE
+              fi
+    - parallel:
+       {% for parallel_step in parallel_steps -%}
+        - step:
+           name: {{ parallel_step.name }}
+           services:
+             - docker
+           script:
+           {% for scriptCommand in parallel_step.scriptCommands -%}
+              - {{ scriptCommand }}
+           {% endfor %}
+       {% endfor %}