CICube

How to Pass Information Between Jobs in GitHub Actions

CICube's photo
CICube
·

5 min read

cicube.io

Introduction

In a complex CI/CD workflow, various data would often have to be exchanged across jobs. Anything from generated environment variables down to important build artifacts would be fair game. GitHub Actions supports this via what are called job outputs, which can make your workflows much more modular and flexible. Be it API credentials, a database ID, something dynamically generated, job outputs provide a structured way to carry information forward between dependent jobs.

Definition of Job Outputs

Outputs of a job in GitHub Actions are defined inside of the job using the outputs field. These will map to the results of steps in the same job and can be referenced from other jobs utilizing the dependsOn keyword. Outputs will always be strings, and will be evaluated at the closing of the job execution. Outputs are particularly useful for when you want to be able to pass dynamic information such as tokens, IDs, or version numbers across your workflow.

jobs:
  job1:
    runs-on: ubuntu-latest
    outputs:
      output1: ${{ steps.step1.outputs.result }}
    steps:
      - id: step1
        run: echo "result=hello" >> "$GITHUB_OUTPUT"
  job2:
    runs-on: ubuntu-latest
    needs: job1
    steps:
      - run: echo "Job 1 output: ${{ needs.job1.outputs.output1 }}"

In the example above, job1 defines an output named output1 that is set to the result of the first step. Then in job2 you can use the value of that output using the needs context in order to pass the value between jobs seamlessly.

Using Outputs in Matrix Jobs

Things get somewhat more interesting if you are employing a matrix approach. Matrix jobs create one output for each instance of the job. Best to make sure, though, that the names for these outputs are unique, especially if you are using a combination of multiple versions or configurations since the order of execution for matrix jobs is not guaranteed.

jobs:
  job1:
    runs-on: ubuntu-latest
    strategy:
      matrix:
        version: [1, 2, 3]
    outputs:
      output_${{ matrix.version }}: ${{ steps.generate_output.outputs.version }}
    steps:
      - id: generate_output
        run: echo "version=${{ matrix.version }}" >> "$GITHUB_OUTPUT"

  job2:
    runs-on: ubuntu-latest
    needs: job1
    steps:
      - run: echo "Matrix outputs: ${{ toJSON(needs.job1.outputs) }}"

Here we define a three-version matrix. Each of the matrix jobs outputs a job displaying its version number. job2 can then utilize all matrix outputs en bloc due to the needs context.

Monitoring GitHub Actions Workflows

CICube is a GitHub Actions monitoring tool that provides you with detailed insights into your workflows to further optimize your CI/CD pipeline. With CICube, you will be able to track your workflow runs, understand where the bottlenecks are, and tease out the best from your build times. Go to cicube.io now and create a free account to better optimize your GitHub Actions workflows!

Passing Sensitive Data Between Jobs

Sometimes, you want to pass some secret information between jobs, be it API keys, passwords, or tokens. GH Actions creates functionality for handling secrets, such as their redaction in logs, out of the box. You can also explicitly mask sensitive information that you pass as job outputs using the add-mask command.

jobs:
  job1:
    runs-on: ubuntu-latest
    outputs:
      api_key: ${{ steps.generate_key.outputs.key }}
    steps:
      - id: generate_key
        run: |
          key="my-sensitive-api-key"
          echo ":add-mask::$key"
          echo "key=$key" >> "$GITHUB_OUTPUT"
  job2:
    runs-on: ubuntu-latest
    needs: job1
    steps:
      - run: echo "Using API key: ${{ needs.job1.outputs.api_key }}"

Here, in the above example, API key created in job1 is masked before passing to job2 so that it won't show up in the logs.

Passing Database Credentials -Real-World Use Case

Consider an applied view-a real-world example-only integrating a database in a CI pipeline. You would create, in a job, the database and then pass the credential or the connection string to another job for the testing purpose.

jobs:
  create-db:
    runs-on: ubuntu-latest
    outputs:
      db_id: ${{ steps.db_setup.outputs.db_id }}
      db_password: ${{ steps.db_setup.outputs.db_password }}
    steps:
      - id: db_setup
        run: |
          db_id=$(uuidgen)
          db_password=$(openssl rand -base64 32)
          echo "db_id=$db_id" >> "$GITHUB_OUTPUT"
          echo "db_password=$db_password" >> "$GITHUB_OUTPUT"

  test-db:
    runs-on: ubuntu-latest
    needs: create-db
    steps:
      - run: |
          echo "Testing database with ID: ${{ needs.create-db.outputs.db_id }}"
          echo "Using password: ${{ needs.create-db.outputs.db_password }}"

Here, create-db outputs a database id and password. These are then given as arguments to test-db, which can use them in conducting integration tests.

Best Practices for Job Outputs

  1. Keep Outputs Simple: Outputs should be simple strings: for example, IDs, paths or tokens. If your use requires more complex data to be passed, encode it first (for example Base64) before passing as an output.

  2. Mask Sensitive Data: Generally, the outputs must not be sensitive; by example, passwords, API keys must not appear in logs using add-mask.

  3. Using Unique Output Names in Matrices: If output names are used within a matrix, use unique names to avoid name conflicts when jobs run on matrix.

  4. Limitation in Size of Output: The outputs that could be provided to a single action are not more than 1 MB maximum and also cannot exceed a total of 50MB for all the outputs in one workflow run. Larger data is to be considered to be passed using artifacts instead.

Conclusion

Passing information between jobs in GitHub Actions can really clean up and streamline your CI/CD workflows. Whether you are passing API keys, dynamic values, or build artifacts, job outputs give you flexible and secure ways to handle data across jobs. By following best practices, add-mask features, and matrix strategies, you'll be in a position to create workflows that are secure, maintainable, and efficient.

DevopsGitHubgithub-actions