Reusable Workflows - Best Practices and Strategies

Introduction

Reusable workflows in GitHub Actions offer numerous benefits, including standardization, improved developer productivity, reduced code duplication, and increased maintainability. By centralizing common tasks and configurations, reusable workflows simplify the development process and ensure consistency across projects.

Key Benefits of Reusable Workflows

  • Standardization and Guardrails
  • Improved Developer Productivity
  • Reduced Code Duplication
  • Increased Maintainability

GitHub reusable workflows provide a uniform approach to CI/CD practices within your repositories. This means fewer errors and higher-quality deployments. With pre-built workflows, your team can focus more on writing code and less on setting up common tasks. This increases efficiency. GitHub reusable workflows remove the need to repeat the same code across multiple workflows, simplifying maintenance and reducing inconsistencies. By centralizing workflows in a GitHub repository, updates and improvements become easier. Changes to a single workflow will apply to all pipelines that utilize it.

Setting Up a Central Repository

It is recommended to create a dedicated GitHub repository to store reusable workflows. This allows easy access and use of existing workflows across different projects. GitHub's inherent version control enables tracking changes and reverting to previous versions if needed.

Example Repository Structure - Deployment Workflows

graph TD
    A[Central Repository]
    -->B[Reusable Workflows]
    B-->C[Deployment Workflow]
    B-->D[Testing Workflow]
    B-->E[Build Workflow]
    C-->F[Environment: Production]
    C-->G[Environment: Staging]
    D-->H[Environment: Testing]
    E-->I[Environment: Build]
    F-->J[Actions: Deploy to Production]
    G-->K[Actions: Deploy to Staging]
    H-->L[Actions: Run Tests]
    I-->M[Actions: Build Application]
    J-->N[Parameters: Production Server, Production Database]
    K-->O[Parameters: Staging Server, Staging Database]
    L-->P[Parameters: Test Cases, Test Data]
    M-->Q[Parameters: Source Code, Build Tools]

Versioning and Rollout for Reusable Workflows

Version Control

Use GitHub's built-in version control for your reusable workflows. This allows for easy tracking of changes, reverting to previous versions, and collaborative improvements.

Rollout Strategy

Have a defined plan for rolling out new versions of reusable workflows. Test new versions in a non-production environment before deploying them into production pipelines.

Semantic Versioning

Semantic versioning is a versioning scheme for software that aims to convey meaning about the underlying changes with each release. This versioning method is widely adopted in the software industry and is very suitable for versioning your reusable workflows. Semantic versioning uses a three-part version number like this: MAJOR.MINOR.PATCH. When you make changes, you increment:

  • the MAJOR version when you make incompatible API changes,
  • the MINOR version when you add functionality in a backwards-compatible manner, and
  • the PATCH version when you make backwards-compatible bug fixes.

For example, if you have a workflow file named workflow.yml, you can version it by tagging the commit with the version number. The tag can be something like v1.0.0. Then, in other workflows, you can refer to this version of your workflow file using the uses keyword with the version tag, like so:

uses: my-org/my-repo/.github/workflows/workflow.yml@v1.0.0

Example process/procedure

  1. Development: Create a new branch in your workflows repository for developing your reusable workflows updates. This keeps the main branch stable and ensures your changes are isolated for testing.
  2. Testing: Once the development is done, run the workflows in a non-production environment or on non-critical repositories. This allows you to catch any potential issues before the changes affect your main projects.
  3. Code Review & Merge: After successful testing, create a pull request for the changes. Your team can review the new workflows and provide feedback. Once the review is completed and any necessary adjustments are made, merge the changes to the main branch.
  4. Versioning: Create a new release on GitHub with a new version number after the merge. Use semantic versioning (major.minor.patch) to indicate the level of changes in the new version. This helps users understand the impact of updating to the new version.
  5. Announcement: Notify your team about the new version, its changes, and the impact of those changes. This can be done through email, team communication tools, or through GitHub notifications.
  6. Adoption: Each team can now update their projects to use the new version. They can do this manually by changing the version number in their workflow files, or, if you have a system in place for automatic updates, they can use that instead.

Example Versioning Strategy

graph TD
    A[Central Repository]
    -->B[Main Branch]
    B-->C[Development Branch]
    B-->D[Version 1.0.0]
    B-->E[Version 1.1.0]
    B-->F[Version 1.2.0]
    C-->G[Feature Development]
    D-->H[Stable]
    E-->I[Testing]
    F-->J[Testing]
    H-->K[Production]
    I-->L[Testing]
    J-->M[Testing]
    K-->N[Production]
    L-->O[Testing]
    M-->P[Testing]
    N-->Q[Production]
    O-->R[Testing]
    P-->S[Testing]
    Q-->T[Production]

Repository Rules and Required Workflows

Repository Rules/rulesets

Use GitHub's repository rulesets to enforce the use of certain reusable workflows. This ensures all developers adhere to established standards and processes.

Required Workflows

As your team evolves and adopts more advanced processes, make certain reusable workflows "required" for specific pipelines. This rule enforces the use of standardized workflows for critical tasks like security checks and code formatting. For example, if the required workflow does not pass, the pull request will not be merged.

Handling Secrets

Minimize Hardcoded Secrets

Avoid including sensitive information like API keys or passwords directly within your reusable workflows. Use GitHub Secrets instead to enhance security.

Secret Scanning

We recommend using GitHub's secret scanning feature. Secret scanning helps prevent accidental commits of secrets and sensitive information to your GitHub repositories. It scans your repositories for known types of secrets, like Azure secrets, AWS secrets, and more.

When secret scanning identifies a potential secret in a commit, it generates an alert. You can view these alerts in the security tab of your repository, allowing you to quickly react and revoke or rotate the compromised secrets.

Benefits of using secret scanning include:

  • Prevention of accidental secret commits: Secret scanning helps catch secrets before they become a security vulnerability.
  • Wide range of secret types: GitHub's secret scanning can identify a wide range of secret types, making it a versatile tool for many different projects.
  • Automated scanning: Secret scanning automatically scans every commit, so you don't have to remember to do it manually.
  • Integration with secret providers: GitHub works with many secret providers to ensure that if a secret is committed, the provider is notified so they can invalidate the secret if necessary.

Parameterize Secrets

Use parameters in your workflows so developers can securely input secrets during workflow execution.

Secrets Management Tools

GitHub Secrets provides a secure way to store and handle secrets directly within your GitHub environment.

Environment-Specific Secrets

If your workflows run across different environments (e.g., staging, production), consider having separate sets of secrets for each environment. This way, you can avoid accidentally using production secrets in a staging environment.

Regular Rotation

Regularly rotate your secrets to minimize the potential damage if a secret is compromised. The frequency of rotation will depend on your organization's security policy.

Least Privilege Principle

Only provide the minimum access required for a secret. This reduces potential damage if a secret is compromised.

Audit Logs

Regularly review your GitHub audit logs to monitor access to secrets. This can help you identify and respond to any unauthorized access promptly.

Secrets in Forks

Be aware that GitHub Secrets are not passed to runs on forked repositories. If a workflow runs on a fork and requires a secret, you'll need to handle this in your workflow logic.

Encrypted Secrets

Secrets are encrypted in GitHub and can only be accessed by GitHub Actions running in the same repository. They are not exposed in logs or available to users with write access to the repository.

Breaking Down Workflows for Flexibility

Granularity

Break down complex workflows into smaller, reusable pieces. This improves maintainability, promotes code reuse, and simplifies troubleshooting.

Example

graph LR
    A[Complex Workflow: Build, Test, Containerize, Deploy]
    -->|Transformation| B[Reusable Workflows]
    B --> C[Reusable Workflow: Build and Test<br>Inputs: buildConfiguration, token]
    B --> D[Reusable Workflow: Containerize<br>Inputs: imageName, tag]
    B --> E[Reusable Workflow: Deploy with OpenID Connect<br>Inputs: environment, oidcCredentials]

Focus on Specific Tasks

Each reusable workflow should ideally focus on one specific task. This makes them easier to understand, reuse, and integrate into different pipelines.

Restricting workflows

  • Access Control: You can control who has access to your repository. Only those with write access to the repository can create or edit workflows.
  • Branch Protection Rules: You can set up branch protection rules to prevent workflows from being edited in protected branches. This can prevent unauthorized changes to workflows.
  • Workflow Permissions: With GitHub Actions, you can set the permissions for the GITHUB_TOKEN used in workflows. This allows you to restrict what actions the workflows can perform.
  • Environment Protection Rules: For more sensitive tasks, you can use environments with protection rules. Workflows that reference a protected environment must have their runs approved by a designated individual.
  • Code Owners: You can use the CODEOWNERS file to require review from specific individuals or teams when a change is made to workflow files.

Overall Benefits

Implementing best practices for GitHub reusable workflows fosters efficiency and collaboration within your development team, leading to:

  • Reduced Time to Market: Streamlined pipelines speed up code deployments.
  • Improved Code Quality: Standardized workflows enforce code quality checks and best practices.
  • Reduced Maintenance Overhead: Reusable workflows simplify maintenance and streamline updates.
  • Enhanced Team Collaboration: Standardized processes promote better communication and understanding within the development team.
  • Increased Security: Secure handling of secrets and standardized workflows enhance security practices.