GitHub Actions Runners: A Comparison

This document provides a comparison between GitHub-hosted runners and self-hosted runners for GitHub Actions. It aims to help customers understand the differences between the two options and make an informed decision based on their specific needs.

Quick Comparison Table

Comprehensive Comparison

Costing

GitHub-hosted runners

• Free for public repositories.
• For private repositories, usage incurs costs based on runner type and usage hours.
• Each GitHub plan provides a certain amount of free minutes, with additional usage billed accordingly.
• Large runners, which offer more resources, may incur additional costs compared to standard runners.
• More information on pricing can be found in the GitHub pricing documentation.

Self-hosted runners

• Free to use, but you are responsible for the cost of maintaining the machine hosting them, including infrastructure costs such as cloud provider charges or hardware maintenance.

Throttling/Priority/Resiliency

GitHub-hosted runners

• GitHub-hosted runners offer a shared pool of resources. Workflows are queued and run as resources become available. This allows for a high degree of parallelism and can help to reduce the time it takes for your workflows to complete.
• GitHub enforces usage limits on these runners to prevent abuse. These limits depend on the type of repository (public or private) and the type of GitHub account.
• There is no public system for prioritizing jobs on GitHub-hosted runners. Jobs are run in the order they are received.
• The resiliency of GitHub-hosted runners is managed by GitHub, which aims to provide high availability and redundancy. However, the resiliency of your workflows also depends on how they are designed and implemented.
• GitHub-hosted runners are maintained by GitHub, including updates and security patches. This reduces the maintenance burden on you and helps to ensure that the runners are secure and up-to-date.
• GitHub-hosted runners are free for public repositories. For private repositories, each GitHub plan comes with a certain amount of free minutes, and usage beyond this is billed.

Self-hosted runners

• With self-hosted runners, you have complete control over the runner's environment. This includes resource allocation, job prioritization, and resiliency strategies.
• Even with self-hosted runners, GitHub enforces some usage limits to prevent abuse. These limits include a maximum job execution time.
• You are responsible for scaling the infrastructure for self-hosted runners as needed. This gives you the flexibility to scale up or down based on the demands of your workflows.
• You are also responsible for maintaining self-hosted runners. This includes tasks like applying updates and security patches.
• Self-hosted runners are free to use, but you are responsible for the cost of the infrastructure that hosts the runners.

Network Access

GitHub-hosted runners

• By default, outbound network access is restricted to specific allowed domains and ports for security reasons. GitHub actively manages these allowed entries, and users cannot directly modify them.
• Using GitHub-hosted runners within your Azure Virtual Network allows enhanced security and control over network access.
• While GitHub-hosted runners don't use a proxy server by default, you can configure them to use a custom proxy server if needed.
• SSH and FTP protocols are currently not supported by GitHub-hosted runners.
• GitHub does not provide a publicly accessible list of outbound IP addresses used by their hosted runners for security reasons. However, you can utilize the GitHub Enterprise Network Access feature (available in GitHub Enterprise) to whitelist larger runners, allowing them to access specific internal services by specifying approved IP addresses or ranges.

Self-hosted runners

• Full control over network access, allowing for unrestricted outbound and inbound traffic.
• Network access is the same as any other application on the host machine.
• Restricting network access for self-hosted runners is achieved through runner groups. These groups allow specifying approved IP addresses or domain names, controlling the runners' communication with external resources.
• Runner groups can be defined at the organization or enterprise level, not repository level, and can be applied to specific runner groups, not individual workflows or jobs.

Security

GitHub-hosted runners

• GitHub implements security measures such as regular security updates, sandboxing environments, and restricted access controls.
• Clean environment for each job execution, but since they are shared, may not be suitable for sensitive tasks.
• Runner sandboxing: Runners are isolated from each other and from the host machine, limiting the potential damage of a security incident.
• GitHub-hosted runners you can use containerization technology, to ensure a consistent and secure environment.
• GitHub Actions permissions: You can control which actions can be executed by runners, limiting the potential damage of a malicious action.
• Environment variables encryption: Sensitive data, like API keys, can be encrypted and stored as environment variables.
• Workflow logs: Detailed logs of workflow execution, allowing for auditing and troubleshooting.
• Regular security updates: GitHub keeps runners up-to-date with the latest security patches and updates.
• Token Security: GitHub uses a token to authenticate your runner to GitHub. The token is automatically invalidated when the job is finished.
• Access Control: GitHub-hosted runners can only be used by GitHub Actions workflows running in the same repository, which reduces the risk of unauthorized access.
• Compliance and regulatory certifications: GitHub maintains various certifications, like SOC 2 and HIPAA, to ensure compliance with industry standards.

Self-hosted runners

• You have full control over the infrastructure, allowing for potentially higher security levels.
• However, they can pose risks if used to run workflows from public repositories, as they could execute malicious code.

Flexibility and Control

GitHub-hosted runners

• Increased Resources: Large runners offer more computational resources, including CPU and memory, which means your workflows can run faster and handle more demanding tasks.
• Parallelism: You can run multiple jobs in parallel, speeding up your overall workflow execution time.
• Maintenance-Free: Since GitHub manages these runners, you don't have to worry about server maintenance, updates, or security patches.
• OS Flexibility: GitHub-hosted runners can run workflows on different operating systems, including Windows, Linux, and macOS.
• Pre-installed Software: GitHub-hosted runners come with a variety of pre-installed software, including programming languages, tools, and utilities.
• Integrated with GitHub: GitHub-hosted runners are tightly integrated with GitHub making it seamless to set up your continuous integration and continuous delivery (CI/CD) pipelines.

Self-hosted runners

• Limited Control: While you have complete control over the runner's environment, this also means you are responsible for managing resource allocation, job prioritization, and resiliency strategies, which can be complex and time-consuming.
• Customization: You can customize the runner's environment to meet your specific needs, including installing custom software and tools. However, this requires additional setup and maintenance effort compared to GitHub-hosted runners.
• Network Access: You can control the runner's network access, allowing for unrestricted outbound and inbound traffic. But this also means you are responsible for managing network security and performance.
• Security: You can implement your own security measures, such as firewall rules, access controls, and encryption. However, in many cases, this places the burden of maintaining security on you, unlike with GitHub-hosted runners where GitHub manages security.
• Cost: Self-hosted runners are free to use, but you are responsible for the cost of the infrastructure that hosts the runners. This includes not only the direct costs of the hardware or cloud services, but also indirect costs such as electricity, cooling, and maintenance.

Decision Tree

graph TD
    A[Start: Need to choose a runner type]
    B{Sensitive tasks?}
    C{Need full control over environment?}
    D{Willing to handle maintenance and updates?}
    E{Need advanced security features?}
    F{Prefer ephemeral and declarative setup?}
    G[GitHub-Hosted Runner]
    H[Self-Hosted Runner]
    A --> B
    B -- Yes --> C
    B -- No --> G
    C -- Yes --> D
    C -- No --> G
    D -- Yes --> E
    D -- No --> G
    E -- Yes --> F
    E -- No --> H
    F -- Yes --> G
    F -- No --> H

Additional Considerations

• GitHub offers various pre-configured runners with different specifications, including larger runners with expanded resources for demanding workflows.
• Runner management tools are provided to facilitate the addition, removal, and organization of runners.
• Utilizing GitHub-hosted runners within your Azure Virtual Network can provide enhanced security and network access control for your workflows.