# Egress firewall

import { Aside } from '@astrojs/starlight/components';

The egress firewall controls which destinations your runner VMs can connect
to. You write an ordered list of allow and deny rules. Avrea enforces them
outside the VM, so a job cannot switch them off, and records for every job
where it connected and what was blocked.

Typical uses are stopping a compromised dependency or action from sending
secrets out of a build, and meeting a requirement that CI reaches only
approved destinations.

## How rules work

Rules are checked from top to bottom and the first match decides. When no rule
matches, the default rule applies. Every organization starts with the default
`ALLOW *`, which allows all outbound traffic. Switch it to `DENY *` to allow
only what your rules list.

A rule has these parts:

| Part | Values |
| --- | --- |
| Match | A hostname such as `api.example.com`, or a network in CIDR form such as `203.0.113.0/24` (IPv4 or IPv6) |
| Protocol | Any, TCP, UDP or ICMP. An ICMP rule does not affect ping today, see [What rules cannot change](#what-rules-cannot-change) |
| Ports | One port or a range. TCP and UDP only |
| Action | Allow or deny |

A hostname rule matches that name and every subdomain under it.
`*.example.com` is also accepted and matches the same set as `example.com`.

Wildcards over a suffix shared by many customers are rejected, for example
`*.com`, `*.co.uk`, `*.github.io` and `*.s3.amazonaws.com`. Use a name that
only you control, such as `*.my-bucket.s3.amazonaws.com`, or list the exact
hostnames.

## Set organization rules

1. Open your organization in the [Avrea console](https://console.avrea.com),
   then **Settings** → **Egress firewall**.
2. Choose the default: `ALLOW *` or `DENY *`.
3. Under **Add rule**, pick **FQDN** or **CIDR**, enter the destination, and
   optionally a protocol and ports. Select **Add**.
4. Use **Move up** and **Move down** to order the rules.

Organization Admins can change rules. Other members can view them. Rule
changes are recorded in the organization audit log.

The console adds rules with the action opposite to the default: deny rules
under `ALLOW *`, allow rules under `DENY *`. To mix both kinds in one list,
use the [CLI](#cli) or the [API](#api).

Changes apply to jobs that start after the change. A running job keeps the
rules it started with.

## Repository rules

A repository can have its own rules and its own default. They are managed
with the [CLI](#cli) or the [API](#api); the console edits organization rules
only. For a job in that repository, Avrea checks in this order:

1. The repository's rules.
2. The organization's rules.
3. The repository's default if it has one, otherwise the organization's
   default.

A repository can therefore be stricter or looser than the organization. For
example, the organization can run with `ALLOW *` while one repository that
handles release signing runs with `DENY *` and a short allow list.

Changing repository rules requires the organization Admin role and admin
access to the repository on GitHub.

## What rules cannot change

Some traffic is decided before your rules are checked.

**Always reachable.** GitHub Actions needs a set of GitHub endpoints to
register the runner, check out code, and upload logs, artifacts and caches.
They stay reachable under `DENY *`, and a deny rule for them has no effect.
They include `github.com`, `api.github.com`, `codeload.github.com`,
`ghcr.io`, `*.actions.githubusercontent.com`, `*.pkg.github.com` and
`*.blob.core.windows.net`. Avrea's own cache services also stay reachable.

<Aside type="caution" title="Azure Blob Storage stays reachable">
GitHub Actions stores logs, artifacts and caches in Azure Blob Storage under
changing `*.blob.core.windows.net` names, so that whole suffix is allowed. It
covers every Azure storage account, not only GitHub's. A `DENY *` default does
not stop a job from connecting to another Azure Blob Storage account.
</Aside>

**Always blocked.** An allow rule cannot open these:

- Cloud metadata and workload credential endpoints: `169.254.169.254`
  (Google Cloud, AWS EC2 and Azure), `169.254.170.2` (AWS ECS),
  `169.254.170.23` (AWS EKS Pod Identity), `100.100.100.200`
  (Alibaba Cloud) and `fd00:ec2::254` (EC2 IPv6 metadata).
- Other runner VMs.
- Private and other non-public address ranges, such as `10.0.0.0/8`,
  `172.16.0.0/12`, `192.168.0.0/16` and `100.64.0.0/10`. A VPN tunnel that
  your job opens from inside the VM is not affected.
- Outbound mail on TCP ports 25, 465 and 587.
- DNS over TLS on port 853.

Ping is rate limited by Avrea and passes before your rules run, so neither an
ICMP rule nor a `DENY *` default affects it today. On macOS runners ICMP rules
never match.

## How hostnames are enforced

Hostname rules are enforced twice. A denied name does not resolve. Under
`DENY *`, the VM can also connect only to the addresses that its own DNS
lookups returned for allowed names. This has a few consequences:

- **DNS goes through Avrea's resolver.** Queries a job sends to another
  resolver, such as `8.8.8.8`, are answered by Avrea's resolver. On macOS
  runners they are blocked instead, so use the VM's default resolver there.
  Well-known DNS over HTTPS hostnames do not resolve.
- **Under `DENY *`, add a CIDR rule for anything reached by IP address.** A
  connection to a hard-coded address is blocked unless a CIDR rule allows it.
- **Under `ALLOW *`, a deny rule for a hostname with no protocol or ports
  works at the DNS level.** The name stops resolving, but a job that connects
  straight to the address is not stopped. Use a CIDR deny rule for addresses
  that must be blocked.

## See what a job reached

Open a job in the console and expand **Job firewall**. A workflow run has the
same panel as **Run firewall**, covering all of its jobs. The data appears
after the job's VM has stopped. It shows:

- **Top destinations**, with the latest hostname seen for each address and the
  traffic volume.
- **Denied DNS**: names the job tried to resolve that your rules deny.
- **IP blocks**: connections that were dropped, and the rule that dropped
  them.
- **Download sources**: where inbound bytes came from.

**Show only blocked** limits a run with several jobs to those that had
something blocked.

### Move to deny by default

1. Keep the default at `ALLOW *` and run your usual workflows.
2. Read **Top destinations** for those runs and list what the build needs,
   such as your package registries and container registries.
3. Add an allow rule for each destination with the [CLI](#cli) or the
   [API](#api). Allow rules change nothing while the default is `ALLOW *`, so
   they can be in place before the switch. The console only adds allow rules
   once the default is `DENY *`.
4. Switch the default to `DENY *`.
5. Run the workflows again and check **Denied DNS** and **IP blocks** for
   anything you missed.

To limit the effect of a missing rule, try this on one repository first by
giving it a `DENY *` default and allow rules of its own.

## CLI

The [`avr` CLI](/cli/) manages organization and repository rules. Commands act
on the organization unless you pass `--repo` with a repository ID.

```sh
# List the organization's rules, or one repository's rules
avr firewall list
avr firewall list --repo <repo-id>

# Allow HTTPS to a package registry
avr firewall add --action allow --fqdn registry.npmjs.org --proto tcp --ports 443

# Give one repository its own allow rule, then make it deny by default
avr firewall add --repo <repo-id> --action allow --fqdn registry.npmjs.org --proto tcp --ports 443
avr firewall set-default --repo <repo-id> --action deny

# Show the merged list a new job in that repository gets
avr firewall show --repo <repo-id>

# Show recent jobs where something was blocked
avr firewall flow-summaries --repo <repo-id> --with-drops
```

`avr firewall move` reorders a rule and `avr firewall delete` removes one. Run
`avr firewall --help` for every command and option.

## API

Rules are served by the public API. Reading requires organization membership,
and for repository routes read access to the repository.

| Method | Path | Purpose |
|--------|------|---------|
| `GET` | `/orgs/{org_id}/firewall/rules` | List organization rules |
| `POST` | `/orgs/{org_id}/firewall/rules` | Add an organization rule |
| `PATCH` | `/orgs/{org_id}/firewall/rules/{rule_id}` | Change a rule |
| `DELETE` | `/orgs/{org_id}/firewall/rules/{rule_id}` | Delete a rule |
| `POST` | `/orgs/{org_id}/firewall/rules/reorder` | Set the rule order |
| `GET` | `/orgs/{org_id}/repos/{repo_id}/firewall/rules` | List repository rules |
| `POST` | `/orgs/{org_id}/repos/{repo_id}/firewall/rules` | Add a repository rule |
| `PATCH` | `/orgs/{org_id}/repos/{repo_id}/firewall/rules/{rule_id}` | Change a repository rule |
| `DELETE` | `/orgs/{org_id}/repos/{repo_id}/firewall/rules/{rule_id}` | Delete a repository rule |
| `POST` | `/orgs/{org_id}/repos/{repo_id}/firewall/rules/reorder` | Set the repository rule order |
| `GET` | `/orgs/{org_id}/repos/{repo_id}/firewall/resolved` | The merged list a new job in this repository gets |
| `GET` | `/orgs/{org_id}/repos/{repo_id}/firewall/vm-flow-summaries` | Network summaries per job VM |

A rule sets either `fqdn` or `cidr`, never both:

```json
{
  "action": "allow",
  "fqdn": "registry.npmjs.org",
  "protocol": "tcp",
  "port_start": 443,
  "port_end": 443
}
```

`protocol` is `any`, `tcp`, `udp` or `icmp` and defaults to `any`. Ports are
accepted only with `tcp` or `udp`. `PATCH` changes only the fields you send.
`reorder` takes `{"rule_ids": [...]}` with every rule of that organization or
repository in the order you want.

To give a repository its own default, add a rule with `"is_default": true`
and no `fqdn`, `cidr`, protocol or ports. A default rule cannot be deleted
afterwards. Its action can be switched between allow and deny.

## Availability

See [pricing](https://avrea.com/pricing) for the plans that include the egress
firewall, or contact [support@avrea.com](mailto:support@avrea.com) for help
with a rule set.