Bringing Slack to AWS GovCloud

When Slack launched GovSlack in July 2022, the service gave government agencies a compliant collaboration platform supporting FedRAMP High, DoD IL4, and ITAR standards. GovSlack runs in AWS GovCloud data centers, enables external collaboration through Slack Connect with other GovSlack organizations, and provides custom encryption keys, enterprise-grade admin controls, and a curated app directory including DLP and eDiscovery tools. The service is maintained and supported by US personnel.

Before that launch, the Cloud Foundations team spent nearly two quarters preparing the infrastructure. GovSlack was Slack's first service on AWS Gov infrastructure, which meant the team had to learn the differences between standard and Gov AWS, then adapt tooling and platform components accordingly.

Account Structure and Billing

Slack had recently been migrating from a single AWS account to a child account model, with significant changes to global network infrastructure along the way. Those learnings carried directly into the GovSlack network buildout.

A key structural difference in GovCloud: Gov accounts have no billing capability. Resource costs in Gov accounts propagate to a linked shell commercial AWS account, which is automatically created when a Gov account is requested. The team had to request a Gov root AWS account from their root payer commercial account first. While requesting the account itself was simple, adding Gov accounts to existing AWS agreements took several weeks.

New GovCloud child accounts are requested through the commercial AWS API using the create-gov-cloud-account call. Once created, you can assume the OrganizationAccountAccessRole in the child account from the GovCloud root account's OrganizationAccountAccessRole.

Provisioning Pipelines

The account creation process runs through a pipeline on the commercial side and produces both a new GovCloud account and its linked commercial account. The linked commercial account is immediately moved to a highly restricted organizational unit, blocking it from creating any AWS resources.

On the Gov partition, a Jenkins pipeline handles the child account configuration. The Gov Jenkins services reside in a dedicated child account, so the pipeline first updates the new child account's OrganizationAccountAccessRole trust policy to allow the Jenkins workers to assume it. That step must complete before any other configuration work proceeds.

Dev/Prod Separation

Since only US persons can access the Gov production environment, Slack stood up two Gov environments: "GovProd" for production and a separate environment known as "GovDev." The GovDev environment is accessible to anyone and allows teams to test services before US personnel deploy them to GovProd.

Isolation between the two is complete. The environments use a full shared-nothing approach, operating in completely different AWS organizations. The layer 3 networking mesh (Nebula) is fully disconnected, keeping the networks entirely segregated.

One wrinkle in the isolation model: new child account provisioning requires assuming the OrganizationAccountAccessRole from the Gov root account. Since the Gov root account can access GovProd child accounts, only US personnel can access it. That means even GovDev account creation must be kicked off by US personnel using the GovProd Jenkins infrastructure, at least for the initial provisioning steps.

Working Around GovCloud Limitations

GovCloud lacks some AWS services, including CloudFront and public Route53 zones. The AWS CLI also defaults to commercial regions, so every CLI invocation in GovCloud requires the --region flag or an AWS_DEFAULT_REGION environment variable set to a Gov region.

Certificates and DNS

Gov services rely on AWS ACM for load balancer SSL certificates. Email-based certificate validation was avoided because it doesn't support auto-renewal. ACM DNS validation does support auto-renewal but requires public DNS records. The same dedicated commercial DNS account used for other purposes validates ACM certificates, and access to that account is restricted to US personnel.

Public Route53 zones are unavailable in GovCloud, though private zones are supported. The team created GovDev and GovProd DNS accounts for hosting private zones, attaching them to shared VPCs connected across regions via AWS Transit Gateways, following the same network architecture Slack uses in its commercial environment. Records added to those private zones resolve immediately within the VPCs.

Because GovCloud lacks public DNS, public GovSlack DNS records are hosted in a dedicated commercial AWS account, also restricted to US personnel.

Cross-Partition Artifact Transfer

AWS does not support assuming roles between standard and GovCloud partitions. To pass objects between the two, Slack built a mechanism that pulls objects from the standard partition into GovCloud using IAM credentials. Those credentials are stored securely on the GovCloud partition itself.

Reusable Terraform Modules

Slack uses Terraform modules to build infrastructure as interdependent resources such as VPCs, Internet Gateways, Transit Gateways, and route tables, and wanted the same patterns for Gov infrastructure. A key difference across partitions: resource ARNs. Commercial ARNs begin with arn:aws; Gov ARNs begin with arn:aws-us-gov.

A simple module called aws_partition was created to address this. Its outputs let the team programmatically build ARNs and detect which partition they're operating in.

data "aws_caller_identity" "current" {}

data "aws_arn" "arn_details" {
  arn = data.aws_caller_identity.current.arn
}

output "partition" {
  value = data.aws_arn.arn_details.partition
}

output "is_govcloud" {
  value = replace(data.aws_arn.arn_details.partition, "gov", "") != data.aws_arn.arn_details.partition ? true : false
}

The module plug into infrastructure code so the same Terraform definitions work across both partitions, referencing partition-specific ARN prefixes automatically.

module "aws_partition" {
  source = "../modules/aws/aws_partition"
}

data "aws_iam_policy_document" "example" {
  statement {
	effect = "Allow"

	actions = [
	  "s3:GetObject",
	]

	resources = [
	  "arn:${module.aws_partition.partition}:iam::*:role/some-role",
	]
  }
}

resource "aws_config_config_rule" "example" {
  count = module.aws_partition.is_govcloud ? 1 : 0

  name = "example-rule"

  source {
	owner             = "AWS"
	source_identifier = "S3_BUCKET_SERVER_SIDE_ENCRYPTION_ENABLED"
  }
}

VPC Endpoints

Slack has been expanding the use of VPC endpoints for accessing native AWS services in its commercial environment, reducing latency, increasing resiliency, and lowering networking costs. The same advantages apply in GovCloud, but the transition comes with a complication: AWS doesn't support every service in every availability zone. Systems need to access AWS services both with and without VPC endpoints, which creates edge cases that are difficult to account for. Even as AWS rolls out endpoints at the AZ level, coverage still isn't at 100% for all services across the regions and AZs Slack runs in.

AWS-SSO

The initial Gov environment used IAM users as a bootstrap measure only. AWS-SSO had recently become available in Gov, and this greenfield environment was a chance to experiment with it. Unlike standard IAM roles, AWS-SSO permission sets are org-wide global resources rather than account-level ones, which changes how they're built and deployed. After deploying AWS-SSO in GovCloud, the team back-ported the learnings to its commercial environment, where the new system streamlined access management compared to the prior SSO setup.

Buildout Takeaways

Rebuilding Slack's network infrastructure from the ground up in GovCloud gave the team a chance to test tooling, processes, and Terraform modules under real conditions. The exercise surfaced a number of hardcoded values that were cleaned up and made reusable. It also forced a deeper review of processes, tools, and AWS footprint, effectively providing an opportunity to rebuild Slack from scratch and gain a fuller understanding of the platform's foundations.