Open Source’s Long-Term Storage: The Arctic Code Vault
In 2019, GitHub announced its Archive Program with a bold objective: preserve open source software for future generations by storing code in a facility built to last a millennium. On February 2, 2020, GitHub took a snapshot of all active public repositories. Since then, archive partner Piql has written this data—21TB of repository content—onto 186 reels of piqlFilm, a digital photosensitive archival film. The original plan called for GitHub’s team to escort the code personally to Norway, but the global pandemic required a change in approach. After coordination with partners on the ground, the code was successfully deposited in the Arctic Code Vault on July 8, 2020.
The Path to the Vault
The code began its trip at Piql’s facility in Drammen, Norway. The 186 reels were shipped to Oslo Airport and loaded onto a passenger plane bound for Svalbard, an island group roughly 600 miles (1000 km) north of the European mainland. Svalbard had only recently reopened to visitors from the Schengen Area and the European Economic Area, making the transport window a narrow one.
After landing in Longyearbyen—a town of a few thousand people on Svalbard—the boxes were met by a local logistics company and placed in secure overnight storage. The following morning, they were moved to a decommissioned coal mine set into a mountain, and then to a chamber hundreds of meters deep in permafrost. This is where the code now resides, fulfilling its mission of preserving the world’s open source software for over 1,000 years.
Recognizing Contributors
Millions of developers contributed to the repositories stored in the vault. To honor these efforts, GitHub designed the Arctic Code Vault Badge, displayed in the highlights section of a developer’s profile. Hovering over the badge reveals some of the repositories to which an individual contributed.
Partner Updates
Internet Archive
The Internet Archive, a non-profit digital library, began its ongoing archive of GitHub public repositories on April 13. The organization is taking a two-pronged approach. First, its Wayback Machine is capturing raw GitHub data as WARC (Web ARChive) files, with approximately 55TB archived to date. Second, it is working toward making entire archived GitHub repositories available via git clone, while keeping repository comments, issues, and other metadata accessible on the web. This second initiative is underway, with initial archiving expected to begin shortly.
Software Heritage Foundation
Software Heritage, a non-profit, multi-stakeholder initiative launched by Inria in collaboration with UNESCO, collects and preserves the source code of the world’s software commons. The organization already archives more than 130 million projects with full development history, of which 100 million come from GitHub. The archival engine is being improved to keep pace with GitHub’s growth. For projects not yet archived—or needing a newer version archived—users can trigger the archival process directly at https://save.softwareheritage.org.
Project Silica
Microsoft’s Project Silica is developing storage technology built for cloud-scale, long-lived data. Using ultrafast laser optics, data is written into quartz glass by permanently altering the physical structure of the material. Quartz glass offers data lifetimes measured in tens of thousands of years and is resistant to electromagnetic interference, water, and heat. As a partner in the GitHub Archive Program, Project Silica has archived 6,000 of the world’s most popular repositories as a proof of concept for future archives.
What’s Next
The Tech Tree
Every reel in the archive contains a copy of the “Guide to the GitHub Code Vault,” written in five languages and available in the Archive Program’s GitHub repository. In addition, the archive will include a separate human-readable reel documenting the technical history and cultural context of its contents. This is called the Tech Tree.
Inspired by the Long Now Foundation’s Manual for Civilization, the Tech Tree will consist primarily of existing works selected to offer a detailed understanding of modern computing, open source, software development practices, and popular programming languages. It will also cover foundational technologies that make software possible: microprocessors, networking, electronics, semiconductors, and pre-industrial technologies. The goal is to allow future readers to understand today’s world and, potentially, to recreate the computers needed to run the archived software.
The Tech Tree is expected to evolve over time. GitHub will soon publish an initial draft list of selected works to the Archive Program’s repository, along with a request for community input. Suggestions from the GitHub community will be incorporated before the Tech Tree is added to the Arctic Code Vault.
