GitHub in the enterprise
During the last year alone, over 56 million developers created more than 60 million new repos and made more than 1.9 billion contributions on GitHub. These developers are building the…
How Facebook keeps its large-scale infrastructure hardware up and running
Facebook’s services rely on fleets of servers in data centers all over the globe — all running applications and delivering the performance our services need. This is why we need to make sure our server hardware is reliable and that we can manage server hardware failures at our scale with as little disruption to our […]
The State of Ruby Static Typing at Shopify - Shopify
Shopify changes a lot. We merge around 400 commits to the main branch daily and deploy a new version of our core monolith 40 times a day. The Monolith is also big: 37,000 Ruby files, 622,000 methods, more than 2,000,000 calls. At this scale with a dynamic language, even with the most rigorous review process and over 150,000 automated tests, it’s a challenge to ensure everything works properly. Dev
Your DevOps guide to GitHub Universe
GitHub Universe is almost here. For more on what to expect from this year’s stream, we sat down with virtual host, Brian Douglas, for a quick Q&A on GitHub Actions,…
The evolving role of operations in DevOps
GitHub’s team delves into answering the question “what are operations roles in the development and operations (DevOps) environments”. From automating the role of QA in DevOps and more for smaller, faster delivery cycles.
The ReadME Project shares innovative paths to DevOps success
GitHub has the privilege of partnering with thousands of organizations to improve their DevOps practices, from established enterprises transforming the way they work to born-in-the-cloud businesses. We often connect developers,…
Vouching for Docker Images - Shopify
If you were using computers in the ‘90s and the early 2000s, you probably had the experience of installing a piece of software you downloaded from the internet, only to discover that someone put some nasty into it, and now you’re dragging your computer to IT to beg them to save your data. To remedy this, software developers started “signing” their software in a way that proved both who they were a
A Byzantine failure in the real world
At Cloudflare, we are always on the lookout for Single Points of Failure. In this post, we explore the role a failure mode known as a Byzantine fault played in a a real-world incident.
Building Black Friday e-commerce experiences with JAMstack and Cloudflare Workers
Cloudflare Workers continues to excel as a JAMstack deployment platform, and be used to power e-commerce experiences, integrating with familiar tools like Stripe, Nuxt.js, and Sanity.io.
Migrating Slack Airflow to Python 3 Without Disruption
Last year, we migrated Airflow from 1.8 to 1.10 at Slack (see here) and we did a “Big bang” upgrade because of the constraints we had. This year, due to Python 2 reaching end of life, we again had a major migration of Airflow from Python 2 to 3 and we wanted to put our…
Don't put data science notebooks into production
We've come across many clients who are interested in taking the computational notebooks developed by their data scientists, and putting them directly into the codebase of production applications. Data science ideas do need to move out of notebooks and into production, but trying to deploy that notebooks as a code artifact breaks a multitude of good software practices. Predictably, that results in
FioSynth: A representative I/O benchmark and data visualizer for data center workloads
What it is: FioSynth is a benchmark tool used to automate the execution of storage workload suites and to parse results. It contains a base set of block level storage workloads, synthesized from production I/O traces, that simulate a diverse range of Facebook production services. It is useful for predicting how a storage device will […]
Automated Origin CA for Kubernetes
Today we're releasing origin-ca-issuer, an extension to cert-manager integrating with Cloudflare Origin CA to easily create and renew certificates for your account's domains.
Building a ubiquitous shared infrastructure using Twine
What the research is: Twine is our homegrown cluster management system, which has been running in production for the past decade. A cluster management system allocates workloads to machines and manages the life cycle of machines, containers, and workloads. Kubernetes is a prominent example of an open source cluster management system. Twine has helped convert […]
A Continuous Integration and Deployment Setup with CircleCI and Coveralls
Continuous Integration (CI) and Continuous Deployment (CD) are crucial development practices, especially for teams. Every project is prone to error,
GitHub Availability Report: October 2020
In October, we experienced one incident resulting in significant impact and degraded state of availability for multiple services.
Getting started with DevOps automation
This is the second post in our series on DevOps fundamentals. For a guide to what DevOps is and answers to common DevOps myths check out part one. What role…
DevOps best practices Q&A: Automated deployments at GitHub
How GitHub measures and improves reliability, security, and developer happiness with automated deployments.
PPL Bench: Creating a standard for benchmarking probabilistic programming languages
[…]
Handling Continuous Integration And Delivery With GitHub Actions — Smashing Magazine
In this article, Aleem Isiaka explains how you can deploy a basic NodeJS application to Heroku, automate and schedule a deployment to run at certain intervals of the day, and use other actions from the Github Actions community.
How to get your organization started with containerized deployments
This is our second post on cloud deployment with containers. Looking for more? Join our upcoming GitHub Actions webcast with Sarah, Solutions Engineer Pavan Ravipati, and Senior Product Manager Kayla…
Pachyderm and the power of GitHub Actions: MLOps meets DevOps
This is a guest post by Jimmy Whitaker, senior data science evangelist at Pachyderm In the last few years, DevOps has begun to shift—from a culture of continuous integration and…
Introducing resctl-demo: Better resource control with simulation
What it is: We all want our workloads to run on our machines as efficiently as possible. With the Facebook resource control demo (resctl-demo), developers can simulate system resource conflicts and test ways to resolve them. It’s like going on a guided tour through a system’s resource control, complete with live demos and detailed explanations. […]
Testing cloud apps with GitHub Actions and cloud-native open source tools
See this post in action during GitHub Demo Days on October 16. What makes a project successful? For developers building cloud-native applications, successful projects thrive on transparent, consistent, and rigorous…
Nemo: Data discovery at Facebook
Large-scale companies serve millions or even billions of people who depend on the services these companies provide for their everyday needs. To keep these services running and delivering meaningful experiences, the teams behind them need to find the most relevant and accurate information quickly so that they can make informed decisions and take action. Finding […]
GitHub Availability Report: September 2020
In September, we experienced one incident resulting in significant impact to the GitHub Pages service.
What is DevOps? A guide to common methods and misconceptions
By now, most people in technology are familiar with the term DevOps. What we call “DevOps” will often differ between organizations, yet one thing remains the same: DevOps is defined…
Keeping your data pipelines healthy with the Great Expectations GitHub Action
This post is the second in our series on using GitHub for MLOps and data science. Just joining in? Get started with part one. Most continuous integration (CI) tools only…
GitHub Welcomes the OpenJDK Project!
Earlier this month we were thrilled to welcome the OpenJDK Community to GitHub. The communities migration effort, codenamed Project ‘Skara’, brought JDK 16 main-line development into GitHub. The JDK project is at the…
Lightning Q&A: DevSecOps in five with Maya Kaczorowski
In this interview, we dig deeper with Maya Kaczorowski on what DevSecOps is, and how to apply it. It’s a mindset shift in how development teams think about security. DevSecOps is about making all parties who are part of the application development lifecycle accountable for security of the application.
Mark Harman elected Fellow of the Royal Academy of Engineering
The U.K.’s Royal Academy of Engineering has elected Facebook Research Scientist Mark Harman as a Fellow for his achievements in academia and industry, including his work on search-based software engineering (SBSE), intelligent software testing tools, and web-enabled simulation (WES) approaches. Election to the Academy is by invitation only, and it is one of the highest […]
The next decade: How Facebook is stepping up the fight against climate change
In 2018, we set an ambitious goal of achieving a 75 percent absolute reduction in operational greenhouse gas (GHG) emissions and to support our global operations with 100 percent renewable energy by the end of 2020. On both counts, we are well on track: These commitments have spurred the construction of over 5,400 megawatts (MW) […]
An Introduction To Running Lighthouse Programmatically — Smashing Magazine
Being able to run Google’s Lighthouse analysis suite programmatically provides a lot of advantages, especially for larger or more complex web applications. Using Lighthouse programmatically allows engineers to set up quality monitoring for sites that need more customization than straightforward applications of Lighthouse (such as Lighthouse CI) allow. This article contains a brief introduction to
Fault tolerance through optimal workload placement
As our infrastructure has expanded, we have seen an exponential growth in failures that affect a subset of capacity in a data center. These may stem from software or firmware errors, or issues in the mechanical or electrical equipment in the data center. As our data center footprint grows, so does the frequency of such […]
Containerizing ZooKeeper with Twine: Powering container orchestration from within
Hardware fails, networks partition, and humans break things. The job of our infrastructure engineers is to abstract these realities away and provide a reliable, stable production environment nonetheless. Two of the technologies we deploy in this pursuit are Twine, our internal container orchestrator, and Apache ZooKeeper. Twine maintains service availability by managing services in containers
August 30th 2020: Analysis of CenturyLink/Level(3) outage
Today CenturyLink/Level(3), a major ISP and Internet bandwidth provider, experienced a significant outage that impacted some of Cloudflare’s customers as well as a significant number of other services and providers across the Internet.
A Bit on CI/CD
I'd say "website" fits better than "mobile app" but I like this framing from Max Lynch:
Automate releases and more with the new Sentry Release GitHub Action
This is a guest post by Rahul Chhabria, Director of Product Marketing at Sentry. At Sentry, we believe that code is the center of every experience and when code works,…
Scaling services with Shard Manager
Over the years, as we’ve expanded in scale and functionalities, Facebook has evolved from a basic web server architecture into a complex one with thousands of services working behind the scenes. It’s no trivial task to scale the wide range of back-end services needed for Facebook’s products. And we found that many of our teams […]
Introducing Deploy Buttons
Deploy Buttons help you deploy a project to the Workers Platform without even needing to set up a local development environment. Now, it’s as easy as clicking a Deploy Button and three short steps to deploy using our new web-based deploy tool.
Asynchronous computing @Facebook: Driving efficiency and developer productivity at Facebook scale
People use our apps and services every day for a wide spectrum of use cases, including sharing pictures; following the latest news and sports updates; sharing life-changing events, such as the birth of a child; or marking themselves safe via Safety Check. Now, more than ever, people are also using our services to stay connected […]
Karuna 2020: Using GitHub Actions in the COVID-19 response efforts
GitHub Actions gives you the power to automate your workflow. Connect with the tools you know and love. Have more freedom to innovate and be creative. Deploy to any cloud,…
How We Improved Developer Productivity for Our DevOps Teams
Across Spotify, our teams diligently strive to fulfill our mission to “unlock the potential of human creativity by giving millions of creative artists the opportunity to live off their work, and billions of fans the opportunity to enjoy and be inspired by it”. As product managers in the Platform Developer Experience (PDX) Tribe, part of Spotify’s Technology Infrastructure Group, we focus on unlock
Using GitHub Actions to deploy applications to IBM Cloud Kubernetes Service
In this post, hear from @stevemar, a Senior Technical Staff Member at IBM, about a new GitHub Starter Workflow for developers deploying containerized applications to IBM Cloud Kubernetes Service. Here…
The Migration of Legacy Applications to Workers
As Cloudflare Workers, and other Serverless platforms, continue to drive down costs while making it easier for developers to stand up globally scaled applications, the migration of legacy applications is becoming increasingly common.
Scalable data classification for security and privacy
What the research is: We’ve built a data classification system that uses multiple data signals, a scalable system architecture, and machine learning to detect semantic types within Facebook at scale. This is important in situations where it’s necessary to detect where an organization’s data is stored in many different formats across various data stores. In […]
Private matching for compute: New solutions to the problem of enabling compute on private set intersections
Matching records is one of the most basic data analysis operations. There are many cases where data needs to be aligned across some common value — whether that’s joining between two different tables in a database or across two data sets stored in a file, or matching data sets between two separate entities. For example, […]
Introducing the GitHub Availability Report
What is the Availability Report? Historically, GitHub has published post-incident reviews for major incidents that impact service availability. Whether we’re sharing new investments to infrastructure or detailing site downtimes, our…
Introducing our 2019 Sustainability Report
As the world navigates the COVID-19 pandemic, companies are focusing not only on their business strategy, but also on their approach to key environmental and social issues. One of several areas we are focusing on now is our commitment to sustainable business practices and reporting. Today, we are taking another important step toward increased transparency […]
Retrie: Haskell refactoring made easy
What’s new: We’ve open-sourced Retrie, a code refactoring tool for Haskell that makes codemodding faster, easier, and safer. Using Retrie, developers can efficiently rewrite large codebases (exceeding 1 million lines), express rewrites as equations in Haskell syntax instead of regular expressions, and avoid large classes of codemodding errors. Retrie’s features include the ability to rewrite […] R
Leveraging Mobile Infrastructure with Data-Driven Decisions
TL;DR The pursuit wasn’t always easy, but “putting data first” has helped Spotify dramatically improve the infrastructure team’s decision-making process while improving company productivity. Raul Herbster describes the benefits, especially for mobile DevOps teams.
All Hands on Deck
This story speaks to the process behind incident response at Slack and uses the May 12th, 2020 outage as an example. For a deeper technical review of the same outage, read Laura Nolan’s post, “A Terrible, Horrible, No-Good, Very Bad Day at Slack” Slack is a critical tool for millions of people, so it’s natural when…
A Terrible, Horrible, No-Good, Very Bad Day at Slack
This story describes the technical details of the problems that caused the Slack downtime on May 12th, 2020. To learn more about the process behind incident response for same outage, read Ryan Katkov’s post, “All Hands on Deck”. On May 12, 2020, Slack had our first significant outage in a long time. We published a summary…
Courier: Dropbox migration to gRPC
GitHub Action Hero: Victoria Drake
GitHub Actions allows you to automate your workflow. With GitHub Actions, you can deploy to any cloud, build containers, automate messages, and do so much more. Use any tool you…
Dependabot now updates your Actions workflows
GitHub Actions makes it easy to automate all your software workflows, from continuous integration and delivery to issue triage and more. Whether you want to build a container, deploy a…
Accelerometer and SoftSKU: Improving hardware platform performance for diverse microservices
What the research is: New strategies to improve the performance of hardware platforms running Facebook’s microservices. SoftSKU is a novel mechanism that tailors an existing server processor to optimize it for a specific microservice without requiring any additional hardware. Accelerometer is an analytical model that predicts gains from these optimizations even before the custom hardware […] Read
Why strace doesn't work in Docker
Why strace doesn't work in Docker
Releasing kubectl support in Access
Starting today, you can use Cloudflare Access and Argo Tunnel to securely manage your Kubernetes cluster with the kubectl command-line tool. SSO requirements and a zero-trust model to your Kubernetes management in under 30 minutes.
GitHub Actions: Community momentum, enterprise capabilities, and developer improvements
GitHub Actions continues its community momentum and ships new features for enterprises and developers.
