Why Request Header Control Matters in UI Testing

HTTP(S) headers are key-value pairs that carry supplementary information about a request or response. A header name is case-insensitive and followed by a colon (:) and its value; values are case-sensitive, and multiple values are separated by commas. While headers often go unnoticed during routine browsing, they become critical in testing scenarios where you need to bypass authentication, set cookies, simulate guest sessions, or enable disabled application features by injecting a custom request header.

A concrete example: many sites refuse to render inside an iframe because of the x-frame-options header (set to deny or sameorigin) or the frame-ancestors directive in the content-security-policy header. To test such pages in an embedded context, you need to override or strip those headers—something that reveals the broader utility of header manipulation in automated UI testing.

Where Selenium Falls Short

Selenium WebDriver is the dominant framework for end-to-end browser tests: it is open source, supports all major programming languages, runs cross-platform, and simulates keyboard and cursor input. However, it has a known gap: no native API to modify request headers, add request parameters, or block requests. The Selenium project has indicated no plans to add this capability, so testers must look outside the core driver.

Approach 1: Selenium Wire for Python

Selenium Wire extends the official Selenium Python bindings with APIs for inspecting and mutating browser traffic. It retains the familiar Selenium authoring style while adding interceptors that can modify requests and responses on the fly, block requests, and mock responses.

To set a request header:

# interceptor function intercepts the network request
# If one arg is provided, requests are intercepted
# and can be modified
def interceptor(request):
    request.headers['New-Header'] = 'Some Value'
# setting the driver's request_interceptor to equal
# the customised interceptor
driver.request_interceptor = interceptor
driver.get(<URL_where_to_modify_the_header>)

# All requests will now contain New-Header

Note that duplicate header names are legal in HTTP requests, so you must first remove any existing header with del before adding your replacement; otherwise both headers will be sent.

Response headers are handled with a two-argument interceptor:

# A response interceptor takes two args which
# then allows to tinker with the response
def interceptor(request, response):  
    if request.url == 'https://server.com/some/path':
        response.headers['New-Header'] = 'Some Value'
driver.response_interceptor = interceptor
driver.get(<URL_where_to_modify_the_header)

# Responses from https://server.com/some/path will now contain 
# the New-Header

The main constraint is that Selenium Wire is a Python-only module; users of other Selenium language bindings can't use it.

Approach 2: Browser Extensions in Selenium

Requestly is a browser extension suite that intercepts and modifies network traffic, supporting header changes, URL redirection, host switching, API response mocking, request delay, and custom script injection. Requestly ships an npm wrapper, @requestly/selenium, that lets Selenium tests drive the extension in Chrome, Firefox, and Edge.

Installation:

npm i @requestly/selenium

Rules are authored in the Requestly web UI at app.requestly.io/rules rather than in test code. After creating a rule, use the Share button to obtain a shared-list link; for example, one that adds an Access-Control-Allow-Origin header to all requests would yield a URL like this one.

That shared-list URL is then consumed in the WebDriver setup as follows:

require("chromedriver");
const { Builder } = require("selenium-webdriver");
const chrome = require("selenium-webdriver/chrome");
const { getRequestlyExtension, importRequestlySharedList } = require("@requestly/selenium");

const options = new chrome.Options().addExtensions(getRequestlyExtension("chrome"));
const driver = new Builder()
    .forBrowser("chrome")
    .setChromeOptions(options)
    .build();

// Imports Rules in Selenium using Requestly sharedList feature
// importRequestlySharedList(driver, <sharedList_URL>);

importRequestlySharedList(driver, 'https://app.requestly.io/rules/#sharedList/1626984924247-Adding-Headers-Example');

Limitations here are the JavaScript-only npm package and the manual step of creating and sharing rules — the rule set can't be altered programmatically from within the test run.

Approach 3: Puppeteer and Chrome DevTools Protocol

Puppeteer is a Google-backed Node library that controls headless Chrome or Chromium through the DevTools Protocol. It requires no external driver and gives deeper control over browser internals than Selenium typically offers. To customize headers, set them directly with:

const puppeteer = require('puppeteer');

(async () => {
    const browser = await puppeteer.launch();
    const page = await browser.newPage();
    await page.goto('<https://example.com>');
    await page.screenshot({ path: 'example.png' });
    await browser.close();
})();

For removing or conditionally altering headers, enable request interception and handle the request events:

await page.setRequestInterception(true);
page.on('request', request => {
// Override headers
    const headers = Object.assign({}, request.headers(), {
    foo: 'bar', // set "foo" header
    origin: undefined, // remove "origin" header
});

request.continue({headers});

});

Puppeteer's weaknesses are its Chrome-only scope, a smaller community than Selenium's, and JavaScript-only bindings. It is nevertheless a natural fit when you are already Chrome-focused and want header manipulation without a proxy or extension layer.

Choosing a Strategy

Each approach suits a different context:

  • Python Selenium users: Selenium Wire adds extensive traffic-control features on top of the existing WebDriver API.
  • Chrome-only testers: Puppeteer offers first-party DevTools support and built-in header modification.
  • Cross-browser Selenium teams: Load an extension like Requestly into the driver; it also covers runtime script injection and network redirection needs beyond simple header edits.

All three methods bypass Selenium's inherent limitation and give testers dependable control over request and response headers.