How do all the popular browser automation tools work under the hood? Can they incorporate missing features in the near future based on their architecture and how their architecture compares to each other test runners?
The underlying architecture of all the browser automation tools are quite different. To communicate and control the browser and the application under the test, the majority of tools can be categorized into three categories:
- WebDriver protocol
- Chrome DevTools Protocol (CDP)
- Native (JavaScript loaded in the same browser application)
WebDriver Protocol
We can categorize the available browser automation tools into two groups, The ones that operate on the Webdriver and the ones that are based on the Chrome DevTools Protocol (CDP).
Selenium was an independent tool until it merged with a similar tool called WebDriver and hence became Selenium Webdriver. Every supported browser version must provide a driver that Webdriver uses for communication. This makes these frameworks cross-browser compatible. This protocol provides you with better support for running the same tests on different browsers and browser versions.
Unfortunately, Webdriver based tests have the reputation to be flaky which means that if they run under the same circumstances multiple times, they can either fail or succeed. That shows a bad reputation for a testing framework. You spend a good amount of time building something that should guarantee the stability of the application and then you cannot even trust that.
Some of the popular tools implementing this protocol are: Selenium Webdriver, WebdriverIO, Nightwatch
Chrome DevTools Protocol (CDP)
The Chrome DevTools Protocol (CDP) uses WebSocket connection to communicate from the test runner to the browser. DevTools are present in all modern browsers. As the name suggests it is a tool for developers which helps to analyze and debug.
CDP tool is available for Chromium-based browsers (Chrome, Edge, Opera, Brave). You can access this developer tool by using the keyboard shortcut CTRL + SHIFT + I or F12
Two quite popular tools are Puppeteer and Playwright. They do not depend on the Webdriver but instead talk directly to the browser via the Chrome DevTools Protocol (CDP). This provides them with much better control which leads to more stable tests.
Some of the popular tools implementing this protocol are: Playwright, Puppeteer.
Native (JavaScript loaded in the same browser application)
The application and the tests share the same process & domain.
Hence, the test can directly access the
- DOM
- Application’s objects
- Browser APIs.
By sharing the event loop, the test and the app take turns: When the app is running, the test code is waiting, when the test code is running, the app is paused. This guarantees that the application does not change between the test steps.
Some of the popular tools implementing this protocol are: Cypress, TestCafe.
Flakiness vs. Cross-Browser
From a very high-level point of view, the closer we get to the browser the more stable the tests become but very less support for cross-browser. On the other hand, the more abstraction we have between our tests and the browser, the flakier tests tend to become but will get more support for cross-browser.
| Webdriver Protocol | Chrome DevTools Protocol |
| Developed by: W3C | Developed by: Chrome Developer Tools |
| Needs drivers (proxy servers) | No need for drivers |
| Relatively slow and More Flaky | Fast and Less Flaky |
| Supports most of the browsers (More cross-browser support) | Supports only modern browsers (Less cross-browser support) |
| No access to network-related info (does not support API testing) | Has access to the network (Supports API testing) |
| Tools Implemented: Selenium WebDriver 3, WebdriverIO, Nightwatch | Tools Implemented: Playwright, Puppeteer, Selenium Webdriver 4, Cypress v7 |
Future Trends:
- Currently, Selenium Webdriver 4 and WebdriverIO are trying to go beyond JSON HTTP protocol to using a bidirectional WebSocket protocol similar to CDP.
- The Playwright team is patching Firefox and WebKit (Safari) to allow CDP to work with all major browsers.
- Cypress also has CDP support built-in already from v7 onwards.
In the future, all browsers might support CDP, or the upcoming protocol known as WebDriver BiDi which will bring stability into the WebDriver world. Things are still evolving.
Considering it all together, this is the current state of the browser automation protocols used for various popular tools available in the market as of now:
Happy Automation Testing!
