Desktop App Automation Using Playwright & Nut.js 

Introduction: 

Playwright is primarily known for its capabilities in automating interactions with web browsers, but it also provides support for desktop app automation. Playwright can be used to automate Electron based desktop applications across various platforms, including Windows, MacOS, and Linux. 

Nut.js aims to simplify desktop automation tasks by providing a higher-level API compared to Playwright. It abstracts away some of the complexities of interacting with desktop applications, making it easier for developers to write automation scripts. 

In this blog, we are going to automate the Codex desktop application using Playwright and Nutjs. 

Pre-Requisites: 

Before proceeding further, make sure that the dependencies below are installed on your system. 

• VS-Code 
• NodeJs 
• TypeScript

Features: 

Playwright:

1. Cross-browser: Playwright supports all modern rendering engines including Chromium, WebKit, and Firefox. 
2. Cross-platform: Supports test automation on Windows, Linux, and MacOS. 
3. Cross-language: Use the Playwright API in TypeScript, JavaScript, Python, .NET, Java. 
4. Auto-wait: The playwright waits for elements to be actionable prior to performing actions. It also has a rich set of introspection events. The combination of the two eliminates the need for artificial timeouts – the primary cause of flaky tests. 
5. Web-first assertions: Playwright assertions are created specifically for the dynamic web. Checks are automatically retried until the necessary conditions are met. 
6. Test Desktop App: Playwright can be used to automate Electron based desktop applications. 

NutJS: 

1. Automate desktop application 
2. Keyboard shortcut key automation 
3. Mouse interactions 
4. Screen Interactions 
5. Cross-platform 

Installation: 

Initialize a new Node.js project: 

Open your terminal or command prompt and navigate to the directory where you want to create your project. Then, run the following command to initialize a new Node.js project: 

npm init 

Playwright Installation: 

We need a playwright for overall electron-based desktop app automation. Run the following command to install Playwright as a dependency in your project: 

npm i playwright 

OR

npm init playwright@latest 

NutJS Installation: 

We need Nut.js to overcome a few limitations of Playwright for desktop automation in our case it’s right mouse clicks. Run the following command to install NutJS as a dependency in your project: 

npm i @nut-tree/nut-js 

Folder Structure:  

Make sure that the desktop automation framework folder structure is as same as mentioned in below image: 

Configuration: 

Create a new file named playwright.config.ts in the root directory of your project. 

Important Code Snippets: 

Launch Application:  

Launch electron-based desktop application using Playwright.  

Get X and Y coordinates of an element on Screen 

X and Y coordinates are required to perform the right mouse click on an element. Playwright provides a way to get these x and y coordinates.

Right-click on an element

To overcome the playwright’s limitation to perform the right mouse click efficiently on the desktop app, Nut.js is required. 

Testcases:  

Validate users can create, read, update, and delete notes folders and notes pages. 

Testcase1 – Verify user is able to create a notes folder. 

Testcase2 – Verify user is able to create a notes page inside a folder. 

Testcase3 – Verify the user is able to add notes on the page. 

Testcase4 – Verify the user is able to delete the notes page. 

Testcase5 – Verify the user is able to delete the notes folder. 

Tests Execution:

Scripts:

A] To execute all tests: 

npm run test 

B] To generate a report: 

npm run report 

Note: Please make sure to the above scripts in package.json file. 

Watch test execution and test report generation video:  

Conclusion: 

Automating the Codex desktop application through Playwright and Nut.js presents a powerful solution for streamlining processes and enhancing productivity. By leveraging the capabilities of these automation tools, users can achieve efficiency, reliability, and scalability in their software testing and development workflows. With the detailed insights provided in this blog, individuals and teams are equipped to embark on their automation journey with confidence, unlocking new possibilities for innovation and success in their projects.