How we migrate Clicknium to Chrome Manifest V3

Introduction

We started a project called Clicknium this year, a Python automation library for browser and desktop applications. It depends on the Clicknium browser extension to control the browser, and simulate mouse and keyboard actions.

Since January 2022, adding new extensions based on Manifest V2 to the Chrome Web Store has been forbidden. All extensions on Manifest V2 will stop working in early 2023, even those that were added to Chrome Web Store earlier.

We have no choice but to migrate to V3. After we review our code and migration checklist,
the most significant impact on Clicknium is that running remote code will be forbidden in Manifest V3.
It asked to include all logic in the extension's package to review the behavior of the extension better.

How does Clicknium run remote code in Manifest V3? Here is a brief introduction.

How Clicknium runs remote code in V2

This is a simple Clicknium automation case, which executes a JS script on the Bing page to set the value of the search box to "Clicknium", and return "success".

from clicknium import clicknium as cc, locator
import os,sys

def main():
    tab = cc.chrome.open("bing.com")
    ele = tab.find_element(locator.bing.cn.search_sb_form_q)
    result = ele.execute_js("function SetText(st){_context$.currentElement.value = st; return \"success\"}", "SetText(\"Clicknium\")")
    print(result)
if __name__ == "__main__":
    main()

In Manifest V2:

• In the background: Through chrome.runtime.sendMessage sends the received string to the content.
• In the content: Using "new Function(jsCode)" to convert the received string to Javascript's Function,

const innerFunction = new Function('_context$', jsCode);
const ret = innerFunction({
    currentElement: targetElement,
    tabId: msg.tabId
});

In Manifest V3, Function and eval is not available. So the main problem is that how to convert string to Javascript's Function.

How Clicknium run remote code in V3

There are two approaches to achieving it.

Solution 1: Use pop up window to imitate the background scripts/pages.

In Manifest V3, Using Service worker replace background scripts/pages. The window object can't be used in the Service worker.
But we can open a popup window in the Service worker to imitate background scripts/pages.

• Step 1 : Open a popup window in the Service worker.

    // Get the URL of the popup window.
    const backgroundUrl = chrome.runtime.getURL("backgroundPage.html");

    await chrome.windows.create({
        type: "popup",
        state: "minimized",
        url: backgroundUrl,
    });

• Step 2 : In the popup window, when received a message, using chrome.debugger.sendCommand sends to the target of the popup window. And then, get Function from the window object.

    // Get the target of the popup window
    const backgroundUrl = chrome.runtime.getURL("backgroundPage.html");
    const targets = await chrome.debugger.getTargets();
    let target;
    for (const t of targets) {
      if (target.url === backgroundUrl) {
        target = t;
      }
    }

    // Wrap up the Javascript's string, register to the window object
    const jsCodeWrapper = `window["evalFunction"] = function () {
      ${jsCode};
    };`;

    // Attach to the popup window. And send expression.
    await chrome.debugger.attach(target, "1.2");
    await chrome.debugger.sendCommand(target, "Runtime.evaluate", {
      expression: jsCodeWrapper,
    });

    // Get the function just registered in the window
    const wrapperFunc = window["evalFunction"];

Step 3: Using chrome.scripting.executeScript to run Function in content.

    await chrome.scripting.executeScript({
      target: { tabId: msg.tabId, frameIds: [msg.frameId] },
      func:wrapperFunc,
    });

Advantage:

• Remote code works in content and background.
• Support to cross-domain iFrame.

Disadvantage:

• Need to open a popup window.

Solution 2: In Service worker, using chrome.debugger.sendCommand sends to content.

    interface CDPRuntimeEvaluateResult {
        exceptionDetails?: { exception?: { description?: string } };

        result: { value: unknown };
    }

    await chrome.debugger.attach({tabId: msg.tabId}, "1.2");

    const result = (await chrome.debugger.sendCommand(
      { tabId: msg.tabId },
      "Runtime.evaluate",
      {
        expression: js,
      }
    )) as CDPRuntimeEvaluateResult;

Advantage:

• No need to open a popup window.

Disadvantage:

• Not support for cross-domain iFrame. Cross-domain iFrame's target isn't in the list of available debug targets, which uses the chrome.debugger.getTargets to get.
• Remote code can't work in the background.

Summary

As cross-domain iFrame is a critical feature we need, Clicknium choosesSolution 1: use popup window to call chrome.debugger.sendCommand.

Manifest V3 has been supported by Clicknium since version 0.1.10.
You can try it in
pypi website and Clicknium VS Code Extension.

One thing that should be noticed is that the Clicknium browser extension has to work with the Clicknium Python SDK. If we publish the browser extension to Chrome Web Store, all the existing extensions will auto-upgrade to the latest version, and it can not work with the SDK before V0.1.0. So we have to pull it off shelves and wait for a period. If you want to try the M3 version, please upgrade to the latest Python SDK.