JavaScript, events and the Web APIs

JavaScript and the browser are two things#

The JavaScript language (called ECMAScript) gives you variables, functions, arrays, promises and Math. It has no idea what a web page is. Everything that touches the page or the outside world comes from the browser, as objects and functions that already exist when your script starts:

• window, the global object, and document, the page (see selectors and the DOM).
• navigator, facts about the browser and the device.
• fetch, localStorage, setTimeout, console, location, history.

These are Web APIs. Each one is written down in a standard (mostly by the WHATWG and the W3C) so that every browser implements it the same way. The same JavaScript runs in a Node.js server without them: document does not exist there. That is also why an automation tool has to run its script inside the page (see Running your own JavaScript).

The APIs you meet most#

Hundreds more exist (sensors, Bluetooth, payment, speech). New ones appear first in Chromium, which is why a feature you rely on can work in your tests and fail for a user on another browser.

Secure contexts and permissions#

Powerful APIs are limited in two ways.

• Secure context. They work only on https:// pages (and on localhost, so developers can try them). On a plain http:// page the object is missing.
• Permission. Many need the user to say yes, through a prompt that belongs to the browser, not to the page.

That second rule is a trap for automation. A prompt asking for the location is browser interface, outside the page’s DOM, so no selector can click “Allow”. The answer is to settle the permission before the page asks, with the DevTools Protocol: Browser.grantPermissions grants a permission to an origin, and Emulation.setGeolocationOverride sets the position the page will receive. In a headless run with no answer, the request usually just fails with the state denied or stays prompt.

The event loop#

JavaScript in a page runs on one thread. It cannot do two things at once, so the browser keeps a queue of work and a loop that takes one job at a time.

One turn of the loop: run a task, run its microtasks, maybe repaint, repeat.

Three consequences matter for tests.

• Nothing happens “in the middle” of a task. A long computation freezes the page: clicks are queued, and nothing repaints, until it ends.
• setTimeout(fn, 500) means “no sooner than 500 ms”, not “at 500 ms”. The callback waits its turn.
• An action’s effect usually arrives later. A click runs its handler in one task; what the handler asks for (a request, an animation) finishes in later ones. The click has returned long before the page has changed.

Loading things after the page: fetch and XHR#

A modern page rarely arrives complete. The first HTML is a shell, and scripts then ask the server for the data and fill the page in. Two APIs do the asking, XMLHttpRequest (old) and fetch (newer). Both are asynchronous: the call returns at once and the answer arrives later as an event, which the loop turns into a task. The page may fire a dozen of these after the “load” event.

This is why the page load events are only the start of readiness. load says the document and its first files arrived; it says nothing about the data that the scripts are still asking for. A test needs to wait for the thing it wants (an element, a text, a finished request), not for the page.

Single-page apps#

In a single-page application (SPA) the browser loads the app once, and clicking a link does not load a new page. A script changes the address with the History API, asks for data with fetch, and rewrites the DOM. React, Vue, Angular and Svelte work this way. See modern web technologies.

For automation this changes three things. There is no load event after the first, so “wait for the page to load” returns at once. The URL changes without a navigation. And elements are created and replaced all the time, so an element you found a moment ago can be removed from under you.

Observers: telling you when something changes#

A modern page is never “finished”. It loads data after the load event, adds rows while you scroll and swaps parts of itself. Instead of asking the page again and again, observers let you subscribe: the browser calls your function when something happens.

All four work the same way: create the observer with a callback, call observe() on a target, call disconnect() when done. They are asynchronous: the callback runs after the change, batched, on the main thread.

Waiting for a change#

Two ways to wait for “the results are there”:

• Poll. Ask every 100 ms or so, for example “does #results have children yet?”, until yes or a timeout. It is simple, works anywhere, and is what the waits of most tools do.
• Observe. Let the browser tell you. It reacts at once and costs nothing between changes, but you write a small script.

A reliable observer script checks the state first (the change may already have happened), then watches, and always has a timeout:

new Promise((resolve, reject) => {  const target = document.querySelector('#results');  const done = () => target.children.length > 0;  if (done()) return resolve(target.children.length);  const timer = setTimeout(() => { observer.disconnect(); reject(new Error('no rows in 5 s')); }, 5000);  const observer = new MutationObserver(() => {    if (done()) { clearTimeout(timer); observer.disconnect(); resolve(target.children.length); }  });  observer.observe(target, { childList: true });})

How a click or a key press becomes an event#

When you press a mouse button, the operating system reports it to the browser process, which finds out which page and which element is under the pointer and sends the input to that page’s renderer. The renderer builds an event object and delivers it to the DOM in three steps: the capture phase (from the top down to the target), the target itself, and the bubble phase (back up).

One physical click is several events, in a fixed order:

Typing is similar: keydown, then (for a character) keypress and a beforeinput and input event as the text changes, then keyup. A page may react to any of them, and a framework often listens to input to keep its state in step with the field. A field filled by assigning element.value = "x" raises none of these events, so the framework never learns about the text.

Trusted and synthetic events#

Every event object has a property, event.isTrusted.

• true when the event came from the browser itself: a real input, or one injected through the browser’s own input path.
• false when a script created it, with new MouseEvent(...) and dispatchEvent, or with element.click().

A script can only produce false. The page can read the flag, so it can tell. A site can refuse an untrusted click on a payment button, or a cookie banner can ignore it. Some actions are permitted only on a trusted event: opening a file chooser, going fullscreen, playing sound, copying to the clipboard, and anything that needs a user gesture.

Auto-waiting and actionability#

Because of everything above, a click should not be fired the instant the selector matches. A good tool waits until the element is actionable: ready for a person to use it. The usual checks are that the element

1. exists,
2. is visible (it has a size and is not hidden),
3. is not covered by another element at the click point,
4. is enabled, and
5. has stopped moving.

The tool re-checks until all hold or a timeout runs out, then fails with a message saying which check failed. This is called auto-waiting, and it replaces the sleeps (sleep(2)) that make a suite slow and still flaky.

In seleniumbase-rs:

• Pure CDP. A Locator waits for its element to be visible, scrolls it into view and works out its centre before it acts (Locator::click, type_text, hover and the other actions do this). The wait is bounded by the locator’s timeout, which Locator::with_timeout changes. If the element never appears the error is ElementNotFound; if it exists but never becomes visible, ElementNotInteractable; and a plain wait that runs out gives WaitTimeout. You can also wait yourself with locator.wait_for(State::Visible), and State has Present, Absent, Visible and Hidden.
• WebDriver. BaseCase methods wait for the element, for example wait_for_element_visible(selector, 10) takes a timeout in seconds, and wait_for_ready_state_complete() waits for the document’s ready state after a navigation.
• Pages. Page::goto waits for the load event; Page::wait_for_load waits for a load that a click started.

Reading MDN and its compatibility tables#

MDN Web Docs is the reference for Web APIs. Learning to read a page pays off every time you automate something new.

How seleniumbase-rs uses it#

Sending input#

The pure CDP engine (src/sb_cdp/input.rs) sends input through the DevTools Input domain, the browser’s own input path, so every event is trusted. See the DevTools Protocol.

Input with a person’s pace#

Real input is trusted, but a script’s movement is still unlike a person’s: the pointer jumps in a straight line to the exact centre, and letters arrive at the same interval. Page::human(Behavior) returns a Human whose move_to, click(&Locator), type_text(&Locator, text), type_focused and scroll_by move the pointer along a curve, land the click somewhere inside the element and type at an uneven rhythm. The events are the same trusted input; only the path and timing differ. It is slower on purpose, so use it where a page watches how input arrives. See Detection and spoofing and the glossary entry Human-paced input.

use seleniumbase_rs::stealth::behavior::Behavior;let person = page.human(Behavior::builder().typing_wpm(65.0).build()?);person.type_text(&page.locator("#search"), "rust cdp").await?;person.click(&page.locator("button[type=submit]")).await?;

Running your own JavaScript#

Sometimes you need to read something the DOM does not expose, or call a function of the page. Page::evaluate("document.title") runs a script in the page and returns its value, and Page::execute("Runtime.evaluate", ...) sends the raw command (see Any CDP command). The script runs in the page’s own world, with the page’s globals, and is bound by the page’s rules: if the page redefined a function, your script sees the change.

Calling Web APIs from a test#

seleniumbase-rs has no wrapper for these APIs, and does not need one: you call them by running JavaScript in the page. That is also how the library itself reads much of the page.

let theme = sb.execute_script("return localStorage.getItem('theme');").await?;let online = sb.execute_script("return navigator.onLine;").await?;

let rows: u64 = page    .evaluate_as(        r#"new Promise(resolve => {             const target = document.querySelector('#results');             const done = () => target.children.length > 0;             if (done()) return resolve(target.children.length);             new MutationObserver((_, observer) => {               if (done()) { observer.disconnect(); resolve(target.children.length); }             }).observe(target, { childList: true });           })"#,    )    .await?;

Where to go next#