HttpSession

mimic::http_session::HttpSession sends plain HTTP requests that look like the Chrome a browser page came from. A browser spends a second or more on a page; a request spends milliseconds. Once a page has earned its cookies (a login, a passed check), an HttpSession carries them on: the measured TLS ClientHello and HTTP/2 start, the header names, order and spelling of that Chrome on that system, and the user agent, client hints and Accept-Language of the page or identity. A browser or system that was not measured is refused, not approximated.

use seleniumbase_rs_mimic::{HttpOptions, HttpSession};use seleniumbase_rs::sb_cdp::Page;# async fn demo(page: Page) -> Result<(), Box<dyn std::error::Error>> {let session = HttpSession::from_page(&page, None, HttpOptions::new()).await?;let reply = session.get("https://example.com/api/orders").await?;let orders: serde_json::Value = reply.json()?;println!("{orders}");// Write the cookies that responses set back into the browser.session.sync_cookies_back(&page).await?;# Ok(())# }

Making one#

• HttpSession::from_page(&page, egress, options) (async) copies the cookies of the page’s browser profile into the jar, reads navigator.userAgent and navigator.languages from the page, and follows the route its traffic takes: the proxy of its browser context. egress is the exit the page leaves through; None takes the context’s own proxy or goes direct. A browser-wide proxy, such as a MITM proxy the browser was launched with, is not visible to a page, so say so with egress.
• HttpSession::from_identity(&identity, egress, options) builds one from a StealthIdentity: its fingerprint becomes the user agent, hints and languages, and its cookies fill the jar.

Both refuse, before any request, what would contradict the measured handshake:

Requests#

session.get(url) sends a GET. session.request(method, url) returns an HttpRequest; nothing is sent until .send().await, and a mistake in a setter is reported there.

use seleniumbase_rs_mimic::{HttpSession, Method};use seleniumbase_rs_codec::header_spec::RequestShape;# async fn demo(session: HttpSession) -> Result<(), Box<dyn std::error::Error>> {let reply = session    .request(Method::POST, "https://example.com/api/search")    .shape(RequestShape::Fetch)    .header("x-requested-with", "XMLHttpRequest")    .json(&serde_json::json!({ "q": "shoes" }))    .send()    .await?;println!("{}: {}", reply.status(), reply.text()?);# Ok(())# }

• GET and HEAD carry the headers of a navigation (RequestShape::Document), every other method those of a script’s request (RequestShape::Fetch); .shape(..) chooses. .header, .body, .json, .form and .timeout complete the request. Method is wreq::Method, re-exported.
• The profile owns User-Agent, Sec-CH-UA*, Accept-Language, Accept-Encoding, Cookie and priority, and the connection owns Host, Content-Length and the like. Setting one is HttpError::ReservedHeader, not a silent overwrite. Any other header replaces the profile’s value in place or is added. CONNECT, TRACE and TRACK are refused, and so is a URL with credentials in it or a scheme other than http and https.
• A status other than 2xx is a reply, not an error. HttpReply has status(), headers() (ReplyHeaders), bytes(), text(), json::<T>(), url() after redirects (and redacted_url()), redirects(), protocol(), elapsed() and detect().

What the session does#

• Cookies. The jar is the RFC 6265 CookieJar. The session writes the Cookie header and reads Set-Cookie itself; wreq’s own store is off. A cookie for a public suffix, for another host, or against the Secure, __Host- and SameSite rules is refused as a browser refuses it. sync_cookies_back(&page) writes only what this session changed into the browser and returns how many cookies it wrote or removed; if the browser refuses, the changes are kept for the next attempt.
• Redirects are followed by hand, up to 10 by default: the cookies of each hop are stored and sent per URL, 303 becomes a GET, and the headers you set are dropped when a hop leaves the origin (Accept and Content-Type stay). A redirect is followed where it points, a private address included, so a session that fetches URLs for someone else must filter them before it asks.

Limits are HttpOptions, with defaults that are safe on the open web:

Also event_sink, trust(CertStore) for a private root (certificate verification is never turned off), resolve(host, addr), bypass(host), installed_chrome_major(major) and clock(..). HttpOptions::validate refuses a value out of range before a client exists.

• Judging replies. reply.detect() runs the challenge detection. session.assess(&reply, &Attempt, &Rules, &identity) adds what the MitigationPolicy and the SessionPolicy make of it, as a Verdict { detection, action, transition }; session.fetch(url, &attempt, &rules, &identity) sends a GET and judges it in one call (Fetched { reply, verdict }); session.preflight(url, &identity, state, &rules) asks, before a request, whether the identity’s clearance is about to end so that the browser should take the URL first. They are the pieces of your own loop; StealthSession is that loop.

  use seleniumbase_rs_mimic::http_session::{Attempt, Rules};use seleniumbase_rs_mimic::HttpSession;use seleniumbase_rs_scrape::StealthIdentity;# async fn demo(session: HttpSession, identity: StealthIdentity) -> Result<(), Box<dyn std::error::Error>> {let fetched = session    .fetch("https://example.com/", &Attempt::default(), &Rules::default(), &identity)    .await?;if fetched.verdict.is_clear() {    println!("{}", fetched.reply.text()?);} else {    println!("{:?}", fetched.verdict);}# Ok(())# }

• Identity. export_identity(&base) puts the session’s cookies back into a StealthIdentity and refreshes its clearances. egress() is the exit without credentials, cookie_count() and profile() read the state. Cloning gives another handle to the same session.

• Secrets. Cookies, credentials and proxy passwords never reach Debug, logs, events or errors: URLs are redacted, a failure of the HTTP client is reduced to a word, and header values are not quoted.

What to know#

Next steps#