-
Notifications
You must be signed in to change notification settings - Fork 1
Running Tests
Every test runs inside an Embedded Browser (EB). There is no local-Playwright fallback and no remote-runner daemon -- the host never launches its own browser, and if no EB is available the run fails rather than falling back. The EB stack is required even in local development. See CI Trigger CLI for the full execution model.
All execution happens in an Embedded Browser: a containerized Chromium with CDP live streaming back to the UI. EBs come in two flavors:
| Flavor | How | When |
|---|---|---|
| System-managed (default) | EBs auto-provisioned into your cluster -- k3d locally, Kubernetes in production -- from an on-demand pool. One EB per test; the provisioner pre-warms a small pool to keep latency low. |
The default for all dev and prod runs. |
| Bring-your-own (BYO) | You run the lastest-eb container yourself and register it in Settings → Runners. The browser runs on the machine/network/OS you choose. |
Run tests near a private environment, on a specific OS, or on your own hardware -- the replacement for old "remote runners". |
Both flavors support running and recording. EB pods are short-lived (one job, one EB).
Exception: headless API tests (
testType: 'api') run in-process on the server with no browser or EB dispatch, then feed the same step-comparison and verdict pipeline as browser tests. See API Tests.
Builds can be triggered in multiple ways:
Click Run in the UI to trigger a build immediately. You can:
- Run all tests in a repository
- Run a specific test suite
- Run individual tests
- Use the Compose page to cherry-pick specific tests and versions
When a PR is opened or updated:
- GitHub webhooks trigger a build automatically
- GitLab MR webhooks trigger a build automatically
- Results are posted back as PR/MR comments
- GitHub Action -- use the reusable action (see CI/CD Integration)
-
CI Trigger CLI --
npx @lastest/runner triggerfrom any pipeline; it creates the build and polls, while execution runs server-side in the EB pool (see CI Trigger CLI)
- Cron-based -- automate builds on preset schedules (daily, weekly, hourly) or custom cron expressions
- See Scheduled Runs for details
| Status | Meaning |
|---|---|
passed |
All tests passed, no visual changes |
safe_to_merge |
Tests passed, only approved/insignificant changes |
review_required |
Visual changes detected that need review |
blocked |
Unapproved meaningful changes present |
failed |
One or more tests failed to execute |
Run only tests affected by your code changes:
- Select a feature branch (not main/master)
- Lastest compares against the default branch via GitHub/GitLab API
- Tests are matched to changed files by URL patterns and code references
- Only affected tests run, skipping unchanged areas
This dramatically reduces test time for large suites while maintaining coverage.
Step through test execution with live feedback:
- See each step as it executes
- View screenshots at each step
- Identify exactly where failures occur
Replay any test (or test version) with a visible browser window for debugging. Useful when a failure isn't reproducible in headless or you want to watch selectors resolve in real time. Available from the test detail page; honours the same setup/teardown sequence as a normal run.
After a recording finishes, Lastest runs a sanity-check pass that resolves every captured selector on the page before saving the test. Bad selectors are surfaced immediately rather than at the next test run, dramatically reducing brittle-on-day-one failures.
Any build, run, or visual diff can be shared via a public link without requiring login. The shared view embeds a watermark and read-only controls so external reviewers (designers, PMs, customers) can see screenshots and approve/comment without an account on the instance.
Configure the maximum number of tests that run simultaneously:
- Set in Settings Reference
- Bounds the embedded-browser pool per build
- The EB pool size and warm-pool keep-alive are tuned via env (
EB_LAUNCH_INTERVAL_MSetc.) - CNI burst protection throttles pod creation to avoid networking storms
Cherry-pick tests and pin specific test versions per build:
- Go to the Compose page
- Select which tests to include
- Optionally pin specific versions (override the latest version)
- Run the composed build
Lastest manages baselines per branch:
- Fork: When a branch is created, baselines are forked from the parent branch
- Merge: When a PR is merged, baselines merge back to the target branch
- Promote: Promote test versions across branches
- SHA256 carry-forward: Matching hashes mean instant pass without pixel comparison
Run preparation steps before tests execute:
- Playwright scripts -- browser-based setup (login, seed data via UI)
- API scripts -- HTTP requests to seed data programmatically
- Test-as-setup -- use another test as a setup step
Clean up after tests complete:
- Repository-default teardown sequence
- Per-test overrides (skip default steps or add extra ones)
- Teardown errors are non-blocking -- they won't fail your build
Lastest Wiki
Getting Started
Core Features
- Visual Diffing
- AI Configuration
- Fix-the-App Advisor
- Agent Monitoring
- Public Shares
- Gamification
- Scheduled Runs
- Bug Reports
- Settings Reference
Integrations
- GitHub Integration
- GitLab Integration
- Google Sheets Integration
- Custom Webhooks
- VSCode Extension API
- MCP Server
Deployment & CI
Administration
Reference