Get started

Self-host Litmus Check

Litmus Check runs entirely on your own infrastructure. First stand up the open-source server and UI — then plug in the litmus-agent CLI to triage Playwright failures on your machine or inside your CI/CD pipeline.

1 · Self-host the stack→2 · Triage with the CLI (optional)

→ Self-host the stack → Triage CLI

1. Self-host the stack

Required

The foundation. Run the lc-server QA engine and the lc-frontend UI on your own infrastructure — everything else, including the CLI, builds on this.

lc-server — start here

Python · Flask · TypeScript

The AI QA engine. Three services: a Flask backend (test/suite/run management on :6010), the LitmusAgent TypeScript Playwright runner, and a Node.js WebSocket gateway.

You'll need: Python 3.12+, Node.js 18+, PostgreSQL, Redis, and Azure OpenAI + Azure Storage credentials (Docker to build the agent image).

Run the Flask backend:

$ git clone https://github.com/litmus-check/lc-server.git
$ cd lc-server
$ virtualenv venv --python=python3.12 && source venv/bin/activate
$ pip install -r requirements.txt
$ cd src && python app.py            # Flask API on :6010

Create an app.env file in the repo root with your PostgreSQL, Redis, Azure OpenAI, and Azure Storage keys, plus JWT_SECRET_KEY — the full list is in the README. Run the background worker with celery -A tasks worker, and bring up the agent + Redis for local runs with docker-compose up from LitmusAgent/.

lc-server README →

lc-frontend — then the UI

Next.js · TypeScript

The Next.js UI. Deploy it on Vercel or any Node.js host, and point it at the lc-server you just started with environment variables.

$ git clone https://github.com/litmus-check/lc-frontend.git
$ cd lc-frontend
$ pnpm install
$ cp .env.example .env.local   # then edit the values below
$ pnpm run build && pnpm start

Point it at your backend:

.env.local

# lc-frontend .env.local
NEXT_PUBLIC_LITMUSCHECK_URL=https://your-litmus.example.com/api/v1
NEXT_PUBLIC_BASE_API_URL=https://your-litmus.example.com
NEXT_PUBLIC_WEBSOCKET_URL=wss://your-litmus.example.com
NEXT_PUBLIC_ENV=PROD

lc-frontend README →

2. Triage with the CLI

litmus-agent · Node.js

litmus-agent triages Playwright failures: it reads your JSON report and traces and tells you what broke. Ideal for triaging locally and, especially, inside a CI/CD pipeline.

🔑 Requires a running self-hosted Litmus Check. The CLI sends your report to your server for triage, and its LITMUS_API_KEY is generated there. Complete step 1 first.

Install the CLI

Install globally, or add it to your project as a dev dependency.

$ npm install -g litmus-agent

Configure Playwright

The CLI needs the JSON reporter and traces retained on failure. Update your Playwright config:

playwright.config.ts

import { defineConfig, devices } from '@playwright/test';

export default defineConfig({
  testDir: './tests',
  retries: 0,
  reporter: [
    ['json', { outputFile: './reports/report.json' }],
  ],
  use: {
    trace: 'retain-on-failure',
    screenshot: 'only-on-failure',
  },
  outputDir: './traces',
  projects: [
    { name: 'chromium', use: { ...devices['Desktop Chrome'] } },
  ],
});

Set your API key

Generate a LITMUS_API_KEY in your self-hosted Litmus Check instance, then add it to a .env file in your project root.

.env

# .env
LITMUS_API_KEY=your_api_key_here

Triage a run

Point the CLI at your report. Add --pretty to format the output, or -o to write it to a file.

$ lc triage ./reports/report.json --pretty

Triage in CI/CD

Run your suite in CI, then triage the failures automatically. Store the key from your self-hosted instance as a CI secret (LITMUS_API_KEY). Example with GitHub Actions:

.github/workflows/e2e.yml

# .github/workflows/e2e.yml
name: E2E
on: [push]

jobs:
  test:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-node@v4
        with: { node-version: 20 }
      - run: npm ci
      - run: npx playwright install --with-deps

      # Run the suite (don't fail the job yet — we want to triage failures)
      - run: npx playwright test
        continue-on-error: true

      # Triage failures against your self-hosted Litmus Check
      - name: Triage with Litmus
        env:
          LITMUS_API_KEY: ${{ secrets.LITMUS_API_KEY }}
        run: npx litmus-agent triage ./reports/report.json --pretty

Full options and reference: litmus-agent on npm.

Want to contribute?

Issues, PRs, and stars are all welcome.

How to contribute