TestBox Reporters — Comprehensive Reference

When to Use This Skill

• Choosing the right reporter for a use case (CI, development, IDE, browser)
• Configuring reporter-specific options (hideSkipped, IDE links)
• Using the AgentReporter for token-efficient output for AI agents and automation
• Using the StreamingReporter for real-time SSE output
• Building a custom reporter by implementing IReporter

---

Built-In Reporters

Reporter Key	Class	Best For
agent	testbox.system.reports.AgentReporter	AI agents, minimal token usage (TB7.2+)
antjunit	testbox.system.reports.ANTJunitReporter	Ant/legacy CI pipelines
console	testbox.system.reports.ConsoleReporter	CI stdout logs
doc	testbox.system.reports.DocReporter	Living documentation
json	testbox.system.reports.JSONReporter	API / tooling consumption
junit	testbox.system.reports.JUnitReporter	Modern CI (GitHub Actions, Jenkins)
min	testbox.system.reports.MinReporter	Fast dot-notation output
mintext	testbox.system.reports.MinTextReporter	Plain-text minimal (no ANSI)
simple	testbox.system.reports.SimpleReporter	Rich HTML browser view
text	testbox.system.reports.TextReporter	Plain-text verbose output
xml	testbox.system.reports.XMLReporter	XML consumers, legacy tools
streaming	testbox.system.reports.StreamingReporter	SSE real-time output (TB7+)

---

Reporter Details

min — Minimal (default for CLI)

Prints a dot (.) for pass, F for fail, E for error, S for skip. Summary at the end.

./testbox/run --reporter=min
testbox run reporter=min

Output:

.....F.....S...E.......
Tests: 15  Pass: 12  Fail: 1  Error: 1  Skipped: 1  Duration: 234ms

---

mintext — Minimal Text (no ANSI colors)

Same as min but no ANSI escape codes — suitable for log files.

./testbox/run --reporter=mintext

---

console — Console (colored verbose)

Prints each spec name with colored pass/fail indicator. Useful in CI logs where you want readable spec names.

Options

new testbox.system.TestBox(
    directory: { mapping: "tests.specs", recurse: true },
    reporter:  {
        class:   "testbox.system.reports.ConsoleReporter",
        options: { hideSkipped: true }   // suppress skipped spec noise
    }
).run()

# Via CLI
testbox run reporter=console options.hideSkipped=true

---

simple — Simple HTML

Rich HTML output with collapsible suites, color coding, and IDE deep-link support.

Editor Links

Configure Deep Links so clicking a spec name opens the file in your IDE:

reporter: {
    class:   "testbox.system.reports.SimpleReporter",
    options: {
        // Supported editors:
        editor: "vscode"       // vscode://file/{path}:{line}
        editor: "vscode-insiders"
        editor: "sublimetext"  // subl://open?url=file://{path}&line={line}
        editor: "textmate"     // txmt://open?url=file://{path}&line={line}
        editor: "emacs"        // emacs://open?url=file://{path}&line={line}
        editor: "macvim"       // mvim://open?url=file://{path}&line={line}
        editor: "atom"         // atom://open?src={path}&line={line}
        editor: "idea"         // idea://open?file={path}&line={line}
    }
}

In box.json:

{
  "testbox": {
    "runner": "http://localhost:8080/tests/runner.cfm",
    "reporter": {
      "class": "testbox.system.reports.SimpleReporter",
      "options": { "editor": "vscode" }
    }
  }
}

---

json — JSON

Returns the full result set as a JSON string. Useful for programmatic consumption, test dashboards, or custom CI tooling.

var results = new testbox.system.TestBox(
    directory: { mapping: "tests.specs", recurse: true },
    reporter:  "json"
).runRaw()

Typical JSON shape:

{
  "totalDuration": 512,
  "totalSpecs": 42,
  "totalPass": 40,
  "totalFail": 1,
  "totalError": 1,
  "totalSkipped": 0,
  "labels": [],
  "bundleStats": [...]
}

---

agent — Agent (token-efficient JSON)

Compact, single-line JSON built for AI agents: totals plus only the failed/errored specs. A passing run is a few dozen tokens. Prefer it over json when the consumer is an LLM.

./testbox/run --reporter=agent

var report = new testbox.system.TestBox(
    bundles  : "tests.specs.MyTest",
    reporter : { type: "testbox.system.reports.AgentReporter", options: { maxFailures: 10, includeStack: true } }
).run()

Output shape:

{"ok":false,"totals":{"pass":120,"fail":2,"error":1,"skipped":3,"specs":126,"ms":4210},
 "failures":[{"bundle":"tests.specs.FooTest","spec":"Foo > can add","status":"failed","message":"Expected [4] but received [3]","at":"tests/specs/FooTest.cfc:42"}],
 "truncated":0}

Option	Default	Purpose
detail	failures	summary (totals only), failures, or all (adds a compact specs list)
maxFailures	20	Cap on listed failures, 0 = unlimited. Overflow is counted in truncated
maxMessageLength	300	Truncates each failure message, 0 = unlimited
includeStack	false	Adds a stack array of file:line frames to each failure
stackDepth	3	Frames kept when includeStack is true
includeSkipped	false	Adds a skipped array of spec paths
includeDebug	false	Adds a debug array of debug buffer output

Notes: at paths are relative to the web/working root. Bundle-level exceptions (for example a failing beforeAll()) appear in failures with an empty spec. The reporter does not change the process exit code, read ok.

---

junit — JUnit XML

Produces JUnit-compatible XML. Use this for GitHub Actions, Jenkins, CircleCI, or any CI that parses JUnit reports:

./testbox/run --reporter=junit > results/junit.xml

# GitHub Actions
- name: Run Tests
  run: ./testbox/run --reporter=junit > test-results/junit.xml

- name: Publish Test Results
  uses: EnricoMi/publish-unit-test-result-action@v2
  with:
    files: test-results/junit.xml

---

antjunit — ANTJunit XML

Legacy Ant-compatible JUnit XML format. Use only when your build tool requires Ant-style XML.

---

doc — Documentation

Produces a structured HTML report formatted as living documentation — spec names read like sentences describing application behaviour.

---

text — Text

Verbose plain-text output: prints every spec name, full error messages and stack traces. Best for deep-dive debugging.

./testbox/run --reporter=text --stacktrace

---

xml — XML

Generic XML representation of the result tree. Different from JUnit — use when you need to feed custom XML consumers (XSLT transforms, legacy reporting tools).

---

streaming — StreamingReporter (TestBox 7+)

Emits Server-Sent Events (SSE) so results appear in real time in the terminal or browser as specs complete.

./testbox/run --stream
testbox run --streaming

When accessed via HTTP, the Content-Type is text/event-stream. Each event payload is a JSON object describing a spec result:

data: {"specName":"it can create a user","status":"passed","duration":12}

data: {"specName":"it can delete a user","status":"failed","message":"Expected true but got false","duration":8}

---

Spec Attachments and Attempts (TestBox 7.2+)

Files attached to a spec, with attach( path, type = "file", name = "" ) or automatically by browser specs (failure screenshot, trace and video files of BrowserSpec / BrowserTestCase), are stored in the attachments array of the spec stats ({ path, type, name }), for passed and failed specs. Each reporter surfaces them:

Reporter	Attachments
json	Included in the spec stats (attachments), along with the attempts count
simple	Linked under the spec
junit, antjunit	A <system-out> per spec with one [[ATTACHMENT|/abs/path]] line per file (understood by Jenkins and JUnit report actions)
text, console, streaming output	Listed under failed specs

Specs that passed after a retry show "(passed after N attempts)" in the text, console, Simple and stream outputs. See the testbox-browser-testing skill for attach() and retries.

---

Programmatic Reporter Configuration

By String Key

new testbox.system.TestBox(
    directory: { mapping: "tests.specs", recurse: true },
    reporter:  "min"
).run()

By Struct with Options

new testbox.system.TestBox(
    directory: { mapping: "tests.specs", recurse: true },
    reporter: {
        class:   "testbox.system.reports.ConsoleReporter",
        options: { hideSkipped: true }
    }
).run()

By Full Class Path

new testbox.system.TestBox(
    directory: { mapping: "tests.specs", recurse: true },
    reporter:  "testbox.system.reports.JUnitReporter"
).run()

---

Custom Reporter

Implement the IReporter interface to create a fully custom reporter.

Interface Contract

// testbox/system/reports/IReporter.cfc (interface)
interface {
    // Called once before any tests run — return initial output string
    string function init( required results, required testbox )

    // Called once after all tests run — return accumulated output string
    string function runReport( required results, required testbox, struct options={} )
}

Minimal Implementation

// tests/reporters/MyReporter.cfc
component implements="testbox.system.reports.IReporter" {

    function init( required results, required testbox ) {
        return ""
    }

    function runReport( required results, required testbox, struct options={} ) {
        var sb = []

        sb.append( "=== Test Results ===" )
        sb.append( "Total: #results.totalSpecs# | Pass: #results.totalPass# | Fail: #results.totalFail# | Error: #results.totalError# | Skipped: #results.totalSkipped#" )
        sb.append( "Duration: #results.totalDuration#ms" )

        if ( results.totalFail > 0 || results.totalError > 0 ) {
            sb.append( "" )
            sb.append( "FAILURES:" )
            for ( var bundle in results.bundleStats ) {
                for ( var suite in bundle.suiteStats ) {
                    for ( var spec in suite.specStats ) {
                        if ( spec.status == "failed" || spec.status == "error" ) {
                            sb.append( "  [#spec.status.uCase()#] #spec.name#" )
                            if ( spec.keyExists( "failMessage" ) ) {
                                sb.append( "    #spec.failMessage#" )
                            }
                        }
                    }
                }
            }
        }

        return sb.toList( chr(10) )
    }

}

Using Your Custom Reporter

new testbox.system.TestBox(
    directory: { mapping: "tests.specs", recurse: true },
    reporter:  "tests.reporters.MyReporter"
).run()

// or by instance
new testbox.system.TestBox(
    directory: { mapping: "tests.specs", recurse: true },
    reporter:  new tests.reporters.MyReporter()
).run()

---

Reporter Selection Guide

Scenario	Reporter
Fast dev feedback in terminal	min or mintext
Rich browser debugging	simple with editor: "vscode"
CI (GitHub Actions / Jenkins)	junit or antjunit
Log file output	text or mintext
Test dashboard / API	json
AI agents / minimal tokens	agent
Real-time streaming	streaming (or --stream flag)
Living documentation	doc
Custom pipeline	Custom class via IReporter