Plan and Run Tests: Scenarios, Suites and Campaigns

This chapter explains why the ChargeLink test objects exist, what each object controls, and how to use them without guessing. Start here when you need repeatable regression tests rather than a one-off simulator session.

The four objects in plain language

Object	Why it exists	What it contains	When you use it
Scenario	Describes one behavior to prove	Initial state, stimuli, expected state transitions, timeouts and verdict rules	One focused test, such as a normal DC charge or an isolation fault
Suite	Groups related scenarios	Ordered scenario IDs, shared defaults and stop/continue policy	Regression testing a feature area or release
Campaign	Freezes an executable test run	Protocol, role, profile, PICS/PIXIT, evidence mode, repetitions, output path and environment	Producing a reproducible customer or release test result
Package	Delivers a licensed test catalogue and runner	Case catalogue, applicability model, profiles, campaign generation, execution and report schema	Structured protocol conformance-readiness testing

A useful rule is:

A scenario says what behavior to test. A suite says which tests belong together. A campaign says exactly how this run will be executed. A package says which licensed test catalogue and workflow are available.

Before running anything

Open PowerShell as the same Windows user that will run ChargeLink and create a clean working area:


$CL = 'C:\Program Files\ChargeLink Native\bin\chargelink.exe'
$Work = 'C:\ChargeLinkData\tests'
$Run = Join-Path $Work (Get-Date -Format 'yyyyMMdd-HHmmss')
New-Item -ItemType Directory -Force -Path $Run | Out-Null

& $CL --json doctor
& $CL --json profile list
& $CL --json scenario list
& $CL --json suite list

Why this matters:

doctor catches missing runtime folders, license or network prerequisites before a long run.
profile list tells you which charger/vehicle configurations are actually available.
scenario list and suite list prevent copying an obsolete identifier from an old report.
A timestamped output folder prevents evidence from different runs being mixed.

Run one scenario first

Use one scenario to validate the environment before launching a suite.


& $CL --json `
  --artifact-root "$Run\artifacts" `
  --runtime-root "$Run\runtime" `
  scenario run `
  --id <SCENARIO_ID>

Replace <SCENARIO_ID> with an identifier returned by scenario list.

After execution, inspect the result before proceeding:


Get-ChildItem "$Run\artifacts" -Recurse
Get-ChildItem "$Run\artifacts" -Recurse -Filter *.json |
  Select-Object FullName,Length,LastWriteTime

A valid run should make it possible to answer:

Which ChargeLink version executed?
Which protocol, role and profile were selected?
Which scenario and inputs were used?
What was the final verdict?
Which trace and evidence files support that verdict?

Do not continue to a full suite if those answers are not visible in the artifacts.

Run a suite

A suite is appropriate when the individual scenario already works and you need broader coverage.


& $CL --json `
  --artifact-root "$Run\artifacts" `
  --runtime-root "$Run\runtime" `
  suite run `
  --id <SUITE_ID>

Use suite verify before customer or release execution:


& $CL --json suite verify --id <SUITE_ID>

Use suite export when the selected suite definition must be archived with the result:


& $CL --json suite export `
  --id <SUITE_ID> `
  --out "$Run\suite-definition"

Dry-run, replay and live evidence modes

The package and ATS workflows use three evidence modes.

Mode	Why to use it	What it proves	What it does not prove
`dry-run`	Validate selection and applicability quickly	The package, profile, PICS/PIXIT and campaign can be resolved	No live protocol communication
`replay`	Re-evaluate known traces deterministically	Verdict logic against controlled evidence	Current DUT interoperability
`live`	Exercise a real simulator/DUT connection	Current runtime behavior and captured evidence	Accredited certification by itself

Recommended progression:


dry-run → replay → live

This sequence isolates configuration errors before bench time is consumed.

Create a package-scoped PICS/PIXIT selection

PICS states what the system under test supports. PIXIT supplies implementation-specific values needed to execute tests. Generate these per package and role.


$Package = 'iso15118-2-evse'
$Role = 'evse'
$Pics = "$Run\pics.json"
$Pixit = "$Run\pixit.json"

& $CL --json conformance package pics-pixit `
  --package $Package `
  --role $Role `
  --pics $Pics `
  --pixit $Pixit `
  --executable-only

Review both JSON files. Do not accept defaults blindly. Confirm at least:

AC or DC energy transfer mode.
EIM or Plug & Charge authorization.
TLS enabled/disabled as required by the selected profile.
Supported services and optional features.
Network interface and addressing assumptions.
Voltage/current/power limits used by the test environment.
Certificate locations where PnC/TLS is selected.

Generate the campaign


& $CL --json conformance package campaign `
  --package $Package `
  --role $Role `
  --pics $Pics `
  --pixit $Pixit `
  --executable-only

The generated campaign is the frozen execution plan. Keep it with the final report. It should identify the selected package, role, profile, applicable cases and excluded/blocked cases.

Execute a complete package flow

For a first run, limit the number of cases:


& $CL --json `
  --license 'C:\ChargeLinkData\licenses\customer.cllic' `
  conformance package execute `
  --package $Package `
  --role $Role `
  --pics $Pics `
  --pixit $Pixit `
  --out "$Run\package-run" `
  --evidence-mode dry-run `
  --executable-only `
  --max-cases 10

When the dry-run is clean, repeat with replay or live:


& $CL --json `
  --license 'C:\ChargeLinkData\licenses\customer.cllic' `
  conformance package execute `
  --package $Package `
  --role $Role `
  --pics $Pics `
  --pixit $Pixit `
  --out "$Run\package-live" `
  --evidence-mode live `
  --executable-only

Interpreting results

Treat every case as one of these operational outcomes:

Outcome	Meaning	Operator action
Passed	Expected behavior and required evidence were observed	Retain evidence and continue
Failed	Executed behavior violated a verdict rule	Inspect trace, DUT logs and first failing step
Blocked	Required environment, hardware or prerequisite was unavailable	Correct prerequisite; do not call it a DUT failure
Not applicable	PICS/PIXIT excluded the case	Confirm the exclusion accurately represents the DUT
Error	Runner or infrastructure failed	Repair environment and rerun; do not interpret as protocol verdict

A practical regression workflow

Run doctor and save the JSON output.
Record the ChargeLink version and license entitlements.
Run one known-good scenario.
Run the selected suite in dry-run or replay mode.
Fix all infrastructure errors and blocked prerequisites.
Execute the live campaign.
Export the verdict report.
Archive PICS, PIXIT, campaign, report, trace, logs and environment inventory together.
Compare the new result with the last accepted baseline.

Common mistakes

Running a large package immediately: begin with --max-cases 10 so configuration errors are found quickly.

Changing PICS/PIXIT after the run: any change creates a different campaign. Regenerate and rerun.

Calling blocked tests failed: blocked means the test did not reach a valid verdict because a prerequisite was absent.

Mixing outputs: always use a new output directory for each run.

Treating an engineering package as accredited certification: ChargeLink produces conformance-readiness and engineering evidence. Certification claims require the applicable accredited process and laboratory boundary.