Testing strategy

An Archestack app is tested in layers, from the editor that flags a mistake as you type to the pipeline that will not deploy while a test fails. The tests themselves are part of the app: they live in its packages, travel with it to every workspace, and run against a sandbox database of their own, so a test never reads or changes anybody's real data.

Five layers

The first two need no work: the editor checks against the same compiler and the same data model the app runs with, and the Test button of a script module and the simulation of a business event run as a dry run, where writes through Db and Files are checked and recorded but never made, and mail and notifications are recorded but not sent. Anything else a script does itself is real: HTTP calls, System.IO, and anything a NuGet package does, such as a connection to another database. The rest of this page is about the third layer, the tests you write, and how the other two run them.

Test cases: given, when, then

A test case is one claim about the app, in three parts: the rows to put in place (given), the one thing to run (when) and what must be true afterwards (then). Cases are grouped into suites. This one, from the Service Planner demo, proves that the planning board marks a day overbooked only when a technician's planned hours go over their capacity:

{
  "name": "Week load flags a day over capacity",
  "suite": "Service Planner",
  "isEnabled": true,
  "data": {
    "description": "A day is overbooked only when its planned hours go over the technician's capacity.",
    "given": [
      { "table": "service_technician", "rows": [
        { "name": "Ana Peeters", "daily_capacity_hours": 8 },
        { "name": "Bram Maes", "daily_capacity_hours": 8 }
      ] },
      { "table": "service_job", "rows": [
        { "title": "Heat pump install", "technician_id": 1, "planned_date": "2026-09-23", "estimated_hours": 5, "status": "Planned" },
        { "title": "Boiler service", "technician_id": 1, "planned_date": "2026-09-23", "estimated_hours": 4, "status": "Planned" },
        { "title": "Commissioning", "technician_id": 2, "planned_date": "2026-09-23", "estimated_hours": 8, "status": "Planned" }
      ] }
    ],
    "when": { "kind": "ScriptModule", "name": "GetPlannerWeek", "parameters": { "week_start": "2026-09-21" } },
    "then": [
      { "subject": "Result", "operator": "Succeeds" },
      { "subject": "Result", "operator": "Equals", "path": "technicians[0].days[2].overbooked", "expected": "true" },
      { "subject": "Result", "operator": "Equals", "path": "technicians[1].days[2].overbooked", "expected": "false" }
    ]
  }
}

Given

The rows the case needs, table by table, written top to bottom into an empty sandbox. The rows get ids 1, 2, 3 in the order they are written, which is what a later fixture's foreign key points at: above, technician_id 1 is Ana. So a table another fixture points at goes first, and a key the database generates is left out, as are the audit columns and any column whose default will do. In the builder, picking a table fills in sample rows from its columns' names and types, and a foreign key is a picker over the rows the fixtures above it write.

When

Exactly one target, named by its name, never by an id:

• Script module: runs a module with the parameters given.
• Business entity: reads an entity back, optionally filtered, to check what a page would show.
• Business event: fires a trigger on a row, at the moment named (BeforeCreate, AfterUpdate and so on), the way a save would.
• Script: free C# with everything a script module has, for a case that has to do several things before the claim can be checked.

Then

Each check is a subject, an operator and usually an expected value. The subjects are Result (what the target returned), Rows (rows read back from a table afterwards), Output (what the target logged), Error, Duration and Code. The operators are Equals, NotEquals, Contains, NotContains, GreaterThan, LessThan, IsNull, IsNotNull, Matches (a regular expression), RowCount, Succeeds and Fails. A path such as technicians[0].days[2].overbooked walks into a result, and a path that leads nowhere fails the check and says where it stopped.

Testing that something is refused

Many of an app's rules say no: a business event refuses a save that would break something. The case for such a rule tries the save and checks it was refused, for the right reason, and that nothing was written:

// A zero estimate is refused.csx: the target, a save the business event must stop
await Db.CreateAsync("service_job", new { title = "Check radiator valves", estimated_hours = 0 });
return "saved";

"then": [
  { "subject": "Result", "operator": "Fails" },
  { "subject": "Error", "operator": "Contains", "expected": "Estimated hours must be more than zero" },
  { "subject": "Rows", "operator": "RowCount", "table": "service_job", "expected": "0" }
]

The second check is the one that matters. Fails on its own passes on any error at all, a misspelt column included, which is the most common way a red test quietly stops testing anything. Pin the failure to its message.

Code checks

A check that one subject and one operator cannot express is a code check: C# that runs after the target in the same sandbox, with everything a script module has, including the typed sets on Db. Result, Output, Error, DurationMs and Failed hold the target's outcome, and Assert says what it expected and what it found when a check fails (Equal, NotEqual, True, False, Null, NotNull, Contains, Count and Fail). The check fails when it throws, fails an Assert or returns false.

// Planning a job marks it Planned.then.1.csx: a check one subject and one operator cannot say
var job = await Db.ServiceJob.SingleAsync(j => j.Title == "Boiler service");
Assert.Equal("Planned", job.Status);
Assert.NotNull(job.PlannedDate, "a planned job has a date");

var sameDay = await Db.ServiceJob
    .Where(j => j.TechnicianId == job.TechnicianId && j.PlannedDate == job.PlannedDate)
    .SumAsync(j => j.EstimatedHours);
Assert.True(sameDay <= 8, $"the day holds {sameDay} hours");

Reach for one when a check needs several rows at once, arithmetic over them, or a comparison between two tables. In a repository each code check is a file of its own, <Case>.then.<n>.csx, edited with the same C# features as any script.

The sandbox

A test never touches the workspace's data. A run redirects the data database to a sandbox, a separate database made on demand in the shape of the real one, and every table in it is emptied before each case, so the only rows a case sees are its own fixtures and what its target writes. The definitions still come from the workspace, so a test exercises the business entities, business events and scripts that are about to ship. Each workspace has four sandboxes, so four runs can go at once; a fifth waits for one to come free.

• Mail and notifications are recorded, not sent. Each case's result lists what its scripts would have mailed and whom they would have notified, so a case can check that too.
• Files stay as they are. Writes, moves and deletes through Files never reach the workspace's folder: the run keeps them to itself, so reading back a file it just wrote works.
• Outbound calls are real. Anything else a script does itself happens on every run: an HTTP request, an SMTP connection it opens, System.IO, and anything a NuGet package does, such as a connection to another database, writes included. Keep calls to other systems out of what the tests run, or point them at a test endpoint or a test database.
• The bookkeeping is real. A saved run records its result on the case and the suite and writes an Event Log entry. A run of an unsaved draft, from the case editor's Run button or from VS Code, records nothing.

Where tests live

A test belongs to the package it tests. It travels with that package through an export, an import or the Module Store, so a module arrives somewhere new with the proof that it still works, and everything it names (the script module, the business entity, the business event) it names by name, so it points at the right thing in any workspace. In a repository, a package's tests are files: tests/<suite>/suite.json, a JSON file per case under cases/, and a .csx beside a case for a Script target or a code check. They are reviewed and merged like the code they test.

Running tests

• In the builder, the Test Suite page lists the suites and their cases, and runs a case, a suite or everything, with every failed check's expected against actual. The case editor's Run tries the case before it is saved.
• In VS Code, Run Tests runs a case, a suite, a package or every package against the project, four cases at a time, and shows each result in the tree and in the Testing view. See Tests in the VS Code guide.
• On a schedule, a suite with a cron runs by itself, in whichever environment it is installed in: 0 0 2 * * ? is every night at two. Each run writes one Event Log entry with the detail of every check, and its last result is on the suite.
• In the pipeline, every package's tests run against a throwaway Archestack built from the repository before anything is deployed, and one failure stops the deployment. See Deployment strategy.
• From an AI assistant, the MCP server gives a connected model the same test guide and the tools to write a case, run it and fix it until it passes.

Reading a result

What to test first

1. The business events that guard data. Every rule that refuses a save or a delete gets a case that is refused and a case that is allowed, so the rule is neither missing nor too strict.
2. Script modules that calculate. Given a few rows, check the numbers the module returns, and the edges: an empty day, a day filled exactly, a day over.
3. The reads the app depends on. A business entity target with a filter, and a RowCount, proves a page shows the rows it should and none it should not.
4. Flows across several steps. A Script target that does what a user would, and code checks on everything that should have followed.

Tests that stay useful

• One case, one claim. A case that checks ten things says nothing about which of the ten broke.
• Name the case after the rule. "A zero estimate is refused" reads as a specification in the list of results.
• Check behaviour, not the fixture. Counting the rows you just inserted tests the fixture loader.
• Pin every failure to its message, as above.
• Keep fixtures small. Only the rows the claim needs, without generated keys.
• Keep tests with the code. A change and the test that proves it belong in the same package, and in the same pull request.

Where this goes next

Archestack in VS Code covers running tests from the editor, Deployment strategy the pipeline that runs them before every deployment, and the reference the business events and script modules most tests are about.