CI/CD for TwinCAT: Build, Staging, Deploy

TL;DR The earlier posts in this series get a TwinCAT agent honest: tools, branches, locks, restore. They do not decide when a green build becomes a package other people can take. Collapse that into one job and every successful compile becomes a soft release. Split it into three: build for fast feedback and optional snapshot publish, staging for a versioned release candidate you can still reject, deploy for the deliberate step that makes that candidate public. That is the topology we run for our own Framework, and the one we wire for customer farms when the goal is CI/CD that ships libraries safely, not only compiles them.

This is the sixth and final post in our CI/CD for TwinCAT series. The earlier ones cover build tooling, distributing CLIs with Scoop, branching for machine software, exclusive locking on a Windows farm, and dependency restoration with Twinpack. This one is how those pieces sit in an actual release pipeline.

The trap: one green build equals “published”

Once restore and compile work on a locked agent, the next temptation is obvious. Put zkmake build, archive the .library, and twinpack push in the same job. Push to main. Wait for green. Library is on the server. Done.

That works until the day a flaky test was skipped, a TwinCAT version was wrong, or someone meant “does this still compile?” and accidentally shipped 1.4.0.87 to every consumer tracking that branch. For machine software the cost of an accidental publish is not a rollback banner. It is a library that already landed in someone else’s restore, or worse, on a machine that is mid-commissioning.

The fix is not a longer build script. It is a different shape: separate jobs with different jobs to do.

Three jobs, three contracts

Nothing mystical about the names. What matters is that publish to consumers is not a side effect of compile.

That is the same pattern we use for the Zeugwerk Framework on Jenkins: a parameterized build job (also used for snapshots), a staging pipeline that versions, fans that build job out across the matrix, collects candidates (and can trigger deploy when asked), and a deploy job that only ever copies artifacts from a chosen staging build and pushes them.

How the three jobs call each other:

Build is one job definition. Staging calls it many times with different parameters, plus sibling jobs that are not compiles. Deploy never recompiles; it promotes what staging already held.

Build: a reusable job, not only the PR pipeline

Treat build as a parameterized job other pipelines can call, not only as “whatever runs on push.”

On every push and every PR you still want one fast instance of it: tools via Scoop, restore from .Zeugwerk/config.json, compile, inner-loop tests, archive .library / .compiled-library for that commit. Keep that instance short enough that developers do not learn to ignore it.

The same job is what staging will fire many times with different parameters: TwinCAT version, compiled-library patch, skip-test vs run-test, and so on. One implementation, many invocations. That is why build stays boring and why the heavy matrix does not live inside the PR path.

If other projects on the same branch need to consume your work before a release, a successful build can also push those artifacts as a Twinpack Snapshot configuration, not as the configuration your field machines restore from. Snapshot means “latest successful build on this branch, for people who asked for it.” It does not mean “we stood behind this version.”

That split matters for release lines. A machine project on release/1.x should restore released 1.x libraries by default. Engineers integrating a hot library change on the same line can opt into Snapshot. The pipeline does not force that choice on everyone who just wanted a green compile.

Staging: orchestrate a candidate you can still throw away

Staging is where a commit becomes a versioned candidate. It is mostly an orchestrator, not a second copy of the compile script.

In practice:

• Versioning. Derive or bump the four-part TwinCAT version for this candidate. Staging owns the number that deploy will later put on Twinpack and on git tags. A PR build can use a working version; staging is what you might actually ship.
• Fan-out. Call the build job many times in parallel: once per TwinCAT version, once per patch (source library vs compiled-library), often a separate test pass that reuses the same job with compile skipped. Collect every artifact back into the staging workspace. That is how one staging run covers a matrix without stuffing the matrix into every developer push.
• Sibling work. Kick the other release chores in the same window: changelog generation, documentation builds (zkdoc / parse jobs), installers, whatever else must exist before humans look at a candidate. Those are not TwinCAT compiles, but they belong to “is this shippable?” the same way.
• Identity. Write down what you built: commit SHAs of every repo that went into the candidate (a references.json or equivalent). Deploy must not rebuild from “whatever main is now.” It must re-publish this staging output.
• Hold. Archive the collected artifacts on the CI system. Do not push them to the public feed yet.

Staging can still fail. That is the point. A failed staging run is cheap. A published bad library is not.

On success, staging either stops (notify, wait for a human) or, if your policy allows it for that branch, triggers deploy with an explicit flag. On our Framework pipelines, deploy from staging is a parameter, defaulting on for main and off everywhere else. Other teams want a manual “Promote” click on every line. Both are fine. Automatic silent publish from staging is how you recreate the one-job trap with extra steps.

Deploy: make that candidate public

Deploy is deliberately thin and deliberately scary.

It does not compile PLC code again if it can help it. It:

1. Selects a successful staging build (usually the last successful one on that branch, or a pinned build number).
2. Copies its artifacts and the recorded revisions.
3. Pushes to Twinpack under the real configuration (Distribution, Debug, whatever you named the non-snapshot channel), for the branch consumers will restore from.
4. Tags the git repos, uploads zips, updates “latest” pointers, notifies the shop floor docs site: whatever “available to the public” means in your org.

If something goes wrong in deploy, you fix deploy or re-run against the same staging artifacts. You do not invent a new binary by compiling again and hoping it matches what QA tested.

That is also where access control belongs. Not everyone who can push to feature/foo should be able to run deploy on release/1.x. Jenkins credentials, protected branches, and a short list of people who know when a library is allowed out are part of the design, not bureaucracy bolted on later.

How the rest of the series plugs in

This topology only works if the earlier posts are not still open questions.

TcPkg still belongs on the agent as machine setup, not as the project publish path. Twinpack (or your private NuGet feed behind it) is what deploy updates when “public” means “other PLC projects can restore this.”

What you still have to decide

The shape is reusable. The policy is not.

• When does deploy fire? Only main? Every release/* after manual approval? Never automatically?
• What is a snapshot consumer? Internal sibling repos only, or also friendly customers on a pre-release feed?
• How much must staging prove? Compile-only is not a candidate. Unit tests on UMRT, static analysis, multi-TC: pick the bar that matches the damage a bad library does.
• Who may promote? Named people, a chat approval, a protected Jenkins job. Write it down once.

Teams that skip those decisions usually invent them under pressure, on a Friday, with a customer waiting.

Where this leaves the series

Across six posts the path is: pick how you will build TwinCAT at all, get tools and dependencies onto agents honestly, branch the way machines in the field actually live, lock the Windows boxes so parallel work does not corrupt them, then stop treating every green compile as a release.

You can assemble that yourself on Jenkins or GitLab. Plenty of machine builders do. It is also a large surface: farm, locks, Scoop tokens, Twinpack sources, staging identity, deploy credentials. The teams that stay sane either dedicate people to that surface, or stop pretending it is a side project.

Staying on-prem does not mean inventing the TwinCAT half from scratch. zkmake and the rest of DevTools are what the build and staging jobs actually run: restore through Twinpack, compile, test, snapshot updates, the same CLIs we ship to nodes with Scoop. You keep the Windows farm and the Jenkins topology; you buy the tooling that already knows TwinCAT instead of maintaining a private Automation Interface stack. That is the path for air-gapped shops and teams that already own infrastructure but do not want to own every binary under it.

Zeugwerk CI/CD, the managed option, is for the other end: TwinCAT build nodes as a service, without you operating the farm at all. And when the missing piece is the topology itself (which jobs, which gates, how Twinpack publish is wired), that is the consulting work we do with teams: design build / staging / deploy, leave you with a publish step that is intentional.

The contract is the same in every case. Build answers “does this commit work?” Staging answers “is this a candidate?” Deploy answers “may everyone else take it?”

If you want the CLIs on your agents, a managed node, or the jobs wired for your libraries and machine lines, get in touch.