Many Twistlock users of Azure DevOps have employed the simple YAML example for twistcli scanning of container images in our sample-code repo, but we’ve had numerous requests for a native Azure DevOps Extension (plugin) so users could take advantage of features like graphical pipelines and secrets management.

Using twistcli with Azure DevOps

We’re very happy to announce that Mario Weigel—DevSecOps and cloud pro, and active user of both Azure DevOps and Twistlock—has shared his twistcli-based Azure DevOps extension with the Twistlock community. We’ve now made the source code available, added a PR or two to support serverless function scanning, and have published the extension to the Azure DevOps/VSTS marketplace where you can start using it today!

The Twistlock “Twistcli Build and Release Task” extension for Azure DevOps is a wrapper for twistcli, our command line tool, specifically calling twistcli images scan to scan Docker/OCI images or twistcli serverless scan to scan serverless function bundle zip files for vulnerabilities and compliance issues. These capabilities allow devops teams to shift security left to scan every container image and serverless function as part of build pipelines in Azure DevOps, providing early detection before artifacts are promoted to registries or deployed to production. Every container image or serverless function build pipeline in Azure DevOps should employ this kind of “shift left” early detection before artifacts are promoted to registries or deployed to production.

If the detected vulnerabilities or compliance issues exceed defined thresholds, builds can be failed, thus enforcing organizational security policy. Developers aren’t left in the lurch though. Twistlock will provide the information needed by developers to fix the problems, such as identifying offending package versions and providing their fixed version numbers, while devs are still working on the code.

In the above screenshot, you can see Twistlock scan results directly in Azure Devops, while the screenshots below also show these scan results being passed to Twistlock Console, our UI.

Console is the central place where twistcli gets its scan policies from and publishes scan results. To ensure complete compatibility, the twistcli binary version and Console version need to match. Instead of publishing a different extension for every Console/twistcli version out there, or vendoring multiple copies of the twistcli inside, we’ve taken the approach of making twistcli pluggable into the Twistcli Build and Release Task extension. This was Mario’s original design so he wouldn’t be shipping our binary without permission, but we think it’s pretty elegant so decided to keep it.

Installing and maintaining the extension

You’ll need a licensed Twistlock installation with your Twistlock Console network accessible to Azure DevOps. After installing the Twistlock Extension from the marketplace, you’ll create a “feed” where you’ll upload the Linux twistcli binary available in your Twistlock install tar.gz or downloadable from the Twistlock Console.

Azure DevOps’ “Universal Packages” makes it easy to maintain a “feed” of versioned content. In our example we create a “twistcli” feed and then upload a versioned copy of twistcli for any Console versions I have in my environment. Then it’s easy to drop in the correct twistcli version and upgrades become a snap. You do not need to uninstall/reinstall the Twistlock extension from your organizations/projects/pipelines for each new Twistlock version upgrade; simply upload the updated twistcli binary to your feed and select that version to be used in your pipeline.

Note: I used Azure DevOps online, where feeds are part of the “Package Management” aka “Artifacts” extension comes pre-installed with entitlement for 5 users. I simply had to assign Package Management to one of my users to get the “Artifacts” menu item to appear and allow me to create a new feed (Artifacts is the replacement or new name for Package Management, but at the time of writing, there were still places in the interface which only mentioned Package Management).

The screenshots below illustrate the process of giving users in your organization the entitlement for Package Management in particular projects.

Users can use the vsts CLI to upload and download to feeds, but I used the az artifacts universal publish command that has nearly the same syntax.

Note: You need to version your artifacts with semver so the Twistlock product name of “19.03” (year.month) needs to be “19.3” in the feed.

~/Downloads/twistlock_19_03_311 $ ls ./linux/
twistcli

~/Downloads/twistlock_19_03_311 $ cat upload.sh
# --version: note semver requires "19.3" not "19.03"
# --path: all files in ./linux dir uploaded, but only contains "twistcli"

az artifacts universal publish \
  --organization https://dev.azure.com/add-twistlock \
  --feed twistcli \
  --name twistcli \
  --version 19.3.311 # note semver requires "19.3" not "19.03" \
  --description "twistcli binary needed by extension" \
  --path ./linux

~/Downloads/twistlock_19_03_311 $ . ./upload.sh
Universal Packages is currently in preview.
{
  "Description": "twistcli binary needed by extension",
  "ManifestId": "591FA72BD7DDAE63EB41C94D7A73BEE13370A348044C21D012AA954FD6A93A3401",
  "SuperRootId": "CDECF3C52F1E2020971CE6A9A1586E69A232870A1A71B2AE0BADEB90FF375F5F02"
}

Now that I have twistcli uploaded, I can create a Service Connection with the Console address and credentials in an Azure DevOps project, and then build pipelines in that project that use the Twistlock extension.

Adding the Twistlock Service Connection to your project

Service Connections in Azure DevOps allow us to store credentials, URLs, and other important data for connecting to external sources such as git repos, container registries, and in our case, the Twistlock Console. At the Azure DevOps project level, click on the gear icon and click on Pipelines > Service Connections under the Project Settings. With the extension installed, you’ll be able to create a new service connection of type “Twistlock console” as shown in the following screenshots.

Using the extension in a visual pipeline

Now that all of the preparations are made including ensuring I have a version of twistcli in my feed that matches my Twistlock Console version, and creating a Service Connection to the Twistlock Console, I’m ready to create a visual pipeline. In my example, I’m going to bring in a git repo with a Dockerfile, build a container image, and then pull in twistcli from my feed, ensure twistcli is in PATH and executable, and then perform the scan. Let’s take a closer look at the Twistlock scan portion of the pipeline steps.

In the large screenshot below, you can see that we’ve added a task to the Ubuntu Agent Job for Universal Packages download. This is how we download the versioned twistcli binary from our feed. Remember this means that we don’t have to uninstall the extension when upgrading to a new Twistlock version. Instead you simply upload the new version of twistcli to the feed and can select it in your pipelines.

In the next screenshot, we’re moving the twistcli binary into the PATH and ensuring it is executable.

And finally, in the following screenshot we show the Twistlock twistcli scan task with a scan type of “images” selected and vulnerability and compliance thresholds set to “critical”. That means that if any security issues are discovered in the scan that are at the “critical” level of severity, the pipeline will fail, otherwise it will pass the Twistlock twistcli scan task.

After a run of this pipeline, you’ll receive output similar to what was shown at the beginning of this post with details on vulnerabilities and compliance issues right in the Azure DevOps interface and additional information in Twistlock Console.

Conclusion

We’re extremely grateful for the contribution from Mario Weigel of the Twistlock extension for Azure DevOps. The extension makes it easy to incorporate Twistlock scanning of container images and serverless functions into Azure DevOps pipelines to ensure that the cloud native artifacts you’re building meet your organization’s security standards. But shifting security “left” is only the first step. Once they pass CI/CD, the images and functions you build will be protected by Twistlock’s comprehensive set of capabilities—like continuous registry and serverless repo scanning, layer 3 and layer 7 cloud native firewalling, and runtime defense including automatic learning of normal behavior with alerts or active prevention of anomalies.

← Back to All Posts Next Post →