NIST SP 800-204C 日本語訳

NIST Special Publication 800-204C

Ramaswamy Chandramouli

This publication is available free of charge from: https://doi.org/10.6028/NIST.SP.800-204C

NIST Special Publication 800-204C

Computer Security Division

Information Technology Laboratory

This publication is available free of charge from: https://doi.org/10.6028/NIST.SP.800-204C

March 2022

U.S. Department of Commerce Gina M. Raimondo, Secretary

National Institute of Standards and Technology

James K. Olthoff, Performing the Non-Exclusive Functions and Duties of the Under Secretary of Commerce for Standards and Technology & Director, National Institute of Standards and Technology

This publication has been developed by NIST in accordance with its statutory responsibilities under the Federal Information Security Modernization Act (FISMA) of 2014, 44 U.S.C. § 3551 et seq., Public Law (P.L.) 113-283. NIST is responsible for developing information security standards and guidelines, including minimum requirements for federal information systems, but such standards and guidelines shall not apply to national security systems without the express approval of appropriate federal officials exercising policy authority over such systems. This guideline is consistent with the requirements of the Office of Management and Budget (OMB) Circular A-130.

Nothing in this publication should be taken to contradict the standards and guidelines made mandatory and binding on federal agencies by the Secretary of Commerce under statutory authority. Nor should these guidelines be interpreted as altering or superseding the existing authorities of the Secretary of Commerce, Director of the OMB, or any other federal official. This publication may be used by nongovernmental organizations on a voluntary basis and is not subject to copyright in the United States. Attribution would, however, be appreciated by NIST.

National Institute of Standards and Technology Special Publication 800-204C Natl. Inst. Stand. Technol. Spec. Publ. 800-204C, 45 pages (March 2022) CODEN: NSPUE2

This publication is available free of charge from: https://doi.org/10.6028/NIST.SP.800-204C

Certain commercial entities, equipment, or materials may be identified in this document in order to describe an experimental procedure or concept adequately. Such identification is not intended to imply recommendation or endorsement by NIST, nor is it intended to imply that the entities, materials, or equipment are necessarily the best available for the purpose.

原本PDFの抜粋画像

There may be references in this publication to other publications currently under development by NIST in accordance with its assigned statutory responsibilities. The information in this publication, including concepts and methodologies, may be used by federal agencies even before the completion of such companion publications. Thus, until each publication is completed, current requirements, guidelines, and procedures, where they exist, remain operative. For planning and transition purposes, federal agencies may wish to closely follow the development of these new publications by NIST.

Organizations are encouraged to review all draft publications during public comment periods and provide feedback to NIST. All NIST Computer Security Division publications, other than the ones noted above, are available at http://csrc.nist.gov/publications.

Submit comments on this publication to: sp800-204c-comments@nist.gov

National Institute of Standards and Technology Attn: Computer Security Division, Information Technology Laboratory 100 Bureau Drive (Mail Stop 8930) Gaithersburg, MD 20899-8930

All comments are subject to release under the Freedom of Information Act (FOIA).

The Information Technology Laboratory (ITL) at the National Institute of Standards and Technology (NIST) promotes the U.S. economy and public welfare by providing technical leadership for the Nation’s measurement and standards infrastructure. ITL develops tests, test methods, reference data, proof of concept implementations, and technical analyses to advance the development and productive use of information technology. ITL’s responsibilities include the development of management, administrative, technical, and physical standards and guidelines for the cost-effective security and privacy of other than national security-related information in federal information systems.

Cloud-native applications have evolved into a standardized architecture consisting of multiple loosely coupled components called microservices (often typically implemented as containers) that are supported by an infrastructure for providing application services, such as service mesh. Both of these components are usually hosted on a container orchestration and resource management platform. In this architecture, the entire set of source code involved in the application environment can be divided into five code types: 1) application code (which embodies the application logic), 2) application services code (for services such as session establishment, network connection, etc.), 3) infrastructure as code (for provisioning and configuring computing, networking, and storage resources), 4) policy as code (for defining runtime policies such as zero trust expressed as a declarative code), 5) and observability as code (for the continuous monitoring of an application runtime state). Due to security, business competitiveness, and the inherent structure of loosely coupled application components, this class of applications needs a different development, deployment, and runtime paradigm. DevSecOps (consisting of acronyms for Development, Security, and Operations, respectively) has been found to be a facilitating paradigm for these applications with primitives such as continuous integration, continuous delivery, and continuous deployment (CI/CD) pipelines. These pipelines are workflows for taking the developer’s source code through various stages, such as building, testing, packaging, deployment, and operations supported by automated tools with feedback mechanisms. The objective of this document is to provide guidance for the implementation of DevSecOps primitives for cloud-native applications with the architecture and code types described above. The benefits of this approach for high security assurance and for enabling continuous authority to operate (C-ATO) are also discussed.

原本PDFの抜粋画像

container orchestration and resource management platform; DevSecOps; CI/CD pipelines; infrastructure as code; policy as code; observability as code; GitOps; workflow models; static AST; dynamic AST; interactive AST; SCA.

The author would like to express his first thanks to David Ferraiolo of NIST for initiating this effort to provide targeted guidance for the implementation of DevSecOps primitives for the development, deployment, and monitoring of services in microservices-based applications with service mesh. Sincere thanks to Mr. Nicolas Chaillan, CSO US Air Force, for his detailed and insightful review and feedback. Thanks are also due to Zack Butcher of Tetrate, Inc. for suggesting the title for this document. The author also expresses thanks to Isabel Van Wyk of NIST for her detailed editorial review.

NOTICE: The Information Technology Laboratory (ITL) has requested that holders of patent claims whose use may be required for compliance with the guidance or requirements of this publication disclose such patent claims to ITL. However, holders of patents are not obligated to respond to ITL calls for patents and ITL has not undertaken a patent search in order to identify which, if any, patents may apply to this publication.

As of the date of publication and following call(s) for the identification of patent claims whose use may be required for compliance with the guidance or requirements of this publication, no such patent claims have been identified to ITL.

No representation is made or implied by ITL that licenses are not required to avoid patent infringement in the use of this publication.

Cloud-native applications have evolved into a standardized architecture consisting of the following components:

Due to security, business competitiveness, and its inherent structure (loosely coupled application components), this class of applications needs a different application, deployment, and runtime monitoring paradigm – collectively called the software life cycle paradigm. DevSecOps (consisting of acronyms for Development, Security, and Operations, respectively) is one of the facilitating paradigms for the development, deployment, and operation of these applications with primitives such as continuous integration, continuous delivery, and continuous deployment (CI/CD) pipelines.

原本PDFの抜粋画像

CI/CD pipelines are workflows for taking the developer’s source code through various stages, such as building, functional testing, security scanning for vulnerabilities, packaging, and deployment supported by automated tools with feedback mechanisms. For the purpose of this document, the entire set of source code involved in the application environment is classified into five code types:

Thus, separate CI/CD pipelines can be created for all five code types. The functions performed by each of these code types is also described to highlight the roles that they play in the overall execution of the application.

Though cloud-native applications have a common architectural stack, the platform on which the components of the stack run may vary. The platform is an abstraction layer over a physical (bare metal) or virtualized (e.g., virtual machines, containers) infrastructure. In this document, the chosen platform is a container orchestration and resource management platform (e.g., Kubernetes). To unambiguously refer to this platform or application environment throughout this document, it is called the Reference Platform for DevSecOps Primitives, or simply the reference platform.

The objective of this document is to provide guidance for the implementation of DevSecOps primitives for the reference platform. The benefits of this implementation for high security assurance and the use of the artifacts within the pipelines for providing continuous authority to operate (C-ATO) using risk management tools and dashboard metrics are also described.

原本PDFの抜粋画像

Cloud-native applications are made up of multiple loosely coupled components (called microservices typically implemented as containers), operate in perimeter-less network environments requiring zero trust concepts (on-premises or cloud), and are accessed by users from a diverse set of locations (e.g., campus, home office, etc.). Cloud-native applications do not just refer to applications that run in the cloud. They also refer to the class of applications with design and runtime architectures, such as microservices, and a dedicated infrastructure for providing all application services, including security. The incorporation of zero trust principles [1] into this class of application provides techniques wherein access to all protected resources is enforced through identity-based protection and network-based protections (e.g., micro-segmentation), where applicable.

Cloud-native applications require agile and secure updates and deployment techniques for business reasons as well as the necessary resilience to respond to cybersecurity events. Hence, they call for a different application development, deployment, and runtime monitoring paradigm (collectively called the software life cycle paradigm) than the ones used for traditional monolithic or multi-tier applications. DevSecOps (Development, Security, and Operations) is a facilitating paradigm for this class of applications since it facilitates agile and secure development, delivery, deployment, and operations through (a) primitives, such as continuous integration, continuous delivery/continuous deployment (CI/CD) pipelines (explained in section 3); (b) security testing throughout the life cycle; and (c) continuous monitoring during runtime, all of which are supported by automation tools. In fact, the paradigm that meets the above objectives was originally given the term DevOps to indicate the fact that it seeks to remove the silos between development and operations groups and promotes (or drives) increased collaboration. The term DevSecOps was later coined by a portion of the community to emphasize the role of the security team in the whole process. Thus, DevSecOps is the term that denotes a culture and set of practices with automation tools to drive increased collaboration, trust, shared responsibility, transparency, autonomy, agility, and automation across the key stakeholders responsible for delivering software, including development, operations, and security organizations. DevSecOps has the necessary primitives and other building blocks to meet the design goals of cloud-native applications.

It should be noted that there is no community-wide consensus on the term “DevSecOps.” As already stated, the term was primarily coined to emphasize the fact that security must be tested and incorporated in all stages of the software development life cycle (i.e., build, test, package, deploy, and operate). A portion of the community continues to use the term “DevOps” based on the argument that there is no need to define a new term since security must be an integral part of any software life cycle process.

In theory, DevSecOps primitives can be applied to many application architectures but are best suited to microservices-based ones, which permit agile development paradigms due to the fact that the application is made up of relatively small, loosely coupled modules called microservices. Even within microservices-based architectures, the implementation of DevSecOps primitives can take on different forms, depending on the platform. In this document, the chosen platform is a container orchestration and resource management platform (e.g., Kubernetes). The platform is an abstraction layer over a physical (bare metal) or virtualized (e.g., virtual machines, containers) infrastructure. To unambiguously refer to this platform or application environment throughout this document, it is called the Reference Platform for DevSecOps Primitives, or simply the reference platform.

Before describing the implementation of DevSecOps primitives for the reference platform, it is assumed that the following due diligence is applied with respect to deployment of the service mesh component [2]:

The guidance provided for implementation of DevSecOps primitives for the reference platform is agnostic to (a) the tools used in DevSecOps pipelines and (b) the service mesh software, which provides application services, though examples from service mesh offerings, such as Istio, are used to link them to real-world application artifacts (e.g., containers, policy enforcement modules, etc.).

原本PDFの抜粋画像

A slightly more detailed description of the code types (referred to in the executive summary) in the entire application environment presented by the reference platform is given below. Please note that these code types include those that support implementation of DevSecOps primitives.

Code types 3, 4, and 5 may have overlap with code type 2.

This document covers the implementation of pipelines or workflows associated with all five code types listed above. Thus, the entire application environment (not just the application code) benefits from all of the best practices that exist for application code (e.g., agile iterative development, version control, governance, etc.). Infrastructure as code, policy as code, and observability as code belong to a special class called declarative code. When using ‘as code’ techniques, the code that is written (e.g., for provisioning a resource) is managed similar to application source code. This implies that it is versioned, documented, and has access controls defined similar to what is done for application source code repository. Often, domain-specific declarative languages are used: the requirements are declared, and an associated tool converts them into artifacts that make up a runtime instance. For example, in the case of infrastructure as code (IaC), the declarative language models the infrastructure as a series of resources. The associated configuration management tool pulls together these resources and generates what are known as manifests that define the final shape and state of the platform (runtime instance) associated with the defined resources. These manifests are stored in servers associated with a configuration management tool and are used by the tool to create compiled configuration instructions for the runtime instance on the designated platform. Manifests are generally encoded in platform-neutral representations (e.g., JSON) and fed to platform resource provisioning agents through REST APIs.

There are several DevSecOps initiatives in various agencies of the Federal Government with varying emphasis and focus, depending on the processes enabled by software and mission needs. Though not exhaustive, here is a brief overview of those initiatives [3].

原本PDFの抜粋画像

Since DevSecOps primitives span development (build and test for security, package), delivery/ deployment and continuous monitoring (to ensure secure states during runtime), the target audience for the recommendations in this document includes software development, operations, and security teams.

Since the reference platform is made up of container orchestration and resource management platform and service mesh software, the following publications offer guidance for securing this platform as well as provide background information for the contents of this document:

This document is organized as follows:

As stated in Section 1.1, the reference platform is a container orchestration and management platform. In modern application environments, the platform is an abstraction layer over a physical (bare metal) or virtualized (e.g., virtual machines, containers) infrastructure. Before the implementation of DevSecOps primitives, the platform simply contains the application code, which contains the application logic and the service mesh code, which in turn provides application services. This section will consider the following:

Since microservices are typically implemented as containers, a container orchestration and resource management platform are used for deployment, operations, and maintenance of services.

原本PDFの抜粋画像

A typical orchestration and resource management platform consists of various logical (forming the abstraction layer) and physical artifacts for the deployment of containers. For example, in Kubernetes, containers run inside the smallest unit of deployment called a pod. A pod can theoretically host a group of containers, though usually, only one container runs inside a pod. A group of pods are defined inside what is known as a node, where a node can be either a physical or virtual machine (VM). A group of nodes constitutes a cluster. Usually, multiple instances of a single microservice are needed to distribute the workload to achieve the desired performance level. A cluster is a pool of resources (nodes) that is used to distribute the workload of microservices. One of the techniques used is horizontal scaling, where microservices that are accessed more frequently are allocated more instances or allocated to nodes with more resources (e.g., CPUs and/or memory).

Microservices-based applications require several application services, including security services such as authentication and authorization, as well as the generation of metrics for individual pods (monitoring), consolidated logging (to ascertain causes of failures of certain requests), tracing (sequence of service calls for an application request), traffic control, caching, secure ingress, service-to-service (east/west traffic), and egress communication.

原本PDFの抜粋画像

Taking the example of secure communications between containers, specific code needs to be added in order to secure the communications between pods in a platform such as Kubernetes (e.g., with mTLS). Pods that communicate do not apply identity and access management between themselves. Though there are tools that can be implemented to act as a firewall between pods, such as the Kubernetes Network Policy [8], this is a layer 3 solution rather than a layer 7 solution, which is what most modern firewalls are. This means that while one can know the source of traffic, one cannot peek into the data packets to understand what they contain. Further, it does not allow for making vital metadata-driven decisions, such as routing on a new version of a pod

6 based on an HTTP header. There are Kubernetes ingress objects that provide a reverse proxy based on layer 7, but they do not offer anything more than simple traffic routing. Kubernetes does offer different ways of deploying pods that some form of A/B testing or canary deployments, but they are done at the connection level and provide no fine-grained control or fast failback. For example, if a developer wants to deploy a new version of a microservice and pass 10 % of traffic through it, they will have to scale the containers to at least 10 – nine for the old version and one for the new version. Further, Kubernetes cannot split the traffic intelligently and instead balances loads between pods in a round-robin fashion. Every Kubernetes container within a pod has a separate log, and a custom solution over Kubernetes must be implemented to capture and consolidate them.

Although the Kubernetes dashboard offers monitoring features on individual pods and their states, it does not expose metrics that describe how application components interact with each other or how much traffic flows through each of the pods. Consolidated logging is required to determine error conditions that cause an application request or transaction to fail. Tracing is required to trace the sequence of containers that are invoked as part of any application request based on the application logic that underlies a transaction. Since traffic flow cannot be traced through Kubernetes pods out of the box, it is unclear where on the chain the failure for the request occurred.

原本PDFの抜粋画像

This is where the service mesh software can provide the needed application services and much more.

Having looked at the various application services required by microservices-based applications, consider the architecture of service mesh software that provides those services. The service mesh software consists of two main components: the control plane and the data plane.

原本PDFの抜粋画像

The control plane has several components. While the data plane of the service mesh mainly consists of proxies running as containers within the same pod as application containers, the control plane components run in their own pods, nodes, and associated clusters. The following are the various functions of the control plane [9]:

原本PDFの抜粋画像

Overall, the control plane helps the administrator populate the data plane component with the configuration data that is generated from the policies resident in the control plane. The policies for function 3 above may include network routing policies, load balancing policies, policies for blue-green deployments, canary rollouts, timeout, retry, and circuit-breaking capabilities. These last three are collectively called by the special name of resiliency capabilities of the networking infrastructure services. Last but not the least are security-related policies (e.g., authentication and authorization policies, TLS establishment policies, etc.). These policy rules are parsed by a module that converts them into configuration parameters for use by executables in data plane proxies that enforce those policies.

原本PDFの抜粋画像

The data plane component performs three different functions:

The primary component of the data plane that performs all three functions listed above is called the sidecar proxy. This L7 proxy runs in the same network namespace (which, in this platform, is the same pod) as the microservice for which it performs proxy functions. There is a proxy for every microservice to ensure that a request from a microservice does not bypass its associated proxy and that each proxy is run as a container in the same pod as the application microservice. Both containers have the same IP address and share the same IP Table rules. That makes the proxy take complete control over the pod and handle all traffic that passes through it [9,10].

原本PDFの抜粋画像

The first category of functions (secure networking) includes all functions related to the actual routing or communication of messages between microservices. The functions that come under this category are service discovery, establishing a secure (TLS) session, establishing network paths and routing rules for each microservice and its associated requests, authenticating each request (from a service or user), and authorizing the request.

With the example of establishing a mutual TLS session, the proxy that initiates the communication session will interact with the module in the control plane of the service mesh to check whether it needs to encrypt traffic through the chain and establish mutual TLS with the backend or target pod. Enabling this functionality using mutual TLS requires every pod to have a certificate (i.e., a valid credential). Since a good-sized microservice application (consisting of many microservices) may require hundreds of pods (even without horizontal scaling of individual microservices through multiple instances), this may involve managing hundreds of short-lived certificates. This, in turn, requires each microservice to have a robust identity and the service mesh to have an access manager, a certificate store, and a certificate validation capability. In addition, mechanisms for identifying and authenticating the two communicating pods are required for supporting authentication policies.

Other kinds of proxies include ingress proxies [11] that intercept the client calls into the first entry point of application (first microservice that is invoked) and egress proxies that handle a microservice’s request to application modules residing outside of the platform cluster.

The second category of functions that the data plane performs is enforcement of the policies defined in the control plane through configuration parameters in the proxies (policy enforcement service). An example is the use of the information in the JWT token that is part of a microservice request to authenticate the calling service. Another example is the enforcement of access control policies for each request using either the code residing in the proxy itself or by connecting to an external authorization service.

The third category of functions that service proxies perform almost always in collaboration with application service containers are to gather telemetry data, which helps to monitor the health and state of the services, transfer logs associated with a service to the log aggregation module in the control plane, and append necessary data to application request headers to facilitate the tracing of all requests associated with a given application transaction. The application response is conveyed by proxies back to its associated calling service in the form of return codes, a description of response, or the retrieved data.

The service mesh is container orchestration platform-aware, interacts with the API server that provides a window into application services installed in various platform artifacts (e.g., pods, nodes, namespaces), monitors it for new microservices, and automatically injects sidecar containers into the pods containing these new microservices. Once the service mesh inserts the sidecar proxy containers, operations and security teams can enforce policies on the traffic and help secure and operate the application. These teams can also configure the monitoring of microservices applications without interfering with the functioning of the applications.

The provisioning of infrastructure, policy enforcement, and observability services can be automated using declarative code that is part of DevSecOps pipelines. While the development team should be overall aware of the security and management details of deployment of their code, the automation of the above mentioned services provides more time to them to concentrate their efforts on efficient development paradigms, such as code modularity and structuring.

DevSecOps incorporates security into the software engineering process early on. It integrates and automates security processes and tooling into all of the development workflow (or pipeline as later explained) in DevOps so that it is seamless and continuous. In other words, it can be looked upon as a combination of the three processes: Development + Security + Operations [12].

原本PDFの抜粋画像

This section discusses the following aspects of DevSecOps:

DevSecOps is a software development, deployment, and life cycle management methodology that involves a shift from one large release for an entire application or platform to the continuous integration, continuous delivery, and continuous deployment (CI/CD) approach. This shift, in turn, requires changes in the structure of a company’s IT department and its workflow. The most pronounced change involves organizing a DevSecOps group that consists of software developers, security specialists, and IT operations experts for each portion of the application (i.e., the microservice). This smaller team not only promotes efficiency and effectiveness in initial agile development and deployment but also in subsequent life cycle management activities, such as monitoring application behavior, developing patches, fixing bugs, or scaling the application. The composition of this cross-functional team with expertise in three areas forms a critical success factor for introducing DevSecOps in an organization.

DevSecOps is an agile, automated development and deployment process that uses primitives called CI/CD pipelines aided by automated tools to take the software from the build phase to deployment phase and finally to the runtime/operations phase. These pipelines are workflows that take the developer’s source code through various stages, such as building, testing, packaging, delivery, and deployment supported by testing tools in various phases.

A DevSecOps platform denotes the set of resources on which various CI/CD pipelines (for each code type) run. At the minimum, this platform consists of the following components:

(a) Pipeline software - CI software - pulls code from a code repository, invokes the build software, invokes test tools and stores back tested artifacts to Image registry - CD software – pulls out artifacts, packages and based on computing, network and storage resource descriptions in IaC deploys the package (b) SDLC software

- Build Tools (e.g., IDEs) - Testing Tools (SAST, DAST, SCA) (c) Repositories - Source Code Repositories (e.g., GitHub) - Container Image Repositories or Registries (d) Observability or Monitoring Tools - Logging and Log Aggregation Tools - Tools that generate metrics - Tracing Tools (sequence of application calls) - Visualization Tools (combine data from above to generate dashboard/alerts)

In a DevSecOps platform, security assurance is provided during the build and deployment phases through built-in design features, such as zero trust, and testing using a comprehensive set of security testing tools, such as static application security tools (SAST), dynamic security testing tools (DAST), and software composition analysis (SCA) tools. In addition, security assurance is also provided during the runtime/operations phase by continuous behavior detection/prevention tools, some of which may even use sophisticated techniques such as artificial intelligence (AI) and machine learning (ML). Therefore, a DevSecOps platform not only runs during build and deployment phases but also during the runtime/operations phase.

In some DevSecOps platforms, the security tools (e.g., SAST, DAST, and SCAs) that perform application security analysis, such as identifying vulnerabilities and bugs through efficient scanning in the background can be tightly integrated with integrated development environments (IDEs) and other DevOps tools. This feature when present, makes these tools transparent to developers and avoids the necessity for them to call separate APIs for running these tools [13]. Depending on the IDE, the task performed, or the resources consumed by the tool, the tool may alternatively execute separate from the IDE.

The use of SAST, DAST, and SCA tools may not be limited to testing just the application code. DevSecOps may include the use of these tools for other code types such as IaC, as IaC defines the deployment architecture of the application and thus the key avenue to automatically assess and remediate security design gaps.

In summary, a DevSecOps platform delivers the following:

The key primitives and implementation tasks involved are:

DevSecOps, being a methodology or framework for agile application development, deployment, and operations – is made up of stages just like any other methodology [14]. The sequence and flow of information through the stages is called workflow, where some stages can be executed in parallel while others have to follow a sequence. Each stage may require the invocation of a unique job to execute the activities in that stage.

A unique concept that DevSecOps introduces in the process workflow is the concept of “pipelines” [15]. With pipelines, there is no need to individually write jobs for initiating/executing each stage of the process. Instead, there is only one job that starts from the initial stage, automatically triggers the activities/tasks pertaining to other stages (both sequential and parallel), and creates an error-free smart workflow.

The pipeline in DevSecOps is called the CI/CD pipeline based on the overall tasks it accomplishes and the two individual stages it contains. CD can denote either the continuous delivery or continuous deployment stage. Depending on this latter stage, CI/CD can involve the following tasks:

In the former, automation ends at the delivery stage, and the next task of deployment of the modified application in the hosting platform infrastructure is performed manually. In the latter, the deployment is also automated. Automation of any stage in the pipeline is enabled by tools that express the pipeline stage as code.

The workflow process for a CI/CD pipeline is depicted in Figure 1 below:

Figure 1: CI/CD pipeline workflow [16]

The unit and integration tests shown in the diagram use the SAST, DAST, and SCA tools described in Section 3.2. It should be noted that an organization has the option to continue the build process when a test fails. Depending on how the organization balances risk tolerance versus business needs, it may choose to fail-open (log and continue) or fail-closed (stop/break) when a specific test fails. In the fail-closed event, the developer gets the test outcome report, must fix the issues, and restart the CI process.

Continuous integration involves developers frequently merging code changes into a central repository where automated builds and tests run. Build is the process of converting the source code to executable code for the platform on which it is intended to run. In the CI/CD pipeline software, the developer’s changes are validated by creating a build and running automated tests against the build. This process avoids the integration challenges that can happen when waiting for release day to merge changes into the release branch [17].

原本PDFの抜粋画像

Continuous delivery is the stage after continuous integration where code changes are deployed to a testing and/or staging environment after the build stage. Continuous delivery to a production environment involves the designation of a release frequency – daily, weekly, fortnightly, or some other period – based on the nature of the software or the market in which the organization operates. This means that on top of automated testing, there is a scheduled release process, though the application can be deployed at any time by clicking a button. The deployment process in continuous delivery is characterized as manual, but tasks such as the migration of code to a production server, the establishment of networking parameters, and the specification of runtime configuration data may be performed by automated scripts.

Continuous deployment is similar to continuous delivery except that the releases happen automatically [17], and changes to code are available to customers immediately after they are made. The automatic release process may in many instances include A/B testing to facilitate slow rollout of new features so as to mitigate the impact of failures if there is a bug/error.

The distinction between continuous delivery and continuous deployment is shown in Figure 2 below.

原本PDFの抜粋画像

Figure 2 - Distinction Between Continuous Delivery and Continuous Deployment [17]

The primary software for defining CI/CD pipeline resources, building the pipelines, and executing those pipelines is the CI/CD pipeline software. There may be slight variations in the architecture of this class of software depending on the particular offering. The following is an overview of the landscape in which CI/CD tools (pipeline software) operate:

The purpose of creating the CI/CD pipeline is to enable frequent updates to source code, rebuilds, and the automatic deployment of updated modules into the production environment. The key tasks involved are [19]:

The preparatory tasks required are the following: (1) Ensure that all individual components in the DevSecOps platform (Pipeline software, SDLC software, Code Repositories, Observability tools etc.) are available (2) Ensure that these components are secure either through certification, validation or customized testing (3) Integrate CI & CD tools with SDLC tools – Access tokens, Calling scripts, pipeline definitions (4) Set up configuration details in IaC tool (with GitOps) based on deployment environment (i.e., application hosting platform in-premise or in the cloud) (5) Integrate the runtime tools to the deployment environment (6) Design the dashboard and define the events to monitor, the alerts to be generated and the application state variables (e.g., Memory utilization etc.) to monitor through connection to Tools such as log aggregators, metric generators and trace generators.

The execution tasks include [19]:

To reiterate, the three primary stages of the CI/CD process are build/test, ship/package, and deploy. The following features transform this into a pipeline:

The workflow of the CI/CD pipeline should not create the impression that there is no human element involved. The following teams/role players contribute to CI/CD pipeline [21]:

Some of the many activities performed by these teams include the customization, update, and enhancement of the tools employed in the CI/CD pipeline (e.g., updating the static vulnerability analysis tool with the latest database of known vulnerabilities). Caution should be exercised during manual operations so that they do not block pipelines. The targets for mean time to production should be set up while also mitigating risks, such as insider threats, through the use of “merge (GitLab) or pull (GitHub) requests” and multiple approvers for those requests. This pipeline is designed, maintained, and executed by the post-release team who – in addition to monitoring functions – performs other processes, such as compliance management, backup processes, and asset tracking [22].

原本PDFの抜粋画像

Compared to other models of software development, which involve a linear progression from coding to release, DevOps uses a forward process with a delivery pipeline (i.e., build/secure, ship/package, and release) and a reverse process with a feedback loop (i.e., plan and monitor) that form a recursive workflow. The role of automation in these activities is to improve this workflow. Continuous integration emphasizes testing automation to ensure that the application is not broken whenever new commits are integrated into the main branch. Automation results in the following benefits:

原本PDFの抜粋画像

The following strategies are recommended [23] for automation so as to facilitate better utilization of organizational resources and derive the greatest benefit in terms of an efficient, secure application environment.

Choice of Activities to Automate: For example, the following are productive candidates for the automation of testing activities.

After choosing the candidates for automation based on the above criteria, the usual risk analysis must be applied to choose a subset that provides an optimum return on investment and maximizes desirable security metrics (e.g., defense in depth). Some recommended strategies include:

The security automation tools for various functions (e.g., static vulnerability analysis, dynamic vulnerability analysis, software composition analysis) used in CI/CD pipelines need to have different interface and alerting/reporting requirements since they have to operate seamlessly depending on the pipeline stage (e.g., build, package, release) during which they are used. These requirements are:

It must be mentioned that security automation tools come with costs, and hence, the extent of the usage of these tools is based on risk factor analysis.

Various CI/CD pipelines are involved in the reference platform (i.e., microservices-based application with service mesh that provides infrastructure services). Though the reference application is a microservices-based application, the DevSecOps primitives can be applied to monolithic applications as well as applications that are both on-premises and cloud-based (e.g., hybrid cloud, single public cloud, and multi-cloud).

In section 2.1, we referred to the five code types in our reference application environment. We also mentioned that separate CI/CD pipelines can be created for each of these five code types as well. The location of these five code types within the reference platform components will be discussed followed by separate sections that will describe the associated CI/CD pipelines.

Implementation issues for all CI/CD pipelines irrespective of code types will be addressed in the following sections:

This section will also consider the overall benefits of DevSecOps with a subsection on specific advantages in the context of the reference platform and the ability to leverage DevSecOps for continuous authorization to operate (C-ATO) in Sections 4.9 and 4.10, respectively.

A brief description of the five types of codes stated above (i.e., application, application services, infrastructure, policy, and monitoring) is as follows:

Each of the code types has an associated CI/CD pipeline and is described in Sections 4.2 through 4.5. There may be overlaps among application service code, infrastructure as code, policy as code, and observability as code types.

The constituent components of the reference platform hosting the five code types are:

The distribution of policy and observability code types within the components of the service mesh are as follows:

Application code and application services code reside in the container orchestration and resource management platform, and the CI/CD software that implements the workflows associated with it usually resides in the same platform. This pipeline should be protected using the steps outlined in Section 4.6, and the application code under the control of this pipeline should be subject to the security testing described in Section 4.8. Additionally, the orchestration platform on which the application resides should itself be protected using a runtime security tool (e.g., Falco) [27] that can read OS kernel logs, container logs, and platform logs in real-time and process them against a threat detection rules engine to alert users of malicious behavior (e.g., creation of a privileged container, reading of a sensitive file by an unauthorized user, etc.). They usually come with a set of default (predefined) rules over which custom rules can be added. Installing them on the platform spins up agents for each node in the cluster, which can monitor the containers running in the various pods of that node. The advantage of this type of tool is that it complements the existing platform’s native security measures, such as access control models and pod security policies, that prevent violations of security by actually detecting them when they occur [27].

The conventional approach to allocating infrastructure for applications consists of initially provisioning compute and networking resources with configuration parameters and ongoing tasks such as patch management (e.g., OS and libraries), establishing conformity to compliance regulations (e.g., data privacy), and making drift (where the current configuration no longer provides the intended operational state) correction.

Infrastructure as code (IaC) is a declarative style of code that encodes computer instructions that encapsulate the parameters necessary to deploy virtual infrastructure on a public cloud service or private data center via a service’s management APIs [28]. In other words, infrastructures are defined in a declarative way and versioned using the same source code control tools (e.g., GitOps) used for the application code. Depending on the particular IaC tool, this language can either be a scripting language (e.g., JavaScript, Python, TypeScript, etc.) or a proprietary configuration language (e.g., HCL) that may or may not be compatible with standardized languages (e.g., JSON). The basic instructions consist of telling the system how to provision and manage infrastructure (whether that is an individual compute instance or a complete server, such as physical servers or virtual machines), containers, storage, network connections, connection topology, and load balancers. [29]. In some cases, the infrastructure may be short-lived or ephemeral, and the lifespan of the infrastructure (whether immutable or mutable) does not warrant continued configuration management. Provisioning could be tied to individual commits of application code using tools that can connect application code and infrastructure code in a way that is logical, expressive, and familiar to development and operations teams, where application code increasingly defines the infrastructure resource requirements for a cloud application [30].

原本PDFの抜粋画像

IaC thus involves codifying all software deployment tasks (allocation of type of servers, such as bare metal, VMs or containers, resource content of servers) and the configuration of these servers and their networks. The software containing this code type is also called a resource manager or deployment manager. In other words, IaC software automates the management of the whole IT infrastructure life cycle (provisioning and de-provisioning of resources) and enables a programmable infrastructure. The integration of this software as part of the CI/CD pipeline not only results in agile deployment and maintenance but also in a robust application platform that is secure and meets performance needs.

When infrastructure is code as in IaC, it can include bugs and oversights that can potentially become vulnerabilities and, therefore, be exploited just as in application code. Thus, protecting the IaC is protecting the infrastructure definitions and eventually the deployment environment. Any piece of IaC has to be scanned for potential vulnerabilities before it enters the GitOps and is merged.

原本PDFの抜粋画像

In addition, the assurance of a secure application platform can be obtained only if there is a methodical drift management process in place. This assurance can be obtained only if the architecture defined in IaC is what actually exists in the deployed environment since this equivalence could be altered by an inadvertent or intentional change made through a console or CLI – thus bypassing the IaC. Ensuring this equivalence has to be done immediately after deployment and periodically during runtime as any change to the architecture could result in the introduction of security design flaws and may require making changes to IaC.

Infrastructure is often confused with configuration [30], which maintains computer systems, software, dependencies, and settings in a desired, consistent state. For example, putting a newly purchased server onto a rack and connecting it to the switches so that it is connected to the existing networks (or launching a new virtual machine and assigning network interfaces to it) belongs to the definition of “infrastructure.” In contrast, after the server is launched, installing an HTTPS server and configuring it belongs to configuration management.

原本PDFの抜粋画像

Policy as code involves codifying all policies and running them as part of the CI/CD pipeline so that they become an integral part of the application runtime. Examples of policy categories include authorization policies, networking policies, and implementation artifact policies (e.g., container policies). Policy management capabilities in a typical “policy as code software” may come with a set of predefined policy categories and policies and also support the definition of new policy categories and associated policies by providing policy templates [31]. The due diligence required for policy as code is that it should provide protection against all known threats that are relevant for the application environment including the infrastructure, and this can be ensured only if that code is periodically scanned and updated with appropriate changes to counter the threats relevant to the application class (e.g., web application) and the hosting infrastructure. Some examples of policy categories and associated policies are given in Table 1 below.

Table 1: Policy Categories and Example Policies

The policies defined in the “policy as code software” may translate into the following in the application infrastructure runtime configuration parameters:

Observability as code deploys a monitoring agent in each of the application’s service components to collect the three types of data (described in Section 4.1), send them to specialized tools that correlate them, perform analysis, and display the analyzed consolidated data on dashboards to present an overall application-level picture [32]. An example of such consolidated data are log patterns, which provide a view of log data that is presented after the log data are filtered using some criterion (e.g., a service or an event). The data are grouped into clusters based on common patterns (e.g., based on timestamp or range of IP addresses) for easy interpretation. Unusual occurrences are identified, and those findings can then be used to steer and accelerate further investigation [33].

There are some common implementation issues to be addressed for CI/CD pipelines irrespective of code type. Securing the processes involves the assignment of roles for operating the build tasks. Automation tools (e.g., Git Secrets) are available for this purpose. The following security tasks should be considered as a minimum for securing the CI/CD pipeline:

The next common issue involves workflow models. All CI/CD pipelines can have two types of workflow models, which depend on the automated tools that are deployed as part of the pipeline:

In the CI/CD tools that support the push-based model, changes made in one stage or phase of the pipeline trigger changes in the subsequent stages and phases. For example, through a series of encoded scripts, the new builds in the CI system trigger changes to the CD portion of the pipeline and thus change the deployment infrastructure (e.g., Kubernetes cluster). The security downside of using the CI system as the basis for change in deployments is the possibility of exposing credentials outside of the deployment environment in spite of best efforts to secure the CI scripts, which operate outside of the trusted domain of the deployment infrastructure. Since CD tools have the keys to production systems, push-based models are rendered insecure.

In a pull-based workflow model, an operator that pertains to the deployment environment (e.g., Kubernetes Operator, Flux, ArgoCD) pulls new images from inside of the environment as soon as the operator observes that a new image has been pushed to the registry. The new image is pulled from the registry, the deployment manifest is automatically updated, and the new image is deployed in the environment (e.g., cluster). Thus, the convergence of the actual deployment infrastructure state with the state declaratively described in the Git deployment repository is achieved. Additionally, the deployment environment credentials (e.g., cluster credentials) are not exposed outside of the production environment. Therefore, a pull-based model, which typically uses a GitOps repository for storing the source code and builds, is highly recommended.

The GitOps workflow model is an improvement on the CI/CD pipeline (for the delivery portion of the pipeline), which uses a pull-based workflow model instead of the push-based model supported by many CI/CD tools. In this model, the CI portion of the pipeline is unchanged since the CI engine (e.g., Jenkins, GitLab CI) is still used for creating builds for the changed code, regression testing, and integrating/merging with the main source code in the relevant repositories, though it is not used to trigger continuous delivery (push updates directly) in the pipeline. Instead, a separate GitOps operator manages the deployment based on updates to the main (trunk) code.

An operator (e.g., Flux, ArgoCD) is an actor managed by an orchestration platform and can inherit the cluster’s configuration, security, and availability. The use of this actor improves security because an agent that lives inside of the cluster listens for updates to all code and image repositories that it is allowed to access and pulls images and configuration updates into the cluster. The pull approach used by the agent has the following security features:

The last common issue is security testing. Whatever the code type is (e.g., application service, Iac, Pac or observability), the CI/CD pipelines of DevSecOps for microservices-based infrastructure with service mesh should include application security testing (AST) enabled by either automated tools or offered as a service. These tools analyze and test applications for security vulnerabilities. According to Gartner, there are four main AST technologies [35]:

In general, the overall metrics that testing tools (including the specific class of AST tools) should satisfy are [37]:

The functional requirements for AST tools in particular include performing the following types of scans [35]:

The vulnerability scans are to be performed whenever the code in the source code repository is revised to ensure that the current revision does not contain any vulnerable dependencies [38].

The desirable features of AST tools and/or services, along with techniques for behavioral analysis, are [35]:

Scanning application code for security vulnerabilities and misconfiguration as part of CI/CD pipeline tasks should involve the following artifacts:

原本PDFの抜粋画像

Container images include OS packages (e.g., Alpine, UBI, RHEL, CentOS, etc.) and language-specific packages (e.g., Bundler, Composer, npm, yarn, etc.).

原本PDFの抜粋画像

Scanning infrastructure as code for security vulnerabilities reduces the operations workload by preventing those vulnerabilities from making it to production, although it cannot replace checking for runtime security since the risk of drift will always exist. However, the reasons for all post-deployment (runtime) changes to the architecture (due to drift) must be analyzed and addressed by pushing appropriate updates to the IaC so that it becomes part of the pipeline and does not reoccur in subsequent deployments. This approach facilitates the use of runtime checks for remediating security design flaws.

The infrastructure-as-code files can be found in the following:

原本PDFの抜粋画像

Application services code, policy-as-code, and observability-as-code files can be found in the data plane and control plane components of the dedicated application services component (e.g., service mesh) and should be scanned for both security vulnerabilities (e.g., information leakage in authorization policies) and misconfiguration.

原本PDFの抜粋画像

The benefits of DevSecOps include:

Validation of each request and the feedback mechanisms mentioned above are further described below:

In the reference platform, the runtime status or execution state of the entire application system is due to a combination of executions of infrastructure code (e.g., networking routes for inter-service communication, resources provisioning code), policy code (e.g., code that specifies authentication and authorization policies), and session management code (e.g., code that establishes an mTLS session, code that generates JWT tokens) as revealed by the execution of observability as code. The observability as code of the service mesh relays the output from the execution of infrastructure, policy, and session management codes during runtime to various monitoring tools that generate applicable metrics and log aggregation tools and tracing tools, which in turn relay their output to a centralized dashboard. The analytics that are integral to the output of these tools enable system administrators to obtain a comprehensive global view of the runtime status of the entire application system. It is the runtime performance of a DevSecOps platform enabled through continuous monitoring with zero trust design features that provides all of the necessary security assurance for cloud-native applications.

The activities in the DevSecOps pipelines that enable continuous ATO are:

Specific risk assessment features include the capability to generate actionable tasks, specify code-level guidance, and test plans for verifying compliance [38]. In addition, risk assessment tools can provide complete traceability for all of the artifacts displayed in the dashboard, as well as the reporting capabilities needed for continuous ATO.

This document provides comprehensive guidance for the implementation of DevSecOps primitives for a reference platform hosting a cloud-native application. It includes an overview of the reference platform and describes the basic DevSecOps primitives (i.e., CI/CD pipelines), its building blocks, the design and execution of the pipelines, and the role of automation in the efficient execution of workflows in CI/CD pipelines.

The architecture of the reference platform – in addition to the application code and the code for providing application services – consists of functional elements for infrastructure, runtime policies, and continuous monitoring of the health of the application, which can be deployed through declarative codes with separate CI/CD pipelines types. The runtime behaviors of these codes, the benefits of the implementation for high-assurance security, and the use of the artifacts within the pipelines for providing a continuous authority to operate (C-ATO) using risk management tools and dashboard metrics are also described.

[1] Rose S, Borchert O, Mitchell S, Connelly S (2020) Zero Trust Architecture. (National Institute of Standards and Technology, Gaithersburg, MD), NIST Special Publication (SP) 800-207. https://doi.org/10.6028/NIST.SP.800-207

[2] Garrison J, Nova K (2017) Cloud Native Infrastructure (oreilly.com). Available at https://www.oreilly.com/library/view/cloud-native-infrastructure/9781491984291/

原本PDFの抜粋画像

[3] Federal News Network (2021) Expert Edition – DevSecOps. Available at https://federalnewsnetwork.com/wp-content/uploads/2021/12/120821_Expert-Edition_DevSecOps.pdf

[4] Chandramouli R (2019) Security Strategies for Microservices-based Application Systems. (National Institute of Standards and Technology, Gaithersburg, MD), NIST Special Publication (SP) 800-204. https://doi.org/10.6028/NIST.SP.800-204

[5] Chandramouli R, Butcher Z (2020) Building Secure Microservices-based Applications Using Service-Mesh Architecture. (National Institute of Standards and Technology, Gaithersburg, MD), NIST Special Publication (SP) 800-204A. https://doi.org/10.6028/NIST.SP.800-204A

原本PDFの抜粋画像

[6] Chandramouli R, Butcher Z, Aradhna C (2021) Attribute-based Access Control for Microservices-based Applications using a Service Mesh. (National Institute of Standards and Technology, Gaithersburg, MD), NIST Special Publication (SP) 800-204B. https://doi.org/10.6028/NIST.SP.800-204B

[7] Souppaya M, Morello J, Scarfone K (2017) Application Container Security Guide. (National Institute of Standards and Technology, Gaithersburg, MD), NIST Special Publication (SP) 800-190. https://doi.org/10.6028/NIST.SP.800-190

[8] Kubernetes Documentation (2021) Network Policies. Available at https://kubernetes.io/docs/concepts/services-networking/network-policies/

[9] Kubernetes Advocate (2021) Managing Microservices With Istio Service Mesh in Kubernetes. Available at https://medium.com/avmconsulting-blog/managing-microservices-with-istio-service-mesh-in-kubernetes-36e1fda81757

[10] Agarwal G (2020) How to Manage Microservices on Kubernetes With Istio. Available at https://medium.com/better-programming/how-to-manage-microservices-on-kubernetes-with-istio-c25e97a60a59

[11] Ramakani A (2020) Kong API Gateway – From Zero to Production. Available at https://medium.com/swlh/kong-api-gateway-zero-to-production-5b8431495ee

[12] Srinath K (2020) DevSecOps – Baking Security into Development Process. Available at https://medium.com/faun/devsecops-baking-security-into-development-process-9579418ad9a7

原本PDFの抜粋画像

[13] Rubinstein D (2020) AppSec vs. DevSecOps, and what that means for developers. Available at https://sdtimes.com/security/appsec-vs-devsecops-and-what-that-means-for-developers/

[14] Red Hat (2021) What is DevSecOps? Available at https://www.redhat.com/en/topics/devops/what-is-devsecops

[15] Red Hat (2021) What is CI/CD? Available at https://www.redhat.com/en/topics/devops/what-is-ci-cd#:~:text=CI%2FCD%20is%20a%20method,continuous%20delivery%2C%20and%20continuous%20deployment

[16] Ali M (2018) Continuous Release Practices are evolving, Here is our story. Available at https://medium.com/the-telegraph-engineering/continuous-release-practices-are-evolving-here-is-our-story-2a4d164e9cac

[17] Pittet S (2021) Continuous integration vs. continuous delivery vs. continuous deployment. Available at https://www.atlassian.com/continuous-delivery/principles/continuous-integration-vs-delivery-vs-deployment

原本PDFの抜粋画像

[18] Wuestkamp K (2020) K8s–native Jenkins-X and Tekton Pipelines. Available at https://itnext.io/k8s-native-jenkins-x-and-tekton-pipelines-e2b5a61a1d22

[19] aws.amazon.com (2021) Create Continuous Delivery Pipeline. Available at https://aws.amazon.com/getting-started/hands-on/create-continuous-delivery-pipeline/?trk=gs_card

[20] aws.amazon.com (2021) Setting up a CI/CD pipeline by integrating Jenkins with AWS CodeBuild and AWS CodeDeploy. https://aws.amazon.com/blogs/devops/setting-up-a-ci-cd-pipeline-by-integrating-jenkins-with-aws-codebuild-and-aws-codedeploy/

[21] mylocaldevstack.pl (2019) The future of DevOps – Assembly Lines. Available at https://medium.com/@mylocaldevstack/the-future-of-devops-assembly-lines-40227546d750

[22] Socher R (2020) Best Terraform Tutorial Guides: An Overview. Available at https://faun.pub/best-terraform-tutorial-guides-an-overview-65a6fcee0a24

[23] Soni A (2020) GitOps: The Next Big Thing for DevOps and Automation! Available at https://medium.com/searce/gitops-the-next-big-thing-for-devops-and-automation-2a9597e51559

[24] Fallon A (2020) Understand the role of infrastructure as code in DevOps (TechTarget.com). Available at https://searchitoperations.techtarget.com/feature/Understand-the-role-of-infrastructure-as-code-in-DevOps?utm_campaign=20210809_The+next+DevSecOps+challenge%3A+People&utm_medium=EM&utm_source=NLN&track=NL-1841&ad=939963&asrc=EM_NLN_174809933

[25] Ahmed M (2020) Introducing Policy As Code: The Open Policy Agent (OPA). Available at https://www.magalix.com/blog/introducing-policy-as-code-the-open-policy-agent-opa

原本PDFの抜粋画像

[26] Singh P (2021) Tackle Kubernetes observability with the right metrics. Available at https://searchitoperations.techtarget.com/tip/Tackle-Kubernetes-observability-with-the-right-metrics?track=NL-1841&ad=938191&asrc=EM_NLN_153034984&utm_medium=EM&utm_source=NLN&utm_campaign=20210322_DevSecOps+leaves+Excel+in+the+dust

[27] Agarwal G (2020) Kubernetes Security With Falco. Available at https://betterprogramming.pub/kubernetes-security-with-falco-2eb060d3ae7d

[28] Marko K (2021) Terraform cheat sheet: Notable commands, HCL and more. Available at https://searchitoperations.techtarget.com/tip/Terraform-cheat-sheet-Notable-commands-HCL-and-more?utm_campaign=20210726_Infrastructure+as+code+still+a+big+security+buzz&utm_medium=EM&utm_source=NLN&track=NL-1841&ad=939808&asrc=EM_NLN_172629823

[29] Guo T (2021) On DevOps 8: Infrastructure as Code: Introduction, Best Practices and How to choose the Right tool. Available at https://medium.com/4th-coffee/on-devops-8-infrastructure-as-code-introduction-best-practices-and-choosing-the-right-tool-2c8f46d1f34

[30] Pulumi.com (2020) Delivering Cloud Native Infrastructure as Code. Available at https://cdn2.hubspot.net/hubfs/4429525/Content/Pulumi-Delivering-CNI-as-Code.pdf

[31] Chong T (2021) Product In-Depth: Instantly Secure your Cloud Infrastructure with Policy-as-Code. Available at https://www.magalix.com/blog/instantly-secure-your-cloud-infrastructure-with-policy-as-code?utm_medium=email&_hsmi=123795689&_hsenc=p2ANqtz-HbHGaLCOrCV_WXjrKE-JgWUWUX9Tg1s9ot5HSox2Ts1rgAhwquoZkBKqZYo-OcHJmlUHG8eFhC5Tfu7MW_Dn3i-g3Ng&utm_content=123795689&utm_source=hs_email

[32] Pariseau B (2020) Mendix dumps cluttered DevOps monitoring tools for Datadog. Available at https://searchitoperations.techtarget.com/news/252497697/Mendix-dumps-cluttered-DevOps-monitoring-tools-for-Datadog?track=NL-1841&ad=938089&asrc=EM_NLN_151848158&utm_medium=EM&utm_source=NLN&utm_campaign=20210315_Kong+Mesh%27s+built-in+Open+Policy+Agent+simplifies+IT+security+management

[33] Boutet R (2018) Log Patterns: Automatically cluster your logs for faster investigation (datadoghq.com). Available at https://www.datadoghq.com/blog/log-patterns/

[34] Devopscurry (2021) Securing your CI/CD pipelines with DevSecOps in 2021. Available at https://medium.com/devopscurry/securing-your-ci-cd-pipelines-with-devsecops-in-2021-1a6a6e34f2e7

原本PDFの抜粋画像

[35] CI/CD]Gartner (2021) Gardner D, Horvath M, Zumerle D Magic Quadrant for Application Security Testing. Available at https://www.gartner.com/doc/reprints?id=1-262TXQZV&ct=210518&st=sb

[36] Lewkowicz J (2021) A guide to automated testing providers. Available at https://sdtimes.com/test/a-guide-to-automated-testing-providers/

[37] Lewkowicz J (2021) Automated testing is a must in CI/CE pipelines. Available at https://sdtimes.com/test/automated-testing-is-a-must-in-ci-cd-pipelines/

原本PDFの抜粋画像

[38] Alexey K (2020) Ultimate guide to CI/CD security and DevSecOps. Available at https://circleci.com/blog/security-best-practices-for-ci-cd/

[39] Sheldon R (2021) Top 30 DevOps interview questions and answers for 2021. Available at https://whatis.techtarget.com/feature/Top-30-DevOps-interview-questions-and-answers?utm_campaign=20210712_New+Kubernetes+use+case%253A+Hacking&utm_medium=EM&utm_source=NLN&track=NL-1841&ad=939635&asrc=EM_NLN_170130891

[40] Prasad A (2020) Kubernetes Admission Controllers: Request Interceptors. Available at https://medium.com/cloudlego/kubernetes-admission-controllers-request-interceptors-47a9b12c5303