NIST SP 800-183 日本語訳

NIST Special Publication 800-183

Jeffrey Voas

This publication is available free of charge from: http://dx.doi.org/10.6028/NIST.SP.800-183

NIST Special Publication 800-183

Jeffrey Voas

Computer Security Division Information Technology Laboratory

This publication is available free of charge from: http://dx.doi.org/10.6028/NIST.SP.800-183

July 2016

U.S. Department of Commerce Penny Pritzker, Secretary

National Institute of Standards and Technology

Willie May, Under Secretary of Commerce for Standards and Technology and Director

Authority

This publication has been developed by NIST in accordance with its statutory responsibilities under the Federal Information Security Modernization Act (FISMA) of 2014, 44 U.S.C. § 3541 et seq., Public Law (P.L.) 113-283. NIST is responsible for developing information security standards and guidelines, including minimum requirements for federal information systems, but such standards and guidelines shall not apply to national security systems without the express approval of appropriate federal officials exercising policy authority over such systems. This guideline is consistent with the requirements of the Office of Management and Budget (OMB) Circular A-130.

Nothing in this publication should be taken to contradict the standards and guidelines made mandatory and binding on federal agencies by the Secretary of Commerce under statutory authority. Nor should these guidelines be interpreted as altering or superseding the existing authorities of the Secretary of Commerce, Director of the OMB, or any other federal official. This publication may be used by nongovernmental organizations on a voluntary basis and is not subject to copyright in the United States. Attribution would, however, be appreciated by NIST.

National Institute of Standards and Technology Special Publication 800-183 Natl. Inst. Stand. Technol. Spec. Publ. 800-183, 30 pages (July 2016) CODEN: NSPUE2

This publication is available free of charge from: http://dx.doi.org/10.6028/NIST.SP.800-183

Comments on this publication may be submitted to: National Institute of Standards and Technology Attn: Computer Security Division, Information Technology Laboratory 100 Bureau Drive (Mail Stop 8930) Gaithersburg, MD 20899-8930 Email: iot@nist.gov

All comments are subject to release under the Freedom of Information Act (FOIA).

The Information Technology Laboratory (ITL) at the National Institute of Standards and Technology (NIST) promotes the U.S. economy and public welfare by providing technical leadership for the Nation’s measurement and standards infrastructure. ITL develops tests, test methods, reference data, proof of concept implementations, and technical analyses to advance the development and productive use of information technology. ITL’s responsibilities include the development of management, administrative, technical, and physical standards and guidelines for the cost-effective security and privacy of other than national security-related information in federal information systems.

System primitives allow formalisms, reasoning, simulations, and reliability and security risk-tradeoffs to be formulated and argued. In this work, five core primitives belonging to most distributed systems are presented. These primitives apply well to systems with large amounts of data, scalability concerns, heterogeneity concerns, temporal concerns, and elements of unknown pedigree with possible nefarious intent. These primitives are the basic building blocks for a Network of ‘Things’ (NoT), including the Internet of Things (IoT). This document offers an underlying and foundational understanding of IoT based on the realization that IoT involves sensing, computing, communication, and actuation. The material presented here is generic to all distributed systems that employ IoT technologies (i.e., ‘things’ and networks). The expected audience is computer scientists, IT managers, networking specialists, and networking and cloud computing software engineers. To our knowledge, the ideas and the manner in which IoT is presented here is unique.

Internet of Things (IoT); Network of Things (NoT); reliability; security; trust; trustworthiness, sensors; big data; composability; distributed system.

Acknowledgment

Appreciation is given to Matt Scholl, Dave Ferraiolo, Tim Grance, Irena Bojanova, Angelos Stavrou, Rick Kuhn, and Phil Laplante, for the encouragement and support to take on this task. Appreciation is also given to Constantinos Kolia (GMU) for the suggestions in Section 5, and to Svetlana Lukoyanova for the artwork.

From agriculture, to manufacturing, to smart homes, to healthcare, and beyond, there is value in having numerous sensory devices connected to larger infrastructures. This technology advance acknowledges the reality that human society is moving towards ‘smart’ and ‘smarter’ systems. The rapid advances in computer science, software engineering, systems engineering, networking, sensing, communication, and artificial intelligence are converging. The tethering factor is data.

There is no formal, analytic, or even descriptive set of the building blocks that govern the operation, trustworthiness, and lifecycle of IoT. A composability model and vocabulary that defines principles common to most, if not all networks of things, is needed to address the question: “what is the science, if any, underlying IoT?” This document offers an underlying and foundational science to IoT based on a belief that IoT involves sensing, computing, communication, and actuation.

This document uses two acronyms, IoT and NoT (Network of Things), extensively and interchangeably—the relationship between NoT and IoT is subtle. IoT is an instantiation of a NoT, more specifically, IoT has its ‘things’ tethered to the Internet. A different type of NoT could be a Local Area Network (LAN), with none of its ‘things’ connected to the Internet. Social media networks, sensor networks, and the Industrial Internet are all variants of NoTs. This differentiation in terminology provides ease in separating out use cases from varying vertical and quality domains (e.g., transportation, medical, financial, agricultural, safety-critical, security-critical, performance-critical, high assurance, to name a few). That is useful since there is no singular IoT, and it is meaningless to speak of comparing one IoT to another.

Primitives are building blocks that offer the possibility of an answer to the aforementioned questions and statements by allowing comparisons between NoTs. We use the term primitive to represent smaller pieces from which larger blocks or systems can be built. For example, in software coding, primitives typically include the arithmetic and logical operations (plus, minus, and, or, etc.). In this document, we do not employ the restriction that primitives cannot be developed or derived from something else; this is often a common, paraphrased definition for “primitive,” but it is not employed here.

This model does not specify a definition for what is or is not a ‘thing.’ Instead, we consider that each primitive injects a behavior representing that ‘thing’ into a NoT’s workflow and dataflow. ‘Things’ can occur in physical space or virtual space. In physical space, consider humans, vehicles, residences, computer, switches, routers, smart devices, road networks, office buildings, etc. In virtual space, consider software, social media threads, files, data streams, virtual machines, virtual networks, etc. More ideas concerning ‘things’ is presented in Appendix C.

Primitives offer a unifying vocabulary that allows for composition and information exchange among differently purposed networks. They offer clarity regarding concerns that are subtle, including interoperability, composability, and continuously-binding assets that come and go on-the-fly. Because no simple, actionable, and universally-accepted definition for IoT exists, the model and vocabulary proposed here reveals underlying foundations of the IoT, i.e., they expose the ingredients that can express how the IoT behaves, without defining IoT. This offers insights into issues specific to trust.

Further, we employ a paraphrased, general definition for a distributed system: a software system in which components located on networked computers communicate and coordinate their actions by passing messages. The components interact with each other in order to achieve a common goal.¹ NoTs satisfy this definition. Thus we consider IoT to be one type of a NoT and a NoT to be one type of a distributed system.

The primitives of a NoT are: 1) Sensor, 2) Aggregator, 3) Communication channel, 4) external utility (eUtility), and 5) Decision trigger. There may be some NoTs that do not contain all of these, but that will be rare.

Each primitive, along with its definition, assumptions, properties, and role, is presented. We employ a data-flow model, captured as a sequence of four figures in this document, to illustrate how primitives, when composed in a certain manner, could impact a confidence in trustworthiness of NoTs. Although this model may seem overly abstract at first glance, its simplicity offers a certain elegance by not over complicating IoT’s small handful of building blocks.

A sensor is an electronic utility that measures physical properties such as temperature, acceleration, weight, sound, location, presence, identity, etc. All sensors employ mechanical, electrical, chemical, optical, or other effects at an interface to a controlled process or open environment. Basic properties, assumptions, recommendations, and general statements about sensor include:

An aggregator is a software implementation based on mathematical function(s) that transforms groups of raw data into intermediate, aggregated data. Raw data can come from any source. Aggregators help in managing ‘big’ data. Basic properties, assumptions, recommendations, and general statements about aggregator include:

A cluster is an abstract grouping of sensors that can appear and disappear instantaneously. Basic properties, assumptions, recommendations, and general statements about cluster include:

Note item 4 in the above list; it is subtly important – it relates to business competition for highly valued data.

Weight is the degree to which a particular sensor’s data will impact an aggregator’s computation. Basic properties, assumptions, recommendations, and general statements about weight include:

A communication channel is a medium by which data is transmitted (e.g., physical via Universal Serial Bus (USB), wireless, wired, verbal, etc.). Basic properties, assumptions, recommendations, and general statements about communication channel include:

In Figure 1, 15 sensors are shown – the blue sensors indicate that these 2 sensors are ‘somehow’ failing and at specific times, that is, they are not satisfying their purpose and expectations. As mentioned earlier, there could be a variety of sensor failure modes, some temporal, and some related to data quality. Further the temporal failure modes for sensors may be actually a result of the transport of that data failing, and not the sensors themselves. Consider also that the two failing sensors in Figure 1 should probably be assigned lower weights. Figure 1 also shows the 15 sensors clustered into 3 clusters with 5 unique sensors assigned to each. Figure 1 shows the data coming out from each of the three clusters as being inputted to 3 corresponding aggregators. It is now the responsibility of the 3 aggregators to turn those 15 sensor inputs into 3 intermediate data points.

Note the close relationship between clusters and aggregators. For example, in Figure 1, aggregator A₁ might be determining how busy restaurant X is. Five independent sensors in C₁ could be taking pictures from inside and outside (parking lot) of X, room temperature measurement in the kitchen, motion detectors from the dining area, sound and volume sensors, light detectors, etc. So while the sensors are certainly not homogeneous, their data is processed to make a new piece of data to address one question with possible results such as is the restaurant busy, not busy, closed, etc. And aggregators A₂ and A₃ might be doing the same for restaurants Y and Z respectively. Consider also that an aggregator can be associated with different clusters.

Figure 1: The first three primitives

An eUtility (external utility) is a software or hardware product or service.³ The current definition of eUtility is deliberately broad, to allow for unforeseen future services and products that will be incorporated in future types of NoTs yet to be defined. (Breaking this primitive down into sub-primitives is future work.) Basic properties, assumptions, recommendations, and general statements about eUtility include:

Figure 2 illustrates using two cloud eUtilities to execute five aggregator implementations. (Note that there is no suggestion here that aggregators must execute on cloud platforms – this was for illustration only – aggregators can execute on any computing platform.) Figure 2 shows the addition of one non-cloud eUtility, eU (a laptop).

1

Figure 2: eUtility

A decision trigger creates the final result(s) needed to satisfy the purpose, specification, and requirements of a specific NoT. Basic properties, assumptions, recommendations, and general statements about decision trigger include:

Figure 3: Decision trigger

Decision triggers are predicates that must be true to initiate a command or action. They will frequently be represented as if-then rules (although this may not always be the case), and variables within the predicate may represent data or sensor values from parts of the NoT. Reliability is essential, indicating the need for strong testing and assurance of decision triggers. Fortunately, methods and tools for testing access control rules are well developed and may apply to a large set of decision trigger testing problems [Kuhn 2016]. Paralleling the structure of decision triggers, access control rules typically take the form of predicates evaluating to two-valued decisions – either grant or deny in most cases. For example, a building access control system may include a rule such as, “if subject is an employee and time is between 6 am and 9 pm then grant access.” Access control rules are composed into policies, which may include hundreds of rules with a large number of variables interacting in complex ways. Methods and tools exist to check complex access control policies for consistency and completeness, and to generate tests for assurance of policies implemented in code. These access control policy testing tools could be adapted to NoT verification and testing, providing a level of assurance that will be essential for the more critical NoT applications.

Figure 4 shows an alternative to any suggestion that this model of a NoT’s dataflow is necessarily uni-directional; it depicts a decision trigger that actually feeds its computation for g(x, y) back into the NoT, creating a continuous feedback loop. So for example if new sensor data were fed continuously into a NoT’s workflow and dataflow, that data can be combined with the results of previous decision trigger outputs to create updated decision trigger results at later points in time.

Figure 4: Decision trigger with feedback

Now, a few additional points concerning the interplay and relationship between the five primitives are as follows. First, sensor feeds aggregator. Secondly, aggregator executes on various eUtilities of a NoT. Thirdly, communication channels are the veins and arteries that connect sensor, aggregator, eUtility, and decision trigger with the data that flows between them. And fourth, sensor, aggregator, communication channel, eUtility, and decision trigger all have events firing at specific snapshot times; a large challenge for IoT and NoTs is to keep these events in sync, just as for any distributed system.

It is important to treat all events with respect to their temporal location, i.e., their geographical place and at what time. For example, the older events become, the staler or less interesting they may be relative to immediate decision making. Historical information may be useful in other calculations such as statistical reporting or model generation.

To complete this model, we define six elements: environment, cost, geographic location, owner, Device_ID, and snapshot, that although are not primitives, are key players in trusting NoTs. These elements play a major role in fostering the degree of trustworthiness⁵ that a specific NoT can provide.

a. Because a NoT is a distributed system, different events, data transfers, and computations occur at different snapshots.

b. Snapshots may be aligned to a clock synchronized within their own network [NIST 2015]. A global clock may be too burdensome for sensor networks that operate in the wild. Others, however, argue in favor of a global clock [Li 2004]. This publication does not endorse either scheme at the time of this writing.

c. Data, without some “agreed upon” time stamping mechanism, is of limited or reduced value.

d. NoTs may affect business performance – sensing, communicating, and computing can speed-up or slow-down a NoT’s workflow and therefore affect the “perceived” performance of the environment it operates in or controls.

e. Snapshots may be tampered with, making it unclear when events actually occurred, not by changing time (which is not possible), but by changing the recorded time at which an event in the workflow is generated, or computation is performed, e.g., sticking in a delay() function call.

f. Malicious latency to induce delays, are possible and will affect when decision triggers are able to execute.

g. Reliability and performance of a NoT may be highly based on (e) and (f).

Five additional considerations to the previous material that defined the primitives and elements include:

NoTs can be open, closed, or somewhere in between. For example, an automobile can have hundreds of sensors, numerous Central Processing Unit (CPU)s, databases such as maps, wired communication channels throughout the car, and without any wireless access between any ‘thing’ in the car to the outside. This illustrates a closed NoT. Such a NoT mitigates nearly all wireless security concerns such as remotely controlling a car, however there could still be concerns of malware and counterfeit ‘things’ that could result in reduced safety. (There are issues related to wireless transmission ranges that we ignore here for simplification.) A fully open system would essentially be any ‘thing’ interoperating with any ‘thing,’ any way, and at any time. This, from a “trustworthiness” standpoint, is impossible to assure since the NoT is unbounded.

Most NoTs will be between these extremes since a continuum will likely exist. The primitives serve as a guidepost as to where reliability and security concerns require additional mitigation, e.g., testing and other technologies related to validation and verification.

We envision a future demand for design patterns that allow larger NoTs to be built from smaller NoTs, similar to design patterns in object-oriented systems. In essence, these smaller entities are sub-NoTs. Sub-NoTs could speed-up IoT adoption for organizations seeking to develop IoT-based systems by having access to sub-NoT catalogues. Further, the topology of sub-NoTs could impact the security and performance of composite NoTs.

To understand the inescapable trust issues associated with IoT and a specific NoT, consider the attributes of the primitives and elements shown in Table 1. The three rightmost columns are our best guess as to whether the pedigree, reliability, or security of an element or primitive creates a trustworthiness risk. Those with question marks (?) are ones do not think we fully understand at this time.

The following table poses questions such as: what does trust mean for a NoT when its primitives are in continual flux due to natural phenomenon that are in continuous change and while its virtual and physical entities are unknown, partially unknown, or faulty? Or if we have insecure physical systems employing faulty snapshots composed with incorrect assumed environments, where is the trust?

Table 1: Primitive and Element Trust Questions

Such questions demonstrate the difficulty assuring and assessing NoT trustworthiness. We offer the following statement about NoT trustworthiness:

Trust in some NoT A, at some snapshot X, is a function of NoT A’s assets ∈ {sensors (s), aggregator(s), communication channel(s), eUtility(s), decision trigger(s)} with respect to the members ∈ {geographic location, owner, environment, cost, Device_IDs, snapshot} when applicable.

A testability⁷ metric that applies here is titled the Domain Range Ratio [Voas 1993]. This ratio is simply the cardinality of the set of all possible test cases to the cardinality of the set of all possible outputs.

To understand this metric, consider a NoT’s decision trigger that forces an actuator to be in one of two states: (‘1’) or (‘0’). (This situation occurs in Figures 3 and 4.) Further, assume that each output state occurs 50 % of the time. Because of this minimal output space size, a fair coin toss also has a 50-50 chance of providing a correct output for any given input, and that’s likely less expensive than building a complex NoT. Worse, consider the scenario where ‘1’ and ‘0’ are not evenly distributed, e.g., the specification states that for 1 million unique test cases only 10 should produce a ‘1’ and the other 999 990 should produce a ‘0’. Here, you could build a NoT to compute this function or you could write a piece of code that simply says: for all inputs output (‘0’). This incorrect software implementation is still 99.999 % reliable, and you’ll almost certainly not discover the code defect with a handful of random tests sampled from the 1M. In short, testing here has a minimal probability of detecting the faulty logic because each test case has low “detectability” due to the tiny output space and the probability density function for each output.

This shows that because of the decision trigger primitive, specifically purposed NoTs may be inherently untestable without well-placed internal self-tests (assertions⁸) upon the other 4 primitives to increase testability.

As NoTs are likely to rely on factors from outside native environmental boundaries (e.g., sensor readings from a NoT's surrounding external environment, whatever that environment may be), it is equally logical to assume that some form of actuation may be in place to effect or at least influence that external environment as a result of a NoT action. Thus, it is feasible that an eUtility may be more than a software-driven virtual entity as is strongly implied, but could also a cyber-physical or purely a mechanical actuator (e.g. control surface hydraulics on an aircraft) that extends an action to influence a NoT environment as opposed to only influencing the internal workings of the NoT itself. This raises the larger question of how a NoT and its environment interact, not only at the element level which is addressed, but, more significantly, at the NoT’s operating level.

The elements lay out key contextual issues related to trustworthiness of a specific NoT. And the primitives are the building blocks of NoTs. Because trustworthiness is such a broad concept, this document has mainly focused on two “ilities” related to the five primitives: security and reliability. People often ask for simple examples of real or hypothetical use cases relating these two “ilities” to each primitive. The following are examples of simple, hypothetical reliability and security scenarios associated with each primitive

Sensor reliability: A modern car’s speed sensor is exposed to heat, water, and dust (environment). Years later, it starts providing inconsistent readings due to naturally occurring fatigue that induces corrupt sensor data. This is an example of malfunctions caused by environmental conditions.

Sensor security: A smart building’s temperature sensors are easily accessible and this particular system doesn’t provide a means for validating the firmware’s authenticity. An attacker substitutes the firmware with one that responds to remote commands. These sensors then become part of a botnet and can contribute to distributed denial-of-service (DDoS) attacks. This is an example of physical tampering and altering firmware.

Aggregator reliability: In a smart city environment, thousands of sensors transmit data to a series of smart gateways that effectively compress several gigabytes of raw data into meaningful information. A blackout that occurred in part of the city creates an unexpected condition that results in division by zero, which causes the application to keep crashing for the entire duration of the blackout. This is an example of unpredicted conditions that lead to undefined behavior and incorrect output.

Aggregator security: An attacker introduces a rogue sensor into a network that produces fake readings. These readings are passed as inputs to the aggregator function without any validation. The attacker launches a buffer overflow attack to gain root access to the entire middleware infrastructure (gateway). This is an example of an injection attack or buffer overflow.

Communication channel reliability: ‘Smart building’ sensors for regulating lights and temperature communicate wirelessly via IEEE 802.11 with the rest of the building management system. During a conference, a large number of people are gathered inside a room, having enabled Wi-Fi on their smartphones. Due to overpopulation of the channel, there are frequent disconnections and service degradation. As a result, the sensors are unable to provide readings with their predefined frequency. This is an example of loss of service due to overpopulation and connection problems.

Communication channel security: A wearable activity tracker is attached to a person’s wrist and measures heart rate and blood pressure. It communicates via Bluetooth Low Energy (BLE) with the wearer’s smartphone and forwards the data to a physician. Despite the fact that BLE takes specific actions to randomize the MAC address of the devices, the manufacturer neglected this feature. An attacker with a high-gain antenna can track the presence of the wearer in a crowd and create a movement profile. This is an example of eavesdropping on the communication channel.

eUtility reliability: A point-of-sale system conducting automatic smart payments depends on a cloud service for verifying the identity of the person using a card. System maintenance of the cloud server happens to occur during business hours, which causes delays in verification. This is an example of system failures that make the resource unavailable, and therefore service unreliable.

eUtility security: A ‘smart home’ has a security camera installed at the front door that sends data to a corresponding cloud application that then forwards notifications and video footage to the homeowner’s device after motion at the door is detected. An attacker conducts a DDoS attacks on the application provider’s servers for two hours. They’re able to break into the house without the user being notified. This is an example of a DDoS attack.

Decision trigger reliability: The logic implemented, for example, a decision trigger is just a conditional expression, e.g., if a > 100 open garage door. But what if the expression should have been if a > 10 open garage door. This is a reliability problem because the function implemented is incorrect for all values of a between 11 and 99. Decision triggers are likely to be written in code, although this assumption could be relaxed.

Decision trigger security: The decision trigger implementation accepts malicious inputs or potentially the outputs from the trigger are sniffed and released to competitors unbeknownst to the legitimate owner of the trigger. Either way, this is an example of data tampering and a loss of data integrity.

Note that there are relatively few standards or best practices for IoT security design and testing, although some related guidance is being developed by the Cyber-Physical Systems Public Working Group [NIST 2014a] and in documents such as the National Institute of Standards and Technology’s Guidelines for Smart Grid Cybersecurity [NIST 2014b].

This document offers an underlying and foundational science for IoT-based technologies on the realization that IoT involves sensing, computing, communication, and actuation. We presented a common vocabulary to foster a better understanding of IoT and better communication between those parties discussing IoT. We acknowledge that the Internet is a network of networks, but we believe that focusing on restricted NoTs in a bounded way gives better traction to addressing trustworthiness problems that an unbounded Internet does not. Some may argue that every ‘thing’ in a NoT is ultimately a service. The primitives definitely offer services, however because of the combinatorics of mixtures of hardware and software ‘things’, and we prefer to distinguish.

Five primitives and six elements that impact IoT trustworthiness have been presented. Primitives are the building blocks; elements are the less tangible trust factors impacting NoTs. Primitives also allow for analytics and formal arguments of IoT use case scenarios. Without an actionable and universally-accepted definition for IoT, the model and vocabulary presented here expresses how IoT, in the broad sense, behaves.

Use case scenarios employing the primitives afford us quicker recommendations and guidance concerning a NoT’s potential trustworthiness. For example, authentication can be used in addressing issues such as geo-location and sensor ownership, but authentication may not be relevant if an adversary owns the sensors and can obtain that information based on proximity. Encryption can protect sensor data transmission integrity and confidentiality including cloud-to-cloud communication, but it might render the IoT sensors unusable due to excessive energy requirements. While fault-tolerant techniques can alleviate reliability concerns associated with inexpensive, replaceable, and defective third party ‘things’, they can also be insecure and induce communication overhead and increased attack surfaces. In short, primitives and how they can be composed create a design vocabulary for how to apply existing technologies that support IoT trustworthiness.

These primitives are simply objects with attributes. The five, along with the context offered by the six elements, form a design catalog for those persons and organizations interested in exploring and implementing current and future IoT-based technology.

[Kuhn 2016] D. R. Kuhn, V. Hu, D. F. Ferraiolo, R. N. Kacker, and Y. Lei, “Pseudo-exhaustive Testing of Attribute Based Access Control Rules,"

International Workshop on Combinatorial Testing at the 2016 IEEE Ninth International Conference on Software Testing, Verification and Validation Workshops (ICSTW), Chicago, Illinois, April 10-15, 2016.

[Li 2004]

Q. Li and D. Rus, “Global Clock Synchronization in Sensor Networks,” Twenty-third Annual Joint Conference of the IEEE Computer and Communications Societies (INFOCOM 2004), Hong Kong, March 7-11, 2004, pp. 564-574. http://dx.doi.org/10.1109/INFCOM.2004.1354528.

[NIST 2015] M. Weiss, J. Eidson, C. Barry, D. Broman, L. Goldin, B. Iannucci, E. A. Lee, and K. Stanton, Time-Aware Applications, Computers, and Communication Systems (TAACCS), NIST Technical Note (TN) 1867, National Institute of Standards and Technology, Gaithersburg, Maryland, February 2015, 26pp. http://dx.doi.org/10.6028/NIST.TN.1867.

[Voas 1993] J. M. Voas and K. W. Miller, “Semantic metrics for software testability,” Journal of Systems and Software vol. 20, no. 3 (March 1993), pp. 207-216. http://dx.doi.org/10.1016/0164-1212(93)90064-5.

[Voas 1995] J. M. Voas and K. W. Miller, “Software testability: the new verification,” IEEE Software vol. 12, no. 3 (May 1995), pp. 17-28. http://dx.doi.org/10.1109/52.382180.

[NIST 2014a]

Cyber-Physical Systems Public Working Group Workshop [Web page], National Institute of Standards and Technology, August 11-12, 2014. http://www.nist.gov/cps/cps-pwg-workshop.cfm [accessed 7/13/16].

[NIST 2014b]

The Smart Grid Interoperability Panel—Smart Grid Cybersecurity Committee, Guidelines for Smart Grid Cybersecurity, NIST Interagency Report (NISTIR) 7628 Revision 1, National Institute of Standards and Technology, Gaithersburg, Maryland, September 2014, 668pp. http://dx.doi.org/10.6018/NIST.IR.7628r1.