QUALITY CHARACTERISTICS AND METRICS FOR REUSABLE SOFTWARE (Preliminary Report)

The Software Producibility Manufacturing Operations Development and Integration Laboratory (Software Producibility MODIL) was established at the National Institute of Standards and Technology (NIST) in 1992. The Software Producibility MODIL was one of four MODILs instituted at national laboratories by the U.S. Department of Defense/Ballistic Missile Defense Organization (BMDO (1) ). The purpose of the MODILs was to investigate technology that could be transitioned from the research community to improve software development within the BMDO.

The initial focus of the Software Producibility MODIL was software reuse. When software is considered for reuse, especially in the high- integrity (2) applications for the BMDO, the quality of the software is of paramount importance. Organizations considering existing software for reuse must consider many variables to decide if the software is fit for reuse. One variable is the quality of the software; hence, organizations must be able to assess the quality of software. Organizations who are developing new software that is intended for reuse must also be able to assess whether the software meets reusability criteria.

This report on quality characteristics and metrics for reusable software is preliminary. It identifies a set of quality characteristics that are most commonly referenced in technical literature and standards. These quality characteristics are common to all software products (e.g., requirements documentation, design documentation, code, and test documentation.) Different metrics for each product may be used to assess the degree to which the product contains each quality characteristic. This report provides some explanation of the value of each metric for determining the reusability of the software. However, more research is needed to ensure the completeness of the quality characteristics and associated metrics.

This preliminary study alone does not provide sufficient measurement information for determining the reusability of software. For example, the value of each characteristic and each metric for each product should be correlated to a reusability index. The reusability index, which no one yet has defined, should also include process metrics (e.g., effort). Most of the metrics have been used for several years and are well-understood relative to structured design methods but most have not yet been applied to software developed with object-oriented technology. Another important topic to be considered for reusing software in high integrity applications is the knowledge about the software relative to its specific domain and application within that domain and the type of information that must be present for analysts to decide how the software must change to meet new requirements.

Additional research topics include, but are not limited to, the following:

• -- algorithms for relating measures to a reusability index
• -- criteria for selecting specific metrics
• -- user of the data (program manager, technical staff)
• -- purpose for which data will be used
• -- constraints (e.g., language, development environment, products available)
• -- value (e.g., quality characteristic most important)
• -- ability to collect and analyze data (e.g., automated support for specific measures).
• -- application of the quality characteristics and metrics to object- oriented development methods
• -- mapping of characteristics and metrics to existing and draft standards
• -- guidance on collecting measurement data for the metrics
• -- guidance on using the resulting measures

The purpose of this report is to summarize a set of metrics that are useful in measuring certain quality characteristics for software. In particular, these characteristics are applicable in assessing the reusability of software products. The quality characteristics are defined first. For each software product (requirements, design, code, test documentation), several metrics are given that will help to qualify the quality characteristics for that product.

A definition of software quality characteristics as given in [ISO9126] is "A set of attributes of a software product by which its quality is described and evaluated." The set of attributes includes functionality, reliability, usability, efficiency, maintainability, and portability. Several other documents ([SAC], [GPALS]) include other characteristics (e.g. adaptability, complexity) that apply to assessing the reusability of software products.

The metrics listed in this report are product metrics, and therefore, are meant to be applied to software products. The products are requirements documentation, design documentation, code, and test documentation. Because the focus on quality for software in the past has been on code, there are many more code metrics listed in this report.

A complete measure of reusability should also include process metrics applied to the development process. This report does not include process metrics.

One document, [SAC2], gives three criteria that should be used to evaluate the metrics themselves. These criteria are:

Objective

We can base the metric on the physical attribute of the object of interest and minimize the amount of guessing required.



Observable

We can extract the metric from existing products, such as source code, documentation, and maintenance histories.



Predictive

The metric correlates well with an attribute of interest, such as development effort and maintenance cost.

In this report, the attributes of interest are the quality characteristics of the software product. The metrics and characteristics can then be used to assess the reusability of the product, and that is the overall attribute of interest.

Most of the metrics in this report have been used for several years and are well-understood relative to structured design methods, but most have not yet been applied to software developed with object-oriented technology. Another important topic to be considered for reusing software in high integrity applications is the knowledge about the software relative to its specific domain and application within that domain and the type of information that must be present for analysts to decide how the software must change to meet new requirements. Some work on this topic has been conducted by NIST and may be found in [NIST5309].

Section 2 of this report provides definitions for each quality characteristic. Section 3 lists suggested metrics for each quality characteristic which pertain to a specific software product. Section 4 provides a summary of this report.

The definitions in sections 2.1 and 2.2 are stated as they appear in the referenced documents and may be associated with software reuse. Several quality characteristics have more than one definition.

This report makes no preference on the source of the definitions. The definition used for each characteristic in section 3 of this report is an aggregate meaning taken from the various sources, mixed with the intuitive definition.

One standard, [ISO9126], has identified several software quality characteristics: portability, efficiency, reliability, functionality, usability, and maintainability. In this section, completeness and correctness are equivalent to functionality as described in [ISO9126]. Understandability is substituted for usability in this section because the quality characteristics used in this report are meant to be applied to individual software products and not the only final software system.

The other quality characteristics in the following list have been extracted from several documents, including [AFSCP800-14] and [CONTE]. There is no standard set of quality characteristics for assessing software which is widely accepted. The list below is an aggregation of the most common quality characteristics.

adaptability

-- The ease with which software can be modified to meet new requirements. [GPALS] [MODIL]

-- The ease with which software allows differing system constraints and user needs to be satisfied. [MODIL]

-- (Flexibility) The ease with which a system or component can be modified for use in applications or environments other than those for which it was specifically designed. [IEEEGLOSS]

-- The ease with which software can accommodate to change. [NSWC]

completeness

-- The degree to which the component implements all required capabilities. [GPALS]

-- Contains all references and required items. [SOFTECH]

correctness

-- (1) The degree to which a component is free from faults in its specification, design, and implementation; (2) The degree to which a component meets specified requirements or user needs and expectations; (3) The ability of a component to produce specified outputs when given specified inputs, and the extent to which they match or satisfy the requirements. [NISTIR4909]

-- (1) The degree to which a system or component is free from faults in its specification, design, and implementation; (2) The degree to which software, documentation, or other items meet specified requirements; (3) The degree to which software, documentation, or other items meet user needs and expectations, whether specified or not. [IEEEGLOSS] Strict adherence to specified requirements. [NSWC]

efficiency

-- The degree to which a component performs its designated functions with minimum consumption of resources. [IEEEGLOSS]

-- A set of attributes that bear on the relationship between the level of performance of the software and the amount of resources used, under stated conditions. [ISO9126]

generality

-- The breadth of applicability of the component. [GPALS]

-- The degree to which a system or component performs a broad range of functions. [IEEEGLOSS]

maintainability

-- The ease with which a component can be modified to correct faults, improve performance or other attributes, or adapt to a changed environment. [IEEEGLOSS]

-- The ease with which software can be maintained, for example, enhanced, adapted, or corrected to satisfy specified requirements. [FIPS106]

-- Modifiable with minimal impact. [SOFTECH]

-- The ease with which corrections can be made in response to recognized inadequacies. [NSWC]

-- A set of attributes that bear on the effort needed to make specified modifications. [ISO9126]

-- Degree to which perfective, adaptive and corrective changes can be cost effectively made to the component. [SAC2]

modularity

-- The degree to which a system or computer program is composed of discrete components such that a change to one component has minimal impact on other components. [IEEEGLOSS]

-- The way the component is decomposed into sub- components. [GPALS]

portability

-- The ease with which a system or component can be transferred from one hardware or software environment to another. [IEEEGLOSS]

-- The extent to which a module originally developed on one computer or operating system can be used on another computer or operating system. [MODIL]

-- Operating environment independence. [GPALS]

-- Platform independence. [SOFTECH]

-- The ease in transferring software to another environment. [NSWC]

-- A set of attributes that bear on the ability of software to be transferred from one environment to another. [ISO9126]

-- Ease with which a software component can be implemented in new applications and virtual machine environments. [SAC2]

reliability

-- The ability of a component to perform its required functions under stated conditions for a specified period of time. [IEEEGLOSS]

-- The error-free use of software over time. [NSWC]

-- Low error rate. [SOFTECH]

-- A set of attributes that bear on the capability of software to maintain its level of performance under stated conditions for a stated period of time. [ISO9126]

-- Expected time between failures while running the application in it's operational environment. [SAC2]

understandability

-- The degree to which the meaning of a software component is clear to a user. [NISTIR4909]

-- (Clarity, Self-Descriptiveness) Ease of comprehending the meaning of the software (opposite of complexity). [GPALS]

-- Low complexity and Documentation. [SOFTECH]

The definitions in this section are definitions of terms used throughout this report. Several of the terms defined here are characteristics of software products which can be measured (e.g. cohesion and complexity.) These characteristics are used to further define the quality characteristics from section 2.1 in sections 3.1 through 3.5.

application domain

-- The knowledge and concepts that pertain to a particular computer application area. [STARS]

-- An identifiable area or subarea of an organization's software development activities in which similar software requirements occur. [GPALS]

cohesion

-- The binding of statements within a software component. [NSWC]

-- The manner and degree to which the tasks performed by a single software module are related to one another. [IEEEGLOSS]

-- The degree to which the functions or processing elements within a module are related or bound together. [FIPS106]

-- The degree to which a component's structure is unified in support of its function. [MODIL]

cohesiveness

-- The degree to which a component's structure is unified in support of its function. [GPALS]

complexity

-- An abstract measure of work associated with a software component. [NSWC]

-- The degree to which a system or component has a design or implementation that is difficult to understand and verify. [IEEEGLOSS]

-- The degree of complication of a system or system component, determined by such factors as the number and intricacy of interfaces, the number and intricacy of conditional branches, the degree of nesting, the types of data structures, and other system characteristics. [FIPS106]

component

-- One of the parts that make up a system. A component may be hardware or software and may be subdivided into other components. [IEEEGLOSS]

coupling

-- The interdependence among software components. [NSWC]

-- The manner and degree of interdependence between software modules. [IEEEGLOSS] [MODIL]

-- (1) The degree of data or control connectivity between different components of a software system; (2) A measure of the strength of interconnection between one component and another. [MODIL]

-- (Interconnectivity) The degree of connectivity between different components. [GPALS]

-- The degree that modules are dependent upon each other in a computer program. [FIPS106]

domain

-- An area of activity or knowledge. [STARS]

-- A group or family of related systems that share a set of common capabilities and/or data. [SOFTECH]

-- A distinct functional area that can be supported by a class of software systems with similar requirements and capabilities. [MODIL]

expandability

-- (Augmentability) The ability to support expansion of data-storage requirements. [GPALS]

-- (Extendibility, Extensibility) The ease with which a system or component can be modified to increase its storage or functional capacity. [IEEEGLOSS]

-- (Extensibility) The extent to which a component allows new capabilities to be added and existing capabilities to be easily tailored to user needs. [MODIL]

fault

-- (Defect) An incorrect step, process, or data definition in a computer program. [IEEEGLOSS]

functionality

-- A set of attributes that bear on the existence of a set of functions and their specified properties. The functions are those that satisfy stated or implied needs. [ISO9126]

readability

-- The difficulty in understanding a software component. [NSWC]

reusability

-- The degree to which a component can be used in more than one software system, or in building other components, with little or no adaptation. [MODIL]

robustness

-- The degree to which a system or component can function correctly in the presence of invalid inputs or stressful environmental conditions. [IEEEGLOSS]

-- Measure of the breadth of the new problem domain addressed by the current system, its subcharacteristics are generality and expandability. [SAC]

-- Ability to produce correct results despite input errors. [GPALS]

testability

-- The ability to evaluate conformance with requirements. [NSWC]

-- Equipped with test plans. [SOFTECH]

traceability

-- The ease in retracing the complete history of a software component from its current status to its requirements specification. [NSWC]

-- (1) The degree to which a relationship can be established between two or more products of the development process, especially products having a predecessor-successor or master-subordinate relationship to one another; for example, the degree to which the requirements and design of a given software component match; (2) The degree to which each element in a software development product establishes its reason for existing; for example, the degree to which each element in a bubble chart references the requirement that it satisfies. [IEEEGLOSS]

-- The characteristic of software systems or designs or architectures or domain models that identifies and documents the derivations path (upward) and allocation/flowdown path (downward) of requirements or constraints. [STARS]

unit

-- (1) A separately testable element specified in the design of a computer software component; (2) A logically separable part of a computer program; (3) A software component that is not subdivided into other components. [IEEEGLOSS]

usability

-- A set of attributes that bear on the effort needed for use, and on the individual assessment of such use, by a stated or implied set of users. [ISO9126]

This section suggests metrics for each quality characteristic associated with software products (requirements documentation, design documentation, code, test documentation). Each product has several quality characteristics, along with metrics to be used in assessing the quality characteristic for the product.

The analyst must identify the objectives to be achieved from collecting and analyzing metric data. It must be possible to collect the data, and methods should exist for analyzing the data. The analyst must present the results such that recommendations can be made concerning the reusability of the software. Obviously, optimum values for all metrics cannot be achieved simultaneously. Selection criteria for requirements for metrics include tradeoffs between the quality characteristics and their priorities. For example, efficiency and understandability often are in conflict with each other; efficient code may be necessarily coded in an assembly language, which reduces the understandability (and hence, maintainability) of the software.

Another selection criterion is the ability to collect the required data for the metric. The simplicity of data collection is one reason lines-of-code remains a popular metric despite its deficiencies. Many other metrics have data collection requirements that can be automated, and therefore, may be less costly to implement. Examples include defect counts and some complexity metrics. Other metrics require more human involvement, such as requirements tracing, function point analysis, and test coverage.

Section 3.1 identifies the metrics for quality characteristics that may be applied to all software products, with minor adjustment. Sections 3.2 through 3.5 identify metrics for the quality characteristics of products of requirements, design, code, and test.

The metrics for the quality characteristics in this section can be applied to all software products. Most of these metrics are primitive in the sense that they are simple counts of problem report values. Cause and effect graphing and RELY (3) require more analysis of the products.

3.1.1 Completeness

cause and effect graphing [IEEE982.1]

Cause and effect graphing aids in identifying requirements that are incomplete and ambiguous. Also, it explores inputs and expected outputs of a program and identifies ambiguities. For test purposes, it is useful for evaluating test cases for probability of detecting faults.

Primitives:

List of causes: distinct input conditions
List of effects: distinct output conditions or system transformations
A existing = number of ambiguities remaining
A tot = total number of ambiguities identified

The requirements are analyzed and the requirements are broken down into specific causes and effects. Each cause and each effect is assigned a unique node identifier. A Boolean graph is made by connecting the cause and effect nodes based on the semantic content of the requirements. Ambiguities are defined as any cause with no effect, and effect with no cause, and any combination of cause and effects that are inconsistent with the requirements or are impossible to achieve.

Calculation of Cause/Effect Completeness:

When no ambiguities exist, and all causes and effects are represented, then the CE measure is 100%. See [IEEE982.2] for use of this metric.

3.1.2 Correctness

number of problem reports per phase, priority, category, or cause [NIST500-209]

number of reported problems per time period [NIST500-209]

number of open real problems per time period [NIST500-209]

number of closed real problems per time period [NIST500-209]

number of unevaluated problem reports [NIST500-209]

age of open real problem reports [NIST500-209]

age of unevaluated problem reports [NIST500-209]

age of real closed problem reports [NIST500-209]

rate of error discovery [NIST500-209]

3.1.3 Reliability

RELY - Required Software Reliability [IEEE982.1]

RELY can be used to provide a reliability rating for a product through examination of the processes used to develop it. At the early planning phases of a project, RELY can be used to measure the trade-offs of cost and degree of reliability. RELY provides ratings very low, low, nominal, high, and very high. See [IEEE982.2] for a table of reliability factors for requirements and product design, detailed design, code and unit test, and integration and test.

Because requirements documentation is usually written in human readable format, the metrics that have been defined for requirements typically require manual analysis. The data needed for many of the metrics must be gathered by hand (i.e. counting requirements). Many of the metrics are subjective in nature, and it is up to the analyst to decide what is a "good" value. For example, the analyst must determine the acceptable readability level of the documents.

Many of the metrics for quality characteristics in this section can be used for other products. The readability metrics may be applied to preliminary and software system design documents, for example.

3.2.1 Completeness

completeness metric [IEEE982.1] [AFSCP800-14]

This metric uses eighteen primitives (number of functions not satisfactorily defined, number of functions, number of functions not used, etc.). Ten derivatives are defined (ratio of functions satisfactorily defined to functions defined, ratio of defined functions used to defined functions, etc). The completeness measure (CM) is calculated as the weighted sum of the ten derivatives expressed as: (See [IEEE982.2])

requirements traceability [IEEE982.1]

Aids in identifying requirements that are missing from, or in addition to, the original requirements. R1 is number of requirements met by the architecture. R2 is the number of original requirements. The traceability measure (TM) is computed:

                                       R1
                                  TM = -- X 100%
                                       R2

deviation between planned number of System/Segment Design Document (SSDD) software requirements to be documented in the Software Requirements Specification (SRS) and actual number of SSDD software requirements completely documented in the SRS [AFP800-48]

Document Relationships [NSWC2]

Document relationship is defined as "the structure among and within the individual documents of a set of documentation, which ties the documents together into a single, unified set." There are two categories: Decompositional and Referential.

Decompositional relationship is the natural, hierarchal ranking of a set of documentation. Documents at a lower level in the hierarchy give more detail on a given subject. For completeness, there should exist a definite or obvious hierarchial ranking among the documents.

Referential relationships are the links between documents; i.e. a specific cited reference to another document or another section in the same document. Appropriate references to another documents (or within the same document) should be maintained.

3.2.2 Correctness

number of discrepancies as a result of each review [STEP]

number of conflicting requirements [IEEE982.1]

requirements compliance [IEEE982.1]

System Verification Diagrams (SVDs) are used to detect inconsistencies, incompleteness, and misinterpretations in the requirements. Requirement errors detected using the SVDs are:

• N 1 = Number of errors due to inconsistencies
• N 2 = Number of errors due to incompleteness
• N 1 = Number of errors due to misinterpretation

requirement errors reported / total number of requirements [GPALS2]

requirement errors corrected / total number of requirements [GPALS2]

number of requirements faults and structural design faults detected during detailed design [NIST500-209]

3.2.3 Generality

size of the application domain [STARS]

Reverse engineering performed on the requirements documentation is useful in identifying common design and architectural features of existing systems. Reusable requirements specifications must define the boundaries of the problem space, as well as a set of variability descriptions. Requirements documentation can then be assessed for application as subdomains of other domain areas.

3.2.4 Understandability

size metrics
- number of requirements [NIST500-209]
this is often expressed as the number of shalls
- number document pages [AMI] [NIST500-209]
- number of document words [NIST500-209]
- number of functions [NIST500-209]

readability metrics

- number of grammatically incorrect statements

- number of misspellings

- readability indices such as Flesch-Kincaid, Gunning's Fog Index [MURRAY]

These readability formulas are based on sentence length and polysyllable frequency. Gunning's Fog index is used to produce a grade level required for reading the document. For example, an index of 12 means that 12 years of education is required to understand the document.



- physical readability [NSWC2]

Four indicators of physical readability are given: format appropriateness, adequacy of print, format consistency, and module appropriateness.

Format appropriateness assesses the suitability of the presentation style or layout. For example, numeric data should be presented in a table or graph.

Adequacy of print includes quality of reproduction, font sizes, paper quality, and font styles. Assessment can be done by sampling document pages.

Format consistency means that tables should all have the same format and indexes should all have the same format. The same rule should be applied to Tables of Contents, chapters, sections, etc. Again, sampling is used to provide measurements.



Module appropriateness measures the suitability of the physical division of the chapters, sections, paragraphs, tables, figures, and so forth. The assessment is whether the document is physically divided in a manner consistent with the logical division of the material.

complexity

- function points [NIST500-209] [SQE] [SYMONS]

Function point analysis was developed by Alan Albrecht at IBM in the 1970's, and furthered refined by Albrecht and others. As a measurement of size, function points are language independent (as opposed to lines-of-code), and can be applied to requirements specifications. Applying to requirements allows estimates of size and complexity earlier in the life cycle.

The first part of calculating function points is determining information processing size, measured in unadjusted function points (UFPs). The application is divided into five types of components: inputs, outputs, enquiries (combinations of inputs and outputs that give immediate results), interfaces to other applications, and logical internal files. Each component is classified as simple, average, or complex, and is assigned a simple number of weighting points. The UFP count for the system is the sum of the individual UFPs.

Second, a technical complexity adjustment (TCA) is determined by estimating the degree of influence of general application characteristics, such as data communications, performance, on-line update, etc. TCA is calculated: TCA = 0.65 + 0.01 X DI

where DI is the total degree of influence.

Finally, the function point (FP) count is computed as: FP = UFP x TCA


- Mk II Information Processing Size [SYMONS]

This metric looks evaluates the system as a collection of logical transactions, each consisting of input, process, and output components. A logical transaction is defined as a unique input/process/output combination triggered by a unique event, or a need to retrieve information. Mk II Information Processing Size also uses Unadjusted Function Points (UFPs) calculated with the formula:
UFP's = WI X (no. of input data element-types) + WE X (no. of entity-types referenced) + WO X (no. of outputdata element-types)

where:

WI = weighting factor for input data element-types
WE = weighting factor for entity-types
WO = weighting factor for output data element-types


The weighting factors can be determined by calibration.

Next, the technical complexity adjustment (TCA) is calculated: TCA = 0.65 + C X DI

where:

DI = the total degree of influence
C = coefficient obtained by calibration

Finally, the MkII Function Points is calculated as UFP x TCA.

Software design documentation is often divided into three activities: functional allocation, software system design, and unit design. These software design activities occur in three chronological phases: preliminary design, detailed design, and unit design [CARD].

The first phase is preliminary design. Designers collect related requirements into functional groups and identify dependencies among functions. The preliminary design may be represented by data flow diagrams, high-level structure charts, or a simple list of requirements by subsystems [CARD]. The choice of metrics depends on the representation used. Requirements traceability can be applied to any representation to verify that requirements are being met. Data flow complexity can be applied to data flow diagrams to evaluate the understandability of the diagrams.

Next comes detailed design, where the overall architecture of the software system is defined. This step allocates data and functions to individual units or design parts. Internal interfaces must also be specified at this stage [CARD]. A structure chart is commonly used to represent the system design. Some of the metrics applied to this type of design are data or information flow complexity, external (De) complexity, fan-in/fan-out per module, graph-theoretic complexity, and readability metrics.

The final design phase is unit design. In this phase, algorithms and data structures are defined. Application and implementation specific information is added to the design. The design itself is often represented as pseudo-code and module prologues [CARD]. Nearly all of the metrics specified below can be applied to unit design documents. Some metrics specific to unit designs are internal (Di) complexity, document lines of code, and number of states per parameter.

The unit design phase is the first indication of the reusability of individual modules. For example, the complexity of a module can be determined by its design, before any coding is done. Many of the same metrics that are typically applied to code can be applied to unit designs. Modularity can often be assessed at the unit design level. See [CARD] for references to studies on heuristics for achieving modularity. These heuristics are small modules, limited data coupling, medium span of control, and singleness of purpose. Modularity is a good indicator of the reusability of an individual software module.

3.3.1 Completeness

requirements traceability [IEEE982.1] (Applied to all designs)

For design, this metric indicates the percent of requirements that have been documented in the design. See Quality Metrics for Requirements Documentation (section 3.2) for a description of calculating this metric. See [IEEE982.2] for a complete description of this metric.



deviation between planned number of Software Requirements Specification (SRS) requirements to be documented as Computer Software Components (CSC) into the Software Design Document (SDD) and actual number of SRS requirements completely documented as CSCs in the SDD [AFP800-48] (Applied to system design)

3.3.2 Correctness

defect density [IEEE982.1] [NIST500-209] (Applied to unit design)

For design, the defect density is calculated after each design inspection of new development or large block modifications. A structured design language is assumed. See [IEEE982.2] for a description on using this metric.

Primitives:

Di = total number of defects found during ith design inspection
I = total number of inspections to date
KSLOD = number of source lines of design statements in thousands

Calculation of Defect Density:

Design defect density may also be measured in terms of design defects/KSLOC [SPC]. Each defect from later phases that is determined to be a design defect is counted.



number of structural (architectural) design faults detected during detailed design [NIST500-209] (Applied during unit design)

number of design faults associated with each module [NIST500- 209] (Applied to system and unit design)

number of integration test cases planned/executed involving each module [NIST500-209] (Applied to system and unit design)

number of black box test cases planned/executed per module [NIST500-209] (Applied to system and unit design)

number of design errors reported / total number of units [GPALS2] (Applied to all)

number of design errors corrected / total number of units [GPALS2] (Applied to all)

3.3.3 Generality

size of application domain [STARS] (Applied all designs)

Reverse engineering can be done on system designs to identify data structures and data management patterns to aid in separating the functionality into two categories: functionality that supports general domain concepts and functionality that achieves a computer-based solution.



Reverse engineering can be done on unit designs to identify the modularization, relationship among structural elements, declare/set/use patterns for variables, control flow within structural elements, and scoping information. The information extracted can be used to identify solution concepts and their interrelationships. Also, requirement choices can be connected with design choices and the effect of performance, timing, sizing, and functionality.

3.3.4 Efficiency

The percent metrics that follow can be applied to system and unit designs if the target capacity for CPU and I/O usage are known. Likewise, if the target random access memory (RAM) and storage capacities are known, then percent usage calculations may be made. These estimates may then be used to assess the efficiency of the design and its reusability in new environments. However, comparisons made between environments must take into account processor speeds, I/O devices, and operating system effects.



target CPU usage as percent of capacity [STEP]

target I/O usage as percent of capacity [STEP]

target upper bound storage usage [STEP]

percent actual of target upper bound storage usage [STEP]

target upper bound RAM usage [STEP]

percent actual of target upper bound RAM usage [STEP]

3.3.5 Modularity

cohesion metric [NIST500-209] [SQE] [CARD] (Applied to unit design)

Cohesion, or module strength, refers to the relationship among the elements of a module. The cohesion value for a module is assigned using a standard rating chart, that can be found in [SQE]. The best cohesion level is functional, and the worst is coincidental.



A high-strength (functional) module performs one function. A low-strength (coincidental) module includes multiple unrelated functions. Studies referenced in [CARD] find that higher strength modules tend to have lower fault rates, and tend to cost less to develop. Also, the modularity is greater for higher strength modules.



The Software Measurement Guidebook [SPC] gives a strength metric proposed by Cruickshank and Gaffney. Strength is a measure of the degree of cohesiveness of the elements of a software module.



Calculation of Strength:

where:

X = reciprocal of the number of assignment statements in the module
Y = number of unique function outputs divided by number of unique function inputs





coupling [NIST500-209] [SQE] [CARD] (Applied to system and unit designs)

Coupling is a measure of the degree to which modules share data. Module coupling is rated using a standard rating chart, that can be found in [SQE]. Data coupling is the best type of coupling, while content coupling is the worst.



Data coupling is the sharing of data via parameter lists, while common coupling is the sharing of data via global (common) areas. Earlier recommendations stated that common coupling should be avoided. However, later studies have shown that the distribution of error rate does not depend on the coupling mechanism [CARD]. However, In terms of modularity and the modules independence of external factors, a lower coupling value is better.



A coupling metric given in [SPC] and originally proposed by Cruickshank and Gaffney is calculated as follows:

Calculation of Coupling:


Mj = sum of the number of input and output items shared between components i & j

Zi = average number of input and output items shared over m components with component i

n = number of components in the software product

3.3.6 Portability

number of features that are language-specific (Applied to unit design)

number of features that are operating system specific (Applied to unit design)

Features that are operating system, hardware, or interface specific should be isolated in modules that can then be changed for other platforms.

3.3.7 Reliability

cumulative failure profile [IEEE982.1]

For design, a graph is made of the cumulative failures in the software resulting from design deficiencies. The curve of the graph can be used to predict the reliability of the design. (Applied to unit design)

3.3.8 Understandability