Quality A supporting narrative must be provided if this factor is used.
For Systems contracts or orders, consider the following aspects of Contractor performance:
Product Performance. Assess the achieved product performance relative to performance parameters required by the contract or order.
Systems Engineering. Assess the Contractor's effort to transform operational needs and requirements into an integrated system design solution.
Areas of focus should be: the planning and control of technical program tasks; the quality and adequacy of the engineering support provided throughout all phases of contract or order execution; the integration of the engineering specialties; management of interfaces; interoperability, and; the management of a totally integrated effort of all engineering concerns to meet cost, technical performance, and schedule objectives.
System engineering activities ensure that integration of these engineering concerns is addressed up-front and early in the design or development process. The evaluation should cover these disciplines: systems architecture; design; manufacturing; integration and support; configuration control; documentation, and; test and evaluation.
The evaluation for test and evaluation should consider successes, problems, and/or failures in developing test and evaluation objectives; planning (ground, air, or sea) test, simulations and/or demonstrations; in accomplishing those objectives and on the timeliness of coordination and feedback of the test results (simulations or demonstrations) into the design and/or manufacturing process.
Other activities include producibility engineering, logistics support analysis, supportability considerations (maintenance personnel and skills availability, or work-hour constraints, operating and cost constraints, allowable downtime, turn-around-time to service and maintain the system, and standardization requirements), survivability, human factors, reliability, quality, maintainability, availability, inspectability, etc.
The evaluation of systems engineering needs to remain flexible to allow the evaluator to account for program-unique technical concerns, and to allow for the changing systems engineering environment as a program moves through the program phases, e.g., Engineering and Manufacturing Development, Production.
Software Engineering. Assess the Contractor's success in meeting contract or order requirements for all applicable software engineering-based activities and processes.
Software engineering activities include, as appropriate: software development (design, code, and unit test); application of reuse, COTS, and other non-developmental software components; integration (including software component integration, system integration and test, and acceptance test support); and sustainment. Software processes include, for example: software size, effort, and schedule estimation; requirements analysis, development, and management; software configuration management; software risk identification and management; metrics collection and analysis, technical reviews, decision analysis, and software quality assurance and control, each as they specifically address software engineering activities.
Consider the Contractor's success with respect to:
- Planning a software development, integration, and testing effort that includes compatible cost, schedule, and performance baselines
- Delivering expected software driven capabilities on cost and on schedule
- Effective software metrics collection/analysis and status monitoring or reporting that provide the software visibility necessary to identify timely corrective actions and appropriately execute them
- Staffing with the software knowledge, skills, and abilities needed to execute the contract or order across the lifecycle; timely assignment of the appropriate numbers of software staff
- Awareness and control of software size and stability to enable tracking and allowing growth according to vetted enhancements vice scope creep
- Effective testing and integration of developed software within the larger system test and evaluation effort
- Effective processes to acquire, integrate, and test commercial and/or government off-the-shelf (COTS/GOTS) software and to achieve planned software reuse
- Achieving software assurance
- Consistent application of documented software engineering and management processes, including technical reviews, in alignment with contract or order requirements
Logistics Support. Assess the success of the Contractor's performance in accomplishing logistics planning. For example: maintenance planning; manpower and personnel; supply support; support equipment; technical provisioning data; training and support; computer resources support; facilities; packaging, handling, storage and transportation; design interface; the contractor's performance of logistics support analysis activities, and the Contractor's ability to successfully support fielded equipment. When the contract or order requires technical and/or engineering data deliverables, the cognizant cataloging and/or standardization activity comments should be solicited.
Product Assurance. Assess how successfully the Contractor meets program quality objectives (e.g., producibility, reliability, maintainability, inspectability, testability, and system safety, and controls the overall manufacturing process). The Project Manager or Contracting Officer must be flexible in how contractor success is measured (e.g., data from design test or operational testing successes, field reliability and maintainability and failure reports, user comments and acceptance rates, improved subcontractor and vendor quality and scrap and rework rates). These quantitative indicators may be useful later, for example, in source selection evaluations, in demonstrating continuous improvement, quality and reliability leadership that reflects progress in total quality management. Assess the Contractor's control of the overall manufacturing process to include material control, shop floor planning and control, status and control, factory floor optimization, factory design, and factory performance.
Other Technical Performance. Assess all the other technical activity critical to successful contract or order performance. Identify any additional evaluation aspects that are unique to the contract or order, or that cannot be captured in another sub-element.
For Non-Systems contracts or orders, consider the following aspects of Contractor performance:
Assess the Contractor's conformance to contract or order requirements, specifications and standards of good workmanship (e.g., commonly accepted technical, professional, environmental, or safety and health standards). Include, as applicable, information on the following:
- Are reports and data accurate?
- Does the product or service provided meet the specifications of the contract or order?
- Does the contractor's work measure up to commonly accepted technical or professional standards?
- What degree of Government technical direction was required to solve problems that arise during performance?
For Operations Support: Assess how successfully the Contractor meets program or project quality objectives such as producibility, reliability, maintainability and inspectability. The Assessing Official must be flexible in how contractor success is measured; (e.g., using data from field reliability and maintainability and failure reports, user comments and acceptance rates, and scrap and rework rates). These quantitative indicators may be useful later, for example, in source selection evaluations, in demonstrating continuous improvement, quality and reliability leadership that reflects progress in total quality management. Assess the Contractor's control of the overall production process to include material control, shop planning and control, and status.
For Architect-Engineer contracts or orders, consider the following aspects of Contractor performance:
Quality reflects the Contractor's management of the quality control program, as well as the quality of the work itself. Questions which should be addressed are as follows: Has a quality product been provided? Specifically, describe the quality, and the Contractor's quality control system responsible for it. For example:
- Ability to maintain quality control
- Ability to address and review comments
- Independent Technical Review
- Whether plans are coordinated with specifications
- Coordination between disciplines
- Compliance with design criteria
To support the assigned rating, the Assessing Official Comments should contain sufficient comments, based on supporting documentation, and include successes and failures, as well as specific corrective actions, as appropriate.
For Construction contracts or orders, consider the following aspects of Contractor performance:
Quality reflects the Contractor's management of the quality control program, as well as the quality of the work itself. Questions which should be addressed are as follows: Has a quality product been provided? Specifically describe the Contractor's quality control system responsible for it. For example:
- Performance of accessory testing
- Implementation of 3-phase inspection process
- CQC (Contractor Quality Control) documentation
- Identification and correction of deficient work
- Reviews of materials and shop drawings
- Whether there was incorporation of unspecified materials