|
Comments on Level 3 KPA Policies
(Prototype Versions)
Below are comments received on the prototype versions of the Level 3 KPA policies.
GENERAL COMMENTS APPLYING TO ALL POLICIES:
CLEVELAND- Jan 21, 99
- Change references from FSA/DSE to ITD/SEO.
- Change references from FSO to ISO.
- Be consistent about numbering or not numbering goals.
- Run spell check to catch words that are run together.
- Some paragraphs start on the same line as the heading and some don't. However it is done, it should be consistent within the policy and among all the policies.
COLUMBUS - Jan 10, 99
Word all policy statements using active voice and consistent language to describe responsibilities (either "will" or "shall") in the lead sentence. Right now there's a mix of narrative statements and directive statements, and responsibilities are often left to the end of the paragraph.
For example:
Appendix 2 (PM Policy). Refer to KC's earlier submission.
Appendix 9 (SPE Policy).
3c. "Tools to support SE tasks are available" --delete and replace
with the last sentence of that paragraph:
"SEO/IT Directors shall ensure that adequate tools are
available..."
Appendix 10 (IC Policy)
3a. Reword first sentence to read: "All affected groups will
review ...." instead of "The system requirements etc. are reviewed by all affected groups."
2. Wordsmithing changes (even though banned)
APPENDIX 9 (SPE), 2b.
Reword to "Software work products are kept consistent with each other" (delete "are produced..")
APPENDIX 11 (PR), 2.
Delete "plan and schedule peer review activities" so that the goal reads "The goal for peer reviews is to detect and remove errors as early as possible in the life cycle."
3. Content wise. Who can argue with the CMM?
KANSAS CITY - Jan 5, 99
General Comments on ISO Level 3 Policies.
- We feel that the intent of the policies can be carried out by the Level 3 prototype systems without the actual policy documents in place. Rather than publish the documents in their current form, we suggest rewording them to focus more on what the ISO hopes to accomplish rather than just restating the CMM goals and activities. These documents should state the directives from the ISO that will guide the future actions of the ITDs/SEOs. As currently stated, many are not specific in setting the ISOs directives of what should be accomplished. They blur the lines of policy and procedure, often getting into the mechanics of how the KPA activities will be accomplished. Rather than present these documents to the Senior Management sponsors at this time, we suggest reworking them and presenting them at a later time before the Level 3 efforts are rolled out to the rest of the organization.
- A separate policy for Peer Reviews (SM 25) may not be necessary. Peer Review activities could be addressed in the Software Quality Assurance policy (SM 13).
- A separate policy may not be necessary for "Institutionalization of Standard Software Processes" (SM 18). The Organization Process Focus and Definition policy (SM 21) seems to cover this.
Policy Statements
- The goal statements should be more than a simple restatement of the CMM Goals for each KPA. They should state what the policy hopes to accomplish for the ISO, not the general goals of a KPA.
- The policy statements should be directive in nature. Again, these should contain more than just the wording of the KPA Activities within the CMM. They should be specific statements of what the ISO plans to address in these areas.
- The policy statements contain too much in the way of procedural language. The policy statements should state what is to be done. The mechanics of how things will be accomplished should be stated in organizational processes like the SSP.
- The responsibilities for action are buried within the language of the policy statements. These should be explicitly stated in a separate section.
The following are examples of the types of changes we would make to all of the Level 3 policy documents:
SM-12 Project Management Activities:
We suggest:
1. Rewording the goals as follows:
- Establish project management requirements for planning and tracking the software projects developed by the ISO. These requirements include:
- documenting software estimates
- planning and documenting project activities and commitments
- documenting agreement between all groups and individuals affected by the software project
- tracking progress and results against the documented plans
- taking corrective action when results deviate from the documented plans
- Establish the use of the System Modification Scenario (SSP) as the standard upon which all software project's tailor local software development processes.
- Establish the use of the locally tailored software development process as the standard by which software releases are planned and managed.
2. Rewording the policy statements, as follows:
- Keep the current statement #1.
- Systems requirements allocated to software will be used by each AIS as the basis for their software plans, work products and activities.
- Each AIS will create a Software Development Plan (SDP) for each release in accordance with standard S-PM-017 as outlined in the ISO's Standard Software Process (SMS, release 5). This plan will include:
- estimates of the size, cost and effort for the release
- critical computer resources needed for the release
- a schedule, including critical paths and dependencies
- identification and mitigation plans for risks associated with the release
- commitments negotiated for the release
- The AIS's SDP will be used to manage and track the activities for each software release. Corrective action will be taken by the AIS Project Officer when actual status differs from the SDP.
- Each AIS will document the commitments for a release. All affected parties will sign the release commitment.
- Each AIS will tailor a project specific software development process from the SMS (SSP) in accordance with the tailoring guidelines outlined in the SSP.
- Each AIS will collect and report metric data as outlined in ISO policies SM-02, SM-06 and SM-07.
- Each AIS will collect technical and management lessons learned for each release and report them to the ISO for inclusion in the organization's Process Asset Library (PAL).
3. Adding a responsibility section that outlines the responsibilities for the ISO, ITD/SEO Directors and AIS Project Officers.
SM-25 - Peer Reviews Policy
We suggest:
1. Rewording the goals as follows:
- Establish the requirement for all AISs to perform peer reviews for software work products.
- Establish the use of the System Modification Scenario (SSP) as the standard upon which all peer review activities are based.
2. Rewording the policy statements as follows:
- Peer reviews will be conducted on software work products as outlined in the Systems Modification Scenario (SSP).
- Peer reviews will be led by trained peer review leaders.
- Peer reviews will focus on the work product and not on the individual(s) who produce the product.
- Measures on the number, conduct and results of the peer reviews shall be collected and recorded in accordance with the standard software process metrics specifications.
3. Add a section for Responsibilities which would list responsibilities for the ISO, ITD/SEO Directors, AIS Project Officer and Peer Review Leaders.
PENSACOLA - Dec 30, 98
The Level 3 ISO Policies ( 21,22,23,24,25) are closely patterned after the Capability Maturity Model. While this probably satisfies any policy requirements, it does not do much for establishing a common framework that can be used across the DFAS software engineering sites.to draw them into common software engineering business processes.
Throughout the policies, editing should be done to replace references to FSO with ISO and FSAs with ITDs or SEOs.
The policies read as if all employees were directly employed by the ISO. They do not seem to recognize the relationship between DFAS, ISO, ITDs and SEOs. For instance, in Policy Number 22 (Training) there is no mention of DFAS Resource Management, which plays a significant role in the training arena.
SPECIFIC COMMENTS:
SM-11 - REQUIREMENTS MANAGEMENT
Indianapolis - Feb 9, 99
Pensacola - Feb 9, 99
SM-12 - PROJECT MANAGEMENT ACTIVITIES:
Cleveland - Jan 21, 99
Under References - Suggest that #1 be changed to read "SEI Capability Maturity Model for Software, version 1.1" since there are other CMM versions and types developed and being developed.
Columbus - Feb 10, 99
Indianapolis - Feb 9, 99
Kansas City - Jan 5, 99
SM-13 - SOFTWARE QUALITY ASSURANCE
Indianapolis - Feb 9, 99
SM-20 - SOFTWARE SUBCONTRACT MANAGEMENT
Indianapolis - Feb 9, 99
Pensacola - Feb 9, 99
SM-21 - ORGANIZATION PROCESS FOCUS AND DEFINITION
Cleveland - Jan 21, 99
1. Under Policy Statement #8, 2nd sentence - Change "...submit it the their ..." to "...submit it to their...".
2. Under Policy Statements #11 - Suggest the last sentence be deleted. The SPI WT is not responsible for the coordination of the activities at each members site. The local SEPG has that responsibility.
3. Under Policy Statements #13 - Suggest that the entire section be deleted. Issue already stated under #3.
4. Under References - Suggest that #1 be changed to read "SEI Capability Maturity Model for Software, version 1.1" since there are other CMM versions and types developed and being developed.
5. Tailoring Guidelines Attachment, 1st sentence - The ISO standard processes are not and should not be influenced by customer and/or unique business requirements. Customer demands and/or unique business requirements influence the AIS's tailoring of the standard process for any particular project. Suggest that the first sentence be deleted. This information is covered in the following sentences concerned with the tailoring of the standard processes as it should be.
The standard processes are influenced by the current project management capability models and industry standards. I have no problem with this being substituted if so desired.
6. Tailoring Guidelines Attachment, Section #4, Approval procedures - This section is referenced for the approval of deviations from the standard process. Where does the Waiver process come into play when approving deviations. It is not mentioned here.
Columbus - Feb 22, 99
1 SPI Org structure -- this chart needs to be updated. Anita has a few drafts available to use.
2 Maintenance of SSP. The first paragraph is confusing.
a. Does the corporate SEPG in fact develop and maintain the
standard software processes? I would suggest "manage" or "oversee" language.
b. Definition of terms is mixed together with the process.
Suggest reorganizing to address definition/content of SSP first, the process second.
c. I see no need to address the planning steps for the
integration teams here (i.e., "plan tasks, dates for assigned objectives", etc.), or to bring in other SPI objectives. Aren't we just dealing with the SSP here?
Suggest a rewrite of para. 1 and 2 for clarity to read something like this (use bullets for the scenarios);
The ISO SEPG manages the development and maintenance of DFAS standard software processes. These standard processes are categorized as implementable scenarios, which align with life cycles.
Scenarios currently in use are:
= System Development Scenario (or development life cycle)
= System Modification Scenario (modification/maintenance life cycle) = System Operations Scenario (addresses emergency system modification)
= Software Subcontract Management Scenario (managing subcontracts)
Scenarios are developed and maintained in accordance with the Standard Software Process Standard (see Attachment 4). Each scenario contains tasks that satisfy the SEI's CMM. A glossary will also be developed and maintained as a companion to the scenarios.
The ISO SEPG may assemble ad hoc or virtual teams from throughout DFAS, as required, to focus on defining or modifying scenario content. The ISO SEPG and/or these teams plan and perform process integration, which involves merging candidate processes into an implementable scenario. Each implementable scenario will be submitted to the Corporate Steering Committee (CSC) for review and approval.
Issue for discussion: The third paragraph states that the SPI WT will make recommendations about scenario changes to the CSC on a quarterly basis. We've never done that -- have there been any changes to consider yet? Is there a current backlog? Is a quarterly review appropriate? feasible? I'm not sure how this fits together with the activity of ad hoc/virtual teams described in the preceding paragraphs---or if you're describing 2 different avenues for change.
3. PAL submissions. This seems to describe what's happening now.
4. Standard. Last paragraph says that procedures should be identified as DFAS standard etc. Is this being done?
Also, a process is defined here for best practices being nominated by a director to the Corporate SEPG, for submission to the CSC. There is no mention of the SPI WT here. Jim has asked our input on how this process should work -- Should the SPI WT play a role in this? (do we want any placeholder language?)
Denver - Feb 22, 99
I generally agree with all the commentary I have reviewed so far and suggest these additions:
1. The goals in 2a thru 2d do a pretty good job of covering the CMM goals for the associated KPAs with the possible exception of OPF Goal 3, "Org level process development and improvement activities are planned." You might compensate for this by changing goal 2a to say "Continuous software process improvement is planned and supported by a ...".
2. In 3a(4), would this statement be less imperative if it read "Assist in coordinating software process improvement activities with individual SEOs/I&Ts." Right now it sounds like the Corp SEPG may also settle differences. If that was intended - hey, I can be settled.
3. In 3d, delete last sentence - unnecessary, and is up to the respective SEO/I&T.
4. Strongly agree that the attachments need numbering.
5. In 3k, perhaps I missed something, but I couldn't find any mention of the SPI WT in E.C6.2.2.7 (Assessment.).
Indianapolis - Feb 23, 99
GENERAL COMMENTS:
1. There appears to be a transition in progress between the terminology for "Standard Software Process" (SSP) and "System Modification Scenario" (SMS). There are a few places within the Policy on OPF/OPD where the one is referred to and the other is meant. This also seems to be the case on the PAL where the SMS, version 5 contains the SSP and, when adopted, will replace earlier versions of the SMS which were distinct from the SSP.
2. Appendix 7 refers to metrics in at least a couple of places (e.g., E.C6.2.d and E.C6.3.a.(3)). This is not the place or the time to discuss metrics since this could distract us from the purpose of refining the Policy on OPF/OPD. However, we do need to return at some future time to finish our discussion and definition of Metrics Goals.
3. At several places within Appendix 7 reference is made to an attachment. There is reference to five (5) attachments. One document which was printed from the PAL had the following statement for the first (1st) attachment: "EMBED PowerPoint.Slide.4" and the second (2nd) attachment on the very next line below. References to attachments would be more clear if the attachments themselves were numbered in addition to displaying their titles.
SPECIFIC COMMENTS:
1. 3.d.: Delete the word "capability" from the statement, "Each SEO/I&T shall establish a local SEPG capability to coordinate..." so that it reads, "Each SEO/I&T shall establish a local SEPG to coordinate..."
2. 3.h: Replace the definite article "the" by the preposition "to" in the phrase, "AIS Managers shall develop a project training plan and submit it the their SEO/I&T Director for consolidation..." so that the phrase reads, "AIS Managers shall develop a project training plan and submit it to their SEO/I&T Director for consolidation..."
3. 3.k.: There is a typo in the reference to a subparagraph: the reference to E.C6.2.2.7 should be E.C6.2.3.7.
SM-22 - TRAINING PROGRAM
Cleveland - Jan 21, 99
The heading line for this policy is formatted with the name first then the SM number. Other policies are formatted SM number then the policy name. However it is done it should be consistent among the policies.
Columbus - Feb 10, 99
Indianapolis - Feb 9, 99
SM-23 - SOFTWARE PRODUCT ENGINEERING
Cleveland - Jan 21, 99
1. The scope states that this policy does not apply to projects that have waivers from the SPI Program. Aren't all projects whether actively improving processes or not expected to produce correct, consistent software products effectively and efficiently? If this product engineering policy doesn't apply to those with Waivers from the SPI program what ISO policy does?
2. Under Policy Statements #2, - Change the last two words to "organizational standards"..
3. Under References - Suggest that #1 be changed to read "SEI Capability Maturity Model for Software, version 1.1" since there are other CMM versions and types developed and being developed.
4. The Purpose heading does not have a period at the end as the rest of the headings do and there should be a space after it.
5. Some paragraphs start on the same line as the heading and some don't. However it is done it should be consistent within this policy
Columbus - Feb 10, 99
SM-24 - INTERGROUP COORDINATION
Cleveland - Jan 21, 99
Under References - Suggest that #1 be changed to read "SEI Capability Maturity Model for Software, Version 1.1" since there are other CMM versions and types developed and being developed.
SM-25 - PEER REVIEWS
Cleveland - Jan 21, 99
1. Under References - Suggest that #1 be changed to read "SEI Capability Maturity Model for Software, Version 1.1" since there are other CMM versions and types developed and being developed.
2. Under Policy Statement #5 - The first part of the first sentence is a little hard to read. It says that "Data on the conduct of the peer reviews shall be collected". What does this mean?
Columbus - Feb 10, 99
Indianapolis - Feb 9, 99
Kansas City - Jan 5, 99
Please direct questions/comments about this page to the DFAS PAL Administrator at pal.admin@dfas.mil, DFAS-TA/IN
| 
