System Analysis

M A N A G I N G THE D E V E L O P M E N T OF LARGE S O F T W A R E SYSTEMS

Dr. Winston W. Rovce

I N T R O D U C T I O N l am going t o describe my pe,-.~onal views about managing large software developments. I have had

various assignments during the past nit,.: years, mostly concerned w i t h the development of software packages

for spacecraft mission planning, commanding and post-flight analysis. In these assignments I have experienced

d i f f e r e n t degrees o f successwith respect to arriving at an operational state, on-time, and w i t h i n costs. I have

become prejudiced by my experiences and I am going to relate some o f these prejudices in this presentation.

COMPUTER PROGRAM D E V E L O P M E N T F U N C T I O N S There are t w o essential steps c o m m o n to all c o m p u t e r program developments, regardless o f size or

c o m p l e x i t y . There is first an analysis step, f o l l o w e d second by a coding step as depicted in Figure 1. This sort

of very simple i m p l e m e n t a t i o n concept is in fact all that is required if the e f f o r t is sufficiently small and if the

final product is to be operated by those w h o b u i l t it – as is t y p i c a l l y done w i t h c o m p u t e r programs for internal

use. It is also the kind of development e f f o r t for w h i c h most customers are happy to pay, since both steps

involve genuinely creative w o r k which d i r e c t l y contributes to the usefulness of the final product. A n

i m p l e m e n t a t i o n plan to manufacture 13rger software systems, and keyed o n l y to these steps, however, is d o o m e d

• t o f a i l u r e . Many additional development steps are required, none c o n t r i b u t e as d i r e c t l y to the final product as

analysis and coding, and all drive up the development costs. Customer personnel t y p i c a l l y w o u l d rather not pay

for them, and development personnel w o u l d rather not implement them. The prime f u n c t i o n of management

is to sell these concepts to b o t h groups and then enforce compliance on the part of development personnel.

A N A L Y S I S

C O D I N G

Figure 1. I m p l e m e n t a t i o n steps to deliver a small c o m p u t e r program f o r internal operations.

A more grandiose approach to software development is illustrated in Figure 2. The analysis and coding

steps are still in the picture, but they are preceded by t w o levels o f requirements analysis, are separated by a

program design step, and f o l l o w e d by a testing step. These additions are treated separately f r o m analysis and

coding because they are distinctly d i f f e r e n t in the way they are executed. They must be planned and staffed

d i f f e r e n t l y for best u t i l i z a t i o n of program resources.

Figure 3 portrays the iterative relationship between successive development phases for this scheme.

The ordering o f steps is based on the f o l l o w i n g concept: that as each step progresses and the design is further

detailed, there is an iteration w i t h the preceding and succeeding steps but rarely w i t h the more remote steps in

the sequence. The virtue of all o f this is that as the design proceeds the change process is scoped d o w n to

manageable limits. A t any p o i n t in the design process after the requirements analysis is completed there exists

a f i r m and c~seup~ moving baseline to whi(:h to ~ t u r n in the event o f unforeseen design difficulties. What we

have is an effective fallback position that tends to maximize the e x t e n t o f early w o r k that is salvageable and

preserved. Reprinted from Proceedings, IEEE WESCON, August 1970, pages 1-9. Co_pyright © 1_9_70 by The Institute of Electrical and Electronics Et)gineers,, .328 Inc. Originally published by TRW.

 

 

I SYSTE M

I ANALYSIS PROGRAM

DESIGN

I c o o , . o TESTING

I OPERATIONS

Figure 2. I m p l e m e n t a t i o n steps to develop a large computer program f o r delivery t o a customer.

I believe in this concept, but the i m p l e m e n t a t i o n described above is risky and invites failure. The

problem is illustrated in Figure 4. The testing phase which occurs at the end o f the development cycle is the

first event f o r w h i c h timing, storage, i n p u t / o u t p u t transfers, etc., are experienced as distinguished f r o m

analyzed. These phenomena are not precisely analyzable. They are not the solutions to the standard partial

differential equations of mathematical physics for instance. Yet if these phenomena fail to satisfy the various

external constraints, then invariably a major redesign is required. A simple octal patch or redo o f some isolated

code w i l l not f i x these kinds o f difficulties. The required design changes are likely to be so disruptive that the

software requirements upon which the design is based and w h i c h provides the rationale for everything are

violated. Either the requirements must be m o d i f i e d , or a substantial change in the design is required. In effect

the development process has returned to the origin and one can expect up to a lO0-percent overrun in schedule

and/or costs.

One might note that there has been a skipping-over o f the analysis and code phases. One cannot, of

course, produce software w i t h o u t these steps, but generally these phases are managed w i t h relative ease and

have little impact on requirements, design, and testing. In my experience there are w h o l e departments

consumed w i t h the analysis o f o r b i t mechanics, spacecraft a t t i t u d e determination, mathematical o p t i m i z a t i o n

of payload activity and so f o r t h , but when these departments have completed their d i f f i c u l t and c o m p l e x w o r k ,

the resultant program steps i n v o l v e a few lines of serial a r i t h m e t i c code. If in the e x e c u t i o n of their d i f f i c u l t

and complex w o r k the analysts have made a mistake, the correction is invariably implemented by a m i n o r

change in the code w i t h no disruptive feedback into the other development bases.

However, I believe the illustrated approach to be f u n d a m e n t a l l y sound. The remainder of this

discussion presents five a d d i t i o n a l features that must be added to this basic approach to eliminate most o f the

development risks.

329

 

 

I SYSTEM ! REQUIREMENTSIBI~

~ ” ‘ i so,w.,~ I

ANALYSIS

~1111~ I I pRI~OGRAM

~ l l l I CODING Ii

TESTING

OPERATIONS

Figure 3. Hopefully, the ~terat=ve interact=on between the various phases is confined to successive steps.

I SYSTEM “1 .~oo,.~-,..Sl.,~ I so,w..~ !.

I ANALYSIS

PROGRAM DESIGN

I coo,.G I , ~ ! TESTING I

I O .ATO.S ! Figure 4. Unfortunately, for the process illustrated, the design iterations are never confined to the successive steps.

330

 

 

STEP 1: P R O G R A M D E S I G N COMES F I R S T

The first step towards a f i x is illustrated in Figure 5. A preliminary program design phase has been

inserted between the software requirements generation phase and the analysis phase. This procedure can be

criticized on the basis that the program designer is forced to design in the relative vacuum of initial software

requirements w i t h o u t any existing analysis..As a result, his preliminary design may be substantially in error as

compared to his design if he were to w a i t until the analysis was complete. This criticism is correct but it misses

the point. By this technique the program designer assures that the software w i l l not fail because of storage,

timing, and data f l u x reasons. As the analysis proceeds in the succeeding phase the program designer must

impose on the analyst the storage, timing, and operational constraints in such a way that he senses the

consequences. When he justifiably requires more of this kind o f resource in order to implement his equations

it must be simultaneously snatched f r o m his analyst compatriots. In this way all the analysts and all the

program designers w i l l c o n t r i b u t e to a meaningful design process w h i c h w i l l culminate in the proper allocation

of execution time and storage resources. If the total resources to be applied are insufficient or if the e m b r y o

operational design is wrong it w i l l be recognized at this earlier stage and the iteration w i t h requhements and

preliminary design can be redone before final design, coding and test commences.

How is this procedure implemented? The f o l l o w i n g steps are required.

1) Begin the design process w i t h program designers, not analysts or programmers.

2) Design, define and allocate the data processing modes even at the risk o f being wrong. Allocate

processing, functions, design the data base, define data base processing, allocate execution time, define

interfaces and processing modes w i t h the operating system, describe input and o u t p u t processing, and define

preliminary operating procedures.

3) Write an overview d o c u m e n t that is understandable, informative and current. Each and every

w o r k e r must have an elemental understanding o f the system. A t least one person must have a deep understand-

ing o f the system which comes partially f r o m having had to w r i t e an overview document.

/ ALLOCATE ~ A DESCRIBE / sO..oOO,,. / c %

I

Figure 5. Step 1 : Insure that a p r e l i m i n a r y program design is complete before analysis begins.

331

 

 

S T E P 2 : D O C U M E N T THE DESIGN

At this p o i n t it is appropriate to raise the issue of – ” h o w much d o c u m e n t a t i o n ? ” My own view is

” q u i t e a l o t ; ” certainly more than most programmers, analysts, or program designers are w i l l i n g to do if left to

their own devices. The first rule of managing software development is ruthless enforcement of d o c u m e n t a t i o n

requirements.

Occasionally I am called upon to review the progress of other software design efforts. My first step is

to investigate the state of the d o c u m e n t a t i o n , If the d o c u m e n t a t i o n is in serious default my first

recommendation is simple. Replace project management. Stop all activities not related to d o c u m e n t a t i o n .

Bring the d o c u m e n t a t i o n up to acceptable standards. Management of software is simply impossible w i t h o u t a

very high degree of d o c u m e n t a t i o n . As an example, let me offer the f o l l o w i n g estimates for comparison. In

order to procure a 5 m i l l i o n dollar hardware device, I w o u l d expect that a 30 page specification w o u l d provide

adequate detail to control the procurement. In order to p r o c u r e 5 m i l l i o n dollars of software I w o u l d estimate

~ 1[,00 pa~e specification is about right in order to achieve comparable control,

Why so much d o c u m e n t a t i o n ?

1) Each designer must communicate w i t h interfacing designers, w i t h his management and possibly

w i t h thecustorner. A verbal record is t o o intangible to provide an adequate basis for an interface or manage-

m e n t d e c i s i o n . An acceptable w r i t t e n description forces the designer to take an unequivocal position and

provide tangible evidence o f c o m p l e t i o n . It prevents the designer from hiding behind t h e – ” l a m 9 0 – p e r c e n t

f i n i s h e d ” – syndrome m o n t h after month.

2) During the early phase of software development the d o c u m e n t a t i o n i . s t h e specification and i._~.s the

design. U n t i l coding begins these three nouns (documentation, specification, design) d e n o t e a s i n g t e t h i n g . If

the d o c u m e n t a t i o n is bad the design is bad. If the d o c u m e n t a t i o n does not yet exist there is as yet no design,

o n l y people t h i n k i n g and talking about the design w h i c h is of some value, but not much.

3) The real monetary value of good d o c u m e n t a t i o n begins downstream in the d e v e l o p m e n t process

during the testing phase and continues through operations and redesign. The value of d o c u m e n t a t i o n can be

described in terms of three concrete, tangible situations that every program manager faces.

a) During the testing phase, w i t h good d o c u m e n t a t i o n the manager can concentrate personnel on the

mistakes in the program. W i t h o u t good d o c u m e n t a t i o n every mistake, large or small, is analyzed by one man

w h o p r o b a b l y made the mistake in the first place because he is the o n l y man w h o understands the program area.

b) During the operational phase, w i t h good d o c u m e n t a t i o n the manager can use o p e r a t i o n – o r i e n t e d

personnel to operate the program and to do a better job, cheaper. W i t h o u t good d o c u m e n t a t i o n the software

must be operated by those w h o b u i l t it. Generally these people are relatively disinterested in operations and do

not do as effective a job as operations-oriented personnel. It should be p o i n t e d o u t in this connection that in

an operational situation, if there is some hangup the software is always blamed first. In order either to absolve

the software or to f i x the blame, the software d o c u m e n t a t i o n must speak clearly.

c) F o l l o w i n g initial operations, when system improvements are in order, good d o c u m e n t a t i o n permits

effective redesign, updating, and r e t r o f i t t i n g in the field. If d o c u m e n t a t i o n does n o t exist, generally the entire

existing f r a m e w o r k o f operating software must be junked, even for relatively modest changes.

Figure 6 shows a d o c u m e n t a t i o n plan w h i c h is keyed to the steps previously shown. Note that six

documents are produced, and at the time of delivery o f the final product, Documents No, 1, No. 3, No. 4,

No. 5, and No. 6 are updated and current.

332

 

 

/ ,

I0: w Z /oo i ,~ g ~

Irl . . o i 0 . i IIII ~,- ,,*,1 =

• . ~

illl ~ $ ~ m z ~ _ ~ u,

E

X

E 8

” 0

Ill N ~ , .~- r” .2

/ ” z_ ,,,. ~ ~ E ~OLU

a . . ~

N

N I Z ,~,- w i – , < ~

t –

LL

333

 

 

STEP 3: DO IT TWICE A f t e r d o c u m e n t a t i o n , the second most i m p o r t a n t criterion for success revolves around whether the

product is t o t a l l y original. If the computer program in question is being developed for the first time, arrange

matters so that the version f i n a l l y delivered to the customer for operational d e p l o y m e n t is actually the second

version insofar as critical design/operations areas are concerned. Figure 7 iltustrates how this might be carried

out by means o f a simulation. Note that it is simply the entire process done in miniature, t o a t i m e scale that

is relatively small w i t h respect to the overall e f f o r t . The nature of this e f f o r t can vary w i d e l y depending

primarily on the overall time scale and the nature o f the critical problem areas to be modeled. If the e f f o r t runs

30 months then this early development o f a p i l o t model might be scheduled for 10 months. For this schedule,

fairly formal controls, d o c u m e n t a t i o n procedures, etc., can be utilized. If, however, the overall e f f o r t were

reduced to 12 months, then the p i l o t e f f o r t could be compressed to three months perhaps, in order to gain

sufficient leverage on the mainline development. In this case a very special kind o f broad competence is

required on the part of the personnel involved. T h e y must have an intuitive feel for analysis, coding, and

program design. T h e y must q u i c k l y sense the t r o u b l e spots in the design, model them, model their alternatives,

forget the straightforward aspects of the design which aren’t w o r t h studying at this early point, and f i n a l l y

arrive at an error-free program. In either case the p o i n t of all this, as w i t h a simulation, is that questions of

timing, storage, etc. w h i c h are otherwise matters of judgment, can now be studied w i t h precision. W i t h o u t

this simulation the project manager is at the mercy of human judgment. With the simulation he can at least

perform experimental tests of some key hypotheses and scope d o w n w h a t remains for human judgment, which

in the area o f computer program design (as in the estimation of t a k e o f f gross weight, costs to complete, or the

daily double) is invariably and seriously optimistic.

I I,,, 1 I ANALYSIS I

! PROGRAM I I DES,GN I

– U coo,.o I LI .,s.,.o

USAGE

PRELIMINARY I % PROGRAM DESIGN

ANALYSIS

i PROGRAM DESIGN

TESTING

[ OPERATIONS Figure 7. Step 3: A t t e m p t t o do the job twice – the first result provides an early simulation o f the final product.

334

 

 

STEP 4: PLAN, C O N T R O L A N D M O N I T O R T E S T I N G

Without question the biggest user of project resources, whether it be manpower, computer time, or

management judgment, is the test phase. It is the phase of greatest risk in terms o f dollars and schedule. It

occurs at the latest p o i n t in the schedule when backup alternatives are least available, if at all.

The previous three recommendations to design the program before beginning analysis and coding, to

document it completely, and to build a p i l o t model are all aimed at uncovering and solving problems before

entering the test phase. However, even after doing these things there is s t i l l a t e s t phase and there are still

i m p o r t a n t things to be done. Figure 81ists some additional aspects to testing. In planning for testing, I w o u l d

suggest the f o l l o w i n g for consideration.

1) Many parts of the test process are best handled by test specialists w h o did not necessarily

contribute to the original design. If it is argued that o n l y the designer can perform a thorough test because

o n l y he understands the area he built, this is a sure sign o f a failure to document properly. With good

d o c u m e n t a t i o n it is feasible to use specialists in software product assurance w h o w i l l , in my judgment, do a

better job of testing than the designer.

2) Most errors a r e o f an obvious nature that carl be easily spotted by visual inspection. Every bit

of an analysis and every bit of code should be subjected to a simple visual scan by a second party w h o did not

do the original analysis or code but w h o w o u l d spot things like dropped minus signs, missing factors of two,

jumps to w r o n g addresses, etc., which are in the nature o f proofrea0ing the analysis and code. Do not use the

computer to detect this kind o f thing – it is t o o expensive.

3) Test every logic path in the computer program at least once w i t h some kind of numerical check. If

I w e r e a c u s t o m e r , I w o u l d not accept delivery until this procedure was completed and certified. This step w i l l

uncover the m a j o r i t y of coding errors.

While this test procedure sounds simple, for a large, complex computer program it is relatively d i f f i c u l t

to plow through every logic path w i t h controlled values of input. In fact there are those w h o w i l l argue that it

is very nearly impossible. In spite of this I w o u l d persist in my recommendation that every logic path be

subjected to at least one authentic check.

4) A f t e r the simple errors (which are in the majority, and which obscure the big mistakes) are removed,

then it is time to turn over the software to the test area for checkout purposes. A t the proper time during the

course of development and in the hands of the proper person the computer itself is the best device for

checkout. Key management decisions are: when is the time and w h o is the person to do final checkout?

STEP 5: I N V O L V E THE CUSTOMER

For some reason w h a t a software design is going to do is subject to wide interpretation even after

previous agreement. It is i m p o r t a n t to involve the customer i n a formal way so that he has c o m m i t t e d

himself at earlier points before final delivery. To give the contractor free rein between requirement

d e f i n i t i o n and operation is inviting trouble. Figure g indicates three points f o l l o w i n g r e q u i r e m e n t s d e f i n i t i o n

where the insight, judgment, and c o m m i t m e n t of the customer carl bolster the development effort.

S U M M A R Y

Figure 10 summarizes the five steps that I feel necessary to transform a risky development process

into one that w i l l provide the desired product. I w o u l d emphasize that each item costs some a d d i t i o n a l sum

of money. If the relatively simpler process w i t h o u t the five complexities described here w o u l d w o r k

successfully, then o f course the additional money is not well spent. Ii, my experience, however, the simpler

method has never w o r k e d on large software development efforts and the costs to recover far exceeded those

required to finance the five-step process listed.

335

 

 

l ~

~ m ~ I T _ ~ L_ ~.L

I w S i g I ~ o _ ~ E ,

_ I ” o ~ .~ O . . v a W

r

/ ,

336

r –

E

2

o

‘ 1

E 8

t – O

E ” O E

o E

8 t –

e,.

Q..

, m L L

 

 

C

l Q. /

( / )

I– z ~

i , , . u a ,~L r.n r r n

I n . J

i i 1

,< z <

E

337

Z o_ I.- < r , – i l l Q .

o

/’L__J (.-

r –

0 ( J

f –

t.-

0 . E ~n

E E

>o

+.~

I

E o

E 4..,

o r –

Q .

c ~

i i

 

 

I |

I ‘ I I

: i ] . ~ ‘ l l e ~$ ~ ~ i

n | ~ ~ u 8 (

I I .. I s ” ”

O0 0@’

0 O° ~

d

p@@@@@@~S.

I w

R

I.L.

338